Classical views suggest that experienced affect is related to a specific bodily response, whereas recent perspectives challenge this view postulating that similar affective experiences rather evoke different physiological responses. To further advance this debate in the field, we used representational similarity analysis to investigate the correspondence between subjective affect (arousal and valence ratings) and physiological reactions (skin conductance response [SCR], startle blink response, heart rate, and corrugator activity) across various emotion induction contexts (picture viewing task, sound listening task, and imagery task). Significant similarities were exclusively observed between SCR and arousal in the picture viewing task. However, none of the other physiological measures showed a significant relation with valence and arousal ratings in any of the tasks. These findings are discussed within the framework of the Populations hypothesis, suggesting that physiological responses do not depend on the experienced affect but are directly associated with the context in which they are evoked.