Several studies have investigated the neural responses triggered by emotional pictures, but the specificity of the involved structures such as the amygdala or the ventral striatum is still under debate. Furthermore, only few studies examined the association of stimuli's valence and arousal and the underlying brain responses. Therefore, we investigated brain responses with functional magnetic resonance imaging of 17 healthy participants to pleasant and unpleasant affective pictures and afterwards assessed ratings of valence and arousal. As expected, unpleasant pictures strongly activated the right and left amygdala, the right hippocampus, and the medial occipital lobe, whereas pleasant pictures elicited significant activations in left occipital regions, and in parts of the medial temporal lobe. The direct comparison of unpleasant and pleasant pictures, which were comparable in arousal clearly indicated stronger amygdala activation in response to the unpleasant pictures. Most important, correlational analyses revealed on the one hand that the arousal of unpleasant pictures was significantly associated with activations in the right amygdala and the left caudate body. On the other hand, valence of pleasant pictures was significantly correlated with activations in the right caudate head, extending to the nucleus accumbens (NAcc) and the left dorsolateral prefrontal cortex. These findings support the notion that the amygdala is primarily involved in processing of unpleasant stimuli, particularly to more arousing unpleasant stimuli. Reward-related structures like the caudate and NAcc primarily respond to pleasant stimuli, the stronger the more positive the valence of these stimuli is.