Alexithymia is believed to involve deficits in emotion processing and imagery ability. Previous findings suggest that it is especially related to deficits in processing the arousal dimension of emotion, and that discordance may exist between self-report and physiological responses to emotional stimuli in alexithymia. The current study used a well-established emotional imagery paradigm to examine emotion processing deficits and discordance in participants (N = 86) selected based on their extreme scores on the Toronto Alexithymia Scale-20. Physiological (skin conductance, heart rate, and corrugator and zygomaticus electromyographic responses) and self-report (valence, arousal ratings) responses were monitored during imagery of anger, fear, joy, and neutral scenes and emotionally neutral high arousal (action) scenes. Results from regression analyses indicated that alexithymia was largely unrelated to responses on valence-based measures (facial electromyography, valence ratings), but that it was related to arousal-based measures. Specifically, alexithymia was related to higher heart rate during neutral and lower heart rate during fear imagery. Alexithymia did not predict differential responses to action versus neutral imagery, suggesting specificity of deficits to emotional contexts. Evidence for discordance between physiological responses and self-report in alexithymia was obtained from within-person analyses using multilevel modeling. Results are consistent with the idea that alexithymic deficits are specific to processing emotional arousal, and suggest difficulties with parasympathetic control and emotion regulation. Alexithymia is also associated with discordance between self-reported emotional experience and physiological response to emotion, consistent with prior evidence.