Images: EEG montage used, beside international type; experiment overview; waveforms; difference topographies. For a more detailed view of the waveforms per group and electrode, visit: <b>https://pablobernabeu.shinyapps.io/export_files/</b><b>Abstract. </b>The engagement of sensory systems during word comprehension has been extensively documented; yet, the precise relevance of those remains unclear. We probed into this with an event-related potential (ERP) experiment which implemented the conceptual modality switch. This paradigm works as follows. In each trial, participants judge whether a property word can describe a concept word. However, the critical manipulation is the conceptual modality of the trials—e.g., haptic or visual—, as enabled by modality-normed stimuli. Switching across trials in different modalities, compared to maintaining a modality, incurs a switching cost. So far, experiments have measured this either on-line, from ERPs time-locked to the second word of the target trials, or off-line, from response times at the end of those trials. Problematically, both measurements fail to control a possible switch at the first word, as well as the semantic relation between the first and second words. In tackling the actual time frame of lexical and semantic access, we time-locked ERPs to the first word of target trials. Then, the experiment included different types of switch—from auditory to visual, and from haptic to visual—, which were compared to the non-switch—visual to visual. Further, we had a quick response group (<i>n</i> = 21), and a self-paced group (<i>n</i> = 21), alongside a few participants with no speed instructions (<i>n</i> = 5). The results, analyzed with mixed effects models, reveal ERP effects of modality-switching in four typical time windows between 160 and 750 ms after word onset. The effects are generally characterized by a more negative amplitude for modality-switching than not switching, and they arise with both types of switch, in both groups, and in anterior as well as posterior brain regions. In sum, the early start and broad scope of this effect suggest that perceptual simulation contributes fundamentally to word comprehension.