We propose the use of the bootstrap resampling technique as a tool to assess the within-subject reliability of experimental modulation effects on event-related potentials (ERPs). The assessment of the within-subject reliability is relevant in all those cases when the subject score is obtained by some estimation procedure, such as averaging. In these cases, possible deviations from the assumptions on which the estimation procedure relies may lead to severely biased results and, consequently, to incorrect functional inferences. In this study, we applied bootstrap analysis to data from an experiment aimed at investigating the relationship between ERPs and memory processes. ERPs were recorded from two groups of subjects engaged in a recognition memory task. During the study phase, subjects in Group A were required to make an orthographic judgment on 160 visually presented words, whereas subjects in Group B were only required to pay attention to the words. During the test phase all subjects were presented with the 160 previously studied words along with 160 new words and were required to decide whether the current word was “old” or “new.” To assess the effect of word imagery value, half of the words had a high imagery value and half a low imagery value. Analyses of variance performed on ERPs showed that an imagery-induced modulation of the old/new effect was evident only for subjects who were not engaged in the orthographic task during the study phase. This result supports the hypothesis that this modulation is due to some aspect of the recognition memory process and not to the stimulus encoding operations that occur during the recognition memory task. However, bootstrap analysis on the same data showed that the old/new effect on ERPs was not reliable for all the subjects. This result suggests that only a cautious inference can be made from these data.