Recognizing a face as familiar is essential in our everyday life. However, 'familiarity' covers a wide range - from people we see every day to those we barely know. Although face recognition is studied extensively, little is known about how the degree of familiarity affects neural face processing, despite the critical social importance of this dimension. Here we report the results of a multivariate cross-classification EEG experiment, where we study the temporal representational dynamics of the degree of familiarity. Participants viewed highly variable face images of 20 identities. Importantly, we measured face familiarity using subjective familiarity ratings in addition to testing explicit knowledge and reaction times in a face matching task. A machine learning algorithm, trained to discriminate familiar and unfamiliar faces from a separate study, was used to predict the degree of face familiarity from the pattern of the EEG data. We found that the neural representations of the degree of familiarity emerge between 400 - 600 msec post-stimulus onset for famous persons. The correlation between decoding performance and behavioral familiarity was more reliable, occurred earlier and lasted longer when personally familiar and viewers' own faces were included in the analysis. Our findings provide new insights into how the brain represents faces with various degrees of familiarity and show that the degree of familiarity can be decoded reliably from the EEG at a relatively late time window. These results support the idea that representations of familiar faces form part of a general neural signature of the familiarity component of recognition memory processes.