This describes how to calculate the reliability of word ratings (or other performance data, such as RTs). It gives R code and an example of how to use it.