Supervised learning algorithms often require large amounts of labeled data.Creating this data can be time consuming and expensive.Recent work has used untrained annotators on Mechanical Turk to quickly and cheaply create data for NLP tasks, such as word sense disambiguation, word similarity, machine translation, and PP attachment.In this experiment, we test whether untrained annotators can accurately perform the task of POS tagging.We design a Java Applet, called the Interactive Tagging Guide (ITG) to assist untrained annotators in accurately and quickly POS tagging words using the Penn Treebank tagset.We test this Applet on a small corpus using Mechanical Turk, an online marketplace where users earn small payments for the completion of short tasks.Our results demonstrate that, given the proper assistance, untrained annotators are able to tag parts of speech with approximately 90% accuracy.Furthermore, we analyze the performance of expert annotators using the ITG and discover nearly identical levels of performance as compared to the untrained annotators.