Labeling data can be an expensive task as it is usually performed manually by\ndomain experts. This is cumbersome for deep learning, as it is dependent on\nlarge labeled datasets. Active learning (AL) is a paradigm that aims to reduce\nlabeling effort by only using the data which the used model deems most\ninformative. Little research has been done on AL in a text classification\nsetting and next to none has involved the more recent, state-of-the-art Natural\nLanguage Processing (NLP) models. Here, we present an empirical study that\ncompares different uncertainty-based algorithms with BERT$_{base}$ as the used\nclassifier. We evaluate the algorithms on two NLP classification datasets:\nStanford Sentiment Treebank and KvK-Frontpages. Additionally, we explore\nheuristics that aim to solve presupposed problems of uncertainty-based AL;\nnamely, that it is unscalable and that it is prone to selecting outliers.\nFurthermore, we explore the influence of the query-pool size on the performance\nof AL. Whereas it was found that the proposed heuristics for AL did not improve\nperformance of AL; our results show that using uncertainty-based AL with\nBERT$_{base}$ outperforms random sampling of data. This difference in\nperformance can decrease as the query-pool size gets larger.\n