We present our contribution to the EvaLatin shared task, which is the first\nevaluation campaign devoted to the evaluation of NLP tools for Latin. We\nsubmitted a system based on UDPipe 2.0, one of the winners of the CoNLL 2018\nShared Task, The 2018 Shared Task on Extrinsic Parser Evaluation and SIGMORPHON\n2019 Shared Task. Our system places first by a wide margin both in\nlemmatization and POS tagging in the open modality, where additional supervised\ndata is allowed, in which case we utilize all Universal Dependency Latin\ntreebanks. In the closed modality, where only the EvaLatin training data is\nallowed, our system achieves the best performance in lemmatization and in\nclassical subtask of POS tagging, while reaching second place in cross-genre\nand cross-time settings. In the ablation experiments, we also evaluate the\ninfluence of BERT and XLM-RoBERTa contextualized embeddings, and the treebank\nencodings of the different flavors of Latin treebanks.\n