Abstract The annotation scheme of dependency treebanks might have an impact on the results of linguistic analysis, thus leading to different interpretations of linguistic phenomena. This study compares the results of two widely used dependency measures, i.e., dependency direction and dependency distance, based on 18 parallel Universal Dependencies (UD) annotated treebanks and 18 corresponding Surface‐Syntactic Universal Dependencies (SUD) annotated treebanks. The results show that (1) Based on the semantic UD and syntactic SUD, dependency relations between function words and content words share the opposite dependency directions but similar dependency distances; (2) Annotation scheme has a significant impact on dependency direction, though the effect size is small. We find that the proportions of head‐final dependencies based on the syntactic SUD can better group language families than those based on semantic UD; (3) Annotation scheme also affects dependency distance significantly, though its effect size is small. Mean dependency distances (MDDs) based on UD are always higher than those based on SUD. However, the MDDs based on both annotation schemes are within a certain threshold, which shows that the linguistic universal of dependency distance minimization is independent of annotation schemes.