S{\\o}gaard (2020) obtained results suggesting the fraction of trees occurring\nin the test data isomorphic to trees in the training set accounts for a\nnon-trivial variation in parser performance. Similar to other statistical\nanalyses in NLP, the results were based on evaluating linear regressions.\nHowever, the study had methodological issues and was undertaken using a small\nsample size leading to unreliable results. We present a replication study in\nwhich we also bin sentences by length and find that only a small subset of\nsentences vary in performance with respect to graph isomorphism. Further, the\ncorrelation observed between parser performance and graph isomorphism in the\nwild disappears when controlling for covariants. However, in a controlled\nexperiment, where covariants are kept fixed, we do observe a strong\ncorrelation. We suggest that conclusions drawn from statistical analyses like\nthis need to be tempered and that controlled experiments can complement them by\nmore readily teasing factors apart.\n