Abstract We present a computational language learning algorithm that can induce massively probabilistic grammars from treebank data. The paper is based on chapter 6 in my forthcoming thesis “Discontinuous Grammar: A dependency-based model of human parsing and language learning.” We start in section 1 by introducing the prerequisites from probability theory and statistics that are needed in the rest of the paper. In section 2, we define weighted grammars and massively probabilistic grammars, and discuss their relationship to standardly used grammars. In section 3, we address the problem of estimating probability distributions for hierarchically structured categorical data, and present an estimation algorithm that selects a hierarchical partition model by means of local search. We also describe a simulation study that shows that the algorithm performs well unless the distribution that generated the data is highly symmetric. Finally, in section 4, we outline how the algorithm can be used to learn massively probabilistic dependency grammars, exemplified by the task of learning the probabilities of complement structures.