Drawing on Gilles Deleuze and Félix Guattari’s analysis of language and literature, formulated in Kafka: pour une littérature mineure (1975), Mille plateaux (1980) and elsewhere, this article pursues the idea that certain forms of language, such as national languages or major literary discourses, might be conceptualized as states. Each operating as a locus of power, these forms of language codify regulatory rules (grammatical, lexical or stylistic, for example) that serve to stake out the boundaries of their territories; adherence to these rules and norms subsequently identifies belonging or non-belonging to a given linguistic or literary community. Concomitant with the notion of the linguistic state is the notion of linguistic statelessness, which might describe a general sense of alienation within language, such as that produced by the defamiliarizing effects of poetic discourse, or a more localized sense, prompted by the socio-political situation of certain marginalized, regional languages. Taking Deleuze and Guattari’s notion of ‘déterritorialisation’ as a point of departure, this article explores how certain forms of literature dismantle or disrupt dominant linguistic codes, pointing to a decentralized position outside of the established state. Here, literature involves a movement outside of a given linguistic territory, prompting a kind of statelessness within language. Having elaborated these notions of ‘state’ and ‘statelessness’, ‘territories’ and ‘deterritorialisation’ in greater detail, the article traces their configuration in the work of the contemporary French poet, Olivier Cadiot. It considers Cadiot’s first collection of poetry, L’art poétic’ (1988), and then his collaboration with the musician Rodolphe Burger on the album Welche: On n’est pas indiens c’est dommage (2000). In both instances, Cadiot uses ready-made language, employing cut-ups and sampling techniques that rework dominant discourses, deterritorializing them and making them ‘minor’.