Abstract This paper argues that Wittgenstein does not assimilate certainties to either linguistic norms or empirical propositions but assigns them to a liminal space between rule and experience. This liminal space is also brought into play in remarks written at the same time as those compiled in On Certainty, but attributed to different bodies of text ( Remarks on Colour, Last Writings on the Philosophy of Psychology ). The paper maintains that certainties express the agreement and constancy in judgements without which – as Wittgenstein contends in his Philosophical Investigations – rule-following would not be possible. It is shown that this intrinsic relation between rule-following and certainties can explain the liminal status of the latter.