Affect fluctuates in a moment-to-moment fashion, and it reflects the continuous relationship between the individual and the environment. Despite substantial research, there remain important open questions regarding how the continuous stream of sensory input is dynamically represented in experienced affect. Here, approaching affect as a temporally dependent process, we show that momentary affect is shaped by a combination of changes in recent stimuli (i.e. visually presented images for the current studies) and previously experienced affect. We also found that this temporally dependent relationship is influenced by context uncertainty. Participants, in each trial, viewed sequentially presented images and subsequently reported their affective experience, which was modeled based on images’ normative affect ratings and participants’ previously reported affect. Study 1 showed that self-reported valence and arousal in a given trial is partly shaped by the affective impact of the given images and previously experienced affect. In Study 2, we manipulated context uncertainty by controlling occurrence probabilities for normatively pleasant and unpleasant images in separate trials. Increasing context uncertainty (i.e. random occurrence of pleasant and unpleasant images) is associated with increased negative affect when the overall effect context is controlled. In addition, the relative contribution of the most recent image to momentary affect increased with increasing context uncertainty. Taken together, these findings provide clear behavioral evidence that affective experience fluctuates in a temporally dependent and continuous fashion based on recent changes in input variables and previous internal state, and that these fluctuations are sensitive to the affective context and its certainty.