Objectives To use the dot memory task to examine the impact of early-onset depression (ED) on visual working memory (VWM) in later life when compared to those with no history of depression (CON). As an exploratory aim, we examined VWM in remitted vs non-remitted depression when compared to those with no history of depression. Design, Setting, Participants, and Measurements 79 participants aged 55-79 underwent lab-based neuropsychological testing. They were then provided with a study-supplied smartphone and were prompted to complete ecological momentary assessments over 14 consecutive days, receiving 3 separate notifications a day. Prompts consisted of a brief survey regarding recent activities, affect ratings, and use of emotion regulation strategies, followed by 3-5 brief cognitive tasks. VWM was measured with a dot memory task. Both response time and error distance were calculated. Results Mean response rate was 70%. The ED group had significantly slower response times at baseline than the CON group (18.9% slower, 95% CI 2.7–37.6%, p=0.023). Across the study period, the ED group's response time decreased faster relative to CONs (-0.8% per day, 95% CI -1.6 to +0.0%, p=0.057), an effect driven by the actively depressed (AD) subgroup, whose response time decreased significantly faster (AD vs CON: -1.3% per day, 95% CI -2.3 to -0.4%, p=0.007).). There was no significant difference in either the level or the rate of change of error distance between CON and ED participants. Within-person variability (standard deviation) in response time did not differ by group or change over time; within-person variability in error distance declined over the study period in CONs but declined significantly less in the ED group (ED × time: 0.02 units/day, 95% CI 0.001–0.039, p=0.037). Conclusions Our findings suggest that the VWM domain may be a malleable target for state-improvements in cognitive functioning in depressed patients.