Background: Body image dissatisfaction, disordered eating, and eating disorders represent significant public health concerns; however, many affected individuals never access evidence-based support. We co-designed and developed a rule-based chatbot, JEM, which conducts conversations addressing evidence-based psychoeducation and psychotherapeutic microinterventions. We previously demonstrated the feasibility, acceptability, and preliminary satisfaction of the JEM chatbot in a research setting. However, broader satisfaction, experiences, and user-reported outcomes in real-world settings have not yet been investigated. Objective: This study aims to conduct a real-world evaluation of the JEM chatbot in Australia and Canada, the two countries that have hosted a deployment of the chatbot to date. Specifically, we aim to explore user satisfaction and experiences with the chatbot and within-session differences in user mood and body image satisfaction when completing the chatbot's microinterventions. Methods: Respondents were users of the JEM chatbot aged 13 to 64 years who self-selected to complete a web-based overall evaluation survey (N=230; n=122 in Australia and n=108 in Canada) over a 6-month period. This evaluation survey included user demographic characteristics, satisfaction measures, and the System Usability Scale. Respondents for the within-session pre-post analyses were JEM chatbot users who chose to complete brief web-based surveys immediately before and after completing one of the chatbot's microinterventions during the same 6-month period. Sample sizes varied across microinterventions, ranging from 75 to 276 respondents overall (Australia: n=34-146; Canada: n=39-130). These surveys included validated visual analog scales (VAS) measuring mood (anxiety, depression, happiness, confidence) and body image satisfaction (body size satisfaction, body shape satisfaction, physical attractiveness). Results: Demographic characteristics showed that survey respondents were commonly young adult cisgender women and nonbinary individuals across Australia and Canada. Respondent satisfaction with the chatbot was high in both countries (Australia: mean 76.1, SD 22.7; Canada: mean 78.8, SD 14.3), and the usability of the chatbot was rated as "excellent" in both countries (Australia: mean 86.5, SD 16.9; Canada: mean 89.5, SD 11.6) according to the System Usability Scale. Across completed microintervention surveys, patterns of within-session pre-post ratings were broadly similar in Australia and Canada, with effect sizes generally ranging from very small to large across VAS-measured mood and body image outcomes. Conclusions: The JEM chatbot achieved high satisfaction and usability ratings. Among respondents who completed pre-post surveys, immediate within-session differences in mood and body image ratings were observed following the completion of chatbot microinterventions. The study findings were broadly similar across Australia and Canada. These results provide evidence of user experience and within-session differences following engagement with JEM and support continued evaluation in future studies.