Physical effort exertion has a substantial affective influence, which can impact performance and adherence in athletes and the general population. Yet, both the level of effort exertion and affective state during exercise are hard to monitor without the use of questionnaires, which suffer from certain biases and inaccuracies. Here, we examined whether prosodic features, which are among the most prominent characteristics of human expression, reflect the effort level and its related affect during physical exercise. To this end, we extracted prosodic features from verbal affective valence ratings recorded in a previously published study (n = 20; 10 women; nobs = 2,428) of resistance exercises performed until task failure. We found that the mean and SD of the pitch predicted effort-related affective valence and proximity to task failure in the two subsets of the data, and in three separate physical exercises. These results imply that mean pitch elevation and the decrease of the SD of the pitch during effort exertion may serve as a signal of distress as task difficulty increases. Moreover, the consistency of the findings across different types of exercises suggests that the mean and the SD of the pitch may be used to monitor physical effort and affect in various settings and help uncover the nature of physical effort in its different manifestations.