News & Updates

Affective experiences are multifaceted, and researchers often rely on facial expressions and continuous ratings to capture this complexity. Although both modalities are often treated as indicators of similar affective states, prior work has reported differences between them. However, most studies measured them in separate samples, so such differences may instead reflect between-sample heterogeneity rather than complementary information. We addressed this issue by recording facial expressions and ratings concurrently while 37 participants watched films, capturing video of their faces and continuous ratings. Facial valence dynamics were derived with ResMaskNet and compared with ratings using within-subject regression. We found that the association between these two measurements was not significant, r = .05, p = .63, consistent with the view that the two measures index distinct facets of affect. A potential concern is that dual-task demands might distort spontaneous facial responses. To evaluate this, a second cohort of 103 participants viewed the same films without providing ratings. Their facial valence dynamics closely matched those of the rating group, r = .98, p < .001, indicating that concurrent self-reports did not measurably distort facial signals, which primarily reflected stimulus-evoked affect. Our findings showed that data from different modalities provide unique information, highlighting the need for multimodal approaches to understand affective experiences.