Barlian Henryranu Prasetio, Edita Rosana Widasari
This study investigates acoustic correlates of affective prosody in emotional and stress-related speech using a hybrid phonetic-computational framework that combines digital signal processing, mixed-effects statistical modeling, and interpretable AI-assisted analysis. Acoustic features, including fundamental frequency (F0), intensity, temporal measures, and voice-quality indices, were extracted from nine publicly available speech corpora spanning multiple languages, elicitation paradigms, and speaker demographics. The analysis compared speech produced under psychological stress with canonical emotional speech in order to identify both shared and differentiating prosodic patterns across heterogeneous datasets. Across corpora, high-arousal emotions such as anger and fear were associated with elevated pitch, greater intensity, and shorter utterance durations, whereas sadness showed lower pitch, longer durations, and flatter prosodic patterns. In contrast, stress-related speech was more consistently associated with increased F0 variability, greater phonatory perturbation, and broader temporal dispersion, indicating reduced stability and coordination of prosodic realization rather than merely heightened expressive magnitude. Linear mixed-effects models were used as the primary inferential framework to account for speaker- and corpus-level variability, while Time-Delay Neural Networks (TDNNs), Support Vector Machine (SVMs), and SHapley Additive exPlanations (SHAP) were used in a complementary exploratory role to assess feature relevance in multivariate acoustic space. The results suggest that stress-related speech differs from canonical emotional prosody not only in global acoustic magnitude but also in regularity and control of prosodic execution. These findings provide a theory-informed basis for distinguishing stress from emotion in speech and have implications for phonetic analysis, affective computing, and speech assessment under cognitive and physiological load. © 2025 IEEE.
Universitas Brawijaya, Faculty of Computer Science, Malang, 65145, Indonesia
