Advanced Emotion Recognition Using LSTM, RNN, and Transformer Models for Comprehensive Sentiment Analysis

DOI

10.2991/978-94-6239-693-7_95How to use a DOI?

Keywords

Multimodal Emotion Recognition; long short-term memory (LSTM); Recurrent neural networks (RNN); Transformer models; Sentiment Analysis

Abstract

The emotion recognition is a significant role in human-computer interaction, in the field of affective computing and intelligent decision-support systems. Most existing methods of sentiment analysis presuppose one channel as a text or speech one, which cannot be applicable in any real-world context when the emotional cues are manifested in numerous channels. The present paper has suggested a multimodal emotion recognition system integrating audio, video and text modalities by using a hybrid of a LSTM/RNN- based deep learning framework alongside Transformer framework to conduct a general sentiment analysis. The audio emotions are extracted by MFCC-based features representation and processed using the temporal LSTM network to obtain speech dynamics. To acquire spatiotemporal emotion patterns, analyses of facial expressions are performed based on video frame sequences using a deep visual model. At the same time, the textual emotional inferences are made with the assistance of speech transcripts and a Transformer- based language model that is fine-tuned. Three modalities are predicted and then the weighted late-fusion approach combines them to come up with one strong emotion label. The presented system is utilized as a real- time web app and it can be seen to be useful in practice-based affect-aware applications, such as mental health analysis, intelligent tutoring and customer behavior monitoring.

Copyright

© 2026 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - R. Asha
AU  - C. Lipika
AU  - Danush Priyan
PY  - 2026
DA  - 2026/06/16
TI  - Advanced Emotion Recognition Using LSTM, RNN, and Transformer Models for Comprehensive Sentiment Analysis
BT  - Proceedings of the International Conference on Intelligent Systems for a Sustainable Future (ISSF 2026)
PB  - Atlantis Press
SP  - 980
EP  - 992
SN  - 2589-4919
UR  - https://doi.org/10.2991/978-94-6239-693-7_95
DO  - 10.2991/978-94-6239-693-7_95
ID  - Asha2026
ER  -