Machine Learning Models for Predicting Emotional Valence from Brain Activity and Physiological Responses

In this study, we developed machine learning models to predict the valence class and valence rating of emotions experienced by participants based on their brain activity and physiological responses. The ICBHI 2024 Scientific Challenge involves using a rich dataset comprising pre-processed functional Magnetic Resonance Imaging (fMRI), photoplethysmography (PPG), and respiratory data from 20 participants. Each participant watched emotion-provoking video clips categorized into three valence classes (positive, negative, neutral) and rated them on a nine-level scale. Our approach integrates Convolutional Neural Networks (CNNs) for analyzing fMRI data and CNNs + Long Short-Term Memory (LSTM) networks for handling PPG and respiratory data. The models were trained to classify the valence class and predict the valence level, using a categorical cross-entropy as loss functions. Initial results show promising trends, indicating the model’s potential for accurate emotion prediction. fMRI model training and validation accuracy are 0.99 and 0.98 respectively. PPG and respiratory models accuracy are 0.86 and 0.66 on training and 0.80 and 0.56 on validation. However, further fine-tuning and architectural adjustments are necessary to enhance performance. This work aims to contribute to understanding how brain activity and physiological responses can be used to decode emotional states, with potential applications in psychological assessment and therapeutic interventions.