Social-IQ-2.0-Multimodal-with-Emotional-Cues

This repository is the implementation of Multi-Modal Correlated Network with Emotional Reasoning Knowledge for Social Intelligence Question-Answering.

Requirements

Please follow the instructions of the Social-IQ 2.0 benchmark to download the dataset and the required dependecies.

Then, to install requirements of this repository:

pip install -r requirements.txt

Feature Extraction

To extract the features, run the follwing commands in the features folder:

python audio_featureExtraction.py
python video_featureExtraction.py
python emotion_featureExtraction.py

Training

To train the model in the paper, run this command:

python -m torch.distributed.launch --nproc_per_node=4 main.py --ngpu 4

Evaluation

To evaluate the model in the paper, run this command:

python evaluation.py