\chapter{Conclusion}
\label{ch:conclusion}

The aim of this thesis was to design and implement a system that is able to recognize the emotions from the input speech recordings. Although this topic is more than 20 years old, there is still no solution that would reach human-level performance, and none of them is even approaching. Based on this fact, a comprehensive analysis of methods, techniques and contemporary state-of-the-art solutions was required in order to get a good grasp of this topic. The analysis results showed that the current trend in the solutions performing at the state-of-the-art level is the utilization of deep learning models, particularly convolutional and recurrent neural networks.

Because there were a lot of solutions founded on mentioned neural network types and none that utilized the Transformer neural network, which is cutting-edge technology in natural language processing and image classification tasks, the author decided to propose a solution founded on Transformer. Throughout the research was found Audio Spectrogram Transformer (AST) neural network. It is a pure Transformer model (without any convolutional layers or recurrent units), originally proposed for the classification of audio sounds, which accepts mel spectrogram extracted from input recording. It is a derivative of Vision Transformer, pretrained on the ImageNet dataset, with a slightly adjusted linear transformation layer and positional encoding.

The proposed solution is a pipeline that loads speech recording and transforms it to the mel spectrogram with the required shape defined by the size of the mel filter bank and the number of time frames. This spectrogram is processed by AST neural network outputting the probabilities for considered emotion classes.

The implemented neural network was trained on three datasets: RAVDESS, Emo-DB and EMOVO. For each dataset was performed 10-fold cross-validation. The evaluation was conducted for each dataset separately. The goal was to find out how well the proposed neural network is able to classify emotions and check if dataset split influences model performance. Therefore, the methodology consisted of standard classification metrics computation: accuracy, precision, recall, F1 score and specificity. The input for metrics formulas was test predictions evidence that contained predictions for a particular dataset. Despite not achieving state-of-the-art performance, the obtained unweighted accuracy values are promising:  84.5~\% for RAVDESS, 91.6~\% for Emo-DB and 73.8~\% for EMOVO. 

Throughout the training were tracked their environmental impact as a rough estimation of CO\(_2\) mass based on the consumed energy of the used GPU. According to the measured values, there was emitted 1058.3703 grams of CO\(_2\) in total. The tool used for measurement, however, failed in the evaluation and therefore it was conducted in a custom way.

The directions for future work are in testing the AST model with different settings that would allow studying their impact on performance and the utilization of the bigger emotional corpus with natural-based speech recordings that would show model capabilities on real-world data.
 