基于语义相关的多模态社交情感分析

社交平台允许用户采用多种信息模态发表意见与观点，多模态语义信息融合能够更有效地预测用户所表达的情感倾向。因此，多模态情感分析近年来受到了广泛关注。然而，多模态情感分析中视觉与文本存在的语义无关问题，导致情感分析效果不佳。针对这一问题，提出了基于语义相关的多模态社交情感分析（MSSA-SC）方法。采用图文语义相关性分类模型，对图文社交信息进行语义相关性识别，若图文语义相关，则对图文社交信息使用图文语义对齐多模态模型进行图文特征融合的情感分析；若图文语义无关，则仅对文本模态进行情感分析。在真实社交媒体数据集上进行了实验，由实验结果可知，所提方法能够有效降低图文语义无关情况对多模态社交媒体情感分析的影响。与此同时，所提方法的Accuracy和Macro-F1指标分别为75.23%和70.18%，均高于基准模型。 

Multimodal social sentiment analysis based on semantic correlation

Social platforms allow users to express opinions in a variety of information modalities, and multi-modal semantic information fusion can more effectively predict the emotional tendencies expressed by users. Therefore, multimodal sentiment analysis has received extensive attention in recent years. However, in multi-modal sentiment analysis, there is a problem of unrelated semantics between vision and text, resulting in poor sentiment analysis. In order to solve this problem, this paper proposes the Multimodal Social Sentiment Analysis based on Semantic Correlation (MSSA-SC) method. The MSSA-SC firstly adopts the semantic relevance classification model of image and text to identify the semantic relevance of the image-text social media. If the image and text are semantically related, the image and text semantic alignment multimodal model is used for the image-text feature fusion for the image-text social media sentiment analysis. When the image and text semantics are irrelevant, only the sentiment analysis is performed on the text modality. The experimental results on real social media datasets show that the MSSA-SC method can effectively reduce the influence of unrelated image and text semantics on multimodal social sentiment analysis. Moreover, the Accuracy and Macro-F1 of the MSSA-SC method are 75.23% and 70.18%, respectively, and outperform those of the benchmark model.