Kun Ma  Kun Ma

Toward Stance Parameter Algorithm with Aggregate Comments for Fake News Detection

Abstract
In the detection of fake news, the stance of comments usually contains evidence supporting false news that can be used to corroborate the detected results of the fake news. However, due to the misleading content of fake news, there is also the possibility of fake comments. By analyzing the position of comments and considering the falseness of comments, comments can be used more effectively to detect fake news. In response to this problem, we proposed Bipolar Argumentation Frameworks of Reset Comments Stance (BAFs-RCS) and Average Parameter Aggregation of Comments (APAC) to use the stance of comments to correct the prediction results of the Roberta model. We use the Fakeddit dataset for experiments. Our macro-F1 results on 2way and 3way are improved by 0.0029 and 0.0038 compared to the baseline RoBERTa model's macro-F1 results at Fakeddit dataset. The results show that our method can effectively use the stance of comments to correct the results of model prediction errors.

Contributions
Bipolar Argumentation Frameworks of Reset Comments Stance(BAFs-RCS). Considering that some commenters are easily misled by fake news content and comment on content containing false information, we need to adjust the structure of the comment tree according to certain rules. After constructing the comment tree, the BAFs-RCS method adjusts the position of each subtree with the first comment as the root node in the tree according to the authenticity of the comment content. By redistributing the position of the comment subtree, we rationally use the comments containing false information to carry out the subsequent parameter aggregation process and turn the problem of itself as a negative effect into a positive effect.
Average Parameter Aggregation of Comments (APAC). The average parameter aggregation algorithm of comments calculates the degree of posture from the leaf node upwards according to the posture relationship between the sub-layer comments and the parent layer comments. In the process of calculating, we average the stance values of all the child nodes under a node to aggregate the stances of all the comments. Then, we combine the degree of the stance value with the parent node degree of the stance. Finally, stance values that aggregate all the comments are obtained. Thereby, the model classification error is corrected by the comments stance.
Fake news recognition integrated with stance detection. With experiment prove its feasibility and effectiveness in the 2-way classification and 3-way classification tasks. Based on the prediction results of the model, the news comments stance is combined to improve the efficiency of news recognition. We use Roberta to fine-tune the fake news classification task provided by Fakeddit to obtain the model with the best classification effect. Combined with the aggregate comments stance parameter algorithm, the accuracy of fake news recognition is improved.

Code & Data

The data set is Fakeddit v2.0.

Google Drive Link: https://drive.google.com/drive/folders/1jU7qgDqU1je9Y0PMKJ_f31yXRo5uWGFm?usp=sharing

Usage
Please note that results in the paper are based on multimodal samples only (samples that have both text and image). In our paper, only samples that have both image and text were used for the baseline experiments and error analysis. Thus, if you would like to compare against the results in the paper, use the samples in the multimodal_only_samples folder.

If there are Unnamed... columns, you can ignore or get rid of them. Use the clean_title column to get filtered text data.

comments.tsv consists of comments made by Reddit users on submissions in the entire released dataset. Use the submission_id column to identify which submission the comment is associated with. Note that one submission can have zero, one, or multiple comments.


Code & Data
Data: https://drive.google.com/drive/folders/1jU7qgDqU1je9Y0PMKJ_f31yXRo5uWGFm?usp=sharing

Cite

Publication

YinNan Yao, ChangHao Tang, Kun Ma, "Toward Stance Parameter Algorithm with Aggregate Comments for Fake News Detection," International Journal of Grid and Utility Computing, 2021

BiBTeX