Review Comment:
The article introduces the TAXODIS taxonomy. The taxonomy provides 66 concepts for annotating online misinformation. The taxonomy is developed through a systematic review of existing research, and it has been published using the SKOS vocabulary. Overall, the paper is easy to follow, and the TAXODIS model responds to the clear need for more structured misinformation annotation.
Detailed comments:
1) Novelty and relationship to prior work
The article appears to extend two previous contributions ([13] and [19]) by only adding a sixth aspect to the previously published taxonomy and providing a formal SKOS implementation. However, the clear difference between this work and the previous publications is not sufficiently discussed. The difference is currently addressed with a single sentence in the introduction and a short paragraph in the second section.
The authors mention that a previous paper presented an earlier unimplemented version of the taxonomy, but do not discuss what has changed besides the formal implementation. These differences remain unclear, and the discussion fails to highlight the extent of the changes. The paper should be more upfront about what is new and what is not, including where content or findings are reused. These distinctions should be highlighted within the body of the paper (e.g., Section 3) rather than only in the related work.
2) Conceptual Scope
The article focuses on the creation of a taxonomy for NLP and machine learning tasks as a facilitating tool for data annotation. This initial premise is somewhat restrictive, as the proposed taxonomy appears to be designed more as a codebook than a model to aid the broader understanding of online disinformation.
3) Related work
While extensive, the related work seems to largely dismiss previous efforts in building categorisation schemes for misinformation (e.g., "None of the mentioned efforts above propose a shared semantic model"). Furthermore, the discussion does not address in detail existing efforts to extract and annotate misinformation automatically, focusing primarily on LIWC.
There is also a lack of discussion regarding existing datasets and knowledge graphs. For example, the paragraph starting at line 37 on page 3 discusses MultiFC and ClaimsKG but fails to mention CimpleKG, which provides specific misinformation, textual and linguistic features. CimpleKG is only mentioned briefly on page 12 without sufficient context.
4) Modelling
The taxonomy follows established practice by reusing SKOS, which facilitates the integration of TAXODIS into other knowledge sources. However, the use of SKOS may not be the most appropriate choice for an aspect-based taxonomy. Such a model might be better coupled with a more traditional ontological model to avoid a faceted taxonomy structure (e.g., by separating veracity, categorisation, and detection features).
The decision to use SKOS rather than a more formal ontological model should be discussed. For instance, characteristic values and boundaries may not be suitably represented in SKOS, and constraints for veracity grades are not well represented by the taxonomy (see how schema.org represents Rating). The example in Section 4.2 may be better represented using an ontological model built upon the ClaimReview model.
5) Methodology and usage
The methodology is sound and draws from multiple sources with a systematic approach. However, as previously noted, the authors should be clearer about what is drawn from [13] and [19] compared to what is new, as these articles overlap with the presented methodology and findings.
Regarding usage, the authors list potential queries combining Schema.org and OA. While this provides context, it is important to note that some of these queries can already be answered using existing vocabularies and knowledge graphs. For example, ClaimReview can already be used to retrieve "mostly false" claims, while CimpleKG can be queried for misinformation factors and mentions. This should be discussed in greater detail.
6) Evaluation
The evaluation section largely refers to a prior evaluation of the model; there is no new evaluation of the SKOS implementation or its integration with other resources. As a result, the paper provides limited novel insights regarding the real-world usage of the taxonomy implementation. Finally, the maintenance and monitoring plan appears to be an afterthought. For example, ClaimsKG is now largely outdated. More current knowledge graphs exist, such as DBFK and CimpleKG.
|