Review Comment:
This submission studies the conceptual disarray in multi-level taxonomies in Wikidata, with a particular focus on two anti-patterns that capture confusion between subclasses and instances. The paper provides a detailed description of Wikidata's taxonomical organization, and analyzes how common the two anti-patterns are. These results are supplemented by an analysis and an initial tool that can be used to capture anti-patterns proactively at knowledge insertion time.
The overall idea of analyzing anti-patterns and integrating them into the Wikidata ecosystem is an attractive one, as there is a recognized confusion in the community with Wikidata's taxonomical organization. The analysis of the paper is interesting and informative, and the tool is a welcome supplement to help editors and knowledge engineers do their job more effectively.
I suggest that the following aspects should be improved to enhance the quality of the paper.
1. Motivation - the authors indicate that similar prior works exist (one of which is by a subset of the authors), which brings up the question about the novelty of this work. The introduction of the paper does not motivate the need for this study sufficiently. Why is this study needed, what gaps in existing research does it fill? Why is it interesting to run a similar experiment as in 2016 now? Some of these questions are touched on in the Related work section, but I wish that the authors summarized these deltas in the introduction as well to help the reader understand the significance of this work.
2. Generalizability/impact - The paper indicates that there are various anti-patterns, two of which are studied in this work. This brings up the questions: why these two anti-patterns; how many other anti-patterns are there; would these investigations generalize to the other anti-patterns and is that relevant to do in the future? In addition, the anti-patterns are not formally defined (the paper refers the reader to other works for this), which also harms the manuscript. I would suggest including a taxonomy of the patterns, with a formal definition at least of the two that are covered in this work, and a crisp motivation for the choice of these two anti-patterns.
3. Insights - The paper has a lot of interesting examples that manifest the challenge of the conceptual modeling of knowledge in Wikidata. Meanwhile, it would have been good if the authors have organized these examples into high-level insights/findings - this would help the reader have clearer takeaways from this paper, rather than merely reading about several (perhaps isolated) cases. Moreover, the analysis should IMHO provide more insights into the conceptual disarray - for instance, are they typically caused by bots or by manual edits? Do they also correlate with constraint violations in Wikidata?
4. Problems vs solutions - The paper points to many problematic cases, yet, there is very little information about what the solutions to these problems (manual, semi-, or automatic) would look like. This is also reflected in the tool, which points out violations of anti-patterns, but it stops there. Following some of the discussion of different ways to resolve some of the violations, I was expecting that the tool will provide a mechanism to assist editors to mitigate these violations more efficiently. Without that, it is questionable how useful the tool would be in practice. In fact, the concluding remarks section states that the methodology and the computational tool help users, but there is no evaluation of whether this is true for users at all, making this claim unsupported.
Other:
* "On the other hand" requires there to be an "On the one hand" clause before
* Footnotes should come after punctuation marks, not before
|