Review Comment:
Overall evaluation
Select your choice from the options below and write its number below.
== 3 strong accept
== 2 accept
== 1 weak accept
== 0 borderline paper
== -1 weak reject
== -2 reject
-3 strong reject
Reviewer's confidence
Select your choice from the options below and write its number below.
== 5 (expert)
4 (high)
== 3 (medium)
== 2 (low)
== 1 (none)
Interest to the Knowledge Engineering and Knowledge Management Community
Select your choice from the options below and write its number below.
5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor
Novelty
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
3 fair
== 2 poor
== 1 very poor
Technical quality
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
2 poor
== 1 very poor
Evaluation
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
2 poor
== 1 not present
Clarity and presentation
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
2 poor
== 1 very poor
Review
In this paper, authors motivate the problem of categories that change over time, especially in the context of scientific research. In order to solve it they present AdvoCate, a tool that seeks a representation of this dynamic nature of categories, and records their evolution.
Authors address a very interesting problem, and its importance is well motivated from various angles, including philosophy, knowledge representation, databases and the Semantic Web. They present a novel approach, since some assumptions of the state-of-the-art (e.g. the fundamental facets of categories) are interpreted in an unseen way. The implementation of a system that leverages such assumptions also speaks in favour of the paper.
I have, however, important concerns about the clarity and scope of the paper. Authors claim that there is a lack of ‘connection’ between current ontology evolution tools and processes in science, being current approaches only top-down based, although there exists good data-driven work. The research questions addressed remain, thus, unclarified. Section 2 defines what apparently is an alternative framework for studying changing categories, where concepts and categories are considered different entities, the latter being composed of the classic facets of intension, extension and position in the hierarchy. These definitions and decisions are poorly justified (if any), and never formalised, which confuses the reader.
In Section 3 the approach is described, although not in an interesting way from the point of view of knowledge representation and management. Section 3.1 is too verbose about implementation details, while the rest never goes deep into the techniques and methods followed (e.g. the ‘change identification rules’ seem interesting, but no further detail on how this works is given; no explanation is given on the role of classifiers and machine learning in the pipeline). Authors do not compare their change operations with those of [17], defining their own in Table 1. Concise and detailed descriptions of these issues are needed. A pointer to the implementation source code or demo would be appreciated.
In summary, the paper tackles a fundamental topic for the conference, but it severely lacks concision, detail and formalism, so I suggest to reject it.
|