Review Comment:
This paper presents a verbalization approach for OWL axioms that aims at removing redundancy in a sophisticated way to provide more helpful and more concise sentences to show formal facts to users. The overall aim is interesting and the paper contains interesting parts, but also suffers from some major issues: It doesn't appropriately discuss some serious limitations, it is too technical, too complicated, and too long for the type of its contribution, core elements contain errors or are at least very confusing, and the positive evaluation results only seem to stem from an inappropriate choice of baseline. Therefore I cannot suggest to accept the paper.
My main points of criticism:
- The restriction to OWL "individuals and concepts" is not well motivated. OWL axioms involving properties can be expected to be harder to understand, given that properties are logically more complex than individuals and classes. Therefore the restriction to individuals and classes is a serious one that limits the potential impact and usefulness.
- The use of axioms like "cat subClassOf animal" to remove "has as pet an animal" if we already know "has as pet a cat" implies that we are starting from a consistent ontology. This means that we cannot use the presented approach as a method to find the errors in inconsistent ontologies, which actually is an important problem that such verbalization tools could help us with. This assumption and the resulting limitation are not discussed.
- The paper contains much formal notation that can be better explained in text. These formal notations are moreover not always used consistently (see below).
- It is unclear in what sense the generated descriptions are "redundancy-free". Some steps actually seem to add redundancy (see below).
- The definition of strictness is either wrong or highly counter-intuitive (see below).
- Overall, the approach seems overly complex given the relatively simple nature of the problem and lacks elegance and/or concise intuitive notations and descriptions. The paper is also too long in my view and I had difficulties to see the broader picture while reading through the technical descriptions.
- The baseline for the empirical evaluation is not a fair one, and the positive results seem to stem just from that unfair baseline choice (see below).
Detailed comments:
Abstract:
- Remove comma: "show that, semantic-refinement"
Introduction:
- "Web Ontology Language (OWL/DL)": DL part of acronym is not explained
- Remove commas: "so that, an intelligent agent with the help of a reasoning system, can"
- I don't find this to be a helpful or accurate explanation of what ontologies are: "Ontologies play an important role in the development and deployment of the Semantic Web since they help in enhancing the understanding of the contextual meaning of data."
- "complex relational context" ?
- "mainly strive for one-to-one conversion of logical statements to NL texts": This is not true for the cited ACE work [8,9]. These relations are not one-to-one there (different equivalent OWL axioms can lead to the same ACE text).
- The introduction contains many unsupported claims. These need evidence/references:
- "Typically, ontologies are developed by a group of knowledge engineers with the help of domain experts."
- "... the process usually follows a spiral model ..."
- "... the quality of the ontology might degrade"
- "... usually an ontology development cycle is accompanied by a validation phase"
- and many more!
Related Work:
- I don't understand how the last part ("hence ...") follows from the rest: "The Semantic Web Authoring (SWAT) NL verbalization tools have given much importance to the fluency of the verbalized sentences [13], rather than removing redundancies from their logical forms, hence have deficiencies in interpreting the ontology contents"
Section 4:
- "In this paper, we use the words “reduction” and “refinement” interchangeably.": I don't see the benefit of this, and I think the authors should commit to just one of them to avoid confusion.
- Often formal notation is used in a way that rather hinders than clarifies the reader's understanding. For example, Definition 2 is just a complicated way of referring to all inferred relations R(x,y) where R is an atomic role. Algorithm 1 is another example, which could be explained in just one sentence.
- L_O(x) is defined in Definition 1 as a structure that is fixed given x and O. However, in Algorithm 1, this notation is misused as a data structure that can be updated (and therefore can have different values for a given x and O throughout the execution of the algorithm).
- Step 2 of the algorithm on page 6 seems to introduce redundancy, which seems strange for an approach that claims "redundancy-free" representations. At least, I didn't understand what purpose is served by introducing this redundancy at that point.
- "Clearly, including the latter description in the verbalization may confuse a reader." I don't think that this statement is so obvious that it doesn't need justification.
- "if a role restriction R1 is implied by another role restriction R2 (i.e., R2 ⇒ R1), then R1 can be said as a stricter version of R2.": That doesn't seem to be right. If R1 follows from R2 then R1 is less strict and not more. Whenever R1 is violated so is R2, but not necessarily vice versa. So, R2 can be violated while R1 is not, therefore R2 is "stricter" in its intuitive sense.
- I don't find the illustrative example on page 11 ("Illustration of the usefulness of the approach") particularly convincing. The original verbalization for a start seems to suffer from different issues except redundancy, specifically the grammatical error of number agreement in "two advisors who is a teaching staff". Then, more importantly, I learn in the original sentence that Sam is a PhD student, which I am not getting from the refined version, which makes me wonder whether the refined version is really more useful.
- The empirical evaluation doesn't seem to be a fair one, as the "traditional approach" is not really what is normally done. The Harry Potter ontology, for example, doesn't contain axioms that Hermione Granger "has as pet only creature, has at least one creature, has at most one creature, as pet". These are inferred properties, and inferring all such properties of course (by definition) introduces unnecessary redundancy. Other verbalization approaches normally don't do that, and so this is not a valid baseline to compare the proposed approach to. By only verbalizing the explicit axioms one gets a result that is very similar to the "proposed approach" one, and there is no reason to believe that it would have performed worse than the proposed approach with a user evaluation like the presented one. The worse performance of the "traditional approach" can easily be explained by the very artificial and abstract phrases like "has as factor only organism and has as factor something", which are not explicitly in the original ontologies.
|