Review Comment:
Summary of the paper and main contributions
--------------------------------------------
The present submission discusses some visual languages for ontology modeling, and in particular proposes an evaluation thereof, mainly based on nine criteria, adapted from the Physics of Notation Theory to the specificity of visual ontology languages. These criteria range from Completeness and Formality, which, roughly speaking, state that there should be a clear correspondence between objects of the W3C standard OWL2 and the visual symbols used in the graphical language (and vice-versa), to Perceptual Clarity (symbols should be clearly distinguishable with one another), Semantic Transparency (symbol appearance should suggest its meaning), Complexity Management (ability to deal with large ontologies), Totally Visual (no text needed to complement graphics), Parsimony/Polymorphism (use of limited amount of symbols), Cognitive Fit (fitting to the target users), and existence of an Editing Tool able to convert the graphical representation in the standard OWL2 syntaxes. Two additional criteria are also considered by the authors in their evaluation, i.e., the existence of a field experimentation and of a metamodel for the language.
The languages analyzed in the paper are:
- the MOT-OWL visual language (a previous proposal from one of the author of the present submission), which is described a bit more in details with respect to the other languages;
- UML-based visual languages, such as the Ontology Definition Metamodel and OWLGrEd;
- Non-UML visual Languages, such as GrOWL, the Graffoo notation, VOWL, Graphol, and G-OWL, an evolution of MOT-OWL proposed by the authors of the present submission.
Each language is evaluated with respect to the 11 mentioned criteria (the comparisons of the various languages are summarized in Table 1 in the paper). The authors finally present some guidelines on how to improve their G-OWL language, which turned out to be the most promising notation according to study presented in this submission.
Overall Evaluation
-------------------
It is unquestionable in my opinion that a visual notation for ontology representation is a crucial tool for both ontology modeling and its understanding by domain experts or other users that are not specialized ontologists. At the same time, currently there is no standard visual language for ontologies, nor a graphical notation that has proved to be more suited than others or that succeeded in becoming more popular and widely adopted. Thus, the topic studied in this paper is certainly important for the community, and the paper is relevant for the semantic web journal. I also appreciate the idea of trying to evaluate languages according to the above mentioned criteria (in particular the ones adapted from the Physics of Notation Theory). I think that such criteria are very reasonable, and I agree that graphical languages for ontologies should satisfy them as much as possible.
Nonetheless, I found several weaknesses in the paper, which in the end provides a limited contribution to the field. The main reason in my opinion is because the assessments given by the authors are not always based on clear evidences, but are often presented through non-properly supported claims. I think that requirements like semantic transparency or cognitive fit (but not only these) should be analysed through a user evaluation study, which is missing in this submission. Also, the level at which some of the languages are presented (and the lack of adequate bibliographic references, such as for G-OWL) makes it difficult to assess Completeness and Formality of these languages. As the same authors say, the table summarizing the comparisons among visual notations is simply a "brief evaluation...based on the analysis of one or two documents for each notation". I found this a bit disappointing, in particular because the reader has no idea about how complex these documents are, and whether they are significative enough to highlight the main peculiarities of each language, and allow for a fair comparison among languages. Then, I am not sure that some of the diagrams in different languages presented in the paper as equivalent are indeed equivalent (see also the specific comments below). It is also a bit disappointing that the paper does not provide a single running example ontology, described according to all the notations investigated in this paper. This is done only for some cases.
Specific Comments
------------------
- When presenting the nine principles in Section 1.3, the authors should also mention which ones can be measured and how, and for the others you should explicitly say how they can be evaluated. There is something on this point only for Semantic Transparency, which is associated to a scale ranging from -1 to +1, but how this can be established is not discussed. Nothing in this respect is said for the for the other requirements. What about, e.g., for Perceptual Clarity or Cognitive Fit (to name a few)? How can it be said that a language is better than another with respect to these criteria?
- From the bibliographic references provided [29,30 or also 31], it is difficult to evaluate the real usage in the community of the MOT-OWL notation for representing ontologies. This language is around from a while, but it seems to me that it has been limitedly used in applications, and the present submission does not really provide enough new elements that should lead to raise interest on it (the paper does not even provide indication on how to download or access the GMOT-OWL editor). Which is the novelty here?
- I found the presentation of the GMOT-OWL confused, and some of the authors' claims make me wonder whether this language has really the ability of capturing OWL2, or even OWL (I mean, the ability of expressing every OWL ontology). Below I list some unclear aspects and problems that I found:
* which is the meaning of equality/inequality symbols connected to classes?
* the group in the right bottom corner of the picture is not explained. What does it represent?
* The meaning of the label R is unclear: there are several edges in a universal, existential or cardinality restriction, with different roles, but all with the label R. Why? This R stands for "ruled by", according to what the authors say, but the meaning of this for each edge is obscure.
* In a universal or existential or cardinality restriction, it seems that the class that is restricted (as Parent, in Figure 2) is necessarily equivalent to the class that is denoted by the restriction (and it is really weird that the "Equi" label is not used here). This is not always the case in OWL. How can a containment between such classes be expressed? This ability is crucial to specify, for example, the typing of a relation, or the mandatory participation of instances of a class in a relation (and not necessarily these two properties together).
* related to the same example (Figure 2): the authors say "the direction of the R links show “Parent” as a domain of “Property1” and “Person” as its range". However, the OWL translation (as well as the First order logic translation) of the diagram is not saying this. According to the OWL piece of code, there can be individuals occurring in the domain of hasChild that are not instances of Parent (it is sufficient that the filler of the role is not a Person). Similarly for the range. So, what exactly is represented in Figure 2? (BTW, “Property1” should be "hasChild")
* If a class is both the union and the intersection of other classes, must it be repeated twice in the diagram?
Some of the above considerations apply also to G-OWL (which is presented as a sort of evolution of GMOT-OWL). The paper does not present the syntax for this second language, so it is really difficult to make a clear assessment. In particular, the claim that G-OWL respects Formality and Completeness is not supported by adequate explanation, nor there is a bibliographic reference where this is proved.
- Figures are not clearly readable throughout the paper. In particular Fig. 1, 3, 4, 5, 7, 8, 12 and 13 should be enlarged and/or should have a better quality. Figure 11 remains unreadable also looking at the pdf (where it can be zoomed).
- Section 3.3: the authors consider the object-oriented nature of UML (which, for example, allows for the presence of methods in classes) as an obstacle for its usage for ontology representation, since ontology interpretations are based on set theory. I disagree with this statement. Methods in UML classes can be simply ignored. Then, classes in UML can be interpreted as set theoretic. For a formal correspondence between UML and set theory the authors can have a look at:
Daniela Berardi, Diego Calvanese, Giuseppe De Giacomo:
Reasoning on UML class diagrams. Artif. Intell. 168(1-2): 70-118 (2005)
- Below, as examples of what I mean when I say "non-properly supported claims"", I list some of such claims:
* page 10, referring to the Graffoo notation: "The shape and colors are a bit too similar to promote Perceptual Clarity"
* page 11: "Graphol respects well the Parsimony Principle"
* page 11, again referring to Graphol:" Metahuman is **understood more directly** in figure 14 [MOT-OWL representation] as the intersection of the sets of all Human and the set of persons (anonymous class) that have at least one ability of a Superpower."
* page 13: "MOT-OWL and G-OWL...have a Good Cognitive Fit"
How did they authors arrive to these conclusions? More in general, it remains unclear how several of the values given in Table 1 have been established.
- Comments on the Graphol diagram in Figure 12: "a Vilain is a class for which there exists an archenemy that is a super hero." This is not correct. There is no mandatory participation for the instances of the class Vilain in the is_archenemy_of relation (notice that there is no arrow from Vilain to the domain of is_archenemy_of). Thus, there can be villains without archenemies. This error is reflected in the representation of the ontology in MOT-OWL (Figure 13). More in general it seems to me that the authors do not distinguish between typing of roles and mandatory participation in their ontologies. Thus I am not sure the diagram in Figure 12 exactly corresponds to the Graphol diagram in Figure 12 (this comment is also connected to my previous observation on the MOT-OWL notation). Thus, it does not turn out clearly that in this case the MOT-OWL model is more easy to read with respect to the Graphol one, in particular because the former uses less nodes (as the authors affirm). As said, I doubts the two models are equivalent, and my guess is that to have the equivalence the number of nodes in the MOT-OWL diagrams should be incremented. BTW, in their current versions, the Graphol diagram has 26 nodes, the MOT-OWL 2 has 24 nodes. So nothing can be inferred by this situation even by ignoring the possible error.
Minor issues and Typos
----------------------
- page 1: "OWL ontologies are schema that can process" I am not sure that process is the right term here. Do you mean that OWL ontologies can be expressed through RDF triples?
- page 2: "separating class, properties,..." -> "separating classes, properties,..."
- "A comparative survey of many other tools". Reference [9] is quite outdated (even in its 2004 revised version) and should be complemented with some more recent material.
- page 3: "for the use on ontologies " -> "for the use of ontologies "
- page 3: "has been used evaluate" -> "has been used to evaluate"
- page 5 "have at least one value in class "Person" by the property "hasChild" " -> please rephrase.
- page 5: "all of the individual" -> "all of the individuals"
- page 5: owl code in Fig. 2: there are unclosed tags
- page 5: "F,I,R,S for properties" -> "F,I,T,S for properties" (?)
- page 6: "in in overload" -> "in overload"
- page 7: in the last paragraphs of Section 2, Figure 4 and Figure 5 should be instead Figure 3 and Figure 4, respectively. Notice that references to the figures are wrong also in other points of the paper.
- page 7: "those Azaleas having exactly" -> "those individuals having exactly" notice that limitedly to Figure 6, a SingleColoredAzalea is not necessarily an Azalea.
- page 10: "are displayed to together" -> "are displayed together"
- page 11: "Shown on example of figure 12 are other kinds..." -> "Shown on example of Figure 12 there are other kinds..."
- page 11: "text like...are the type of restriction" please rephrase
- page 11: "...is a class for which there exist an archenemy..." -> "...is a class for which there exists an archenemy..."
- page 13: "To each semantic OWL object correspond a unique symbol...." wrong English sentence construction
- page 13: "one-t-one" -> "one-to-one"
- page 13: "Onto-Case4G-OWL Eclipse-based editor" Reference needed
- page 14: "can be linke to" -> "can be linked to"
- page 15: "can be specify" -> "can be specified"
| 
