Feasibility of Automated Foundational Ontology Interchangeability

Overall evaluation
Select your choice from the options below and write its number below.

== 3 strong accept
== 2 accept
== 1 weak accept
== 0 borderline paper
X == -1 weak reject
== -2 reject
== -3 strong reject

Reviewer's confidence
Select your choice from the options below and write its number below.

== 5 (expert)
X == 4 (high)
== 3 (medium)
== 2 (low)
== 1 (none)

Interest to the Knowledge Engineering and Knowledge Management Community
Select your choice from the options below and write its number below.

== 5 excellent
X== 4 good
== 3 fair
== 2 poor
== 1 very poor

Novelty
Select your choice from the options below and write its number below.

== 5 excellent
== 4 good
== 3 fair
X== 2 poor
== 1 very poor

Technical quality
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
X== 3 fair
== 2 poor
== 1 very poor

Evaluation
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
X== 2 poor
== 1 not present

Clarity and presentation
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
X== 3 fair
== 2 poor
== 1 very poor

Review
The authors present the tool SUGOI, used for automatic interchange between different foundational ontologies being mapped to a domain ontology. Foundational ontologies have been developed to facilitate the interoperability between domain ontologies, however, since their number has risen over the years, we are facing a new set of semantic conflicts. SUGOI offers a solution when swapping between different foundational ontologies, with the raw interchangeability measure averaging at 36% (ranges from 2% to 82%, not considering subsumptions). The evaluation is performed on 16 ontologies, linked to DOLCE, BFO and GFO foundational ontologies.

The paper seems to lack clearly defined motivation and interpretation of the obtained results.

It is stated that predefined mapping files are used as input to the SUGOI algorithm, yet it appears that the algorithm produces the same result any existing reasoner would by making implicit or transitive equivalences explicit. The novelty of the proposed algorithm and its contribution in improving the mapping is lacking.

Furthermore, the results' interpretation and findings are not entirely clear. Aren’t the results predictable based on the (already known) quality of the existing mapping files? Since the algorithm itself doesn’t contribute too much in resolving existing semantic issues, isn’t the whole evaluation just dependent on the initial quality of mapping files (which have already been published)?

The problem of interchanging foundational ontologies is indeed validation. Since foundational ontologies refer to philosophical ideas, a mapping between such ontologies has to be created very carefully with respective philosophical foundations in mind. In my opinion it might be not feasible to achieve a proper 1:1 mapping in any case. Regarding the entity-level analysis in the paper bfo:Continuant and gfo:Presential are not interchangeable contrary as proposed, endurants are modeled differently in GFO compared to BFO and DOLCE, while bro:Continuant has a timely dimension gfo:Presential doesn’t have, gfo:Presentials rather are instances of gfo:Persistants which contain the timely extent of such concepts. One possibility to model such mappings would be to introduce interchangeable patterns.

Typos:
* sec 2.1: "… and coverts …"
* sec 3.2 Comparing SUGOI to manual mappings: "Fig. 3" should be "Table 3“
dolce-physial-endurant
* Fig. 2 does not really contribute to the content of the paper

Overall evaluation
== 2 accept

Reviewer's confidence
== 5 (expert)

Interest to the Knowledge Engineering and Knowledge Management Community
== 5 excellent

Novelty
== 4 good

Technical quality
== 4 good

Evaluation
== 4 good

Clarity and presentation
== 4 good

This paper tries to answer the question: To what extent can we take a
domain-specific KB developed using one upper ontology and use it with
another upper ontology?

Since upper ontologies are used by several research groups for creating
domain specific knowledge bases, this sort of analysis is very interesting.
Even though one may never want to port the KB from one upper ontology
to another.

The authors present empirical data on how many of the links to one upper ontology
can be automatically mapped to another. The analysis of the data also reveals
the relative differences in the coverage between different upper ontologies.

The basis of the mapping is a detailed algorithm which covers a variety of cases
such as when no equivalent class exists in the target ontology. Even though I did
not check the correctness and completeness of the algorithm, but based on a few
spot checks, it seemed sound and thorough. This algorithm is implemented in a
tool called SUGOI that can perform the mappings automatically.

The evaluation consider an impressively diverse set of ontologies.

I liked the idea of interchanging the ontologies in both directions and checking if
the original ontology is obtained in the round trip. The results of such round-tripping
were, however, not prominently explained.

It will be interesting hear the authors speculate about whether one can derive
more or less conclusions by changing the upper ontology. How does changing
the upper ontology affects the deductive closure of domain-specific conclusions
that one can derive?

Some detailed and specific comments:

page 3 - what is DMOP? SAO? (these acronyms are not explained until later in the paper)
page 4 - Figure 1, why swap the location of DataType and Strategy
page 7 - change "OWLized" to "OWL version of"
page 8 - pre-processing of ontologies, and then, on page 9, the first
paragraph in the results section are trivial engineering details
and could be dropped

Overall evaluation
Select your choice from the options below and write its number below.

== 3 strong accept
== 2 accept
== 1 weak accept
== 0 borderline paper
== -1 weak reject
== -2 reject
== -3 strong reject
1

Reviewer's confidence
Select your choice from the options below and write its number below.

== 5 (expert)
== 4 (high)
== 3 (medium)
== 2 (low)
== 1 (none)
3

Interest to the Knowledge Engineering and Knowledge Management Community
Select your choice from the options below and write its number below.

== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor
4

Novelty
Select your choice from the options below and write its number below.

== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor
4

Technical quality
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor
4

Evaluation
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 not present
4

Clarity and presentation
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor
4

Review
Please provide your textual review here.

This paper proposes a tool for interchanging foundational ontology when used by some domain ontology. Essentially, this is a fancier version of "find-and-replace" for domain ontologies that use some foundational ontology. I think, the interchangeability metric is also aan important contribution. Technically, I did not spot a significant flaw.

What is unclear to me is what did a rather low interchangeability score tell us. The authors should state more clearly how this score can help better alignment between foundational ontologies. I think this is a more important question than simply asking whether two foundational ontologies can be interchanged. Also, foundational ontologies are typically very complex, so although the tool could provide a list of non-mapped entities to help the ontology maintainers improve their foundational ontologies, it is not clear how such an improvement to those foundational ontologies can be actually done without semantically jeopardizing the carefully designed existing axioms in them.

-- minor --
p3, par 3: that did has been --> that did that has been