Comparison and Evaluation of Ontologies for Units of Measurement

Tracking #: 1775-2987

Authors: 
Jan Martin Keil
Sirko Schindler

Responsible editor: 
Boyan Brodaric

Submission type: 
Survey Article
Abstract: 
Measurement units and their relations like conversions or quantity kinds play an important role in many applications. Thus, many ontologies covering this area have been developed. Consequently, for new projects aiming at reusing one of these ontologies, the process of evaluating them has become more and more time consuming and cumbersome. We evaluated eight well known ontologies for measurement units and the relevant parts of the Wikidata corpus. We automatically collected descriptive statistics about the ontologies and scanned them for potential errors, using an extensible collection of scripts. The computational results were manually reviewed, which uncovered several issues and misconceptions in the examined ontologies. The issues were reported to the ontology authors. This caused new bugfix releases in three cases. In this paper we will present the evaluation results including statistics as well as an overview of detected issues. We thereby want to enable a well-founded decision upon the unit ontology to use. Further, we hope to prevent errors in the future by describing some pitfalls in ontology development—not limited to the domain of measurement units.
Full PDF Version: 
Tags: 
Reviewed

Decision/Status: 
Minor Revision

Solicited Reviews:
Click to Expand/Collapse
Review #1
By Steve Ray submitted on 15/Dec/2017
Suggestion:
Accept
Review Comment:

On the second submission, I see that many of the issues I raised with the first submission have now been fixed. I see, for example, that the authors have simply removed what had been Figures 3-6, which I suppose does eliminate the risk of confusion. They expanded the abbreviations in Table 1. I now understand Table 3 to be read as (for example): ‘There are 275 unit definitions that appear in 3 of the ontologies studied’ and ‘There are 17 unit definitions that appear in all 9 of the ontologies studied’. If this is not the correct interpretation, I’m afraid it needs further explanation.
Overall, I still believe this is a valuable paper that will be helpful to the Semantic Web community, as people consider incorporating units of measure into their models. As a collaborator in the QUDT ontology, I look forward to the analysis of QUDT 2 once it is fully released.

Review #2
By Hajo Rijgersberg submitted on 28/Dec/2017
Suggestion:
Minor Revision
Review Comment:

This manuscript was submitted as 'Survey Article' and should be reviewed along the following dimensions: (1) Suitability as introductory text, targeted at researchers, PhD students, or practitioners, to get started on the covered topic. (2) How comprehensive and how balanced is the presentation and coverage. (3) Readability and clarity of the presentation. (4) Importance of the covered material to the broader Semantic Web community.

(I was already enthusiastic about this paper - see my previous review. And I am still!)

The issues that I have raised in my previous review of this paper have been addressed adequately or I agree with rebuttals; only a few, very minor issues remain:
- Page 1: ‘they cater’ should be switched back to ‘catered’, I think.
- Page 7-8, Section 3.2.4: The rebuttal ‘The heuristic was used in addition to the usage of the prefix relation in the ontologies. The heuristic does not detect “hectare”, but “hectare” is marked as prefix in, e.g., OM, so it is already known as prefixed unit. ” does not address that ‘hecto’ will not be recognized in the string ‘hectare’ in case this unit does not have an explicit prefix relation in an ontology.
- Page 8, Section 3.2.5: "an lexicological" -> "a lexicological"
- Page 14: OM updates have led to releases 1.8.6 and 2.0.6, not 2.0.3 (I always process changes in versions 1 and 2 of OM synchronically).
- Page 14: I still have a bit of a problem with the phrasing “While we started our analysis with version 1.8, the latest version 2.0.3 is still affected by the following issues:” The cited sentence gives a little bit the impression (unintendedly, I am convinced about that) that *still* the following issues have not been addressed. Perhaps the cause is in that the authors are not native English (like me). I have presented this formulation to a native English colleague of mine, and she suggested: “We started our analysis with version 1.8. As of the latest version, 2.0.6, the following issues remain to be addressed:” To my opinion that is a very good formulations, neutral and objective.

I have doubted whether not just to accept the paper, regarding how minor the remaining issues are. However, having stated this, I hope that the authors will perceive this minor revision as a very minor revision. The paper and the work, namely, are very good!

Review #3
Anonymous submitted on 30/Jan/2018
Suggestion:
Minor Revision
Review Comment:

This manuscript was submitted as 'Survey Article' and should be reviewed along the following dimensions: (1) Suitability as introductory text, targeted at researchers, PhD students, or practitioners, to get started on the covered topic. (2) How comprehensive and how balanced is the presentation and coverage. (3) Readability and clarity of the presentation. (4) Importance of the covered material to the broader Semantic Web community.

The authors have addressed all of my comments except one.
Although moving the old paper's Section 3 to an Appendix is good, it seems that they simply did a cut and paste.
Some of the sentences between Definitions make no sense.
It would be better to treat this Appendix as a Glossary, so that all Definitions are standalone; any extraneous remarks should be relegated to footnotes.