Roadmapping and Navigating in the Ontology Visualization Landscape

Tracking #: 755-1965

Authors: 
Marek Dudas
Ondřej Zamazal
Vojtěch Svátek

Responsible editor: 
Guest Editors EKAW 2014 Schlobach Janowicz

Submission type: 
Conference Style
Abstract: 
Proper visualization is essential for ontology development, sharing and usage; various use cases however pose specific requirements on visualization features. We analyzed several visualization tools from the perspective of use case categories as well as low-level functional features and OWL expressiveness. A rule-based recommender was subsequently developed to help the user choose a suitable visualizer. Both the analysis results and the recommender were evaluated via a questionnaire.
Full PDF Version: 
Tags: 
Reviewed

Decision/Status: 
[EKAW] conference only accept

Solicited Reviews:
Click to Expand/Collapse
Review #1
Anonymous submitted on 25/Aug/2014
Suggestion:
[EKAW] combined track accept
Review Comment:

Overall evaluation
Select your choice from the options below and write its number below.
3

== 3 strong accept
== 2 accept
== 1 weak accept
== 0 borderline paper
== -1 weak reject
== -2 reject
== -3 strong reject

Reviewer's confidence
Select your choice from the options below and write its number below.
4
== 5 (expert)
== 4 (high)
== 3 (medium)
== 2 (low)
== 1 (none)

Interest to the Knowledge Engineering and Knowledge Management Community
Select your choice from the options below and write its number below.
5
== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor

Novelty
Select your choice from the options below and write its number below.
5
== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor

Technical quality
Select your choice from the options below and write its number below.
5
== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor

Evaluation
Select your choice from the options below and write its number below.
5
== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 not present

Clarity and presentation
Select your choice from the options below and write its number below.
5
== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor

Review
Please provide your textual review here.

The visualization of ontologies for various purposes is of growing importance. With broader applications, people lesser skilled with Semantic technologies are in touch with ontologies and visualization is important for them. Also, the number and size of ontologies make visual exploration necessary. The paper addresses these issues in a novel way. It is very well written, the approach with features is well-designed. The evaluation is well done. So this is a clear accept. I suggest to include more visual examples in the text. I also am slightly skeptical on quantified qualitative attributes as it is done with calculating on the features.

Review #2
Anonymous submitted on 25/Aug/2014
Suggestion:
[EKAW] conference only accept
Review Comment:

Overall evaluation
Select your choice from the options below and write its number below.

== 3 strong accept
== 2 accept
== 1 weak accept
== 0 borderline paper
== -1 weak reject
== -2 reject
== -3 strong reject
1

Reviewer's confidence
Select your choice from the options below and write its number below.

== 5 (expert)
== 4 (high)
== 3 (medium)
== 2 (low)
== 1 (none)
4

Interest to the Knowledge Engineering and Knowledge Management Community
Select your choice from the options below and write its number below.

== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor
4

Novelty
Select your choice from the options below and write its number below.

== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor
3

Technical quality
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor
3

Evaluation
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 not present
3

Clarity and presentation
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor
4

Review
Please provide your textual review here.

The paper analyzes tools for visualizing ontologies. To this end several description categories are introduced, among them use case scenarios, functional features, and ontology language constructs covered by a tool. Based on the categories and the classification of the various visualization tools into these categories a system for recommending a visualization tool for a given task is presented. Thus the paper is more a "meta" paper, giving an overview and describing the foundation for giving guidelines and recommendations for selecting an ontology visualization tool.

Strong points of the paper:

Ontology visualization is an important but also quite difficult topic. The analysis and insights given by the paper are therefore valuable to the community. Furthermore, the framework used to analyze the visualization tools is also interesting in itself.

Weak points of the paper:

The distinction into use cases is an interesting approach but seems a bit arbitrary. The authors should discuss in more detail the suitability of the use cases for characterizing different kinds of interactions with an ontology. For example, why does it make sense to distinguish between uc7 and uc9? Sure, the use cases are different, but does it also mean that the kinds of interactions with an ontology are considerably different in these two use cases? The same with uc3 and uc4, etc. The same remarks holds for aggregating the use cases into use case categories: Are these categories really helpful to distinguish different kinds of interaction with an ontology? Although the evaluation results show that the distinction into use cases categories is widely accepted by the participants of the survey, the authors should still motivate in more detail why they think that their use case framework is suitable.

The description of the recommender system should explain more clearly how the various criteria introduced are actually used to make up rules. Figures 2 and 3b give some hints but some more elaboration including an example would be helpful. In particular, it should be explained how the suitability scores introduced in Sec.3.2 are used for rule formulation. How are rules combined to come up with a final recommendation?

Minor remark: The left part of Figure 3 is very difficult to read.

Review #3
Anonymous submitted on 27/Aug/2014
Suggestion:
[EKAW] conference only accept
Review Comment:

Overall evaluation
Select your choice from the options below and write its number below.

== 0 borderline paper

== 3 strong accept
== 2 accept
== 1 weak accept
== 0 borderline paper
== -1 weak reject
== -2 reject
== -3 strong reject

Reviewer's confidence
Select your choice from the options below and write its number below.

== 5 (expert)

== 5 (expert)
== 4 (high)
== 3 (medium)
== 2 (low)
== 1 (none)

Interest to the Knowledge Engineering and Knowledge Management Community
Select your choice from the options below and write its number below.

== 4 good

== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor

Novelty
Select your choice from the options below and write its number below.

== 3 fair

== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor

Technical quality
Select your choice from the options below and write its number below.

== 3 fair

== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor

Evaluation
Select your choice from the options below and write its number below.

== 3 fair

== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 not present

Clarity and presentation
Select your choice from the options below and write its number below.

== 3 fair

== 5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor

Review
Please provide your textual review here.

This paper provides a survey of ontology visualization tools and then describes a recommender system to help people decide on which tool to use on the basis of their requirements. The paper is basically OK, however its contribution appears to be marginal. The survey is based on a rather 'static' analysis of features rather than a task-based evaluation, as it is the case in most other analyses in the literature. Unfortunately, providing a particular feature, e.g. editing complex classes, does not necessarily mean that this feature is well supported. To prove this a user evaluation is needed. As a result the paper reads very much like an academic exercise, useful as a basic roadmap for students and practitioners, but not really advancing the state of the art. The same remarks apply to the recommender system. Although the system appears to provide plausible results (how can a result not be plausible here? What would be an incorrect result?) I am not sure many people would actually use the system for selecting a tool to use in a concrete project. Such decisions are influenced by a variety of technical and organizational requirements, only some of which have to do with ontology features.

Another problem is that the characterization of the use cases seem artificial. What if multiple use cases are relvant to my ontology engineering project? Is the recommender system able to cope with this situation? I assume it is, but this aspect is not discussed in the paper – an example of the rather artificial style of analysis.

In sum, a nice piece of work, which has been diligently carried out. However, I am not convinced that the research contribution is significant enough for EKAW.