An Egocentric Semantic Reference System for Affordances

Review 1 by Aldo Gangemi:

Basically the paper is fine as it is. I only ask the authors for the following improvements:

page 15 Sec. 6.2: when you say "loss of ontological and cognitive accuracy", I'd better say that the requirements for your implementation only needed part of the DOLCE pattern you are using. This does not reduce the accuracy of the work, and certainly not using *all* DOLCE axioms is not a measure for accuracy ;).

In the meantime another paper has appeared which is related to this work and it'd be good to cite it:

David Carral, Simon Scheider, Krzysztof Janowicz, Charles Vardeman, Adila A. Krisnadhi, Pascal Hitzler. An Ontology Design Pattern for Cartographic Map Scale. ESWC2013

Finally (really optional, however I couldn't help but saying it ;)), in Sec. 4.1, when talking about procedures performed by observers and their shared environment that avoid a solipsist claim, it might be relevant in the current debate to make a link to cognitive frames or "knowledge patterns" that result to be shared, and ground the possibility to performs experiments like yours for discovering similar reference frames that filter in the most valuable information. Maybe the discussion section is also appropriate for that.

Typos and linguistic inaccuracies:

page 1
"human independent" -> "human-independent"

page 2
"suggest, that" -> "suggest that"
"example of stair" -> "examples of stair"
"implemented in Section 6" -> "implemented as described in Section 6"

page 5
there is a cumbersome phrasing in the last paragraph of Sec. 2.2: you are not possibly saying that Turvey used DOLCE (in 1992!) ...

page 6
"runnig-" -> "running-"

page 7
"craweled" -> "crawled"

page 8
"a reference systems" -> "a reference system"

page 14
probably a footnote is not well implemented in Sec. 6.1 between "Jena" and "HTML"

Review 2 by Giovanni Pezzulo:
Accept as is

Review 3 by Gaurav Sinha:
[Reviewer accepted manuscript but asked to fix typos]

Revised submission after an "accept pending major revisiosn". First round title of the manuscript was "An Egocentric Reference System for Affordances". First round reviews are below.

Review 1 by Giovanni Pezzulo:

The authors tackle a complex and interesting problem, proposing a novel methodology to represent and use egocentric frames of reference, based on the theory of affordances.

Below I list the Pros and Cons of the article (from my egocentric reference frame). The latter should be carefully considered in the paper resubmission.

* Pros

- The overall methodology is interesting and novel in its emphasis on egocentric reference systems, and their 'projections'

- The authors provide a simple yet useful example of this approach, linking it to existing semantic technologies. Although the example is at the moment hard to evaluate (and is implemented with many simplifications) this is useful to illustrate the approach and its potential.

* Cons

- The paper is very verbose and in most parts repetitive (e.g., Warren's example is repeated several times; the 'background sections' are very long) and full of off-topic parts (e.g., it is discussed twice why the authors do not consider this theory to be 'semantic'). This makes it hard to follow. It should be really kept shorter and very focused on the key topics.

- There are three (important) aspects of the authors' methodology that IMO are not completely convincing.

1) It is true that subjects have different frames of reference, but this does not necessarily imply that their ratings are purely subjective. When subjects rate, they could try to comply with some 'ideal' (or prototype) reference value (of the community). An example helps clarifying: when Rita the hiker reports her rating of a hike, she could evaluate it 'as if' a 'prototype' member of the community (would have) judged it, not as she really (subjectively) experienced it. These social/imitative dynamics are often reported in the social psychology literature and could lead to quite stable 'scales of value' in many disciplines, such as hiking, but also skiing, mountaineering, which would be much harder to explain from a purely subjective perspective. Despite this remark, I believe that the method for projecting the different frames of reference is useful (and indeed the way subjects rate is an empirical matter).

2) The formalism used for the transformations, although quite reasonable, has to be justified better. (i) The authors recognize that -although they support a fuzzy approach- their methodology does not use fuzzy logic. So, I see a big discrepancy between the proposed theory and the proposed methodology. Why so? Simply saying that it is hard to use it is not sufficient. What would be the added value of using fuzzy logic? (ii) Even worse (IMO) the formalism does not use any notion of uncertainty and confidence (in the ratings), both of which are very useful. These are indeed the main tools of statistical processing; although the authors object (with some good reasons) to the typical methodology of statistics, the mathematical framework they propose is quite weak and could miss important points (e.g., do you think that people do not use confidence measures when they read reports by other hikers?)

3) The theory of affordances is open to many interpretations (as the authors correctly recognize). I agree that (at list as a first approximation) it is tempting to link them to 'qualities' in DOLCE, in this way the action-based flavor of affordances is lost (affordances are 'invitations to actions', but qualities can be of many kind)

- The authors often mix up theoretical arguments and implementation details (e.g., "The R script" paragraph, pag. 19). The latter could be better placed in an appendix, I believe.

- The Evaluation section is quite weak, in my opinion. It merely repeats arguments already done in the paper, saying that they are good arguments (e.g., the hypothesis is accepted).

Review 2 by Gaurav Sinha:

This paper extends the literature on semantic reference systems. The paper is generally well written and I really applaud the authors' efforts to be as transparent as possible, documenting several limitations of their work clearly. This makes it much easier for the reader to not only come to terms with the idea, but also appreciate the difficulty. They also chose a sufficiently complicate use case (hikeability) that exposed the difficulties in implementing their idea. They could have chosen an easier example and made this work look a lot more advanced at first glance. I commend their efforts invested in teasing out the ontological and practical problems of their use case and repeated reminders of the incompleteness of the work. Although I value this work and recommend the paper for publication, for readers' convenience and for increasing the likelihood of the paper being understood and cited, I would prefer the authors revise the paper based on the comments below.

Section 4 is crucial to the paper since it discusses the idea of the egocentric datum and egocentric reference frame. I think for readers, who are especially not from the GIScience or surveying background, will find it difficult to grasp the discussion. The content is not such that it is imperative to refer to the literature. A simple diagram summarizing the traditional notion of a datum, and coordinate systems and transformations would help information scientists. Similarly, a diagram to schematize the relationship between the environment or real world, the semantic datum, egocentric reference system, the observing agent, and especially the datum and direct transformations would reduce the effort required in understanding the core ideas of this paper.

I found section 4.1 confusing because repeated reference is made to the egocentric reference frame, which is only explained later in the following sub-section (4.2). Another issue is the possible confusion in my mind about how exactly the authors distinguished between the concepts of allocentric vs. grounded reference system. It may be beneficial to explicitly address the difference in a footnote at least. Overall the second paragraph of section 4.1 needs some work and diagrams.

I am not convinced with the cited research (Warren's paper) that states that climbability affordance is just a function of leglength and stair height. It could also depend on the physical condition of the climber—if the person is hurt, handicapped, or suffers from some arthritic problems, the perceived climbability is not going to just be a function of leglength. I bring this up because the authors conclude the third paragraph of section 4.1 stating: "Therefore this article proposes that the observing agent, with its senses and qualities, establishes an egocentric datum that grounds the meaning of the agent's observation results." I wonder if it would make sense to add a qualifier to state that ONLY SOME SHAREABLE senses and qualities of the observing agent establish an egocentric datum, not the whole observing agent. To their credit, the authors do allude to all this in a statement earlier in the paragraph: "In this case, only one particular quality of the person is taken as the reference (i.e. leg length) and not the person as a whole."

The examples and discussion of transformations (section 5) do not clearly state if datum and direct transformations are mutually exclusive or they can both be also needed. If so an example to discuss that situation would be necessary. I would also prefer that if possible the two transformation be discussed using the same example. Currently, climbability example is used for datum transformation, and the hikeability for direct transformation. Is it not possible to work out a discussion where hikeability is also used to discuss datum transformation.

Section 7 seems somewhat too long and unwarranted. It can be collapsed into a page of discussion and caveats, creating space for explanatory diagrams.

Review 3 by Aldo Gangemi:

This article contains a substantial contribution to treat one important source of ambiguity for knowledge systems: the heterogeneity of reference systems used by humans when formulating judgments.
In order to do it, the authors tackle the notions of affordance (in cognitive systems, an opportunity for action), subjectivity, granularity, and transformation between reference systems. The result is impressive and promising in my view, but the work and the text need some revision and additional work to work at full-regime.

As far as the work is concerned, I would like that the authors perform one bit of work that would make their effort really satisfying, i.e. that they make a minimal advancement with respect to the following self-assessment they offer:

"The prototype is not integrated with the original application, nor did it reach a maturity that would allow user testing or a closer investigation of benefits for end users. The advantages have been discussed on a theoretical level and demonstrated through a prototype."

But the authors themselves got a commitment before:

"A successful implementation would verify the hypothesis."

Unfortunately enough, the implemented prototype has not been used even for a small user study, and this is surprising, considering the good amount of data from hickr.org. Scalability and immaturity of the implementation are not an excuse, because a small amount of suggestions can be discovered, similarly to the Rita's one, and then submitted to a small group of users, which could assess it. Another possibility could be to compare automatic results from the prototype with public statements that exist on hickr.org about the usefulness of some reviews.

As far as the paper is concerned, it should be fine-tuned for optimal readability and balance of sections size. Some themes are repeated several times, without adding much to comprehension. The evaluation section
is far too verbose, and essentially repeats many times and with different paraphrases the following claims by the authors:

"The hypothesis was made explicit and its verification/falsification has been outlined clearly throughout the paper."
"The paper first introduces a novel theoretical approach and then demonstrates its value for semantic web applications."

Which are ok, but need to be simplified narratively, and a bit more evidence must be provided. In particular, "demonstrating a value for semantic web applications" is not exactly what this paper does. It actually very well analyzes an important problem of semantic heterogeneity, makes a good state of art review, proposes sound solutions in terms of ontology design and an initial approach to judgment similarity through reference system transformations, and makes a nice self-assessment of what they did.
These contributions are exemplified practically with a semantic application prototype, which *makes us think* that this approach could be useful for better and more sophisticated semantic web applications when dealing with users' viewpoints and judgments. But the authors need to prove that their experiment gives at least sound results to users, i.e. that their notable representation machinery for affordances and egocentric reference systems functionally correspond to users' intuition in practice.

Another area of improvement in the narrative is an intuitive statement of problems and solutions. For example, I'd go like this: "given the requirement of gathering the best reviews from a social web site, e.g. for finding the hiking paths that are most appropriate to someone, our devised solution consists in designing an ontology that helps analyzing social web user reviews to the level needed. Such ontology uses a sophisticated modeling framework that enables to talk about affordances, subjective reference systems, and granularity levels." This is the useful intuition I'd like to read. Only at that point, I can be available to read extensive accounts of cognitive, philosophical, formal ontological, and mathematical methods to deal with that.
On the contrary, the paper tends to concentrate on the description of the scientific problems *behind* the solution, leaving the reader somehow disoriented until the concrete problem faced by the prototype implementation is described.
I agree with the authors that their contribution goes farther than just solving a specific requirement for social web hikers' site, but they would only serve their purpose better in having a more goal-oriented narrative.
As another example of lots of knowledge packed into hard-to-understand sentences: "this article proposes that the observing agent, with its senses and qualities, establishes an egocentric datum that grounds the meaning of the agent’s observation results." I am perfectly fine with the sentence from a philosophical standpoint, but for SWJ I suggest to use the goal-oriented narrative as a primary choice, and when highly synthetic philosophical statements are provided, that they are explicit and analytic instead. E.g. (if I understand well the previous sentence): "in this work we assume that observing agents express subjective observations, which we call "egocentric datums"".

Some technical issues:
– the authors say: "We refrain from calling the theory of reference system a “semantic” theory; the term “semantic” is only used as a technical term in conjunction with information systems and web applications and when talking about machine-interpretable representation of human observations.". This is confusing, and seems just a terminological issue within a paper that addresses practical problems. However, I still disagree that reference system theory is not a semantic theory.
– a more friendly presentation (e.g. with an intuitive example) of the vectorial treatment of reference framework is welcome. In particular, I suggest to skip digressions when describing a method: if authors are explaining X, then continue talking about X, and only when got to the point, add a possible branch Y.
- the authors self-assess their work as "The establishment of a semantic transformation that allows comparisons across users exceed the state of the art and makes human observations of affordances operational in semantic web applications.
Ok, but please provide also hints about how R implementation of required transformation can be integrated with SW technology.
- concerning DOLCE EQQS pattern extension and discussion, it is rather complicated in its presentation, although Figure 3 helps the reader a lot. However, I have some doubt on the choice of using qualia for representing actual ratings, as parts of user rating quality spaces. Quality spaces typically have a metric, or at least such metric needs to be reconstructed if reasoning is needed. However, the authors make the reconstruction external to the model itself, and the results do not fit the hQualia. In general, it seems that EQQS+ has been used as an overlay of the hickr schema, but then the actual processed data are those implicit in the R implementation, so creating a discontinuity between the data and the actual schema assumed, which goes against semantic web principles. I am not sure however if I have understood this well. If so, a D&S representation of the actual ontology used to represent ratings and reason on them might be explored.