Review Comment:
Overall evaluation: -2
Select your choice from the options below and write its number below
== 3 strong accept
== 2 accept
== 1 weak accept
== 0 borderline paper
== -1 weak reject
X -2 reject
== -3 strong reject
Reviewer's confidence: 4
Select your choice from the options below and write its number below.
== 5 (expert)
X 4 (high)
== 3 (medium)
== 2 (low)
== 1 (none)
Interest to the Knowledge Engineering and Knowledge Management Community: 3
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
X 3 fair
== 2 poor
== 1 very poor
Novelty
Select your choice from the options below and write its number below: 2
== 5 excellent
== 4 good
== 3 fair
X 2 poor
== 1 very poor
Technical quality: 3
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
X 3 fair
== 2 poor
== 1 very poor
Evaluation: 3
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
X 3 fair
== 2 poor
== 1 not present
Clarity and presentation: 4
Select your choice from the options below and write its number below.
== 5 excellent
X 4 good
== 3 fair
== 2 poor
== 1 very poor
Review
This paper presents a system that analyzes textual inquiries about products and compares them with entries on social media sites to determine the nature of the reported problems. The approach is to store texts from social sites as triplets, against which inquires are matched to verify if this is a model problem or not. WordNet is used to calculate entity similarities in case there is no perfect match.
The research is well motivated with a good use case. The structure of the paper is fine, and it is well-written and easy to read.
The main contribution of the paper lies in the combination of techniques (POS tagging, Linked Data, triplets, WordNet similarity scores, etc.) to address a clear business need. I also like the fact that they do not complicate their approach unnecessary, but focus on the end result of their system.
However, my main concern is the lack of a substantial research contribution. There are some interesting challenges in entity linking and triplet construction, but the paper does not go into sufficient detail. It is of course fine to concentrate on the overall approach, but then there should be a more extensive evaluation that compares their approach to some keyword-based search solution, or measures some overall benefit over current practice.
The evaluation in Section 4.2 is not clear to me. Since they use WordNet and entity linking to relate entities in triplets, I assume that there may be both perfect matches of entities and entities that are sufficiently related to count as matches to some extent. The results in Table 5, thus, heavily depends on how this entity linking is done. It would be nice if they define "match" properly and explain how this is affected by the quality of entity linking.
Also, there is also an issue of the reliability of social media content. If the system is deployed in full scale, there would also be inconsistent or wrong content out there that would matched against the inquiries. How do they plan to deal with noise in these RDF-based graphs?
Minor issues to be fixed:
- The second format presented in Section 2.2 should be deleted, as it is not used in the rest of the paper.
- Figure 2 is too small
- Page 5: The sentence "The figure is also available..." should be deleted
|