Survey on Challenges of Question Answering in the Semantic Web

This manuscript was submitted as 'Survey Article' and should be reviewed along the following dimensions: (1) Suitability as introductory text, targeted at researchers, PhD students, or practitioners, to get started on the covered topic. (2) How comprehensive and how balanced is the presentation and coverage. (3) Readability and clarity of the presentation. (4) Importance of the covered material to the broader Semantic Web community.

(1) Very
(2) Very
(3) Very
(4) Very

This paper presents an extensive survey over Question Answering Systems on Linked Data (Semantic Question Answering, SQA) for the period of 2010-2015.
The task of SQA is to retrieve answers from structured SW resources for natural language queries. This is the first revision of the paper.

I am happy to see that my comments on the previous version were addressed very nicely. Thus, I vote for accepting the paper since it is an important addition to the literature and nicely serves as a more current survey on the field of SQA.

Below, some minor errors that should be addressed in the final version. I am sure I didn’t find all of them.

Minor remarks
- "don't" -> do not, in various places
- make sure to drop spaces before \footnote{}, e.g. fn 13
- 5.1: an Apache Lucene ndex which implements a Levenshtein Automaton -> Lucene index, which
- 6. than 100 languages which -> than 100 languages, which
AVERBIS

References formatting need to be revisited. See e.g. for a contrast:

[18] R. Blanco, P. Mika, and S. Vigna. Effective and efficient entity search in RDF data. In L. Aroyo, C. Welty, H. Alani, J. Taylor, and A. Bernstein, editors, Proceedings of the 10th International Conference on The Semantic Web—Volume Part I, volume 7031 of Lecture Notes in Computer Science, pages 83–97, Berlin Heidelberg, Germany, 2011. Springer-Verlag. ISBN 978-3-642-25072-9. . URL http://dl.acm.org/citation.cfm?id=2063016.2063023.

versus

[21] A. Both, D. Diefenbach, K. Singh, S. Shekarpour, D. Cherix, and C. Lange34. Qanary–An extensible vocabulary for open Question Answering systems. ESWC, 2016.
[104] A.-C. Ngonga Ngomo. Link discovery with guaranteed reduction ratio in affine spaces with minkowski measures. In Proceedings of ISWC, 2012.

in length/verbosity. There is no right or wrong, its a matter of consistency.

Also, there are some errors. Please carefully check all references, I just list a few:

[14] A. Ben Abacha and P. Zweigenbaum. Medical Question Answering: Translating medical questions into SPARQL queries. In IHI ’12: Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium, pages 41–50, New York, NY, USA, 2012. Association for Computational Linguistics.

No, this is ACM not ACL

[68] B. Hamp and H. Feldweg. Germanet—a lexical-semantic net for German. In I. Mani and M. Maybury, editors, Proceedings of the ACL/EACL ’97 Workshop on Intelligent Scalable Text Summarization, pages 9–15. Citeseer, 1997.

Citeser is not a publisher. Don’t just blindly copy bibtex files from the web please. Also, check title capitalizations, e.g. this one should be {GermaNet}.

[93] Natural Language Processing and Information Systems: 19th International Conference on Applications of Natural Language to Information Systems, NLDB 2014, Montpellier, France, June 18–20, 2014. Proceedings, volume 8455 of Lec- ture Notes in Computer Science, Berlin Heidelberg, Germany, 2014. L’Unité Mixte de Recherche Territoires, Environnement, Télédétection et Information Spatiale (TETIS), Springer.

TETIS???

[102] N. Nakashole, G. Weikum, and F. Suchanek. PATTY: A taxonomy of relational patterns with semantic types. In EMNLP-CoNLL 2012, 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Proceedings of the Conference, pages 1135–1145, Stroudsburg, PA, USA, 2012. Association for Computational Linguistics.

Stroudsburg, PA, USA is not the place of the venue, as in most other references

I disagreed with the other reviewers and thought the paper provided a valuable resource ONLY IF it can be published quickly. Quit nit-picking and get this out there, its a reference not a result.

Overall Rate

The reviewer appreciate the restructuring of the introduction about the overall organisation of Sections 4 and 5. It makes clearer also the terminology adopted in some subsections, that is actually consistent.

(1) Suitability as introductory text, targeted at researchers, PhD students, or practitioners, to get started on the covered topic.

Very high suitability, as outlined in my previous review.

(2) How comprehensive and how balanced is the presentation and coverage.

With proper justifications the authors restrict the focus very well on the wide existing literature, and the discussion is well balanced.

(3) Readability and clarity of the presentation.

The current version has been revised and it is much better at the readability level. It makes the overall picture clearer and more consistent.

(4) Importance of the covered material to the broader Semantic Web community.

The paper tries to discuss the guidelines for future work, just as a summary of current trends. I appreciated the discussion, although it has been concentrated in a too limited space at the end.

Pointwise comments.

Please avoid capital letters after column e.g. pag. 1375-2587
"Our contributions are threefold: First, we complement existing
work with 72 publications about 62 systems ..."
or pag. 20
"This survey follows a strict discovery methodology: Objective inclusion and exclusion"
or pag. 4
"This accellerates the evolution of QA in four different ways: Firstly, new systems"

At pag. 7, please change
"More elaborate normalizations use natural language programming (NLP)"
into
"More elaborate normalizations use natural language processing (NLP)"

Please remove "talks about their" in pag. 9

"... the time consuming querying step, by combining restrictions for each meaning.
talks about their
Each term is mapped to a Dependency-based ..."