Review Comment:
The article reports on GORT (Guesstimation with Ontologies and Reasoning Techniques), a system for formalizing the guestimation process. Guestimates are approximate answers to questions like, “How many solar panels would it take to power all the houses in the UK”. At the heart of GORT is their SINGSIGDIG Calculus (The Single Significant Digit Calculus). GORT retrieves numbers from the semantic web with SINDICE search engine. The paper also discusses some dynamic curation methods employed in GORT, such as how to handle dropped units and inconsistent numerical values.
() It would have been easier for me to understand the definitions and methods underlying GORT by providing one or two simple numerical examples that would be carried throughout the paper. Essentially, the examples I would like to see would involve expanding the solar panel example used in the Count Method on p. 3 throughout the article. These examples could be embedded in the text, or perhaps better would be to illustrate these different definitions and methods with a Figure along the lines of Figure 2.
() The description of the evaluation on p. 5 is sparse. But as far as I can tell, the evaluation only tells us about how GORT performs with human curated data vs. data retrieved from SINDICE. However, this doesn’t let us evaluate the SINGSIGDIG Calculus or GORT’s dynamic curation methods. SINGSIGDIG Calculus could be evaluated by making GORT geustimates for questions that could be verified independently. These could be questions such as:
“How teachers work for the Chicago public school system?”
“How many solar panels are sold in the UK?”
The dynamic curation methods could be evaluated by turning them on and off and re-running the 12 questions currently used to evaluate the system.
() What are readers expected to know? Do the authors expect their readers to know what Hilbert’s operator is? Or perhaps, this is a question more for the editors. OWL and RDF triples are terms that simply will not be familiar to the vast majority of cognitive scientists. As many of these terms will be needed to understand more than one of the articles, it would make sense to have a semantic web primer or recommend one that covers some of the background terms and concepts needed to understand the articles..
() The acronym GORT is used on page 2 and there a mysterious reference to GORT 4.0 on page 4, even though the GORT is not defined until page 5.
Table 4 (“Questions used in evaluation of GORT 4.0”) on page 8 should come before Table 2 (Evaluation results for GORT 4.0”) that shows up page 6.
|
Comments
Submission in response to
Submission in response to http://www.semantic-web-journal.net/blog/call-special-issue-cognitive-sc...