Review Comment:
The authors have done a great job in addressing most of my comments. Especially since the evaluation metrics are now much clearer to me. To help other readers that might get confused like I was, it might be an option to depict a 3x2 confusion matrix ((GT exists, GT does not exist) x (correct replacement, wrong replacement, no replacement)), and illlustrate the measures using that matrix.
There are still a few open (and a few new ;-)) questions open.
Old questions:
* While I see that some of my questions for clarification are now addressed (particularly here: why neighborhoods were not used as candidates, and how salient semantic confusion is), it would be nice to provide some real examples here. The numbers arguing for the neighborhood sizes from the cover letter should be included in the paper, since they provide a hard justification for the approach chosen.
* I see my comment on the neighborhod was eaten by the journal website's HTML encoder ;-) Let me try again: Section 4.3.1: algorithm 1 seems to extract neighborhoods only with statements with the same predicate as the target assertions. For example, if my target assertion was "Germany capital Berlin", the neighboorhood graph would not contain, e.g., "Germany seatOfGovernment Berlin" or "Germany hasPOI Berlin_Wall". Is that really intended?
New questions:
* in Fig. 4: why does the correction rate decline with higher thresholds for DBpedia, but increase for the other two datasets? There seems to be some particularity that DBpedia has, but the other two do not. It would be interesting to dig a bit deeper here.
* Since there are quite efficient implementations of RDF2vec out there (like jRDF2vec, which can be considered the fastest one, or even faster approximations like RDF2vec Light), which scale well to larger graphs as well, it would not have been a too big deal to train RDF2vec on the other two datasets as well. In the final version, I would find it neat to see results on those datasets as well.
Overall, I am very happy with revision. The remaining few questions could be solved in a minor revision.
|