Review Comment:
This paper presents an approach for commonsense knowledge graph
completion using an encoder/decoder model. The authors use an encoding
model based on BERT, and a decoder model with a convolutional network.
The authors show that their method finds more supplementary commonsense
triples than the previous state of the art. They show this with the
Hits@n(n=1, 3, 10) ranking: where the correct triples are ranked in the
top n among all combinations of triples. However, the authors do not
outline how their commonsense graph completion is evaluated on ATOMIC
(which also has triple stores). I was also interested to hear how the
author's model compares to the types of knowledge that is inferred from
Malaviya et al.'s model.
The authors adequately state the parts of their approach, but some
details could be strengthened. For example, many of the figures, like
Figure 2 could use a more descriptive caption. I think much of the text
in Section 4.1 describes the figure, and it may want to be moved (or
repeated) in the caption.
One of the contributions of the papers is an appropriate importance
weight. However, the weight calculation is "the weight of the triples
is set to the size of the threshold," which seems arbitrary (not to
exceed 4 in the paper). If the weights are an important contribution of
the work, then I think they should be evaluated. Further, the authors
use the weights in conceptNet, but I believe these weights are not
normalized, which may prove to induce biases in their model.
The paper could also be strengthened by explaining the types of
commonsense triples their model is able to learn. I found Table 7 in
the Appendix to be useful to understand the types of triples that their
model is able to predict. I'm wondering if there is a subset of these
examples that could be (1) added to the main part of the paper and (2)
explained to show the novelty of the types of information that their
model can learn.
Based on the points above (and the suggestion below). I suggest major
revisions before publication.
1 Minor suggestions
===================
- The first sentence of the abstract: "Commonsense knowledge graphs
have recently gained attention since they contain lots of
commonsense triples" could perhaps be strengthened to say how
commonsense knowledge graphs in a common tripe store are essential
for AI applications.
- I found Table 4: the summary of results to difficult to interpret.
Firstly, the quantification of success, Hits, are described after
the table. Similarly, Table 5 is very difficult to read.
- "still exists implicitly or misses" -> "exists implicitly or is
missing."
- "to solve this incomplete problems" -> "to solve these incomplete
problems" or -> "to solve this incomplete problem"
- Line 29: "Lean more accurate entity semantic representation" ->
"learn more accurate semantic representations" or -> "learn a more
accurate semantic representation".
- Line 39, Page 3: "In 2020, Malaviya et al. [7] propose a model" ->
"In 2020, Malaviya et al. [7] proposed a model"
- Line 31, Page 5 "CoceptNet [23]" -> "ConceptNet [23]"
- Page 7, line 12, missing period: "attention The" -> "attention. The"
2 Larger suggestions
====================
1. Page 2, line 50: "The values of triples in commonsense KGs...have
never bee utilized for CKGC in these previous work. I think there
may be a couple other previous works that the authors may want to
examine:
1. Omeliyanenko, Janna, et al. "Lm4kg: Improving common sense
knowledge graphs with language models." International Semantic
Web Conference. Springer, Cham, 2020.
2. Davison, Joe, Joshua Feldman, and Alexander
M. Rush. "Commonsense knowledge mining from pretrained models."
Proceedings of the 2019 Conference on Empirical Methods in
Natural Language Processing and the 9th International Joint
Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
2. In section 2.2, the authors state that Malaviya et al. [7] are the
first to propose a "specific" model for the complete of the
commonsense knowledge graph. What do they mean the "first time a
specific model" is being used for completion? Here is another
(very recent) paper on KG completion: B. Wang, G. Wang, J. Huang,
J. You, J. Leskovec and C. . -C. J. Kuo, "Inductive Learning on
Commonsense Knowledge Graph Completion," 2021 International Joint
Conference on Neural Networks (IJCNN), 2021, pp. 1-8, doi:
10.1109/IJCNN52387.2021.9534355.
3. In section 2.2, the authors mention that InductivE is the first
benchmark for inductive commonsense KG completion. Before
InductivE, how was commonsense completion evaluated?
4. Page 4, line 42: "Commonsense knowledge is a fact accepted by most
people.." I'm not sure what this means.
5. There is a "similarTo" or "synonym" relation in ConceptNet. How
does this differ between the edges used for the similarity score.
6. In Section 5.1, the authors explain how they use BERT to find
"initial entity embeddings" and they add "some similar edges." How
many? In terms of the percentage of edges?
7. In section 5.1.1, what do the authors mean by ELMO "cannot perform
deep modeling work"
8. What do you mean by BERT is "a kind of representation learning"?
This point wasn't made clear.
9. The element wise activation function, $\sigma$, is that defined in
previous work, or was that defined in this paper?
|