Explainable Zero-shot Learning via Attentive Graph Convolutional Network and Knowledge Graphs

Tracking #: 2547-3761

Authors: 
Yuxia Geng
Jiaoyan Chen
Zhiquan Ye
Wei Zhang
Huajun Chen
Zonggang Yuan

Responsible editor: 
Dagmar Gromann

Submission type: 
Full Paper
Abstract: 
Zero-shot learning (ZSL) which aims to deal with new classes that have never appeared in the training data (i.e., unseen classes) has attracted massive research interests recently. Transferring of deep features learned from training classes (i.e., seen classes) are often used, but most current methods are black-box models without any explanations, especially textual explanations that are more acceptable to not only machine learning specialists but also common people without artificial intelligence expertise. In this paper, we focus on explainable ZSL, and present a knowledge graph (KG) based framework that can explain the transferability of features in ZSL in a human understandable manner. The framework has two modules: an attentive ZSL learner and an explanation generator. The former utilizes an Attentive Graph Convolutional Network (AGCN) to match class knowledge from WordNet with deep features learned from CNNs (i.e., encode inter-class relationship to predict classifiers), in which the features of unseen classes are transferred from seen classes to predict the samples of unseen classes, with impressive (important) seen classes detected, while the latter generates human understandable explanations for the transferability of features with class knowledge that are enriched by external KGs, including a domain-specific Attribute Graph and DBpedia. We evaluate our method on two benchmarks of animal recognition. Augmented by class knowledge from KGs, our framework generates promising explanations for the transferability of features, and at the same time improves the recognition accuracy.
Full PDF Version: 
Tags: 
Reviewed

Decision/Status: 
Accept

Solicited Reviews:
Click to Expand/Collapse
Review #1
By Dagmar Gromann submitted on 30/Sep/2020
Suggestion:
Accept
Review Comment:

Thank you for your very careful consideration of all of my comments of the previous Review #2. I think all of your explanations clarified the points raised in my review. Regarding point 5. thanks for adding the explanation in Section 5.1.2. and I think no further explanations are necessary.

As always, I also have some further minor comments in this round:
p.2 / 49 A serious of templates => series
p.3 / 21 Natural Language Processing
p.3 / 19 machine translation is actually one of the most popular ZSL approaches in comp. ling.
p.3 / 34 Explainable Artificial Intelligence (XAI)
p.3 / 47 with AI expertise. They can be used for system debug to efficiently manage and develop machine learnin gmodels
p.4 / 16 Link Data => Linked Data
p.4 / 18 These works indicate the transferability of features is highly related => These works indicate that
p.5 / 51 Github => GitHub
p.6 / 49 real-value vector => real-valued vector
p.7 / 35 on graph, => on a graph,
p.7 / 35 dependence => dependency
p.8 / 36 , therefore => . Therefore
p.8 / 31-32 layer,these => layer. These
p.8 / 35 By this means, => these
p.9 / 50 outputted => output
p.10 / 44 are we need => are what we need
p.10 / 45 Equation 5
p.11 / 30 line too long
p.12 / 20 They both have sharp ear => ears
p.13 / 7 omit "technique"
p.13 / 8 in these nouns => among these nouns
p.13 / 23 on image classification task => on an ...
p.13 / 24 in => regarding
p.13 / 27 more details please refer to our publishedcode => for more details...
p.15 / 44 most of settings => most settings
p.17 / 16 common attribute set => a common attribute set
p.17 / 48 unseen class set => the unseen class set
p.17 / 47 For unseen class => For the unseen class
p.18 / 41 are => is
p.18 / 13 are more applicable => is
p.18 / 44 those are discriminative => those that are discriminative
p.20 / 37 features can be transferred to them => features that can be...

Review #2
By Michael Cochez submitted on 18/Oct/2020
Suggestion:
Accept
Review Comment:

I am happy with the changes made in the paper. I think the additional experiment, although showing a limitation of the proposed method, is very valuable, as it shows a path for future work.
For comment 19., I suggest you also state that very explicitly in the final version of the article.