SCIENCE@home

Visual Question Generation as Dual Task of Visual Question Answering

Yikang Li,Nan Duan,Bolei Zhou,Xiao Chu,Wanli Ouyang,Xiaogang Wang

Published 2017 in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

ABSTRACT

Visual question answering (VQA) and visual question generation (VQG) are two trending topics in the computer vision, but they are usually explored separately despite their intrinsic complementary relationship. In this paper, we propose an end-to-end unified model, the Invertible Question Answering Network (iQAN), to introduce question generation as a dual task of question answering to improve the VQA performance. With our proposed invertible bilinear fusion module and parameter sharing scheme, our iQAN can accomplish VQA and its dual task VQG simultaneously. By jointly trained on two tasks with our proposed dual regularizes (termed as Dual Training), our model has a better understanding of the interactions among images, questions and answers. After training, iQAN can take either question or answer as input, and output the counterpart. Evaluated on the CLEVR and VQA2 datasets, our iQAN improves the top-1 accuracy of the prior art MUTAN VQA method by 1.33% and 0.88% (absolute increase) respectiely. We also show that our proposed dual training framework can consistently improve model performances of many popular VQA architectures.1

PUBLICATION RECORD

CITATION MAP

EXTRACTION MAP

CLAIMS

CONCEPTS

REFERENCES

Showing 1-39 of 39 references · Page 1 of 1

CITED BY

Showing 1-100 of 176 citing papers · Page 1 of 2

12
Next