模式识别实验室知识库

This paper focuses on the task of RGB-D indoor scene classification. It is a very challenging task due to two folds. 1)
Learning robust representation for indoor scene is difficult
because of various objects and layouts. 2) Fusing the complementary cues in RGB and Depth is nontrivial since there
are large semantic gaps between the two modalities. Most existing works learn representation for classification by training
a deep network with softmax loss and fuse the two modalities by simply concatenating the features of them. However,
these pipelines do not explicitly consider intra-class and interclass similarity as well as inter-modal intrinsic relationships.
To address these problems, this paper proposes a Discriminative Feature Learning and Fusion Network (DF2Net) with
two-stage training. In the first stage, to better represent scene
in each modality, a deep multi-task network is constructed to
simultaneously minimize the structured loss and the softmax
loss. In the second stage, we design a novel discriminative
fusion network which is able to learn correlative features of
multiple modalities and distinctive features of each modality.
Extensive analysis and experiments on SUN RGB-D Dataset
and NYU Depth Dataset V2 show the superiority of DF2Net
over other state-of-the-art methods in RGB-D indoor scene
classification task.