Being a Supercook: Joint Food Attributes and Multimodal Content Modeling for Recipe Retrieval and Exploration
Min, Weiqing1; Jiang, Shuqiang1; Sang, Jitao2; Wang, Huayang1; Liu, Xinda3; Herranz, Luis1
2017-05-01
发表期刊IEEE TRANSACTIONS ON MULTIMEDIA
卷号19期号:5页码:1100-1113
文章类型Article
摘要This paper considers the problem of recipe-oriented image-ingredient correlation learning with multi-attributes for recipe retrieval and exploration. Existing methods mainly focus on food visual information for recognition while we model visual information, textual content (e.g., ingredients), and attributes (e.g., cuisine and course) together to solve extended recipe-oriented problems, such as multimodal cuisine classification and attributeenhanced food image retrieval. As a solution, we propose a multimodal multitask deep belief network (M3TDBN) to learn joint image-ingredient representation regularized by different attributes. By grouping ingredients into visible ingredients (which are visible in the food image, e.g., "chicken" and "mushroom") and nonvisible ingredients (e. g., "salt" and "oil"), M3TDBN is capable of learning both midlevel visual representation between images and visible ingredients and nonvisual representation. Furthermore, in order to utilize different attributes to improve the intermodality correlation, M3TDBN incorporates multitask learning to make different attributes collaborate each other. Based on the proposed M3TDBN, we exploit the derived deep features and the discovered correlations for three extended novel applications: 1) multimodal cuisine classification; 2) attribute-augmented cross-modal recipe image retrieval; and 3) ingredient and attribute inference fromfood images. The proposed approach is evaluated on the constructed Yummly dataset and the evaluation results have validated the effectiveness of the proposed approach.
关键词Cuisine Classification Recipe Image Retrieval Ingredient Inference Multitask Deep Belief Network
WOS标题词Science & Technology ; Technology
DOI10.1109/TMM.2016.2639382
关键词[WOS]BOLTZMANN MACHINES
收录类别SCI
语种英语
项目资助者National Natural Science Foundation of China(61322212 ; National High Technology Research and Development 863 Program of China(2014AA015202) ; Beijing Municipal Commission of Science and Technology(D161100001816001) ; Lenovo Outstanding Young Scientists Program ; National Program for Special Support of Eminent Professionals ; China Post-doctoral Science Foundation(2016M590135) ; National Program for Support of Top-Notch Young Professionals ; 61532018 ; 61550110505 ; 61602437 ; 61373122)
WOS研究方向Computer Science ; Telecommunications
WOS类目Computer Science, Information Systems ; Computer Science, Software Engineering ; Telecommunications
WOS记录号WOS:000404056000017
引用统计
被引频次:5[WOS]   [WOS记录]     [WOS相关记录]
文献类型期刊论文
条目标识符http://ir.ia.ac.cn/handle/173211/15237
专题模式识别国家重点实验室_多媒体计算与图形学
作者单位1.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China
2.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China
3.Beihang Univ, State Key Lab Virtual Real Technol & Syst, Beijing 100191, Peoples R China
推荐引用方式
GB/T 7714
Min, Weiqing,Jiang, Shuqiang,Sang, Jitao,et al. Being a Supercook: Joint Food Attributes and Multimodal Content Modeling for Recipe Retrieval and Exploration[J]. IEEE TRANSACTIONS ON MULTIMEDIA,2017,19(5):1100-1113.
APA Min, Weiqing,Jiang, Shuqiang,Sang, Jitao,Wang, Huayang,Liu, Xinda,&Herranz, Luis.(2017).Being a Supercook: Joint Food Attributes and Multimodal Content Modeling for Recipe Retrieval and Exploration.IEEE TRANSACTIONS ON MULTIMEDIA,19(5),1100-1113.
MLA Min, Weiqing,et al."Being a Supercook: Joint Food Attributes and Multimodal Content Modeling for Recipe Retrieval and Exploration".IEEE TRANSACTIONS ON MULTIMEDIA 19.5(2017):1100-1113.
条目包含的文件
条目无相关文件。
个性服务
推荐该条目
保存到收藏夹
查看访问统计
导出为Endnote文件
谷歌学术
谷歌学术中相似的文章
[Min, Weiqing]的文章
[Jiang, Shuqiang]的文章
[Sang, Jitao]的文章
百度学术
百度学术中相似的文章
[Min, Weiqing]的文章
[Jiang, Shuqiang]的文章
[Sang, Jitao]的文章
必应学术
必应学术中相似的文章
[Min, Weiqing]的文章
[Jiang, Shuqiang]的文章
[Sang, Jitao]的文章
相关权益政策
暂无数据
收藏/分享
所有评论 (0)
暂无评论
 

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。