CASIA OpenIR  > 智能机器人系统研究
Multimodal Unknown Surface Material Classification and Its Application to Physical Reasoning
Wei, Junhang1,2; Cui, Shaowei1,2; Hu, Jingyi1,2; Hao, Peng2,3; Wang, Shuo2,3,4; Lou, Zheng5
Source PublicationIEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS
ISSN1551-3203
2022-07-01
Volume18Issue:7Pages:4406-4416
Abstract

Unknown surface material classification (SMC) can inform a robot about material properties, enabling it to interact with environments appropriately. Recent research has leveraged multimodal data using deep learning to improve the performance of SMC. In this article, we present a deep learning model, multimodal temporal convolutional neural network (MTCNN), which integrates energy spectrum, dilated convolutions, and sequence poolings into a unified network architecture. The proposed model can learn material representations from auditory and multitactile (i.e., acceleration, normal force, and friction force) data generated by dragging a tool along surfaces, and distinguish unknown object surface materials into categories. For surface material data collection, a tool is also designed to detect different object surfaces. The performance of MTCNN is evaluated on a public dataset and the highest classification accuracy is 87.55%. A robotic curling example is provided to illustrate how the presented model helps the robot in manipulation.

KeywordRobots Convolutional neural networks Visualization Informatics Feature extraction Task analysis Haptic interfaces Auditory and haptic information deep learning multimodal fusion physical reasoning unknown surface material classification (USMC)
DOI10.1109/TII.2021.3126601
WOS KeywordTACTILE ; GENERATION
Indexed BySCI
Language英语
Funding ProjectNational Key R&D Program of China[2018AAA0103003] ; National Natural Science Foundation of China[61773378] ; National Natural Science Foundation of China[U1913201] ; National Natural Science Foundation of China[U1713222] ; Strategic Priority Research Program of Chinese Academy of Science[XDB32050100] ; Beijing Advanced Discipline Fund
Funding OrganizationNational Key R&D Program of China ; National Natural Science Foundation of China ; Strategic Priority Research Program of Chinese Academy of Science ; Beijing Advanced Discipline Fund
WOS Research AreaAutomation & Control Systems ; Computer Science ; Engineering
WOS SubjectAutomation & Control Systems ; Computer Science, Interdisciplinary Applications ; Engineering, Industrial
WOS IDWOS:000784218500013
PublisherIEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Citation statistics
Document Type期刊论文
Identifierhttp://ir.ia.ac.cn/handle/173211/48323
Collection智能机器人系统研究
Corresponding AuthorWang, Shuo
Affiliation1.Univ Chinese Acad Sci, Sch Future Technol, Beijing 100049, Peoples R China
2.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China
3.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China
4.Chinese Acad Sci, Ctr Excellence Brain Sci & Intelligence Technol, Shanghai 200031, Peoples R China
5.Chinese Acad Sci, State Key Lab Superlattices & Microstruct, Inst Semicond, Beijing 100083, Peoples R China
First Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Corresponding Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Recommended Citation
GB/T 7714
Wei, Junhang,Cui, Shaowei,Hu, Jingyi,et al. Multimodal Unknown Surface Material Classification and Its Application to Physical Reasoning[J]. IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS,2022,18(7):4406-4416.
APA Wei, Junhang,Cui, Shaowei,Hu, Jingyi,Hao, Peng,Wang, Shuo,&Lou, Zheng.(2022).Multimodal Unknown Surface Material Classification and Its Application to Physical Reasoning.IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS,18(7),4406-4416.
MLA Wei, Junhang,et al."Multimodal Unknown Surface Material Classification and Its Application to Physical Reasoning".IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 18.7(2022):4406-4416.
Files in This Item: Download All
File Name/Size DocType Version Access License
Multimodal_Unknown_S(3954KB)期刊论文作者接受稿开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Wei, Junhang]'s Articles
[Cui, Shaowei]'s Articles
[Hu, Jingyi]'s Articles
Baidu academic
Similar articles in Baidu academic
[Wei, Junhang]'s Articles
[Cui, Shaowei]'s Articles
[Hu, Jingyi]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Wei, Junhang]'s Articles
[Cui, Shaowei]'s Articles
[Hu, Jingyi]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: Multimodal_Unknown_Surface_Material_Classification_and_Its_Application_to_Physical_Reasoning-2.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.