毕业生知识库

机器人灵巧操作是智能机器人、智能制造与人工智能等领域相交叉的重要研究方向，目前已成为国际学术研究的前沿热点，具有广阔的应用前景。其中，视觉、触觉融合是机器人感知与灵巧操作技术迈向更加自主、智能的关键所在，特别是在非结构化、接触丰富的机器人操作场景中。本文主要研究内容如下：

一、面向机器人触觉感知，设计了基于双目视觉的 GelStereo 触觉传感器，提出了三维触觉点云以及接触表面稠密三维几何传感信息建模方法，已应用到机器人指尖触觉传感平台。首先通过对标记点的追踪及双目立体匹配，实现了对传感器接触表面的高空间分辨率（小于 1 mm）三维触觉点云实时获取，并在此基础上开展了滑动感知方法研究。为提升传感器对接触表面细节纹理的感知能力，通过自监督学习的方式设计了双目视差估计网络，在不依赖接触深度真值标签的条件下实现了对传感器接触表面的稠密三维几何形状传感信息建模。

二、针对视觉、触觉双模态融合感知，提出了基于注意力机制的融合学习方法，用以提取双模态融合学习任务中任务相关的模态特有与跨模态特征。面向时刻和时序视触传感数据分别构建了基于自注意力机制和基于 Transformer 的融合深度神经网络，并分别在抓取结果预测以及滑动检测任务中进行了方法的验证，实验结果表明所提方法不仅相比传统融合方法具有更高的模型性能，且适用于包含图像及阵列数据在内的多种触觉数据格式，并能够进一步处理非对齐的视触时序数据，具备较强的通用性。

三、面向机器人操作任务，提出了基于视触感知的机器人操作状态感知与作业控制框架。该框架主要由操作任务规划器、抓取状态反馈控制器以及掌内物体姿态反馈控制器三部分组成。操作任务规划器通过具体的操作任务先验策略结合视触感知确定当前操作动作；抓取状态反馈控制器通过视触感知确定当前操作对象的抓取状态，并通过控制夹持力调整抓取状态；掌内物体姿态反馈控制器通过计算操作对象的掌内位姿，并通过控制末端执行器 6 自由度位姿调整目标姿态。本文在该框架基础上实现了机器人自适应抓取、USB 接口装配、瓶盖旋拧等作业任务。

Dexterous robotic manipulation is an important research direction that intersects the fields of robotics, intelligent manufacturing, and artificial intelligence. It has become a frontier of international academic research and has broad application prospects. Visual-tactile fusion perception is a key step towards more autonomous and intelligent robotics perception and manipulation, especially in unstructured and contact-rich manipulating tasks. The main contents of this thesis are as follows:

For robotic tactile sensing, a novel visuotactile sensor named GelStereo is first designed, and two 3D tactile point cloud and dense 3D geometric sensing methods are proposed. The GelStereo tactile sensing method has been applied to a robotic fingertip platform. The real-time 3D tactile point coud with high spatial resolution (less than 1 mm) is implemented by tracking the markers on the sensor surface and stereo matching. Meanwhile, the slip perception method using the tactile point cloud is studied. To improve the sensing ability of detailed texture, this thesis also proposes a binocular disparity estimation network by self-supervised learning.

Secondly, the fusion learning methods based on attention mechanism are proposed to extract task-related modal-specific and cross-modal features for visual-tactile perception. Specifically, two self-attention and Transformer-based fusion networks are presented for momentary and temporal visual-tactile sensing data, respectively, and the methods are verified in multi-modal grasping outcomes prediction and slip detection tasks, respectively. The experimental results show that the proposed methods not only have higher model performance than traditional fusion methods, but also is applicable to many types of tactile sensing data, including tactile images and array tactile data, and has strong generality.

Thirdly, a framework for robot dexterous manipulation control based on visualtactile perception is proposed for various robotic manipulation tasks. The framework consists of a manipulation task planner, a gripping state feedback controller, and an in-hand object pose feedback controller. The manipulation task planner determines the current manipulation action through the priori strategy from a specific task combined with visual-tactile perception. The gripping state feedback controller determines the current gripping state of the manipulating object through visual-tactile perception and adjusts the gripping state by controlling the gripping force. The in-hand object pose feedback controller calculates the in-hand pose of the manipulating object and adjusts its pose by controlling the 6-DOF pose of the end-effector. Finally, the proposed manipulating framework has been successfully applied in several manipulation tasks, including adaptive grasping, USB interface peg-in-hole assembly, and bottle-cap screwing.