柔性电极微创植入机器人识别定位与测量对准研究
梁文亮
2023-05-22
Pages99
Subtype硕士
Abstract

脑机接口(Brain-machine Interface, BMI)(Brain-machine Interface, BMI)作为是脑科学、神经科学、信号检测与处理、模式识别等多学科交叉融合的技术,一直受到研究人员广泛关注。脑机接口是指在大脑与外部设备间建立直接信息通道,实现神经系统和外部设备间信息交互与功能整合。。脑机接口通过将受试者大脑神经元活动转换为特定指令以控制外部设备,实现人与外部环境的交互,在康复医学、人体增强、军事作战、教育娱乐等诸多领域有着广阔的应用场景。本文以柔性电极微创植入机器人为具体应用,重点研究柔性电极植入过程过程中的自动聚焦、色彩复原、识别定位、位姿测量和对准控制等关键技术,研制出面向侵入式脑机接口应用的柔性电极植入机器人,在实验中成功将柔性电极沿颅骨微孔植入大鼠颅脑皮层。本文的主要研究工作与贡献如下:

(1)植入机器人系统硬件平台和可视化交互软件界面的搭建,机器人具备显微测量单元信息读取、植入单元控制和系统状态读取等功能。提出基于双目显微相机聚焦平面相交约束的植入针、柔性电极和颅骨微孔快速聚焦方法,可有效提升植入机器人操作效率。针对显微视觉同轴光照明偏色严重的现象问题,提出基于深度卷积网络的显微图像色彩复原方算法,可实现在曝光、弱光照、不同倍率条件下图像的色彩复原,为后续视觉定位和位姿测量提供了必要的基础有效提升图像质量。

(2)针对显微视觉场景下目标易受光线变化、离焦、倍率变化和遮挡等因素影响的情况,提出研究ResNet与U-Net型网络结合的提取多尺度融合特征提取方法,并引入注意力机制增加模型对重点区域和通道的关注能力,通过可变性卷积实现形状感知的自适应空间采样,可有效增加模型对目标形状及尺寸变化的适应能力。提出的基于深度卷积神经网络模型可同时对植入针和柔性电极进行关键点定位和角度估计,结果可用于后续植入机器人伺服控制。针对植入机器人操作过程中,植入针容易移出显微相机狭小测量单元视野的问题,重点研究显微测量单元和植入单元的映射关系,并采用列文伯格-马夸尔特法(Levenberg-Marquard,,LM)算法求解,建立显微测量单元和植入单元之间的视野跟随模型,确保植入针始终处于显微测量单元视野内。

(3)提出基于采用极线约束和几何形状约束结合的颅骨微孔法向矢量测量方法测量颅骨微孔法向矢量,并将微孔法向矢量投影到两路显微图像空间,以实现并投影到两路显微图像空间为后续植入针和颅骨微孔的位姿对准控制提供参考信息。同时,为了兼顾植入操作全局视野和目标细节,基于镜头倍率、运动驱动器和目标特征尺寸信息,标定出不同倍率下的图像雅克比矩阵,用于植入针与柔性电极和颅骨微孔的快速接近和精准对准。基于研制的柔性电极植入机器人系统样机,将柔性电极成功植入小鼠颅脑、颅骨微孔模型以及大鼠颅骨微孔,验证了本文所提出位姿测量和对准控制方法的有效性。

 
Other Abstract

Brain-machine interface (BMI), as a multidisciplinary technology of brain science, neuroscience, signal detection and processing, pattern recognition, etc., has received wide attention from researchers. BMI refers to the establishment of direct information channels between the brain and external devices, to achieve information interaction and functional integration between the nervous system and external devices. This thesis  presents a flexible electrodes implantation robotic system for invasive brain-machine interface applications by focusing on the key techniques of automatic focusing, color recovery, recognition and localization , pose measurement and alignment control during flexible electrodes implantation, and successfully implant the flexible electrodes into the rat cranial cortex along the cranial micro-holes in the experiment. The main research work and contributions of this thesis   are as follows:

(1) The hardware platform of the implantation robotic system and the visual interaction software interface are built, which include the functions of microscopic measurement information reading, implantation unit control and system status reading. The method of fast focusing of implantation needle, flexible electrodes and cranial micro-holes is proposed based on the focused planes intersection constraint of binocular microscope, which can effectively improve the operation efficiency of the implantation robot. For the serious color bias problem in coaxial light illumination of microscopic vision, a microscopic image color recovery method based on deep convolutional network is proposed, which can realize the color recovery of images under over-exposure, low light and different magnification conditions and effectively improve image quality.

(2) For microscopic vision scenes where the target is susceptible to light changes, out-of-focus, magnification changes and occlusions, this thesis  studies the combination of ResNet and U-shaped networks to extract multi-scale fusion features, and introduce the attention mechanism to increase the model's ability to focus on key regions and channels, and realize adaptive spatial sampling for shape perception through deformable convolution, which can effectively increase the model's ability to adapt to target shape and size. The proposed deep convolutional neural network model can simultaneously perform keypoint localization and angle estimation for the implantation needle and flexible electrode, and the results can be used for servo control of the implantation robot. To address the problem that the implantation needle easily moves out of the narrow field-of-view (FOV) of the microscope camera during the operation of the implantation robot, this thesis  focuses on the mapping relationship between the microscopic measurement unit and the implantation unit, and use the Levenberg-Marquard (LM) algorithm to solve the problem and establish the FOV following model between the microscopic measurement unit and the implantation unit to ensure that the implantation needle is always within the FOV of the microscopic measurement unit.

 (3) A method of normal vector measurement of cranial micro-holes is proposed based on polar lines and geometry constraints. The micro-hole normal vector is projected into two microscopic image spaces for pose alignment control of the implantation needle and the cranial micro-holes. Meanwhile, considering the global FOV of the implantation operation and the target details, the image Jacobian matrices at different magnifications are calibrated based on the lens magnification, motion driver and target feature size information for fast approach and accurate alignment of the implantation needle and the cranial micro-hole. Based on the developed prototype of the flexible electrodes implantation robotic system, the flexible electrodes can be successfully implanted into the mouse cranial brain, cranial micro-holes model and rat cranial micro-holes, which verifies the effectiveness of the proposed pose measurement and alignment control method.   

Keyword脑机接口 柔性电极植入 关键点定位 位姿测量 对准控制
Language中文
Sub direction classification机器人感知与决策
planning direction of the national heavy laboratory其他
Paper associated data
Document Type学位论文
Identifierhttp://ir.ia.ac.cn/handle/173211/52219
Collection中国科学院工业视觉智能装备工程实验室_精密感知与控制
毕业生_硕士学位论文
Recommended Citation
GB/T 7714
梁文亮. 柔性电极微创植入机器人识别定位与测量对准研究[D],2023.
Files in This Item:
File Name/Size DocType Version Access License
论文最终版.pdf(4690KB)学位论文 开放获取CC BY-NC-SA
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[梁文亮]'s Articles
Baidu academic
Similar articles in Baidu academic
[梁文亮]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[梁文亮]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.