CASIA OpenIR  > 复杂系统管理与控制国家重点实验室
平行手术的平台构建及其关键技术研究
孟祥冰
Subtype博士
Thesis Advisor王飞跃 ; 耿征
2019-05-20
Degree Grantor中国科学院大学
Place of Conferral中国科学院大学自动化研究所
Degree Name工学博士
Degree Discipline模式识别与智能系统
Keyword平行手术 增强现实系统 Slam 增强传感 上采样 配准定位 增强现实 深度感知 数据驱动
Abstract

随着医疗技术的飞速发展,外科手术经历了从大开口式开放手术到基于增强现实系统的智能机器人辅助手术,其跨入了一个崭新的时代。然而,目前手术中仍然存在着由于医生经验的差异、术中微创手术视野受限导致操作难度增大以及术后的并发症难以预估等问题。本研究采用平行手术理论拟为解决上述问题提供新的思路。平行手术利用ACP(Artificial Societies,Computational Experiments,Parallel Execution)理论,构建与实际手术系统平行演化的虚拟手术系统,并通过计算实验过程实现对于手术方案的制定与评价从而得到最优的手术方案,最后通过平行执行过程与实际手术过程实现虚实互动,引导手术执行,同时将手术过程中的反馈应用于虚拟手术系统,从而实现在线化、长期化的迭代优化。本研究针对平行手术框架中涉及的关键技术进行了研究,其中增强现实系统是实现平行执行中虚实互动的有效手段,以数据为驱动的建模预测方案是虚拟手术系统建模以及计算实验实现的有效手段。具体内容概述如下:

1)增强现实系统为解决平行执行过程中的虚实互动的关键技术,包含增强现实过程以及增强传感过程。本研究以微创手术中增强现实系统为例,利用SLAM(Simultaneous Localization and Mapping)技术完成增强传感的信息采集和处理,为增强现实过程提供必要的场景信息,指出基于光度差的稠密SLAM系统是微创手术中增强现实系统的核心内容,提出稠密SLAM系统的整体系统框架,以贝叶斯理论解释了稠密SLAM这一过程,并针对定位及建图给出了实验验证。接着,提出了基于多目相机的稠密SLAM系统,提升了系统的初始化效率并且获得了场景的真实的尺度,通过实验验证了系统的定位精度及对于噪声抗干扰能力。最终,提出可实现在线化、长期化自我更新、自我学习的平行感知的系统框架理论,并指出未来感知系统的发展方向。

2)针对增强现实系统中增强传感过程的相关技术进行了改进研究,主要集中于深度信息增强以及配准定位的相关技术。对于深度信息增强,分析了深度信息重建的方法及其问题所在,并基于此提出了深度信息上采样的解决方案。

针对上采样任务,提出了使用图的方案来表示基于马尔科夫随机场(Markov Random Field,MRF)上采样过程及数据置信度和关系置信度的概念。然后针对上采样任务,构建了由关系置信度和数据置信度约束的上采样算法并通过实验证明了方案的有效性。进而,提出了一种以置信度为核心的金字塔式结构化上采样方案,其中改进了置信度的衡量策略,提出了基于置信度的多线索融合的上采样方案,解决了上采样过程中深度信息细节恢复的问题,并通过实验证明了此方案在含有噪声和无噪声的数据集上均具有着明显的优势。

对于配准定位,为了降低增强现实系统中配准定位过程对于场景深度信息的依赖,提出了利用轮廓信息并结合稠密纹理信息来获取虚拟物体和实际场景的相对位姿信息的方案。通过场景信息中目标物体的轮廓和虚拟物体的二维投影一致性以及由虚拟物体表面深度建立起的多目相机间投影的纹理(光度)一致性约束对相对位姿进行求解,实验证明了此方案的有效性。

3)对于增强现实系统增强现实过程的相关技术进行了改进研究,主要针对可见表面和不可见表面的深度感知技术。采用真三维显示技术解决了可见表面的深度感知问题,其中基于多平面镜的多投影技术可在有限空间中实现最广的视角范围,然而由于不可避免的平面镜拼接倾斜导致显示畸变,本研究通过数学分析寻找出了导致畸变的根源,并提出了校正显示的算法,通过实验证明了方案的准确性。对于不可见表面的深度感知,基于真三维显示,提出了剔除遮挡不可见表面的可见表面信息以及更改此可见表面的纹理属性等方案,并针对这些方案进行了实验验证,证明其对于增强不可见表面的深度感知有较为优异的结果。

4)最后,构建了平行手术中虚拟人工手术系统,并实现了计算实验执行的相关技术。对于虚拟人工手术系统的构建,利用了以数据为驱动建模方案,包含基于统计学的方案的显性建模以及基于深度学习理论来构建相关模型的隐性建模方案,通过实验证明了两种手段对于建立人工手术系统的有效性。最后,通过利用以数据为驱动的机器学习的方案对于手术结果进行了预测,并通过模型分析了影响手术结果的重要性因素,为优化手术设计中的参数选择提供了参考。

综上所述,本研究利用增强现实系统探索了平行手术中平行执行的实现方案,并通过以数据为驱动的建模和预测方式探索了虚拟人工手术系统的搭建和计算实验的进行。希望通过本课题能为推动平行手术的发展做出相应的贡献。

Other Abstract

With the rapid development of medical technology, surgery has entered a new era from open surgery to intelligent robot assisted surgery based on augmented reality (AR) system. However, there are still some problems, such as the difference in experience of doctors, the difficulty in operation due to the limited visual field in minimally invasive surgery, and the difficulty in predicting the postoperative complications. In this study, we aim to find the way to solve the above problems using the theory of parallel surgery. Parallel surgery, using ACP (Artificial Societies, Computational Experiments, Parallel Execution) theory, constructs the virtual artificial surgery system (A) that evolves in parallel with the real actual operation system. Then, the formulation and evaluation of the operation scheme are realized by computational experiments (C). Finally, the artificial surgery system interacts with actual operation process through parallel execution (P) process, further to guide the surgery. At the same time, the feedback in the operation process turns back to the artificial surgery system to realize the on-line and long-term iterative optimization system. This study investigates the key technologies involved in the framework of parallel surgery, where the AR system is an effective means to realize the virtual and real interaction in parallel execution and the data-driven modeling and prediction scheme is an effective means of virtual surgical system modeling and computational experiment implementation. The specific contents are summarized as follows:

1) AR system is the key technology of virtual-real interaction in parallel execution, including augmented reality process and augmented sensing process. In this study, the AR system in minimally invasive surgery is taken as an example, and the SLAM (Simultaneous Localization and Mapping) technology is used to realize the information extraction and processing in the augmented sensing process, and provide the necessary scene information for the AR process. We have proposed that dense SLAM system based on luminosity difference is the core technology of AR system in minimally invasive surgery. We have studied the whole system framework of dense SLAM system and explained the process of dense SLAM by Bayesian theory. The experimental verification of localization and mapping is given. Then, we propose a dense SLAM system based on multi-camera, which improves the initialization efficiency of the system and obtains the real scale of the scene, and experiments results have verified the accuracy and robustness of it. Finally, we propose a systematic framework theory of parallel perception, which is on-line, long-term self-updating and self-learning, and predict the development direction of future perceptual system.

2) Core technologies of the augmented sensing of AR system are improved, which focus on the depth information enhancement and registration. Firstly, for the depth information enhancement task, we analyze the method of depth information reconstruction and its problems, and propose depth information upsampling method to solve the problems. A graph-based scheme is proposed to represent the upsampling process based on MRF, the concepts of data confidence and relational confidence. Then, for the upsampling task, we construct the upsampling algorithm constrained by the relational confidence and the data confidence, and prove the effectiveness of the scheme by experiments. Furthermore, we propose a pyramid-structured upsampling scheme with confidence as the core, where we improve the measurement strategy of confidence and propose a multi-cue combined upsampling scheme based on confidence. The problem on the restoration of depth details in the upsampling process is solved, and the experimental results shows that the proposed scheme has obvious advantages in both noisy and non-noisy data sets. Secondly, for the registration task, in order to reduce the dependence of registration on scene depth information in AR system, we propose a strategy to obtain the relative pose information between virtual object and real scene by using contour information and dense texture information. This strategy solves the relative pose by the consistency between the contour of the target object in the scene information and the two-dimensional projection of the virtual object and the texture (luminosity) consistency constraint of the projection between the multi-cameras established by the surface depth of the virtual object. The experimental results show that the strategy is effective.

3) Core technologies of the augmented reality of AR system are improved, which focus on the depth perception of visible surface and invisible surface. For depth perception of visible surface, we use true 3D display technology, in which multi-plane-mirror-based multi-projection technology can achieve the broadest range of views in limited space. However, there are distortions in this system for the inevitable plane mirror splicing. This study analyzes the reason of distortion through mathematical analysis, and puts forward an algorithm for correcting display. The accuracy of the strategy is proved by experiments. Finally, on the basis of true three-dimensional display, we propose strategies to enhance the depth perception of invisible surface, including removing visible surface information which is the occlusion of invisible surface, and changing texture attribute of invisible surface. Experiments are carried out to verify these strategies for excellent results of enhancing depth perception of invisible surfaces.

4) The virtual artificial surgery system in parallel surgery is constructed and computational experiments are implemented. For the construction of virtual artificial surgery system, we use the data-driven modeling strategy, including the explicit modeling based on statistics and the implicit modeling strategy based on deep learning theory. The experimental results show that the two methods are effective for the establishment of artificial surgery system. Finally, we predict the surgical results by using the data-driven machine learning strategy, and then analyze the important factors affecting the surgical results through the model, which provides guide for surgical design on parameters.

In summary, this study investigates the implementation of parallel execution in parallel surgery through the AR system, the construction of virtual artificial surgery system, and calculation experiments through data-driven modeling and prediction. It may contribute to the development of parallel surgery.

Pages180
Language中文
Document Type学位论文
Identifierhttp://ir.ia.ac.cn/handle/173211/23953
Collection复杂系统管理与控制国家重点实验室
Recommended Citation
GB/T 7714
孟祥冰. 平行手术的平台构建及其关键技术研究[D]. 中国科学院大学自动化研究所. 中国科学院大学,2019.
Files in This Item:
File Name/Size DocType Version Access License
Thesis_final_vision.(15308KB)学位论文 开放获取CC BY-NC-SAApplication Full Text
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[孟祥冰]'s Articles
Baidu academic
Similar articles in Baidu academic
[孟祥冰]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[孟祥冰]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.