CASIA OpenIR  > 毕业生  > 博士学位论文
平行手术中增强现实关键技术的研究
王蓉1,2
2018-05-23
学位类型工学博士
中文摘要增强现实技术是手术导航系统的一次重大革命,其应用十分广泛,特别是在内窥镜手术中。借助于增强现实技术,医生可以获得肉眼不可见的组织结构信息和更好的空间感知能力,大大缩短手术时间并提升手术精度。实际中的医学增强现实系统涉及到很多方面的问题,包括跟踪、配准和可视化技术等。跟踪是视点变化时的定位问题;配准是术前医学模型的定位问题;可视化是增强现实如何呈现的问题。这当中的每一项技术都存在着很多的问题和难点。本文旨在自主搭建完整的医学增强现实系统,并对其中关键性技术存在的问题进行针对性的研究,提出相应改进方案。进一步提出更具智能的平行手术框架,指出它是未来手术导航系统发展的方向。本文具体研究如下:
(1)在增强现实的跟踪技术中,围绕基于视觉的相机跟踪展开研究。我们从两个角度分别给出解决方案。第一个角度我们认为只有相机运动,针对缺乏特征的腹腔场景首次将稠密的同时定位与地图构建技术应用到内窥镜场景中。针对其中尺度缺乏和初始化困难的问题,利用多目内窥镜进行解决。这一系统结合三维配准可实现一套多目内窥镜增强现实系统。第二个角度我们认为只有目标物体运动,为了充分利用医学影像分割重建得到的三维模型,首次把原相机跟踪问题转化为已知三维模型的物体跟踪问题,从而将跟踪和配准统一到同一框架中。提出将互补的区域和稠密线索进行结合,提升跟踪的准确性和鲁棒性。进一步提出表观模型更新方法和遮挡处理方法来有效识别和处理遮挡情况。应用这一算法可直接实现一套单目增强现实系统。
(2)在增强现实的配准技术中,围绕三维配准展开研究。打破传统配准算法中点云的顶点间存在理想一一对应的假设,提出通过线性加权插值得到子顶点的方式建立对应点对,并分别设计了全局和局部配准的改进算法。在全局配准算法中,提出使用演化博弈理论进行子顶点权重的计算,这种权重计算方式相比于简单加权的优点是每个点的权重需要通过与其它点相互作用而得到。进一步,我们将相同的思想用于改进传统迭代最近点算法,提出加权顶点对应的局部配准改进算法。
(3)在增强现实的可视化技术中,围绕深度感知问题展开研究。为了提升可视化中的深度感知,我们设计了融合遮挡线索的不同可视化模式。进一步探索了融合深度线索的三维增强现实显示,并通过感知实验的方式探讨了深度感知问题。最后,设计了基于增强现实的虚拟手术刀交互训练系统,它一方面可以作为测试平台进行定量感知实验的测试,另一方面作为训练平台提升医生对当前内窥镜增强现实系统的视觉感知能力。
(4)为了使增强现实系统具有处理复杂场景和突发情况的能力,我们创新地将增强现实系统与平行理论相结合。首先,提出将平行理论应用到增强现实的同时定位与地图构建技术中的平行感知框架,目的是建立从底层视觉算法到高层决策与分析的全新理论。其次,提出包含增强现实的平行手术框架,它是平行智能在手术场景中的具体应用,也是未来手术导航系统发展的方向。
总的来说,本文对医学增强现实及其关键技术进行了深入的研究,提出了多种有效方法。希望本课题的研究能为推动该领域的发展做出贡献。
英文摘要Augmented reality technique is a significant revolution of surgical navigation system. It can be widely used, especially in endoscopic surgery. Through augmented reality, surgeons can perceive physically invisible structures and improve the ability of spatial perception, and hence increase surgical efficiency and accuracy. Medical augmented reality system includes many aspects, including tracking, registration, visualization and so on. Tracking is the localization when viewpoint changes. Registration is the localization of preoperative medical models. Visualization is the problem of how to present augmented reality results. And in each aspect, there are also many problems. In this thesis, we want to build a whole medical augmented reality system, and provide some improvements on its key techniques. Furthermore, we propose the framework of parallel surgery, which we believe is the future direction of surgical navigation system. Our contributions are listed below:
(1) In tracking technique of augmented reality, we mainly focus on visual based camera tracking. We design solutions from two perspectives. In the first perspective where only the camera is deemed in motion, in order to deal with relatively featureless abdominal scenes we first apply the dense tracking and mapping technique in endoscopic scene. Since it usually lacks absolute scale and has difficulty in system initialization, multi-camera strategy has been adopted. Besides the above system, registration is also required for a complete endoscopic augmented reality system. In the second perspective where only the target organ is deemed in motion, in order to make full use of the preoperative 3D model we first convert the original camera tracking to a known model based tracking. In such a way, we can merge tracking and registration in a unified manner. We propose to combine the complementary region and dense cues in our model based tracking to improve accuracy and robustness. We further propose an appearance model adaption method and an occlusion processing method to effectively detect and handle occlusions. This tracking strategy can be directly used to form a monocular augmented reality system.
(2) In registration technique of augmented reality, we mainly focus on 3D surface registration. We break the assumption in traditional registration that each vertex on a surface has an ideal corresponding vertex on another surface, propose the linear weighted combination to obtain more accurate sub-vertex correspondence and apply this strategy to both global and local registration algorithms. In global registration, the weights for constructing sub-vertex are computed in the framework of evolutionary game. In such a way, the weight of every candidate vertex is not considered independently, but in pairwise. Furthermore, the same idea is also adopted to improve iterative closest point algorithm, whose innovation is to compute correspondence through weighted average of the entire points.
(3) In visualization technique of augmented reality, we mainly focus on depth perception. In order to improve depth perception, we use occlusion cues to design different visualization modes. We further consider depth cues to implement 3D augmented reality display. We perform perceptual tests to evaluate depth perception of 3D augmented reality display. Finally, we design a 3D augmented reality training system where a virtual instrument is integrated for interaction. This training system on one hand can act as a test platform for quantitative perceptual tests, on the other hand can act as a training platform to improve surgeons' visual perception.
(4) In order to make augmented reality systems have the ability to handle complex scene and unexpected situation, we innovatively combine augmented reality and parallel theory together. First, we introduce the parallel theory into simultaneous localization and mapping technique in augmented reality and propose the framework of parallel perception. Parallel perception aims to construct a novel theory ranging from low-level vision algorithm to high-level decision and analysis. Then, we propose the concept of parallel surgery which contains augmented reality in it. Parallel surgery is the specific application of parallel intelligence in surgery, and it is also the future direction of surgical navigation system.
In summary, in this thesis, we have made a lot of significant progresses on medical augmented reality and its key techniques. I hope this study can make contribution to promote the development of this area.
关键词增强现实 内窥镜手术 视觉跟踪 三维配准 深度感知 平行理论
文献类型学位论文
条目标识符http://ir.ia.ac.cn/handle/173211/21011
专题毕业生_博士学位论文
作者单位1.中国科学院自动化所
2.中国科学院大学
推荐引用方式
GB/T 7714
王蓉. 平行手术中增强现实关键技术的研究[D]. 北京. 中国科学院研究生院,2018.
条目包含的文件
文件名称/大小 文献类型 版本类型 开放类型 使用许可
平行手术中增强现实关键技术的研究_王蓉_(38096KB)学位论文 限制开放CC BY-NC-SA
个性服务
推荐该条目
保存到收藏夹
查看访问统计
导出为Endnote文件
谷歌学术
谷歌学术中相似的文章
[王蓉]的文章
百度学术
百度学术中相似的文章
[王蓉]的文章
必应学术
必应学术中相似的文章
[王蓉]的文章
相关权益政策
暂无数据
收藏/分享
所有评论 (0)
暂无评论
 

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。