Games Engineering Programs

Implementing an Infrared-Based Rigid Body Pose Estimation System on HoloLens 2 for Neurosurgical Contexts

HCI Master Thesis
Created by
Thore Keser


Supporting medical professionals with Augmented Reality during surgery is not a new concept to research. The general idea is to overlay the patient body with computed tomography (CT) scan data, allowing surgeons to explore internal structures in 3D. However, most approaches are limited by performance, registration errors, such as drift and offset, or long-lasting setup procedures.


The project shows an approach to identify small reflective spheres through the infrared camera of HoloLens 2 and calculate the pose of the patient from them. The system promises simple setup and portability, and is extensible through a Unity integration. An expert evaluation will be conducted to yield qualitative feedback concerning practicability, quality, and perceived usability of the system in regard to the surgical domain.

Legal Information