A systematic review of techniques used to validate the registration of AR images using a head-mounted device to navigate surgery

Published:October 06, 2022DOI:



      Augmented Reality (AR) Head Mounted Devices (HMD) allow the wearer to have digital images superposed onto their field of vision. It is being used to superpose annotations onto the surgical field akin to a navigation system. This review examines the published validation studies on HMD-AR systems, their reported protocols and outcomes. The aim is to establish commonalities and an acceptable registration outcome.

      Materials and Methods

      Multiple databases were systematically searched for relevant articles between January 2015 and January 2021. Studies examining registration of AR content, using a HMD to guide surgery were eligible for inclusion. The country of origin, year of publication, medical speciality, HMD device, software, method of registration were recorded. A meta-analysis of the mean registration error was calculated.


      4784 papers were identified, of which 23 met the inclusion criteria. This included studies using Hololens (22) and nVisor ST60 (1). 66% of studies were in hard tissue specialities. Eleven studies reported registration error using pattern markers (mean 2.6mm SD1.8mm). Four studies reported registration error in studies using surface markers (mean 3.8mm SD3.7mm). Three studies reported registration error using manual alignment (mean 2.2mm SD1.3mm).


      The majority of studies in this review used in-house software with a variety of registration methods and reported errors. The mean registration error calculated in this study can be considered as a minimum acceptable standard. It should be taken into consideration when procedural applications are selected.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to British Journal of Oral and Maxillofacial Surgery
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • McCulloch P.
        The IDEAL framework for ensuring safety and effectiveness of medical devices.
        BMJ. 2020; 370m3183
      1. Chegini S. REGISTRY OF SYSTEMATIC REVIEWS/META-ANALYSES DETAILS. 2022; Accessed 14/07/2022.

        • Nguyen N.Q.
        • Cardinell J.
        • Ramjist J.
        • et al.
        Augmented reality systems for improved operating room workflow.
        Clinical Neurosurgery. 2019; 66: 101
        • Mitsuno D.
        • Ueda K.
        • Itamiya T.
        • et al.
        Intraoperative Evaluation of Body Surface Improvement by an Augmented Reality System That a Clinician Can Modify.
        Plastic and Reconstructive Surgery-Global Open. 2017; 5
        • Agten C.A.
        • Dennler C.
        • Rosskopf A.B.
        • et al.
        Augmented Reality-Guided Lumbar Facet Joint Injections.
        Invest Radiol. 2018; 53: 495-498
        • Nuri T.
        • Mitsuno D.
        • Otsuki Y.
        • et al.
        Augmented Reality Technology for the Positioning of the Auricle in the Treatment of Microtia.
        Plastic and Reconstructive Surgery-Global Open. 2020; 8
        • Frantz T.
        • Jansen B.
        • Duerinck J.
        • et al.
        Augmenting Microsoft's HoloLens with vuforia tracking for neuronavigation.
        Healthc Technol Lett. 2018; 5: 221-225
        • Jiang T.
        • Yu D.
        • Wang Y.
        • et al.
        HoloLens-Based Vascular Localization System: Precision Evaluation Study With a Three-Dimensional Printed Model.
        J Med Internet Res. 2020; 22: e16852
        • Moreta-Martinez R.
        • García-Mato D.
        • García-Sevilla M.
        • et al.
        Augmented reality in computer-assisted interventions based on patient-specific 3D printed reference.
        Healthc Technol Lett. 2018; 5: 162-166
        • Zhou Z.Y.
        • Yang Z.Y.
        • Jiang S.
        • et al.
        Design and validation of a surgical navigation system for brachytherapy based on mixed reality.
        Medical Physics. 2019; 46: 3709-3718
        • Gibby J.T.
        • Swenson S.A.
        • Cvetko S.
        • et al.
        Head-mounted display augmented reality to guide pedicle screw placement utilizing computed tomography.
        International Journal of Computer Assisted Radiology and Surgery. 2019; 14: 525-535
        • Gao Y.
        • Lin L.
        • Chai G.
        • et al.
        A feasibility study of a new method to enhance the augmented reality navigation effect in mandibular angle split osteotomy.
        J Craniomaxillofac Surg. 2019; 47: 1242-1248
        • Liu H.
        • Auvinet E.
        • Giles J.
        • et al.
        Augmented Reality Based Navigation for Computer Assisted Hip Resurfacing: A Proof of Concept Study.
        Ann Biomed Eng. 2018; 46: 1595-1605
        • van Doormaal T.P.C.
        • van Doormaal J.A.M.
        • Mensink T.
        Clinical Accuracy of Holographic Navigation Using Point-Based Registration on Augmented-Reality Glasses.
        Oper Neurosurg (Hagerstown). 2019; 17: 588-593
        • Liebmann F.
        • Carrillo F.
        • Farshad M.
        • et al.
        First experiences of using mixed-reality for surgical navigation of corrective osteotomies.
        Swiss Medical Weekly. 2018; 148: 29S-S
        • Muller F.
        • Roner S.
        • Liebmann F.
        • et al.
        Augmented reality navigation for spinal pedicle screw instrumentation using intraoperative 3D imaging.
        Spine Journal. 2020; 20: 621-628
        • Viehöfer A.F.
        • Wirth S.H.
        • Zimmermann S.M.
        • et al.
        Augmented reality guided osteotomy in hallux Valgus correction.
        BMC Musculoskelet Disord. 2020; 21: 438
        • Jiang J.G.
        • Huang Z.Y.
        • Qian W.
        • et al.
        Registration Technology of Augmented Reality in Oral Medicine: A Review.
        Ieee Access. 2019; 7: 53566-53584
        • Liu H.
        • Wu J.
        • Tang Y.
        • et al.
        Percutaneous placement of lumbar pedicle screws via intraoperative CT image-based augmented reality-guided technology.
        Journal of Neurosurgery: Spine. 2020; 32: 542-547
        • Zhu M.
        • Liu F.
        • Zhou C.
        • et al.
        Does intraoperative navigation improve the accuracy of mandibular angle osteotomy: Comparison between augmented reality navigation, individualised templates and free-hand techniques.
        J Plast Reconstr Aesthet Surg. 2018; 71: 1188-1195
        • Wang J.
        • Shen Y.
        • Yang S.
        A practical marker-less image registration method for augmented reality oral and maxillofacial surgery.
        International Journal of Computer Assisted Radiology and Surgery. 2019; 14: 763-773
        • Chen G.
        • Zeng W.
        • Yin H.
        • et al.
        The Preliminary Application of Augmented Reality in Unilateral Orbitozygomatic Maxillary Complex Fractures Treatment.
        J Craniofac Surg. 2020; 31: 542-548
      2. Urakov TM, Wang MY, Levi AD. Workflow Caveats in Augmented Reality-Assisted Pedicle Instrumentation: Cadaver Lab. World Neurosurg. 2019;126:e1449-e1455.

        • Rose A.S.
        • Kim H.
        • Fuchs H.
        • et al.
        Development of augmented-reality applications in otolaryngology-head and neck surgery.
        Laryngoscope. 2019; 129: S1-S11
        • Pepe A.
        • Trotta G.F.
        • Mohr-Ziak P.
        • et al.
        A Marker-Less Registration Approach for Mixed Reality-Aided Maxillofacial Surgery: a Pilot Evaluation.
        J Digit Imaging. 2019; 32: 1008-1018
        • Gibby J.
        • Cvetko S.
        • Javan R.
        • et al.
        Use of augmented reality for image-guided spine procedures.
        European Spine Journal. 2020; 29: 1823-1832
        • Condino S.
        • Carbone M.
        • Piazza R.
        • et al.
        Perceptual Limits of Optical See-Through Visors for Augmented Reality Guidance of Manual Tasks.
        Ieee Transactions on Biomedical Engineering. 2020; 67: 411-419
        • Meulstee J.W.
        • Nijsink J.
        • Schreurs R.
        • et al.
        Toward Holographic-Guided Surgery.
        Surgical Innovation. 2019; 26: 86-94
        • Kuhlemann I.
        • Kleemann M.
        • Jauer P.
        • et al.
        Towards X-ray free endovascular interventions - using HoloLens for on-line holographic visualisation.
        Healthc Technol Lett. 2017; 4: 184-187
        • Andress S.
        • Johnson A.
        • Unberath M.
        • et al.
        On-the-fly augmented reality for orthopedic surgery using a multimodal fiducial. Journal of Medical.
        Imaging. 2018; 5
        • Liu J.
        • Al'Aref S.J.
        • Singh G.
        • et al.
        An augmented reality system for image guidance of transcatheter procedures for structural heart disease.
        PLoS One. 2019; 14: e0219174