Feasibility of intraprocedural augmented reality visualisation of 3D rotational angiography in congenital cardiac catheterisation

Abstract Three-dimensional rotational angiography has become a mainstay of congenital cardiac catheterisation. Augmented reality is an exciting and emerging technology that allows for interactive visualisation of 3D holographic images in the user’s environment. This case series reports on the feasibility of intraprocedural augmented reality visualisation of 3D rotational angiography in five patients with CHD.

Feasibility of intraprocedural augmented reality visualisation of 3D rotational angiography in congenital cardiac catheterisation Arash Salavitabar 1 , Wendy Whiteside 2 and Jeffrey D. Zampi 2 1 The Heart Center, Nationwide Children's Hospital, Columbus, OH, USA and 2 University of Michigan Congenital Heart Center, Ann Arbor, MI, USA Abstract Three-dimensional rotational angiography has become a mainstay of congenital cardiac catheterisation. Augmented reality is an exciting and emerging technology that allows for interactive visualisation of 3D holographic images in the user's environment. This case series reports on the feasibility of intraprocedural augmented reality visualisation of 3D rotational angiography in five patients with CHD.
Three-dimensional rotational angiography has been a valuable addition to the congenital cardiac catheterisation laboratory and provides information that is otherwise unattainable with standard 2D angiography. However, viewing a 3D image on a 2D computer screen remains a major limitation of this technology. Extended reality technologies, such as augmented reality, have the potential to bridge available 3D imaging acquisition and advanced visualisation technologies by projecting interactive 3D models into the procedural environment. [1][2][3][4] This technology has tremendous potential within the complexity and heterogeneity of CHD and interventional cardiology.
This case series describes the prospective use and feasibility of intraprocedural augmented reality visualisation of three-dimensional rotational angiography in the congenital cardiac catheterisation laboratory.

Methods
This was a single-centre, prospective study in which patients who presented to the congenital cardiac catheterisation laboratory who had a plan for 3D rotational angiography were prospectively enrolled and consented. All 3D rotational angiograms were performed within a Canon (formerly Toshiba) Infinix-I (Tochigi, Japan) fluoroscopic platform, and 3D reconstructions were performed solely using the Vitrea 3D workstation (Vital Solutions, Canon). Models were optimised within Vitrea by selecting the "Remove Fragment" option, followed by the "Visible" function to select areas of interest and "Erode" option to further remove minor artefact. The models were saved as STL files, which were then transferred directly to a Microsoft Hololens 2 mixed reality headset (Microsoft Corporation, Washington, USA) for intraprocedural visualisation while maintaining sterility ( Fig 1A). The primary operators identified discrepancies between the augmented reality and computer models and described additional advantages of the augmented reality model. This study was approved by the institutional review board.

Cases
Augmented reality visualisation of 3D rotational angiography was utilised in five patients with CHD, including right and left heart obstructive lesions, as well as single-ventricle palliation ( Table 1). The mean age was 8.1 ± 9.5 years and mean weight was 40.3 ± 45.1 kg. A mean total injection volume of 2.3 ± 1.2 cc/kg was used for 3D rotational angiography with a median contrast dilution to 50% (range 50-66%). Apnoea was performed for four of five 3D rotational angiograms and all five were performed with a 1-second fluoroscopic delay and without rapid ventricular pacing. To maintain a sterile field, the Hololens 2 headset was easily placed on the operators and removed by a separate staff member in all cases.
Augmented reality visualisation was feasible in all cases (Figs 1B and 1C, Supplementary  Fig 1), and the primary operator was able to maintain sterility, perform manipulations, and share the view and manipulations of the model with a second operator (Supplemental Video). The median time from acquisition of 3D rotational angiography to 3D reconstruction was 2.8 minutes (range 1.5-5.4), and the median time from 3D reconstruction to visualisation of the 3D model in augmented reality was 4.2 minutes (range 2.8-4.6). Interventions were  performed in four of five cases (Table 1). Only one case showed a minor discrepancy between the augmented reality and computer models, with no major discrepancies reported.

Discussion
This case series reports on the use and feasibility of intraprocedural augmented reality visualisation of 3D rotational angiography in the congenital cardiac catheterisation laboratory. This exciting application of mixed reality technology allows the operator to appreciate the 3D model in its true form, rather than relying on a 2D computer screen to represent 3D structures and relationships. In this series of various forms of complex CHD, a wireless mixed reality headset was easily placed on the primary operator while remaining sterile, with full visibility of the procedural environment and with minimal additional procedural time. The models were visualised without significant discrepancies and were able to be manipulated and "sliced" through with simple movements of a finger. This takes the concept and benefits of 3D rotational angiography [5][6][7] and combines them with the added visualisation advantages of 3D printing, 8,9 to allow for advanced, real-time intraprocedural 3D modelling and an ultimate appreciation of the patient's anatomy for the operators. While holographic visualisation of 3D rotational angiography has been reported, 10 its application in the procedural environment has been limited by the need for a handheld tool to manipulate the image at a display workstation. Our study is the first report and demonstration to our knowledge of augmented reality visualisation of 3DRA in a "hands-free" procedural environment that allowed for continued operator sterility, full mobility, and preserved ability to visualise the patient, data, and catheterisation laboratory while utilising voice commands and hand gestures to control the model (Supplementary Video). Additionally, a second operator can interact with the same model through a separate headset, allowing for opportunities for enhanced education and unique team engagement, as well as giving the opportunity to include this technology in post-procedural counselling of the patient and family. Important next steps include development and validation of intraprocedural augmented reality simulation of interventions, such as stent, valve, and device implantation, and the eventual incorporation of computational flow dynamics. Following validation of this tool, future studies should prove its utility for regular intraprocedural decision-making. Practical and feasible potential educational opportunities include facilitating "live case" transmissions for international conferences, remote proctoring for new device technologies and junior faculty, family counselling, and trainee instruction. Industry support for integration of this technology into specific fluoroscopic platforms will be a key future addition, thus allowing for gantry angles and other features to be further utilised within the augmented reality model.

Conclusion
Intraprocedural augmented reality visualisation of 3DRA is feasible in the cardiac catheterisation laboratory. This technology allows for an enhanced appreciation of 3D anatomy in its true form, complementing the existing resources that exist with 3DRA post-processing. Exciting opportunities include live intraprocedural sharing of imaging with a first-person view of the procedural environment, remote proctoring, and critical future development of intraprocedural simulation of transcatheter interventions. These features would provide the ability to acquire, simulate, and intervene all in one setting to provide the ultimate personalisation of patient care.
Supplementary material. To view supplementary material for this article, please visit https://doi.org/10.1017/S1047951122002153