Kelley, Brendan, authorPlabst, Lucas, authorPlabst, Lena, authorACM, publisher2025-03-132025-03-132025-01-19Brendan Kelley, Lucas Plabst, and Lena Plabst. 2024. Segmentation and Immersive Visualization of Brain Lesions Using Deep Learning and Virtual Reality. In The 19th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry (VRCAI 24), December 01 02, 2024, Nanjing, China. ACM, New York, NY, USA, 8 pages. https://doi.org/10.1145/3703619.3706035https://hdl.handle.net/10217/240172Magnetic resonance imaging (MRIs) are commonly used for diagnosing potential neurological disorders, however preparation and interpretation of MRI scans requires professional oversight. Additionally, MRIs are typically viewed as single cross sections of the affected regions which does not always capture the full picture of brain lesions and can be difficult to understand due to 2D's inherent abstraction of our 3D world. To address these challenges we propose a immersive visualization pipeline that combines deep learning image segmentation techniques using a VGG-16 model trained on MRI fluid attenuated inversion recovery (FLAIR) with virtual reality (VR) immersive analytics. Our visualization pipeline begins with our VGG-16 model predicting which regions of the brain are potentially affected by a disease. This output, along with the original scan, are then volumentrically rendered. These renders can then be viewed in VR using an head mounted display (HMD). Within the HMD users can move through the volumentric renderings to view the affected regions and utilize planes to view cross sections of the MRI scans. Our work provides a potential pipeline and tool for diagnosis and care.born digitalarticleseng©Brendan Kelley, et al. ACM 2024. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in VRCAI 24, https://dx.doi.org/10.1145/3703619.3706035.deep learningimage segmentationmedical imagingmedical applicationsvirtual realityimmersive analyticsimmersive visualizationTexthttps://doi.org/10.1145/3703619.3706035