Mixed reality in surgical mechatronics: Innovations and challenges in bioengineering education and clinical practice

Realidad mixta en mecatrónica quirúrgica: Innovaciones y retos en la educación en bioingeniería y práctica clínica | 手术机械电子学中的混合现实:生物工程教育和临床实践中的创新与挑战

Authors

  • Adrian Nacarino Instituto de Investigaciones en Ciencias Biomédicas (INICIB), Universidad Ricardo Palma, Lima, Perú
  • Anderson La-Rosa Instituto de Investigaciones en Ciencias Biomédicas (INICIB), Universidad Ricardo Palma, Lima, Perú.
  • Bryan Sanchez Instituto de Investigaciones en Ciencias Biomédicas (INICIB), Universidad Ricardo Palma, Lima, Perú.
  • Jose Cornejo Instituto de Investigaciones en Ciencias Biomédicas (INICIB), Universidad Ricardo Palma, Lima, Perú.
  • Mariela Vargas Instituto de Investigaciones en Ciencias Biomédicas (INICIB), Universidad Ricardo Palma, Lima, Perú
  • Jorge Cornejo Instituto de Investigaciones en Ciencias Biomédicas (INICIB), Universidad Ricardo Palma, Lima, Perú. Mayo Clinic, USA
  • Ricardo Palomares Grupo de Investigación en Robótica y Mecatrónica Avanzada (GI-ROMA), Universidad Ricardo Palma, Lima, Perú.

DOI:

https://doi.org/10.25176/RFMH.v24i4.6807

Keywords:

Realidad Virtual, Realidad Aumentada, Realidad Mixta, Cirugía, Mecatrónica

Abstract

Background: Mixed reality (MR), a combination of augmented reality (AR) and virtual reality (VR), has emerged as a key tool in surgical mechatronics, improving precision and visualization in clinical procedures. Methods: This study presents a literature review of recent MR advances from the Scopus database, taking into account its impact on surgery and medical education. Technologies such as Microsoft HoloLens and Magic Leap, as well as neurosurgery and orthopedic simulators, are reviewed. Results: MR enhances surgical precision and reduces operating times, with an average decrease from 121.34 to 97.62 minutes. Despite its potential, challenges include discomfort from prolonged device use and low battery autonomy. Conclusions: MR has the potential to transform surgery and medical education; however, its widespread adoption will depend on overcoming technological and financial barriers, especially in Latin America, where infrastructure is still limited.

Downloads

Download data is not yet available.

Author Biographies

Jose Cornejo, Instituto de Investigaciones en Ciencias Biomédicas (INICIB), Universidad Ricardo Palma, Lima, Perú.

Professional School of Mechatronic Engineering, Faculty of Engineering, Ricardo Palma University, Lima, Peru
 Advanced Robotics and Mechatronics Research Group (GI-ROMA), Ricardo Palma University, Lima, Peru

Ricardo Palomares, Grupo de Investigación en Robótica y Mecatrónica Avanzada (GI-ROMA), Universidad Ricardo Palma, Lima, Perú.

Advanced Robotics and Mechatronics Research Group (GI-ROMA), Ricardo Palma University, Lima, Peru
Details: Works in advanced robotics and mechatronics research.

References

McKnight RR, Pean CA, Buck JS, Hwang JS, Hsu JR, Pierrie SN. Virtual Reality and Augmented Reality—Translating Surgical Training into Surgical Technique. Curr Rev Musculoskelet Med. 2020;13(6):663–74. doi:10.1007/s12178-020-09667-3

Tepper OM, Rudy HL, Lefkowitz A, Weimer KA, Marks SM, Stern CS, et al. Mixed Reality with HoloLens: Where Virtual Reality Meets Augmented Reality in the Operating Room. Plast Reconstr Surg. 2017;140(5):1066. doi:10.1097/PRS.0000000000003802

Yoon JW, Chen RE, Kim EJ, Akinduro OO, Kerezoudis P, Han PK, et al. Augmented reality for the surgeon: Systematic review. Int J Med Robot. 2018;14(4):e1914. doi:10.1002/rcs.1914

Vervoorn MT, Wulfse M, Doormaal TPCV, Ruurda JP, Kaaij NPV der, Heer LMD. Mixed Reality in Modern Surgical and Interventional Practice: Narrative Review of the Literature. JMIR Serious Games. 2023;11(1):e41297. doi:10.2196/41297

Cornejo J, Cornejo-Aguilar JA, Vargas M, Helguero CG, Milanezi de Andrade R, Torres-Montoya S, et al. Anatomical Engineering and 3D Printing for Surgery and Medical Devices: International Review and Future Exponential Innovations. BioMed Res Int. 2022;2022(1):6797745. doi:10.1155/2022/6797745

Zhang J, Yu N, Wang B, Lv X. Trends in the Use of Augmented Reality, Virtual Reality, and Mixed Reality in Surgical Research: a Global Bibliometric and Visualized Analysis. Indian J Surg. 2022;84(1):52–69. doi:10.1007/s12262-021-03243-w

Carmigniani J, Furht B. Augmented Reality: An Overview. En: Furht B, editor. Handbook of Augmented Reality [Internet]. New York, NY: Springer; 2011 [citado el 29 de agosto de 2024]. p. 3–46. doi:10.1007/978-1-4614-0064-6_1

Allcca D, Nacarino A, Sanchez B, Castro R, La-Rosa A, Cornejo J, et al. Mechatronics Bio-Design of Hip Prosthesis using Mechanic of Materials Analysis and Finite Element Method: A Proof of Concept. En: 2024 5th International Conference on Recent Trends in Computer Science and Technology (ICRTCST) [Internet]. 2024 [citado el 31 de agosto de 2024]. p. 438–43. doi:10.1109/ICRTCST61793.2024.10578526

Coutrix C, Nigay L. Mixed reality: a model of mixed interaction. En: Proceedings of the working conference on Advanced visual interfaces [Internet]. New York, NY, USA: Association for Computing Machinery; 2006 [citado el 29 de agosto de 2024]. p. 43–50. (AVI ’06). doi:10.1145/1133265.1133274

Liao T. Future directions for mobile augmented reality research: Understanding relationships between augmented reality users, nonusers, content, devices, and industry. Mob Media Commun. 2019;7(1):131–49. doi:10.1177/2050157918792438

Rokhsaritalemi S, Sadeghi-Niaraki A, Choi S-M. A Review on Mixed Reality: Current Trends, Challenges and Prospects. Appl Sci. 2020;10(2):636. doi:10.3390/app10020636

Carmigniani J, Furht B, Anisetti M, Ceravolo P, Damiani E, Ivkovic M. Augmented reality technologies, systems and applications. Multimed Tools Appl. 2011;51(1):341–77. doi:10.1007/s11042-010-0660-6

Moser T, Hohlagschwandtner M, Kormann-Hainzl G, Pölzlbauer S, Wolfartsberger J. Mixed Reality Applications in Industry: Challenges and Research Areas. En: Winkler D, Biffl S, Bergsmann J, editores. Software Quality: The Complexity and Challenges of Software Engineering and Software Quality in the Cloud. Cham: Springer International Publishing; 2019. p. 95–105. doi:10.1007/978-3-030-05767-1_7

Adams L, Krybus W, Meyer-Ebrecht D, Rueger R, Gilsbach JM, Moesges R, et al. Computer-assisted surgery. IEEE Comput Graph Appl. 1990;10(3):43–51. doi:10.1109/38.55152

Cornejo J, Cornejo J, Vargas M, Carvajal M, Perales P, Rodríguez G, et al. SY-MIS Project: Biomedical Design of Endo-Robotic and Laparoscopic Training System for Surgery on the Earth and Space. Emerg Sci J. 2024;8(2):372–93. doi:10.28991/ESJ-2024-08-02-01

Cornejo J, Cornejo-Aguilar JA, Gonzalez C, Sebastian R. Mechanical and Kinematic Design of Surgical Mini Robotic Manipulator used into SP-LAP Multi-DOF Platform for Training and Simulation. En: 2021 IEEE XXVIII International Conference on Electronics, Electrical Engineering and Computing (INTERCON) [Internet]. 2021 [citado el 31 de agosto de 2024]. p. 1–4. doi:10.1109/INTERCON52678.2021.9532965

Cornejo J, Barrera S, Ruiz CAH, Gutierrez F, Casasnovas MO, Kot L, et al. Industrial, Collaborative and Mobile Robotics in Latin America: Review of Mechatronic Technologies for Advanced Automation. Emerg Sci J. 2023;7(4):1430–58. doi:10.28991/ESJ-2023-07-04-025

Dagnino G, Kundrat D. Robot-assistive minimally invasive surgery: trends and future directions. Int J Intell Robot Appl [Internet]. 2024 [citado el 7 de octubre de 2024]; doi:10.1007/s41315-024-00341-2

Cornejo J, Cornejo-Aguilar JA, Sebastian R, Perales P, Gonzalez C, Vargas M, et al. Mechanical Design of a Novel Surgical Laparoscopic Simulator for Telemedicine Assistance and Physician Training during Aerospace Applications. En: 2021 IEEE 3rd Eurasia Conference on Biomedical Engineering, Healthcare and Sustainability (ECBIOS) [Internet]. 2021 [citado el 31 de agosto de 2024]. p. 53–6. doi:10.1109/ECBIOS51820.2021.9510753

Cornejo J, Perales-Villarroel JP, Sebastian R, Cornejo-Aguilar JA. Conceptual Design of Space Biosurgeon for Robotic Surgery and Aerospace Medicine. En: 2020 IEEE ANDESCON [Internet]. 2020 [citado el 31 de agosto de 2024]. p. 1–6. doi:10.1109/ANDESCON50619.2020.9272122

De Mauro A, Raczkowsky J, Halatsch ME, Wörn H. Mixed Reality Neurosurgical Microscope for Training and Intra-operative Purposes. En: Shumaker R, editor. Virtual and Mixed Reality [Internet]. Berlin, Heidelberg: Springer Berlin Heidelberg; 2009 [citado el 20 de agosto de 2024]. p. 542–9. (Lecture Notes in Computer Science; vol. 5622). doi:10.1007/978-3-642-02771-0_60

Ujiie H, Chiba R, Yamaguchi A, Nomura S, Shiiya H, Fujiwara-Kuroda A, et al. Developing a Virtual Reality Simulation System for Preoperative Planning of Robotic-Assisted Thoracic Surgery. J Clin Med. 2024;13(2):611. doi:10.3390/jcm13020611

Deng Z, Xiang N, Pan J. State of the Art in Immersive Interactive Technologies for Surgery Simulation: A Review and Prospective. Bioengineering. 2023;10(12):1346. doi:10.3390/bioengineering10121346

Lin Z, Lei C, Yang L. Modern Image-Guided Surgery: A Narrative Review of Medical Image Processing and Visualization. Sensors. 2023;23(24):9872. doi:10.3390/s23249872

Galati R, Simone M, Barile G, De Luca R, Cartanese C, Grassi G. Experimental Setup Employed in the Operating Room Based on Virtual and Mixed Reality: Analysis of Pros and Cons in Open Abdomen Surgery. J Healthc Eng. 2020;2020(1):8851964. doi:10.1155/2020/8851964

Xiang N, Liang H-N, Yu L, Yang X, Zhang JJ. A mixed reality framework for microsurgery simulation with visual-tactile perception. Vis Comput. 2023;39(8):3661–73. doi:10.1007/s00371-023-02964-1

Jain S, Timofeev I, Kirollos RW, Helmy A. Use of Mixed Reality in Neurosurgery Training: A Single Centre Experience. World Neurosurg. 2023;176:e68–76. doi:10.1016/j.wneu.2023.04.107

Palumbo A. Microsoft HoloLens 2 in Medical and Healthcare Context: State of the Art and Future Prospects. Sensors. 2022;22(20). doi:10.3390/s22207709

Cornejo J, Cornejo-Aguilar JA, Palomares R. Biomedik Surgeon: Surgical Robotic System for Training and Simulation by Medical Students in Peru. En: 2019 International Conference on Control of Dynamical and Aerospace Systems (XPOTRON) [Internet]. 2019 [citado el 31 de agosto de 2024]. p. 1–4. doi:10.1109/XPOTRON.2019.8705717

Zari G, Condino S, Cutolo F, Ferrari V. Magic Leap 1 versus Microsoft HoloLens 2 for the Visualization of 3D Content Obtained from Radiological Images. Sensors. 2023;23(6):3040. doi:10.3390/s23063040

Westermeier F, Brübach L, Wienrich C, Latoschik ME. Assessing Depth Perception in VR and Video See-Through AR: A Comparison on Distance Judgment, Performance, and Preference. IEEE Trans Vis Comput Graph. 2024;30(5):2140–50. doi:10.1109/TVCG.2024.3372061

Anaya-Sánchez R, Rejón-Guardia F, Molinillo S. Impact of virtual reality experiences on destination image and visit intentions: the moderating effects of immersion, destination familiarity and sickness. Int J Contemp Hosp Manag [Internet]. 2024 [citado el 31 de agosto de 2024];ahead-of-print(ahead-of-print). doi:10.1108/IJCHM-09-2023-1488

Zaragoza Pérez R, Cuevas Escudero AL. Realidad aumentada en la enseñanza. RDU UNAM [Internet]. 2020 [citado el 31 de agosto de 2024]; doi:10.22201/cuaieed.16076079e.2020.21.6.9

Arroyo-Berezowsky C, Jorba-Elguero P, Altamirano-Cruz MA, Quinzaños-Fresnedo J. Usefulness of immersive virtual reality simulation during femoral nail application in an orthopedic fracture skills course. J Musculoskelet Surg Res. 2019;3:326. doi:10.4103/jmsr.jmsr_78_19

De Jesus Encarnacion Ramirez M, Chmutin G, Nurmukhametov R, Soto GR, Kannan S, Piavchenko G, et al. Integrating Augmented Reality in Spine Surgery: Redefining Precision with New Technologies. Brain Sci. 2024;14(7):645. doi:10.3390/brainsci14070645

Medellín-Castillo HI, Govea-Valladares EH, Pérez-Guerrero CN, Gil-Valladares J, Lim T, Ritchie JM. The evaluation of a novel haptic-enabled virtual reality approach for computer-aided cephalometry. Comput Methods Programs Biomed. 2016;130:46–53. doi:10.1016/j.cmpb.2016.03.014

da Cruz JAS, dos Reis ST, Cunha Frati RM, Duarte RJ, Nguyen H, Srougi M, et al. Does Warm-Up Training in a Virtual Reality Simulator Improve Surgical Performance? A Prospective Randomized Analysis. J Surg Educ. 2016;73(6):974–8. doi:10.1016/j.jsurg.2016.04.020

de Faria JWV, Teixeira MJ, de Moura Sousa Júnior L, Otoch JP, Figueiredo EG. Virtual and stereoscopic anatomy: when virtual reality meets medical education. J Neurosurg. 2016;125(5):1105–11. doi:10.3171/2015.8.JNS141563

Almirón Santa-Bárbara R, García Rivera F, Lamb M, Víquez Da-Silva R, Gutiérrez Bedmar M. New technologies for the classification of proximal humeral fractures: Comparison between Virtual Reality and 3D printed models—a randomised controlled trial. Virtual Real. 2023;27(3):1623–34. doi:10.1007/s10055-023-00757-4

Morales-Mere J, Chessa JJ, Palomares R, Cornejo J. Mixed Reality System for Education and Innovation in Prehospital Interventions at Peruvian Fire Department. En: 2020 IEEE XXVII International Conference on Electronics, Electrical Engineering and Computing (INTERCON) [Internet]. 2020 [citado el 1 de septiembre de 2024]. p. 1–4. doi:10.1109/INTERCON50315.2020.9220259

Wilson AB, Miller CH, Klein BA, Taylor MA, Goodwin M, Boyle EK, et al. A meta-analysis of anatomy laboratory pedagogies. Clin Anat. 2018;31(1):122–33. doi:10.1002/ca.22934

Cornejo-Aguilar JA, Cornejo J, Vargas M, Sebastian R. The revolution of robotic surgery in latin america and the future implementation in the healthcare system of Peru: La revolución de la cirugía robótica en latino américa y la futura implementación en el sistema de salud del perú. Rev Fac Med Humana. 2019;19(1):5. doi:10.25176/RFMH.v19.n1.1800

Cornejo J, Cornejo Aguilar JA, Perales Villarroel JP. INTERNATIONAL INNOVATIONS IN MEDICAL ROBOTICS TO IMPROVE THE PATIENT MANAGEMENT IN PERU: INNOVACIONES INTERNACIONALES EN ROBÓTICA MÉDICA PARA MEJORAR EL MANEJO DEL PACIENTE EN PERÚ. Rev Fac Med Humana. 2019;19(4):1. doi:10.25176/RFMH.v19i4.2349

Mendoza GAA, Lewis F, Plante P, Brassard C. Estado del arte sobre el uso de la realidad virtual, la realidad augmentada y el video 360° en educación superior. Edutec Rev Electrónica Tecnol Educ. 2023;(84):35–51. doi:10.21556/edutec.2023.84.2769

Cornejo J, Vargas M, Cornejo-Aguilar JA. Robotics and Biomedical Innovative Applications in Public Health during the COVID-19 Pandemic: Aplicaciones Innovadoras De La Robótica Y Biomédica En La Salud Pública Durante La Pandemia Del COVID-19. Rev Fac Med Humana [Internet]. 2020 [citado el 2 de septiembre de 2024];20(4). doi:10.25176/RFMH.v20i4.3042

Clínica Internacional adquiere el primer robot quirúrgico en el Perú | empresas | breca | cirugías | medicina | cirujanos | Intuitive Surgical | ECONOMIA | GESTIÓN [Internet]. [citado el 7 de octubre de 2024]. Disponible en: https://gestion.pe/economia/empresas/clinica-internacional-adquiere-el-primer-robot-quirurgico-en-el-peru-empresas-breca-cirugias-medicina-cirujanos-intuitive-surgical-noticia/#google_vignette

Vargas M, Cornejo J, De La Cruz-Vargas JA. Health without Borders: INICIB-URP leading Medical Education through Bioengineering and Mechatronic Technologies: Salud sin Fronteras: INICIB-URP liderando la Educación Médica a través de Bioingeniería y Tecnologías Mecatrónicas. Rev Fac Med Humana. 2024;24(3):07–11. doi:10.25176/RFMH.v24i3.6701

Vargas M, Cornejo J, Correa-López LE. Ingeniería biomédica: la revolución tecnológica para el futuro del sistema de salud peruano. Rev Fac Med Humana [Internet]. 2017 [citado el 2 de septiembre de 2024];16(2). doi:10.25176/RFMH.v16.n3.659

Caballero-Garriazo JA, Rojas-Huacanca JR, Sánchez-Castro A, Lázaro-Aguirre AF. Systematic review on the application of Virtual Reality in University Education. Rev Electrónica Educ. 2023;27(3):1–18. doi:10.15359/ree.27-3.17271

Morimoto T, Kobayashi T, Hirata H, Otani K, Sugimoto M, Tsukamoto M, et al. XR (Extended Reality: Virtual Reality, Augmented Reality, Mixed Reality) Technology in Spine Medicine: Status Quo and Quo Vadis. J Clin Med. 2022;11(2):470. doi:10.3390/jcm11020470

Zhang R, Jin X, Liu M, Tong HY. The Effectiveness of Augmented Reality/Mixed Reality in Medical Education: A Meta- Analysis [Internet]. 2024 [citado el 30 de agosto de 2024]. doi:10.21203/rs.3.rs-4549366/v1

Wu Q, Wang Y, Lu L, Chen Y, Long H, Wang J. Virtual Simulation in Undergraduate Medical Education: A Scoping Review of Recent Practice. Front Med [Internet]. 2022 [citado el 30 de agosto de 2024];9. doi:10.3389/fmed.2022.855403

Sadek O, Baldwin F, Gray R, Khayyat N, Fotis T. Impact of Virtual and Augmented Reality on Quality of Medical Education During the COVID-19 Pandemic: A Systematic Review. J Grad Med Educ. 2023;15(3):328–38. doi:10.4300/JGME-D-22-00594.1

Martel Cervantes C, Sandoval C, Palomares R, Borja Arroyo J, Murillo Manrique M, Cornejo J. BIOMEDICAL ANTHROPOMETRIC EVALUATION AND CONCEPTUAL MECHANICAL DESIGN OF A ROBOTIC SYSTEM FOR LOWER LIMBS PASSIVE-REHABILITATION ON POST-STROKE PATIENTS. Revista de la Facultad de Medicina Humana. 2024 Apr 1;24(2). doi: 10.25176/rfmh.v24i2.6550

Sandoval C, Martel C, Palomares R, Arroyo JB, Manrique MFM, Cornejo J. Conceptual mechatronic design of ankle-foot exoskeleton system for assisted rehabilitation of pediatric patients with spastic cerebral palsy. In: 2023 IEEE MIT Undergraduate Research Technology Conference (URTC). IEEE; 2023. doi: 10.1109/URTC60662.2023.10534922

Kang D, Choi JH, Hwang H. Autostereoscopic 3D display system for 3D medical images. Applied Sciences. 2022 Apr 24;12(9):4288. doi: 10.3390/app12094288

Published

2024-10-31

How to Cite

Nacarino, A., La-Rosa, A., Sanchez, B., Cornejo, J., Vargas, M., Cornejo, J., & Palomares, R. (2024). Mixed reality in surgical mechatronics: Innovations and challenges in bioengineering education and clinical practice: Realidad mixta en mecatrónica quirúrgica: Innovaciones y retos en la educación en bioingeniería y práctica clínica | 手术机械电子学中的混合现实:生物工程教育和临床实践中的创新与挑战. Revista De La Facultad De Medicina Humana, 24(4), 215–223. https://doi.org/10.25176/RFMH.v24i4.6807