Overview
Auscultation — listening to the sounds of the heart and lungs — is a foundational clinical skill that is hard to learn from textbooks and risky to practise only on real patients. This mixed-reality application lets learners practise on virtual patients in realistic clinical scenarios, combining high-quality audio samples of heartbeats and breath sounds with interactive 3D models of cardiac and pulmonary anatomy.
Built for Meta Quest devices (Quest Pro, Quest 3, and Quest 3S with colour passthrough), the platform lets users select clinical scenarios and practise distinguishing normal from pathological sounds. It supports diverse learning styles, offers anytime/anywhere practice, and provides an engaging interface that helps students build confidence and competence before they ever touch a real stethoscope. As Lead Researcher I directed the design and development of the platform, in collaboration with the ICU Follow-up Care Research Lab of the Nursing Department.
Highlights
- Realistic sound simulation of both physiological and pathological heart and lung conditions.
- Interactive 3D anatomical models of the heart and lungs, aligned with the audio.
- Flexible, selectable clinical scenarios ranging from common sounds to rare conditions.
- Runs on Meta Quest Pro / Quest 3 / Quest 3S using mixed-reality colour passthrough.
- Interdisciplinary collaboration between engineering (CoNSeRT) and nursing (ICU Lab).
Gallery
The MR Auscultation App
The application running on Meta Quest — listening, placement, and feedback in mixed reality.
Clinical Scenarios & 3D Anatomy
The interactive 3D heart and lung models and the selectable clinical scenarios.
Lab & Demonstrations
Demonstrations and testing with the ICU Follow-up Care Research Lab.
At a glance
Role
Lead Researcher
Partners
CoNSeRT Lab × ICU Follow-up Care Research Lab
Timeline
Jul 2024 — Present
Platform
Meta Quest Pro / 3 / 3S
Team
with K. Evgenikos & D. Papageorgiou