It is late afternoon in a busy hospital. A team of specialists gathers for the weekly Multidisciplinary Tumor Board. On the agenda is a list of complex cancer cases where the next step is anything but obvious. For each patient, the group has only a few minutes to review scans, lab values, pathology reports, guidelines, and the personal situation behind the case - and then agree on a recommendation that will shape that person’s future.

In practice, much of this precious time is lost to chasing information. Doctors switch between different programs and screens, scrolling through the electronic health record, opening the imaging archive, searching for the right report. High-resolution 3D scans are reduced to flat 2D projections, and the discussion must be documented manually during or after the meeting. This creates stress for the team and increases the risk that the full story of the patient is not clearly visible.

The VR-MTB project was created to tackle exactly this problem. Our goal is to build an Extended Reality tumor board environment where all relevant data for a case comes together in one place: imaging, reports, timelines, and notes. In this immersive “virtual tumor board room”, specialists can explore 3D anatomy together while AI tools capture and structure the discussion in the background - so that the focus stays where it belongs: on making the best possible decision for each patient.

The Core Challenge: Why Traditional Multidisciplinary Tumor Boards Fall Short

Cancer care relies fundamentally on Multidisciplinary Tumor Boards (MTBs), specialized meetings where experts collaborate to formulate optimal treatment plans. Research confirms that this collaborative review is critical, often resulting in changes to nearly 40 per cent of initial treatment plans.

However, traditional MTB processes are hindered by systemic inefficiencies that compromise their effectiveness. A major challenge is data fragmentation: essential patient information - including medical history from the Electronic Health Record (EHR), high-resolution scans from the Picture Archiving and Communication System (PACS), and laboratory results from the Laboratory Information System (LIS) - is typically stored in isolated hospital systems. Clinicians must manually retrieve and compile these disparate data points, a labor-intensive process that can consume between 15 and 90 minutes for a single complex case.

During the actual meeting, which often averages just 5 to10 minutes per case, teams struggle with outdated visualization tools. Diagnostic data, such as CT and MRI scans, are inherently volumetric and complex, yet they are typically displayed on conventional 2D screens. This limitation hinders spatial comprehension, making it challenging for surgeons and radiation oncologists to accurately assess tumor margins and plan treatment targets. Furthermore, the observation of current clinical sessions confirms that meeting documentation is often managed manually, requiring dedicated personnel to type final decisions into the hospital system, adding to the administrative load and risking transcription errors.

Introducing the VR-MTB Vision: An Immersive Research Platform

To revolutionize how oncology teams manage data, visualize cases, and collaborate, the research team at the European Campus Rottal-Inn initiated the VR-MTB project.

This project is a translational research initiative focused on developing an advanced digital platform for oncology. Our goal is to create an interactive virtual environment that fundamentally changes the nature of tumor board collaboration, moving it from a fragmented, 2D experience to an immersive, integrated workspace. The project is fully supported with €870,000 in funding from the German Federal Ministry for Research, Technology, and Space (BMBF) and is scheduled to run for three years, from March 2025 to January 2028. The VR-MTB project is currently in the research and innovation phase.

Core Technological Pillars

The platform incorporates Extended Reality (XR) and Artificial Intelligence (AI) to address clinical workflow bottlenecks directly:

• Immersive XR Visualization: We utilize XR technology to enable high-quality visualization of complex medical data. This allows the multidisciplinary team to collaboratively view and manipulate volumetric images, such as DICOM scans, in real-time 3D representations within a shared virtual space, significantly improving spatial awareness and comprehension for diagnostic and planning purposes.
   
• Intelligent AI Integration: The platform incorporates AI-driven tools based on a Large Language Model (LLM) to automate critical documentation tasks. This functionality streamlines documentation by transcribing and summarizing discussions, categorizing clinical data, and preparing it for integration into the EHR. This innovation drastically reduces the clerical load currently associated with manual note-taking.
   
• Seamless Interoperability: A major focus is ensuring the platform’s secure compatibility with existing hospital infrastructure.

A Structured Pathway from Research to Reality

The project follows a rigorous roadmap designed to translate the prototype from a lab-tested concept into a clinically relevant solution.

General Objectives

1. 1. Enhance Patient-Centered Oncology Care: Developing an advanced VR-based platform that allows oncology teams to analyze complex cancer cases in an immersive 3D environment, ultimately aiming to improve treatment outcomes.
    
2. 2. Improve Transdisciplinary Collaboration and Decision-Making: Facilitating smooth communication between multidisciplinary teams by securely integrating various hospital data sources to optimize treatment planning.
    
3. 3. Facilitate the Integration of Advanced Digital Technologies: Ensuring the platform’s compatibility with existing HIS, EHR, PACS, and CDSS to support the adoption of VR in clinical settings.

Partnerships for Real-World Validation

A cornerstone of the VR-MTB project is the close partnership between DIT-ECRI and Rottal-Inn Kliniken. As an associated clinical partner, Rottal-Inn Kliniken contributes invaluable real-world clinical expertise, especially in oncology workflows. The hospital's staff will be crucial during the clinical evaluation phase, ensuring that the technology is practical, user-friendly, and aligns seamlessly with existing processes, including their use of the ORBIS EHR system. The project is led by the DIT-ECRI research team and funded by the BMBF.

Key Takeaways: The Need for VR-MTB

• Preparation Burden: Clinicians spend significant, valuable time (15 to  90 minutes per case) manually gathering and compiling disparate data from systems like EHR, PACS, and LIS.
   
• Visualization Gap: Conventional 2D screens fail to provide the spatial comprehension necessary for accurate interpretation and planning related to complex 3D radiological data.
   
• Interoperability and Automation: Existing tumor board tools often struggle with fragmented data, highlighting a crucial unmet need for automated data integration and AI-assisted documentation in real-time.

Navigating the Roadblocks: Lessons Learned

Our research into existing digital health implementations and dedicated stakeholder interviews identified significant hurdles that must be addressed for successful adoption.

MTB Gaps and the Promise of XR/AI

In our literature review and observational studies, we found that traditional meetings, while medically necessary, struggle with workflow efficiency. Case presentation often occurs via multiple screens, and the documentation relies on manual typing into the Hospital Information System (HIS), leading to minor delays and clerical fatigue. Furthermore, many digital tools are narrowly focused and lack comprehensive features, complicating the overall workflow.

The promise of our technology lies in its potential to solve these systemic issues:

• Visualization: XR/VR offers detailed 3D visualization, which enhances spatial understanding - a major need, especially for surgical planning - going far beyond the limits of 2D visualization.
   
• Efficiency: AI can automate data aggregation and summarization, replacing labor-intensive manual preparation and increasing meeting efficiency.
   
• Collaboration: VR creates a shared, immersive environment that improves communication and remote participation, mitigating issues caused by specialist absenteeism and the lack of interactive tools in basic video conferencing.

Organizational and Technical Barriers

We are actively mitigating known barriers that often impede the integration of new digital systems in healthcare:

• Resistance and Training: Clinicians frequently exhibit resistance to change, preferring familiar tools over new digital systems they perceive as complex or disruptive to their existing workflows. This necessitates user-centered design, customized training modules, and strong support.
   
• Cost and Infrastructure: The initial investment required for specialized XR hardware (headsets) and integrating them with complex existing systems poses a significant financial constraint for many healthcare organizations.
   
• Technical Interoperability: Even robust technical solutions fail if they cannot securely and seamlessly communicate with heterogeneous legacy IT systems (HIS, PACS, LIS). Relying on standardized data protocols (FHIR, DICOM) and well-defined APIs is the mandatory technical prerequisite to ensuring data flows smoothly.
   
• Usability and Comfort: User experience issues, such as the possibility of motion sickness (or cybersickness) from head-mounted displays (HMDs), must be addressed during the prototyping phase to ensure the long-term comfort and acceptance of the technology by clinicians.

Ethical and Regulatory Framing

As a university research initiative, we are focused on developing a secure and reliable technological blueprint. The VR-MTB platform is a research prototype and must not be confused with a commercially cleared medical device.

Our work is built on principles of Safety by Design, aligning with crucial European regulatory standards.

• Data Protection: We ensure full compliance with the General Data Protection Regulation (GDPR), mandating robust controls for data encryption, access control, pseudonymization, and secure storage of sensitive patient information within the multi-user environment.
   
• Medical Device Standards: The development adheres to the principles of the EU Medical Device Regulation (MDR), particularly for software that provides information for clinical decisions or planning. We are also tracking forthcoming obligations under the EU AI Act, ensuring that AI components meet requirements for transparency and accountability.

Upon successful development and rigorous prototype demonstration, the VR-MTB system would seek appropriate conformity assessment, such as CE certification, confirming it meets the required safety and performance benchmarks for potential future medical deployment.

What’s Next for VR-MTB?

The project is currently focused on the development cycle, specifically working on the fully functional Demonstrator and preparing for the critical Clinical Demonstration and Evaluation. This stage will involve rigorous testing with our associated clinical partner, Rottal-Inn Kliniken, to validate the platform’s usability, efficiency gains, and functional integrity within a live hospital environment.

We are confident that by continuously integrating clinical feedback and adhering to strict technical and ethical standards, the VR-MTB platform will set a new benchmark for multidisciplinary oncology collaboration.

We invite clinicians, health-IT specialists, and administrative leaders who share our passion for digital innovation in cancer care to learn more about the project, follow our progress, and express their interest in future testing opportunities via our official THD VR-MTB page: https://th-deg.de/vr-mtb. Updates will continue to be shared through university news channels and events like the annual DigiHealthDay.

In conclusion, the integration of cutting-edge XR visualization, secure interoperable data, and intelligent AI promises to make tumor board decisions clearer and faster, responsibly transforming oncology for the benefit of both healthcare providers and patients worldwide.

Grigor Chakmishyan

Grigor Chakmishyan is a Research Assistant at ECRI (European Campus Rottal-Inn), Deggendorf Institute of Technology, with a background in Medicine and Molecular Biology and a completed M.Sc. in Digital Health. Now based in Germany, he works on digital health applications and data-driven care; his master’s thesis examined the structural and economic feasibility of 10P Health. Grigor is a health enthusiast, chess player, and a beginner AI artist. For a dose of inspiration, check out his AI art on Instagram: @eaglesgift_