Back to Case Studies
Simulations

XR Medical Simulations

Immersive XR medical training simulation on Meta Quest 3, delivering step-by-step procedural learning with spatial anchors, voice commands, and rich multimedia.

VR/XRHealthcare & ResearchTraining

Type

XR Medical Training Simulation

Platform

Meta Quest 3

Engine

Unity (XR Interaction Toolkit)

Technology Stack

UnityMeta QuestXR Interaction ToolkitUnity Input SystemUnity AddressablesUniTaskAVPro VideoBest.HTTPNewtonsoft.JsonFMODOAuthSpatial AnchorsC#
XR Medical Simulations

Project overview

XR Medical Simulations was developed for Softura and their client, the University of Michigan, as an immersive procedural training platform for medical students and residents on Meta Quest 3.

The product combines step-by-step guided flows, interactive XR content, and rich media delivery to improve procedural understanding while maintaining scalability for future training modules.

Gameplay Videos

Procedure and Course Flow

Step-by-step training path with module navigation, media playback, and guided XR interactions.

OAuth Institutional Login

Secure deep-link authentication flow for institutional account access.

Spatial Anchors Workflow

Persistent, stable placement of content and UI in mixed environments.

Goals and Challenges

Goals

  • •Deliver an immersive XR simulation platform for step-by-step medical procedure training.
  • •Structure content as courses, modules, tasks, and steps for clear procedural progression.
  • •Support rich mixed-media learning including text, 2D/3D content, audio, video, and quizzes.
  • •Provide robust XR interaction models including ray/hand inputs, voice commands, and object alignment.
  • •Maintain readable, stable XR UI in passthrough and mixed-context environments.

Challenges

  • •Integrating with provided backend APIs while operating without direct backend ownership.
  • •Maintaining reliable real-time interactions and object recognition in XR contexts.
  • •Managing multi-format media with stable headset performance and memory behavior.
  • •Implementing OAuth and deep-link authentication with consistent cross-platform behavior.
  • •Adapting quickly to evolving XR SDK and headset software changes during development.

Approach and Technology

Content Runtime

JSON-driven course/module/task/step pipeline

Runtime schema support enabled scalable procedural content orchestration.

Interaction Layer

XR inputs + voice + anchors

Unified interaction model across controller, hand, and anchored UI contexts.

Media System

Single reusable panel for mixed media types

Text, quizzes, 2D/3D assets, and AV content rendered through one extensible UI flow.

Integration Stack

API + OAuth + progression tracking

Institutional login, remote media resolution, and session continuity in one path.

Design and Development

  • •Implemented JSON schema support to define runtime procedures, modules, and media logic.
  • •Built data-driven training progression with reusable flow control and progress tracking.
  • •Unified media delivery into reusable panel components for mixed content experiences.
  • •Integrated spatial anchoring and guided XR interaction for procedural clarity.
  • •Maintained a modular architecture optimized for future in-app course expansion.

Creative and Technical Innovations

  • •Single adaptive media panel that supports text, quizzes, images, 3D content, audio, and video.
  • •Voice-based controls for playback actions such as play, stop, seek, and volume adjustment.
  • •Spatial anchor workflows for consistent, persistent placement of guided training content.
  • •Remote media URL resolution from APIs to support streaming without packaging every asset locally.
  • •Deep-link OAuth login flow adapted for institutional XR usage scenarios.
  • •Gamified FMOD audio cues to reinforce learner feedback without compromising immersion.

Feature Highlights

Auth Flow Screenshot 1
Click to expand

Auth Flow Screenshot 1

Initial institutional authentication stage in XR onboarding.

1 / 11

Results and Impact

  • •Delivered a functional, data-driven XR medical training application aligned with institutional requirements.
  • •Established a stable XR foundation that remained resilient through SDK and platform shifts.
  • •Achieved reliable media-heavy performance with spatial anchoring and mixed interaction systems.
  • •Simplified course authoring workflows through flexible JSON structures and reusable UI systems.
  • •Integrated secure OAuth login and progression tracking into one cohesive learner experience.

Key Takeaways

  • •A unified data-driven XR framework can scale procedural learning without major refactors.
  • •Reusable multi-format media systems dramatically reduce implementation complexity in training apps.
  • •Readable and stable XR UI is critical in passthrough-driven instructional environments.
  • •Combining voice, spatial anchors, and modular progression improves both immersion and retention.

Ready to Build Something Great?

Let's discuss how we can bring your vision to life with cutting-edge game development, interactive experiences, or custom solutions.

Start Your ProjectView More Case Studies