We built HOMECOMING as an interactive mixed-reality system that combines real-time 3D environments, embodied interaction, machine learning, and choreographic media. The experience places the player in a body-less state inside a surreal desert environment, navigating memory fragments through movement-based interaction rather than traditional controls.

The core interaction revolves around golden ornaments distributed throughout the environment. Each ornament represents a spatial pattern the player must reproduce through hand gestures. We developed a custom image-classification machine learning model to recognize these movement patterns in real time, enabling the system to validate player input and trigger progression events. Successful pattern matching unlocks narrative content and advances the experience.

Narrative memories are delivered as 360° dance videos, integrated directly into the 3D world. These videos are triggered dynamically and spatially anchored, allowing choreography to function as both narrative content and interactive reward. The system synchronizes player state, ML recognition output, and media playback to ensure smooth transitions between exploration and memory reveal.

The 3D environment was constructed entirely from scratch using an iterative pipeline that moved from concept sketches to modular world-building tools. We implemented custom shaders, lighting setups, and visual effects to support navigation cues, interaction feedback, and narrative pacing. These effects were designed not as decoration, but as functional UX elements that guide the player’s attention and reinforce state changes in the system.

From a production standpoint, the project required tight integration between choreography, narrative design, and technical systems. We collaborated closely with the artistic team to translate thematic concepts—identity, loss, and memory—into clear interaction logic, system states, and feedback loops. The experience reinterprets the myth of Eurydice through interactive mechanics, using progression, repetition, and embodied input as narrative devices rather than linear storytelling.

HOMECOMING was produced in two formats:

• Live Mixed Reality Performance: 60 minutes

• Immersive Installation: 50 minutes

Both versions share the same underlying technical architecture, adapted for different audience flow, interaction density, and performance constraints.