XR For Unity — 3 — Mixed Reality

4 min readJust now

I am Samuel Asher Rivello, a professional game developer with 25 years of game development experience including 10+ years with the Unity game engine. Unity is a powerful and popular suite of tools (Project IDE, Code IDE, and run-time).

Unity For XR

Unity 6 builds upon the strengths of its predecessors while introducing groundbreaking features that elevate the game development process. Unity 6 also offers deeper integration with modern hardware, making it easier to optimize games for cutting-edge platforms like AR, VR, and XR.

XR Technologies

Extended reality (XR) is the umbrella under which we find AR, VR, and MR technologies. These are the new realities.

Augmented Reality (AR) — Layers computer generated imagery on top of the users view of the real world. Typically the user looks through a mobile phone’s live camera feed or the user wears Ar glasses. The included software may rely on SLAM.
Virtual Reality (VR) — Replaces (blocks) the users real world view with a completely digital surroundings. The input can vary, but generally includes at least 6 degrees of head tracking (positions & rotations). Typically the user wears goggles.
Mixed Reality (MR) — Blends AR and VR concepts into the user’s environment. Think of a 3D tooltips hovering over real world objects that respect the position, scale, and movement of the objects.

Unity For Gaming

Unity has long been a cornerstone of the game development industry, providing developers with a versatile and powerful engine capable of delivering experiences across multiple platforms.

With a user-friendly interface, a massive asset store, and a thriving community, Unity has enabled indie developers and large studios alike to create stunning games without the need for massive budgets or teams.

Mixed Reality For Unity

Mixed Reality (MR) in Unity is distinct from both Virtual Reality (VR) and Augmented Reality (AR) in that it combines elements from both to create an experience where virtual objects interact seamlessly with the real world. Unlike AR, which primarily overlays virtual objects onto the real environment, or VR, which immerses users in a completely virtual environment, MR blends both by allowing virtual objects to respond to the real world’s physical properties (e.g., movement, scale, and lighting) while maintaining immersive interactions similar to VR.

Quick Start: https://docs.unity3d.com/Packages/com.unity.template.mixed-reality@2.0/

To set up MR in Unity, developers should use the XR Plugin Management and AR Foundation packages, just as they would for VR and AR, but with additional considerations for MR’s unique features. Unity’s XR Interaction Toolkit provides tools for creating interactions that combine both real-world elements and fully immersive virtual elements. Developers can begin by configuring spatial tracking, enabling the application to understand and interact with physical objects in the user’s environment.

Additionally, features like hand tracking and spatial awareness are critical for MR, as users often need to interact with both the virtual and real environments simultaneously. MR setups in Unity often rely on the use of depth sensors and environmental scanning to allow virtual objects to blend into the user’s physical surroundings. Unity’s integration with major MR hardware, such as Microsoft HoloLens and Magic Leap, simplifies these tasks, providing pre-built components for tracking and interaction.

Best Practices for MR Development

Developing for Mixed Reality requires unique considerations that blend elements of both AR and VR. One of the primary differences between MR and the other technologies is that virtual objects are more interactive with the physical environment. This means that developers need to ensure that virtual objects not only appear correctly in the user’s view but also respect real-world physics, such as gravity and collision with physical objects.

Several headsets work specifically for MR, offering advanced capabilities beyond standard AR or VR headsets. Notable MR headsets include the Microsoft HoloLens 2, which allows users to interact with holograms that respond to physical touch and environmental conditions, and the Magic Leap 2, known for its advanced spatial computing abilities. Both of these headsets integrate well with Unity’s XR features, providing developers with the necessary tools for high-quality MR experiences. Mixed Reality also supports devices like the Varjo XR-3, which offers ultra-high-fidelity visuals for enterprise applications.

Resources

🦜 Contact

Samuel Asher Rivello has over 20 years of game dev XP. He is available for remote, contract hire as a game developer and game dev educator.
Contact Sam today to say hi and discuss your projects!