Search Unity

Holographic Emulation is a new feature that vastly reduces iteration time when developing holographic applications in Unity. Developers creating applications for Microsoft HoloLens will immediately benefit by being able to prototype, debug, and iterate on design directly from the Unity Editor without getting bogged down by long build and deploy times. Holographic Emulation works in two different modes: Remoting and Simulation.

holographicremotinganim-2-50

Holographic Remoting allows you to run your application directly from the Editor after connecting to a Windows Holographic device (Microsoft HoloLens). The application will behave as if it were deployed to the device (with full sensor data and positioning), but will actually be running on the host machine. The Game view window will also allow you to see what is being rendered on the device (absent, of course, anything that the wearer of the device sees from the real world).

Holographic Simulation goes one step further, allowing you to run on a simulated device directly in the editor — in other words, no physical device is necessary. This is a great option for development when you have a limited number of devices to share among your team, or you want to get started early with holographic development before getting physical hardware.

Requirements

In order to use Holographic Emulation, you’ll need to be running Unity 5.5 (now currently in beta) on a machine with Windows 10 Anniversary update (or later) installed.

Getting Started

To enable remoting or simulation, open the Holographic Emulation control window from the Editor Windows menu by selecting “Holographic Emulation”:

holographic-emulation-image-0

The Holographic Emulation control window can be moved and docked like any other window; we recommend keeping it visible during development because it has a few settings you’ll want access to when starting your application.

holographic-emulation-image-1

Initially the Emulation Mode will be set to None, which means that your application will run in the Editor without any of the Holographic API functionality.

Holographic Remoting

To enable Remoting, change the Emulation Mode to “Remote to Device”. The interface will change to reflect the additional options available when remoting.

holographic-emulation-image-2To connect to your device, you’ll need to do three things. First, you’ll need to install and run the Remoting Player, available from the Windows Store. Launch the application on the device, and it will enter a waiting state and show the IP address of the device. Additional information about the Remoting Player, including how to enable connection diagnostics, can be found on the Microsoft Windows Dev Center.

holographic-emulation-image-3

Second, enter the IP address of your device in the Remote Machine field. The dropdown button to the right of the field will allow you to select recently used addresses.

Finally, press the “Connect” button.  The connection status should change to a green light with a connected message. You’re now ready to hit Play in the Editor and run your device remotely. You can pause, inspect objects, and debug just like you are running an app in the Editor, because that’s essentially what’s happening, but with video, audio, and device input transmitted back and forth across the network between the host machine and the device. Remoting isn’t useful for validating performance (since it’s using your host machine for all the heavy lifting), but it is a great way to iterate quickly during development.

Holographic Simulation

Running in Simulation mode is even simpler than Remoting. Just change the mode to “Simulate in Editor” and press the Play button. Your application will start with a emulator built into the Editor — no headset required. You can choose from one of five virtual rooms (the same as those supplied with the XDE HoloLens Emulator) as well as specify which virtual hand (left or right) performs gestures.

holographic-emulation-image-4When in Simulation mode, you control a virtual human with a game controller.

holographicsimulationcontrol

Note that in order to use the game controller, the game window will need to have focus. Just clicking the Game view window once after doing anything else with the UI should give it focus.

Known Limitations

  • Speech (PhraseRecognizer) is not currently supported via Holographic Remoting, and instead will intercept speech from the Editor-hosted machine.
  • PhotoCapture can be performed during Holographic Simulation, but since the capture will be performed via a local camera (such as a web cam), you will not be able to retrieve a matrix with TryGetProjectionMatrix or TryGetCameraToWorldMatrix.
  • During Remoting, all audio on the hosted PC (not just that from the app) will redirect to the device.

22 Comments

Subscribe to comments

Comments are closed.

  1. Please, what’s the pipeline to prepare a scene offline in Simulation mode ? Is there a way to input one’s own xef map and initialize the virtual objects’ positions in it ?

  2. Please, how can I make the selected room appear in my Unity3D Scene window in play mode ?

    1. You won’t be able to see the actual room, but you can display the spatial mapping data. You can either implement your own renderer/shader for this, or use the built-in Spatial Mapping components: https://docs.unity3d.com/550/Documentation/Manual/windowsholographic-sm-component.html

      1. Thank you, the Spatial Mapping component is what I was looking for.

  3. Hi Peter, could we get a list of the keyboard controls? Here’s what I’ve found so far:

    Keyboard controls:

    WASD/Arrow keys – move forward/move backward/left strafe/right strafe
    Shift – enable mouse pan
    Space bar – perform tap gesture with virtual hand

    Awesome stuff!

  4. It’s Not workiong on my PC :(

    1. It looks like a bug was introduced in b10 which prevents loading the DLLs necessary to support Holographic Emulation. A fix is on the way to next week’s beta. Reverting to b9 should allow you to test and experiment with Simulation and Remoting.

      1. Even with b9 it just sits at “Connecting” and never connects to the HMD.

        1. Check to make sure that: 1) that you are running the remoting app and trying to connect at the address shown on the app, and 2) that you can connect to the device at that IP address — you should be able to connect via WebB to that address.

  5. I’m getting the “Failed to get HolographicSpace from SimulationManager” when I’m running it, any one encountered it as well?

    1. Please submit a bug report so we can get some information about your OS configuration.

      1. It looks like a bug was introduced in b10 which prevents loading the DLLs necessary to support Holographic Emulation. A fix is on the way to next week’s beta. Reverting to b9 should allow you to test and experiment with Simulation and Remoting.

        1. Cool, I also sent a bug report to you guys. I’ll be reverting back to b9 for the meantime. Thanks!

        2. Doesn’t seem fixed in b11 either. I still have the message :(

        3. The fix for the issue preventing Holographic Emulation from working unfortunately did not get in in time for b11, but will be in the next release of 5.5.

  6. Pardon the newbie question, but how do we enable this? I have installed the beta and selected “virtual reality supported” in Project Settings, but the Holographic Emulation window keeps giving me this message: “You must enable Virtual Reality support in settings and add Windows Holographic to the device to use Holographic Emulation.”

    How do I do the latter…? There’s no listing for Windows Holographic in the VR SDKs dropdown.

    1. Nevermind. Figured it out. I didn’t include the Windows Store component in the beta installation.

  7. Most awaited feature. Ah, what a relief!

  8. This is amazing! It will dramatically reduce iteration time for the holographic app development.
    Thank you for the updates!

  9. And it requires Windows 10 Pro, not Home, right?

    1. U can use Pro or Education version of W 10