Holographic Emulation is a new feature that vastly reduces iteration time when developing holographic applications in Unity. Developers creating applications for Microsoft HoloLens will immediately benefit by being able to prototype, debug, and iterate on design directly from the Unity Editor without getting bogged down by long build and deploy times. Holographic Emulation works in two different modes: Remoting and Simulation.
Holographic Remoting allows you to run your application directly from the Editor after connecting to a Windows Holographic device (Microsoft HoloLens). The application will behave as if it were deployed to the device (with full sensor data and positioning), but will actually be running on the host machine. The Game view window will also allow you to see what is being rendered on the device (absent, of course, anything that the wearer of the device sees from the real world).
Holographic Simulation goes one step further, allowing you to run on a simulated device directly in the editor — in other words, no physical device is necessary. This is a great option for development when you have a limited number of devices to share among your team, or you want to get started early with holographic development before getting physical hardware.
Requirements
In order to use Holographic Emulation, you’ll need to be running Unity 5.5 (now currently in beta) on a machine with Windows 10 Anniversary update (or later) installed.
Getting Started
To enable remoting or simulation, open the Holographic Emulation control window from the Editor Windows menu by selecting “Holographic Emulation”:
The Holographic Emulation control window can be moved and docked like any other window; we recommend keeping it visible during development because it has a few settings you’ll want access to when starting your application.
Initially the Emulation Mode will be set to None, which means that your application will run in the Editor without any of the Holographic API functionality.
Holographic Remoting
To enable Remoting, change the Emulation Mode to “Remote to Device”. The interface will change to reflect the additional options available when remoting.
To connect to your device, you’ll need to do three things. First, you’ll need to install and run the Remoting Player, available from the Windows Store. Launch the application on the device, and it will enter a waiting state and show the IP address of the device. Additional information about the Remoting Player, including how to enable connection diagnostics, can be found on the Microsoft Windows Dev Center.
Second, enter the IP address of your device in the Remote Machine field. The dropdown button to the right of the field will allow you to select recently used addresses.
Finally, press the “Connect” button. The connection status should change to a green light with a connected message. You’re now ready to hit Play in the Editor and run your device remotely. You can pause, inspect objects, and debug just like you are running an app in the Editor, because that’s essentially what’s happening, but with video, audio, and device input transmitted back and forth across the network between the host machine and the device. Remoting isn’t useful for validating performance (since it’s using your host machine for all the heavy lifting), but it is a great way to iterate quickly during development.
Holographic Simulation
Running in Simulation mode is even simpler than Remoting. Just change the mode to “Simulate in Editor” and press the Play button. Your application will start with a emulator built into the Editor — no headset required. You can choose from one of five virtual rooms (the same as those supplied with the XDE HoloLens Emulator) as well as specify which virtual hand (left or right) performs gestures.
When in Simulation mode, you control a virtual human with a game controller.
Note that in order to use the game controller, the game window will need to have focus. Just clicking the Game view window once after doing anything else with the UI should give it focus.
Known Limitations
- Speech (PhraseRecognizer) is not currently supported via Holographic Remoting, and instead will intercept speech from the Editor-hosted machine.
- PhotoCapture can be performed during Holographic Simulation, but since the capture will be performed via a local camera (such as a web cam), you will not be able to retrieve a matrix with TryGetProjectionMatrix or TryGetCameraToWorldMatrix.
- During Remoting, all audio on the hosted PC (not just that from the app) will redirect to the device.
ahtur
十一月 21, 2016 3:55 下午Please, what’s the pipeline to prepare a scene offline in Simulation mode ? Is there a way to input one’s own xef map and initialize the virtual objects’ positions in it ?
ahtur
十一月 15, 2016 12:50 下午Please, how can I make the selected room appear in my Unity3D Scene window in play mode ?
Peter Freese
十一月 15, 2016 8:14 下午You won’t be able to see the actual room, but you can display the spatial mapping data. You can either implement your own renderer/shader for this, or use the built-in Spatial Mapping components: https://docs.unity3d.com/550/Documentation/Manual/windowsholographic-sm-component.html
ahtur
十一月 21, 2016 10:26 上午Thank you, the Spatial Mapping component is what I was looking for.
TOBY
十一月 7, 2016 7:59 上午Hi Peter, could we get a list of the keyboard controls? Here’s what I’ve found so far:
Keyboard controls:
WASD/Arrow keys – move forward/move backward/left strafe/right strafe
Shift – enable mouse pan
Space bar – perform tap gesture with virtual hand
Awesome stuff!
christian
十一月 3, 2016 6:08 上午It’s Not workiong on my PC :(
Peter Freese
十一月 3, 2016 6:44 下午It looks like a bug was introduced in b10 which prevents loading the DLLs necessary to support Holographic Emulation. A fix is on the way to next week’s beta. Reverting to b9 should allow you to test and experiment with Simulation and Remoting.
Pixelplacement
十一月 4, 2016 6:39 上午Even with b9 it just sits at “Connecting” and never connects to the HMD.
Peter Freese
十一月 4, 2016 9:40 下午Check to make sure that: 1) that you are running the remoting app and trying to connect at the address shown on the app, and 2) that you can connect to the device at that IP address — you should be able to connect via WebB to that address.
Philip
十一月 2, 2016 8:03 上午I’m getting the “Failed to get HolographicSpace from SimulationManager” when I’m running it, any one encountered it as well?
Peter Freese
十一月 2, 2016 8:00 下午Please submit a bug report so we can get some information about your OS configuration.
Peter Freese
十一月 3, 2016 6:43 下午It looks like a bug was introduced in b10 which prevents loading the DLLs necessary to support Holographic Emulation. A fix is on the way to next week’s beta. Reverting to b9 should allow you to test and experiment with Simulation and Remoting.
Philip
十一月 4, 2016 7:19 上午Cool, I also sent a bug report to you guys. I’ll be reverting back to b9 for the meantime. Thanks!
ahtur
十一月 15, 2016 10:17 上午Doesn’t seem fixed in b11 either. I still have the message :(
Peter Freese
十一月 15, 2016 8:16 下午The fix for the issue preventing Holographic Emulation from working unfortunately did not get in in time for b11, but will be in the next release of 5.5.
ragamoffyn
十月 31, 2016 10:05 上午Pardon the newbie question, but how do we enable this? I have installed the beta and selected “virtual reality supported” in Project Settings, but the Holographic Emulation window keeps giving me this message: “You must enable Virtual Reality support in settings and add Windows Holographic to the device to use Holographic Emulation.”
How do I do the latter…? There’s no listing for Windows Holographic in the VR SDKs dropdown.
ragamoffyn
十月 31, 2016 10:50 上午Nevermind. Figured it out. I didn’t include the Windows Store component in the beta installation.
Pravin Yadav
十月 30, 2016 12:47 下午Most awaited feature. Ah, what a relief!
cre8ivepark
十月 29, 2016 4:23 上午This is amazing! It will dramatically reduce iteration time for the holographic app development.
Thank you for the updates!
Linojon
十月 28, 2016 3:09 下午And it requires Windows 10 Pro, not Home, right?
Michal
十月 28, 2016 3:13 下午U can use Pro or Education version of W 10
roboto
十月 28, 2016 9:29 下午Enterprise, Pro or Education, yup. Key word is Hyper-V…
https://msdn.microsoft.com/en-us/virtualization/hyperv_on_windows/quick_start/walkthrough_compatibility