Unity 2019.3 will add support for using Unity as a library controlled by native Android/Java and iOS/Objective C apps so you can easily insert AR and other Unity features.
We know there are times when developers using native platform technologies (like Android/Java and iOS/Objective C) want to include features powered by Unity in their apps and games. Starting with Unity 2019.3.a2, you can integrate the Unity runtime components and your content into a native platform project so you can use Unity as a library.
This means that you can now insert features powered by Unity, such as augmented reality (AR), 3D/2D real-time rendering, 2D mini-games, and more, directly into your native mobile apps.
How it works: Changes to project structure
The Unity runtime library exposes controls to manage when and how to load/activate/unload within the native application.
The mobile app build process overall is still the same. Unity creates the iOS Xcode and Android Gradle projects.
However, to enable this feature, we changed the structure of the generated iOS Xcode and Android Gradle projects as follows:
- A library part – iOS framework and Android Archive (AAR) file – that includes all source files and plugins
- A thin launcher part that includes app representation data and runs the library part
You can find out more details about the changes to the Xcode/Android Gradle projects in this forum post.
We also have step-by-step instructions on how to integrate Unity as a library on iOS and Android, including basic sample projects.
Please note that using Unity as a library requires that you deeply understand the architecture of Android/Java and iOS/Objective C applications.
Some limitations
We tested many scenarios for Unity as a library hosted by a native app. But since Unity no longer controls the lifecycle of the runtime, we cannot guarantee it will work in all possible use cases. For example:
- Unity as a Library supports full-screen rendering only; rendering on only a part of the screen is not supported.
- Loading more than one instance of the Unity runtime is not supported.
- You may need to adapt third-party plugins (native or managed) for them to work properly.
Powering the technology that is revolutionizing marketing campaigns
Unity as a library is a tremendous advantage for various use cases across industries. In particular, brands and creative agencies can now easily insert AR directly into their native mobile apps.
Cutting-edge brands see the value in adding AR to their traditional marketing campaigns. With Unity support for Unity as a library, the process is streamlined. Brands and creative agencies no longer have to rebuild their app to insert AR or hack together a solution to use Unity as a library.
Consumers are ready for AR marketing. AR lets brands foster a personal relationship with consumers, and we’re excited to offer a solution that powers the evolution of marketing.
To learn more about how brands and creative agencies are using Unity, check out our solutions page.
David
June 19, 2019 at 9:19 pm> Unity as a Library supports full-screen rendering only; rendering on only a part of the screen is not supported.
Not supported as in “and will never be” or as in “but will be in the future”? :D
JC Cimetiere
June 20, 2019 at 1:41 amHi David,
Not supported for now.
We are considering it but it’s a tricky scenario where Unity needs to share resources with the native app. We don’t have a timeline yet, and are looking for feedback on what scenario make sense for you.
Please jump on the forum post to share ideas.
JC
Joni
June 19, 2019 at 8:36 pmIt would interesting to have it running under a Cordova hibrid App
David
June 19, 2019 at 6:50 pmI can use this for AR in a lot of my applications so I’m excited! My only request is please release a tutorial or youtube video on an example of how to put this together as it might be a bit confusing doing guesswork once it releases.
JC Cimetiere
June 20, 2019 at 1:44 amwe are making more samples but the one shared on the forum linked in this blog post should give you all the building block you need.
don’t forget, as I wrote:
“Please note that using Unity as a library requires that you deeply understand the architecture of Android/Java and iOS/Objective C applications.”
Liz
June 19, 2019 at 5:05 pmHonestly, this was a lot more impressive until I saw “supports full-screen rendering only.” Really, every potential idea I had up until that point was cast aside.
JC Cimetiere
June 20, 2019 at 1:47 amHi Liz,
Please share your ideas on the forum. we are listening ;-)
As responded on another similar question:
We are considering it but it’s a tricky scenario where Unity needs to share resources with the native app. We don’t have a timeline yet, and are looking for feedback on what scenario make sense for you.
FlyingHighUp
June 18, 2019 at 4:28 pmCould I write a Apple Watch app with Unity with this change?
Like running the engine on the watch itself without a phone companion. Ideally I’d like my game to run both the watch and my phone so I can have a shared codebase.
JC Cimetiere
June 19, 2019 at 12:35 amHi,
No, Unity does not support Watch OS, and no plan for now.
Logo Victoria
June 18, 2019 at 1:40 pmThank you for sharing this good information. Mobile Application Development is an interesting field. Our website have many packages about App Development. You can visit:
App Development Packages
VanTa
June 18, 2019 at 10:33 amIs it planed to be just for mobile?
It could be amazing to use Unity rendering capabilities as a runtime for native desktop apps.
I’m thinking in the direction of embedding Unity as a set of nodes for TouchDesigner for example.
nxrighthere
June 18, 2019 at 11:29 amTake a look at this thread https://forum.unity.com/threads/partial-integration-with-the-engine-and-api-that-powers-dots-using-c-c.674905/#post-4658300
Gregory Pierce
June 18, 2019 at 3:44 amUnity, just go ahead and build us something for Python. The ML interface is Python, the new UI scripting stuff supports python – just go ahead and give us python support @ the engine and be done with it :)
Ed
June 18, 2019 at 7:37 amUsing Python would violate the Performance by Default goal.
Isaac Surfraz
June 18, 2019 at 12:38 pmAside from the terrible performance implications, why would you actually need this? C# really is not that hard to learn and write. You use the right tools for the right job. Python is used for machine learning because its an industry standard and performs well for AI. The same cant be said about python and the rest of use cases, so really putting it for the rest of the engine just takes us back to the point we were with javascript.
If you to write in python, why not just move to an actual python based graphics engine?
Gregory Pierce
June 18, 2019 at 8:40 pmBecause Unity has made a mission of making Unity a platform for more than just games. So while it is true that for regular games it would be a problematic issue – for other use cases (including as a teaching environment), Python would be excellent.
Neoshaman
June 19, 2019 at 3:48 pmThey used to have Boo script, ie it was “python” the same way that unityscript was “javascript”
JC Cimetiere
June 20, 2019 at 1:53 amHi Gregory,
if you care about Python for ML scenarios, please check “Unity ML Agents”:
https://unity3d.com/machine-learning
Unity Machine Learning Agents, the first of Unity’s machine learning product offerings, trains intelligent agents with reinforcement learning and evolutionary methods via a simple Python API.
We don’t plan to add any new language to Unity for now. C# it is.
Gregory Pierce
June 20, 2019 at 8:47 amCertainly JC, that’s where I currently interface with Unity for a number of RL things. Even tried to build something with IronPython a while back because Python is a good teaching/learning language. But I know you guys are busy rebuilding pretty much all of the Unity that I used to know (and that’s a good thing), so maybe when you’re done with that ;)
Ehsan Sayyad
June 18, 2019 at 1:50 amCan’t wait for a python interface! it would be great for machine learning research projects!
JC Cimetiere
June 20, 2019 at 1:54 amlook at “Unity ML Agents”:
https://unity3d.com/machine-learning
Unity Machine Learning Agents, the first of Unity’s machine learning product offerings, trains intelligent agents with reinforcement learning and evolutionary methods via a simple Python API.
steve blake
June 17, 2019 at 10:55 pmthis one is cool i really love these updates by Unity, i have a past of game development but now i am an full time app developer at https://appdevelopmentagency.com.au/. working in a professional environment boosts my creativity i am a fan of Unity.
Ray
June 17, 2019 at 7:51 pmAny word on how big the framework and aar will be for an empty project?
JC Cimetiere
June 19, 2019 at 12:42 amYou can consider that the size of the final binary is the same than a normal Unity project.
h4ri
June 19, 2019 at 6:59 amSee bottom of the README.md
https://github.com/EgorBo/XamarinWithUnityApp
Stefan Maton
June 17, 2019 at 5:44 pmNow, this would also be great on PC. We currently prototype the integration of Unity in a WPF application and had to setup a client/server structure to handle communication between the app and Unity…
Steve Lanuit
June 17, 2019 at 6:29 pm@Stefan Maton @Unity
For some of our projects, we have implemented the same type of TCP communication protocol between a Unity application and a WPF / WinForm application. With the addition of the management of the inclusion of a Unity window in a WPF / WinForm Panel (SetParent, …). This works well but it would be comfortable to have native support for this type of integration.
Semicolon
June 17, 2019 at 8:25 pmIt would certainly be great to have an easier integration of Unity into desktop apps too, e.g. Qt programs. Currently, we have a setup that uses some shared memory and communicates over a lot of callbacks. But this is super cumbersome.
nxrighthere
June 18, 2019 at 7:45 amThis is something that I’m asking for since this May https://forum.unity.com/threads/partial-integration-with-the-engine-and-api-that-powers-dots-using-c-c.674905/
Nick Rapp
June 20, 2019 at 5:10 amThis is in fact already possible for PC, and has been for some time now, though admittedly it isn’t well documented. We are planning to include documentation for how to embed Unity as a library for desktop platforms soon. Stay tuned.