Search Unity

ML-Agents Toolkit v0.6: Improved usability of Brains and Imitation Learning

, December 17, 2018

We’ve added a two new enhancements in the Unity ML-Agents Toolkit v0.6 based on your feedback! The first enhancement improves the usability of Brains by turning them from MonoBehaviors to ScriptableObjects. The second enhancement provides a better user workflow for Imitation Learning by enabling you to record expert demonstrations and use them for offline training.

Scriptable Object Brains

In earlier versions of the ML-Agents Toolkit, Brains were GameObjects that were attached as children to the Academy GameObject. Consequently, it was difficult to re-use Brains across Unity scenes within the same project. To alleviate this, in this latest v0.6 release, we turned Brains into Scriptable Objects, so they can be managed as standard Unity assets. This makes it easy to use them across scenes and to create Agents’ Prefabs with Brains pre-attached. To complete this transition, we introduced a new Learning Brain Scriptable Object that replaces the previous Internal and External Brains. For completeness, we also introduced Player and Heuristic Brain Scriptable Objects to replace the Player and Heuristic Brain Types, respectively.

Here is a before and after on implementing Brains in the 3DBall environment.

On the left, the old ML-Agents v0.5 Brain GameObject. On the right, new ScriptableObject Brain in ML-Agents v0.6

In version 0.6, you can no longer change the type of Brain with the Brain Type drop down. You will need to create a different Brain for Player and Learning. However, we made it easy to copy Brain Parameters from one Brain to another with the “Copy Brain Parameters from” selection box in the Brain Inspector. You can drag and drop any other Brain into this field and the corresponding Brain Parameters will be copied over to the new Brain.

Additionally, since Brains are no longer part of the Scene, you can create them from the Assets menu.

Creating Brains from the Asset Menu

Lastly, we made it easy to keep track of which Brains are being trained. This can be accomplished with the BroadcastHub in the Academy Component. You can now drag Brain assets into this component in order to expose the Brains to the external training process.

Control Checkbox

Checking the Control checkbox for a Brain is equivalent using the External Brain Type in earlier versions of the ML-Agents Toolkit. Essentially, the Brains with Control checked are the Brains that get trained. If you do not want to run training, simply uncheck the Control checkboxes on all the Brains in the BroadcastHub.

Please see an intro video below on how to create a Scriptable Object Brain.

For more information, check out the documentation on Github.

Demonstration Recorder

When playing your game, you can now record the actions and observations of an Agent and either use them to train Agents at a later time via Imitation Learning or analyze the data. This allows you to reuse training data for multiple training sessions, rather than capturing it every time. When you want to record an Agent, add the Demonstration Recorder component to your Agent, check Record and give the demonstration a name.

Demonstration Recorder Component

Enter Play Mode and perform the actions you would like to see the agent perform. When you are done recording, quit Play Mode and a Demonstration asset will be created in the Demonstrations folder.

Demonstration Asset

To train an Agent with this recording, modify the Hyperparameters in the training configuration.

For more information, check out documentation on this on Github.

Next steps

Your opinion matters a great deal to us. Only by hearing your thoughts and ideas about the Unity ML-Agents Toolkit can we continue to improve and grow. Please take a few minutes to let us know about it by filling out our survey.

For any feedback or questions regarding the ML-Agents Toolkit, feel free to reach out to us on our ML-Agents GitHub issues page, or email us directly at

Interested in joining Unity Machine Learning, check our career opportunities here.

4 replies on “ML-Agents Toolkit v0.6: Improved usability of Brains and Imitation Learning”

For the “Control” textbox, what’s behind that naming choice? It’d be great to know to help me remember the purpose it serves.

For example, does it stand for something like “External Control”?

I do get it that it’s function is to flag the brain for external training.

Thanks for any details.

Great, thank you! Please implement RND ( ) from OpenAI for ML Agents next! It’s an extension or improvement on PPO which uses two extra networks for curiousity, but it makes such a HUGE positive difference, especially in sparse reward scenarios which are quite common in unity! I tried it on a number of gym environments (atari, nes) and it works really well, much better than PPO, PPO2 or AC3 ever could! Implementing that for unity would open so much possibilities!

ML-Agents is my favourite thing about Unity Engine. it is simple and elegant. I wish we could train our agents without leaving Unity environment but still, I love it. Thanks a lot for this awesome project.

Comments are closed.