Search Unity

ML-Agents Toolkit v0.6: Improved usability of Brains and Imitation Learning

December 17, 2018 in Engine & platform | 3 min. read
Share

Is this article helpful for you?

Thank you for your feedback!

We’ve added a two new enhancements in the Unity ML-Agents Toolkit v0.6 based on your feedback! The first enhancement improves the usability of Brains by turning them from MonoBehaviors to ScriptableObjects. The second enhancement provides a better user workflow for Imitation Learning by enabling you to record expert demonstrations and use them for offline training.

Scriptable Object Brains

In earlier versions of the ML-Agents Toolkit, Brains were GameObjects that were attached as children to the Academy GameObject. Consequently, it was difficult to re-use Brains across Unity scenes within the same project. To alleviate this, in this latest v0.6 release, we turned Brains into Scriptable Objects, so they can be managed as standard Unity assets. This makes it easy to use them across scenes and to create Agents’ Prefabs with Brains pre-attached. To complete this transition, we introduced a new Learning Brain Scriptable Object that replaces the previous Internal and External Brains. For completeness, we also introduced Player and Heuristic Brain Scriptable Objects to replace the Player and Heuristic Brain Types, respectively.

Here is a before and after on implementing Brains in the 3DBall environment.

On the left, the old ML-Agents v0.5 Brain GameObject. On the right, new ScriptableObject Brain in ML-Agents v0.6

In version 0.6, you can no longer change the type of Brain with the Brain Type drop down. You will need to create a different Brain for Player and Learning. However, we made it easy to copy Brain Parameters from one Brain to another with the “Copy Brain Parameters from” selection box in the Brain Inspector. You can drag and drop any other Brain into this field and the corresponding Brain Parameters will be copied over to the new Brain.

Additionally, since Brains are no longer part of the Scene, you can create them from the Assets menu.

Creating Brains from the Asset Menu

Lastly, we made it easy to keep track of which Brains are being trained. This can be accomplished with the BroadcastHub in the Academy Component. You can now drag Brain assets into this component in order to expose the Brains to the external training process.

Control Checkbox

Checking the Control checkbox for a Brain is equivalent using the External Brain Type in earlier versions of the ML-Agents Toolkit. Essentially, the Brains with Control checked are the Brains that get trained. If you do not want to run training, simply uncheck the Control checkboxes on all the Brains in the BroadcastHub.

This content is hosted by a third party provider that does not allow video views without acceptance of Targeting Cookies. Please set your cookie preferences for Targeting Cookies to yes if you wish to view videos from these providers.

Demonstration Recorder

When playing your game, you can now record the actions and observations of an Agent and either use them to train Agents at a later time via Imitation Learning or analyze the data. This allows you to reuse training data for multiple training sessions, rather than capturing it every time. When you want to record an Agent, add the Demonstration Recorder component to your Agent, check Record and give the demonstration a name.

Demonstration Recorder Component

Enter Play Mode and perform the actions you would like to see the agent perform. When you are done recording, quit Play Mode and a Demonstration asset will be created in the Demonstrations folder.

Demonstration Asset

To train an Agent with this recording, modify the Hyperparameters in the training configuration.

For more information, check out documentation on this on Github.

Next steps

Your opinion matters a great deal to us. Only by hearing your thoughts and ideas about the Unity ML-Agents Toolkit can we continue to improve and grow. Please take a few minutes to let us know about it by filling out our survey.

For any feedback or questions regarding the ML-Agents Toolkit, feel free to reach out to us on our ML-Agents GitHub issues page, or email us directly at ml-agents@unity3d.com.

Interested in joining Unity Machine Learning, check our career opportunities here.

December 17, 2018 in Engine & platform | 3 min. read

Is this article helpful for you?

Thank you for your feedback!