ML-Agents Toolkit v0.6: Improved usability of Brains and Imitation Learning
We’ve added a two new enhancements in the Unity ML-Agents Toolkit v0.6 based on your feedback! The first enhancement improves the usability of Brains by turning them from MonoBehaviors to ScriptableObjects. The second enhancement provides a better user workflow for Imitation Learning by enabling you to record expert demonstrations and use them for offline training.
Scriptable Object Brains
In earlier versions of the ML-Agents Toolkit, Brains were GameObjects that were attached as children to the Academy GameObject. Consequently, it was difficult to re-use Brains across Unity scenes within the same project. To alleviate this, in this latest v0.6 release, we turned Brains into Scriptable Objects, so they can be managed as standard Unity assets. This makes it easy to use them across scenes and to create Agents’ Prefabs with Brains pre-attached. To complete this transition, we introduced a new Learning Brain Scriptable Object that replaces the previous Internal and External Brains. For completeness, we also introduced Player and Heuristic Brain Scriptable Objects to replace the Player and Heuristic Brain Types, respectively.
Here is a before and after on implementing Brains in the 3DBall environment.
In version 0.6, you can no longer change the type of Brain with the Brain Type drop down. You will need to create a different Brain for Player and Learning. However, we made it easy to copy Brain Parameters from one Brain to another with the “Copy Brain Parameters from” selection box in the Brain Inspector. You can drag and drop any other Brain into this field and the corresponding Brain Parameters will be copied over to the new Brain.
Additionally, since Brains are no longer part of the Scene, you can create them from the Assets menu.
Lastly, we made it easy to keep track of which Brains are being trained. This can be accomplished with the BroadcastHub in the Academy Component. You can now drag Brain assets into this component in order to expose the Brains to the external training process.
Checking the Control checkbox for a Brain is equivalent using the External Brain Type in earlier versions of the ML-Agents Toolkit. Essentially, the Brains with Control checked are the Brains that get trained. If you do not want to run training, simply uncheck the Control checkboxes on all the Brains in the BroadcastHub.
Please see an intro video below on how to create a Scriptable Object Brain.
For more information, check out the documentation on Github.
When playing your game, you can now record the actions and observations of an Agent and either use them to train Agents at a later time via Imitation Learning or analyze the data. This allows you to reuse training data for multiple training sessions, rather than capturing it every time. When you want to record an Agent, add the Demonstration Recorder component to your Agent, check Record and give the demonstration a name.
Enter Play Mode and perform the actions you would like to see the agent perform. When you are done recording, quit Play Mode and a Demonstration asset will be created in the Demonstrations folder.
To train an Agent with this recording, modify the Hyperparameters in the training configuration.
For more information, check out documentation on this on Github.
Your opinion matters a great deal to us. Only by hearing your thoughts and ideas about the Unity ML-Agents Toolkit can we continue to improve and grow. Please take a few minutes to let us know about it by filling out our survey.
Interested in joining Unity Machine Learning, check our career opportunities here.