Categories & Tags
Archive

Unity Serialization

October 25, 2012 in Technology by

So you are writing a really cool editor extension in Unity and things seem to be going really well. You get your data structures all sorted out are really happy with how the tool you have written works.

Then you enter and exit play mode.

Suddenly all the data you had entered is gone and your tool is reset to the default, just initialized state. It’s very frustrating! “Why does this happen?” you ask yourself. The reason has to do with how the managed (mono) layer of Unity works. Once you understand it, then things get much easier :)

What happens when an assembly is reloaded?
When you enter / exit play mode or change a script Unity has to reload the mono assemblies, that is the dll’s associated with Unity.

On the user side this is a 3 step process:

  • Pull all the serializable data out of managed land, creating an internal representation of the data on the C++ side of Unity.
  • Destroy all memory / information associated with the managed side of Unity, and reload the assemblies.
  • Reserialize the data that was saved in C++ back into managed land.

What this means is that for your data structures / information to survive an assembly reload you need to ensure that it can get serialized into and out of c++ memory properly. Doing this also means that (with some minor modifications) you can save this data structure to an asset file and reload it at a later date.

How do I work with Unity’s serialization?
The easiest way to learn about Unity serialization is by working through an example. We are going to start with a simple editor window, it contains a reference to a class which we want to make survive an assembly reload.

using UnityEngine;
using UnityEditor;

public class MyWindow : EditorWindow
{
	private SerializeMe m_SerialziedThing;

	[MenuItem ("Window/Serialization")]
	static void Init () {
		GetWindow ();
	}

	void OnEnable ()
	{
		hideFlags = HideFlags.HideAndDontSave;
		if (m_SerialziedThing == null)
			m_SerialziedThing = new SerializeMe ();
	}

	void OnGUI () {
		GUILayout.Label ("Serialized Things", EditorStyles.boldLabel);
		m_SerialziedThing.OnGUI ();
	}
}
using UnityEditor;

public struct NestedStruct
{
	private float m_StructFloat;
	public void OnGUI ()
	{
		m_StructFloat = EditorGUILayout.FloatField("Struct Float", m_StructFloat);
	}
}

public class SerializeMe
{
	private string m_Name;
	private int m_Value;

	private NestedStruct m_Struct;

	public SerializeMe ()
	{
		m_Struct = new NestedStruct();
		m_Name = "";
	}

	public void OnGUI ()
	{
		m_Name = EditorGUILayout.TextField( "Name", m_Name);
		m_Value = EditorGUILayout.IntSlider ("Value", m_Value, 0, 10);

		m_Struct.OnGUI ();
	}
}

When you run this and force an assembly reload you will notice that any value in the window you have changed will not survive. This is because when the assembly is reloaded the reference to the ‘m_SerialziedThing’ is gone. It is not marked up to be serialized.

There are a few things that need to be done to make this serialization work properly:
In MyWindow.cs:

  • The field ‘m_SerializedThing’ needs to have the attribute [SerializeField] added to it. What this tells Unity is that it should attempt to serialize this field on assembly reload or similar events.

In SerializeMe.cs:

  • The class ‘SerializeMe’ needs to have the [Serializable] attribute added to it. This tells Unity that the class is serializable.
  • The struct ‘NestedStruct’ needs to have the [Serializable] attribute added to it.
  • Each (non public) field that you want to be serialized needs to have the [SerializeField] attribute added to it.

After adding these flags open the window and modify the fields. You will notice that after an assembly reload that the fields retain their values; that is apart from the field that came from the struct. This brings up the first important point, structs are not very well supported for serialization. Changing ‘NestedStruct’ from a struct to a class fixes this issue.

The code now looks like this:

using UnityEngine;
using UnityEditor;

public class MyWindow : EditorWindow
{
	private SerializeMe m_SerialziedThing;

	[MenuItem ("Window/Serialization")]
	static void Init () {
		GetWindow ();
	}

	void OnEnable ()
	{
		hideFlags = HideFlags.HideAndDontSave;
		if (m_SerialziedThing == null)
			m_SerialziedThing = new SerializeMe ();
	}

	void OnGUI () {
		GUILayout.Label ("Serialized Things", EditorStyles.boldLabel);
		m_SerialziedThing.OnGUI ();
	}
}

using System;
using UnityEditor;
using UnityEngine;

[Serializable]
public class NestedStruct
{
	[SerializeField]
	private float m_StructFloat;
	public void OnGUI ()
	{
		m_StructFloat = EditorGUILayout.FloatField("Struct Float", m_StructFloat);
	}
}

[Serializable]
public class SerializeMe
{
	[SerializeField]
	private string m_Name;
	[SerializeField]
	private int m_Value;
	[SerializeField]
	private NestedStruct m_Struct;

	public SerializeMe ()
	{
		m_Struct = new NestedStruct();
		m_Name = "";
	}

	public void OnGUI ()
	{
		m_Name = EditorGUILayout.TextField( "Name", m_Name);
		m_Value = EditorGUILayout.IntSlider ("Value", m_Value, 0, 10);

		m_Struct.OnGUI ();
	}
}

Some Serialization Rules

  • Avoid structs
  • Classes you want to be serializable need to be marked with [Serializable]
  • Public fields are serialized (so long as they reference a [Serializable] class)
  • Private fields are serialized under some circumstances (editor).
  • Mark private fields as [SerializeField] if you wish them to be serialized.
  • [NonSerialized] exists for fields that you do not want to serialize.

Scriptable Objects
So far we have looked at using normal classes when it comes to serialization. Unfortunately using plain classes has some issues when it comes to serialization in Unity. Lets take a look at an example.

using System;
using UnityEditor;
using UnityEngine;

[Serializable]
public class NestedClass
{
	[SerializeField]
	private float m_StructFloat;
	public void OnGUI()
	{
		m_StructFloat = EditorGUILayout.FloatField("Float", m_StructFloat);
	}
}

[Serializable]
public class SerializeMe
{
	[SerializeField]
	private NestedClass m_Class1;

	[SerializeField]
	private NestedClass m_Class2;

	public void OnGUI ()
	{
		if (m_Class1 == null)
			m_Class1 = new NestedClass ();
		if (m_Class2 == null)
			m_Class2 = m_Class1;

		m_Class1.OnGUI();
		m_Class2.OnGUI();
	}
}

This is a contrived example to show a very specific corner case of the Unity serialization system that can catch you if you are not careful. You will notice that we have two fields of type NestedClass. The first time the window is drawn it will show both the fields, and as m_Class1 and m_Class2 point to the same reference, modifying one will modify the other.

Now try reloading the assembly by entering and exiting play mode… The references have been decoupled. This is due to how serialization works when you mark a class as simply [Serializable]

When you are serializing standard classes Unity walks through the fields of the class and serializes each one individually, even if the reference is shared between multiple fields. This means that you could have the same object serialized multiple times, and on deserialization the system will not know they are really the same object. If you are designing a complex system this is a frustrating limitation because it means that complex interactions between classes can not be captured properly.

Enter ScriptableObjects! ScriptableObjects are a type of class that correctly serializes as references, so that they only get serialized once. This allows complex class interactions to be stored in a way that you would expect. Internally in Unity ScriptableObjects and MonoBehaviours are the same; in userland code you can have a ScriptableObject that is not attached to a GameObject; this is different to how MonoBehaviour works. They are great for general data structure serialization.

Let’s modify the example to be able to handle serialization properly:

using System;
using UnityEditor;
using UnityEngine;

[Serializable]
public class NestedClass : ScriptableObject
{
	[SerializeField]
	private float m_StructFloat;

	public void OnEnable() { hideFlags = HideFlags.HideAndDontSave; }

	public void OnGUI()
	{
		m_StructFloat = EditorGUILayout.FloatField("Float", m_StructFloat);
	}
}

[Serializable]
public class SerializeMe
{
	[SerializeField]
	private NestedClass m_Class1;

	[SerializeField]
	private NestedClass m_Class2;

	public SerializeMe ()
	{
		m_Class1 = ScriptableObject.CreateInstance ();
		m_Class2 = m_Class1;
	}

	public void OnGUI ()
	{
		m_Class1.OnGUI();
		m_Class2.OnGUI();
	}
}

The three changes of note here are that:

  • NestedClass is now a ScriptableObject.
  • We create an instance using the CreateInstance<> function instead of calling the constructor.
  • We also set the hide flags… this will be explained later

These simple changes mean that the instance of the NestedClass will only be serialized once, with each of the references to the class pointing to the same one.

ScriptableObject Initialization
So now we know that for complex data structures where external referencing is needed it is a good idea to use ScriptableObjects. But what is the correct way to work with ScriptableObjects from user code? The first thing to examine is HOW scriptable objects are initialized, especially from the Unity serialization system.

  1. The constructor is called on the ScriptableObject.
  2. Data is serialized into the object from the c++ side of unity (if such data exists).
  3. OnEnable() is called on the ScriptableObject.

Working with this knowledge there are some things that we can say:

  • Doing initialization in the constructor isn’t a very good idea as data will potentially be overridden by the serialization system.
  • Serialization happens AFTER construction, so we should do our configuration stuff after serialization.
  • OnEnable() seems like the best candidate for initialization.

Lets make some changes to the ‘SerializeMe’ class so that it is a ScriptableObject. This will allow us to see the correct initialization pattern for ScriptableObjects.

// also updated the Window to call CreateInstance instead of the constructor
using System;
using UnityEngine;

[Serializable]
public class SerializeMe : ScriptableObject
{
	[SerializeField]
	private NestedClass m_Class1;

	[SerializeField]
	private NestedClass m_Class2;

	public void OnEnable ()
	{
		hideFlags = HideFlags.HideAndDontSave;
		if (m_Class1 == null)
		{
			m_Class1 = CreateInstance ();
			m_Class2 = m_Class1;
		}
	}

	public void OnGUI ()
	{
		m_Class1.OnGUI();
		m_Class2.OnGUI();
	}
}

On the surface it seems that we have not really changed this class much, it now inherits from ScriptableObject and instead of using a constructor has an OnEnable(). The important part to take note of is slightly more subtle… OnEnable() is called AFTER serialization; because of this we can see if the [SerializedFields] are null or not. If they are null it indicates that this is the first initialization, and we need to construct the instances. If they are not null then they have been loaded into memory, and do NOT need to be constructed. It is common in OnEnable() to also call a custom Initialization function to configure any private / non serialized fields on the object, much like you would do in a constructor.

HideFlags
In the examples using ScriptableObjects you will notice that we are setting the ‘hideFlags’ on the object to HideFlags.HideAndDontSave. This is a special setup that is required when writing custom data structures that have no root in the scene. This is to get around how scene loading works in Unity.

When a scene is loaded internally unity calls Resources.UnloadUnusedAssets. If nothing is referencing an asset the garbage collector will find it. The GC uses the scene as ‘the root’ and traverses the hierarchy to see what can get GC’d. Setting the HideAndDontSave flag on a ScriptableObject tells Unity to consider that object as a root object. Because of this it will not just disappear because of an assembly reload. The object can still be destroyed by calling Destroy().

Some ScriptableObject Rules

  • ScriptableObjects will only be serialized once, allowing you to use references properly.
  • Use OnEnable to initialize ScriptableObjects.
  • Don’t ever call the constructor of a ScriptableObject, use CreatInstance instead
  • For nested data structures that are only referenced once don’t use ScriptableObject as they have more overhead.
  • If your scriptable object is not rooted in the scene set the hideFlags to HideAndDontSave.

Concrete Array Serialization
Lets have a look at a simple example that serializes a range of concrete classes.

using System;
using System.Collections.Generic;
using UnityEditor;
using UnityEngine;

[Serializable]
public class BaseClass
{
	[SerializeField]
	private int m_IntField;
	public void OnGUI() {m_IntField = EditorGUILayout.IntSlider ("IntField", m_IntField, 0, 10);}
}

[Serializable]
public class SerializeMe : ScriptableObject
{
	[SerializeField]
	private List m_Instances;

	public void OnEnable ()
	{
		hideFlags = HideFlags.HideAndDontSave;
		if (m_Instances == null)
			m_Instances = new List ();
	}

	public void OnGUI ()
	{
		foreach (var instance in m_Instances)
			instance.OnGUI ();

		if (GUILayout.Button ("Add Simple"))
			m_Instances.Add (new BaseClass ());
	}
}

This basic example has a list of BaseClasses, by clicking the ‘Add Simple’ button it creates an instance and adds it to the list. Due to the SerializeMe class being configured properly for serialization (as discussed before) it ‘just works’. Unity sees that the List is marked for serialization and serializes each of the List elements.

General Array Serialization
Lets modify the example to serialize a list that contains members of a base class and child class:

using System;
using System.Collections.Generic;
using UnityEditor;
using UnityEngine;

[Serializable]
public class BaseClass
{
	[SerializeField]
	private int m_IntField;
	public virtual void OnGUI() { m_IntField = EditorGUILayout.IntSlider("IntField", m_IntField, 0, 10); }
}

[Serializable]
public class ChildClass : BaseClass
{
	[SerializeField]
	private float m_FloatField;
	public override void OnGUI()
	{
		base.OnGUI ();
		m_FloatField = EditorGUILayout.Slider("FloatField", m_FloatField, 0f, 10f);
	}
}

[Serializable]
public class SerializeMe : ScriptableObject
{
	[SerializeField]
	private List m_Instances;

	public void OnEnable ()
	{
		if (m_Instances == null)
			m_Instances = new List ();

		hideFlags = HideFlags.HideAndDontSave;
	}

	public void OnGUI ()
	{
		foreach (var instance in m_Instances)
			instance.OnGUI ();

		if (GUILayout.Button ("Add Base"))
			m_Instances.Add (new BaseClass ());
		if (GUILayout.Button ("Add Child"))
			m_Instances.Add (new ChildClass ());
	}
}

The example has been extended so that there is now a ChildClass, but we are serializing using the BaseClass. If you create a few instance of the ChildClass and the BaseClass they will render properly. Issues arise when they are placed through an assembly reload. After the reload completes every instance will be a BaseClass, with all the ChildClass information stripped. The instances are being sheared by the serialization system.

The way to work around this limitation of the serialization system is to once again use ScriptableObjects:

using System;
using System.Collections.Generic;
using UnityEngine;
using UnityEditor;

[Serializable]
public class MyBaseClass : ScriptableObject
{
	[SerializeField]
	protected int m_IntField;

	public void OnEnable() { hideFlags = HideFlags.HideAndDontSave; }

	public virtual void OnGUI ()
	{
		m_IntField = EditorGUILayout.IntSlider("IntField", m_IntField, 0, 10);
	}
}

[Serializable]
public class ChildClass : MyBaseClass
{
	[SerializeField]
	private float m_FloatField;

	public override void OnGUI()
	{
		base.OnGUI ();
		m_FloatField = EditorGUILayout.Slider("FloatField", m_FloatField, 0f, 10f);
	}
}

[Serializable]
public class SerializeMe : ScriptableObject
{
	[SerializeField]
	private List m_Instances;

	public void OnEnable ()
	{
		if (m_Instances == null)
			m_Instances = new List();

		hideFlags = HideFlags.HideAndDontSave;
	}

	public void OnGUI ()
	{
		foreach (var instance in m_Instances)
			instance.OnGUI ();

		if (GUILayout.Button ("Add Base"))
			m_Instances.Add(CreateInstance());
		if (GUILayout.Button ("Add Child"))
			m_Instances.Add(CreateInstance());
	}
}

After running this, changing some values, and reloading assemblies you will notice that ScriptableObjects are safe to use in arrays even if you are serializing derived types. The reason is that when you serialize a standard [Serializable] class it is serialized ‘in place’, but a ScriptableObject is serialized externally and the reference inserted into the collection. The shearing occurs because the type can not be properly be serialized as the serialization system thinks it is of the base type.

Serializing Abstract Classes
So now we have seen that it’s possible to serialize a general list (so long as the members are of type ScriptableObject). Lets see how abstract classes behave:

using System;
using UnityEditor;
using System.Collections.Generic;
using UnityEngine;

[Serializable]
public abstract class MyBaseClass : ScriptableObject
{
	[SerializeField]
	protected int m_IntField;

	public void OnEnable() { hideFlags = HideFlags.HideAndDontSave; }

	public abstract void OnGUI ();
}

[Serializable]
public class ChildClass : MyBaseClass
{
	[SerializeField]
	private float m_FloatField;

	public override void OnGUI()
	{
		m_IntField = EditorGUILayout.IntSlider("IntField", m_IntField, 0, 10);
		m_FloatField = EditorGUILayout.Slider("FloatField", m_FloatField, 0f, 10f);
	}
}

[Serializable]
public class SerializeMe : ScriptableObject
{
	[SerializeField]
	private List m_Instances;

	public void OnEnable ()
	{
		if (m_Instances == null)
			m_Instances = new List();

		hideFlags = HideFlags.HideAndDontSave;
	}

	public void OnGUI ()
	{
		foreach (var instance in m_Instances)
			instance.OnGUI ();

		if (GUILayout.Button ("Add Child"))
			m_Instances.Add(CreateInstance());
	}
}

This code much like the previous example works. But it IS dangerous. Lets see why.

The function CreateInstance<>() expects a type that inherits from ScriptableObject, the class ‘MyBaseClass’ does in fact inherit from ScriptableObject. This means that it’s possible to add an instance of the abstract class MyBaseClass to the m_Instances array. If you do this and then try and access an abstract method bad things will happen because there is no implementation of that function. In this specific case that would be the OnGUI method.

Using abstract classes as the serialized type for lists and fields DOES work, so long as they inherit from ScriptableObject, but it is not a recommended practice. Personally I think it’s better to use concrete classes with empty virtual methods. This ensures that things will not go bad for you.

When do ScriptableObjects get persisted into scene / prefab files?
GameObjects and their components are saved into a scene by default. Asset types (Materials / Meshes / AnimationClip / SerializedObject’s) that are created from code are saved in the scene so long as a game object or their components in the scene references it.

Asset types can also be explicitly marked as assets using AssetDatabase.CreateAsset. In that case they will not be saved in the scene but simply referenced. If an asset type or game object type is marked as HideAndDontSave it also not saved in the scene.

Questions?

Share this post

Comments (20)

Comments are closed.

25 Oct 2012, 1:48 am

Well this is very helpful, I will try to use this technique in our extension but one time as I remember we tried this before with no luck but it seems we might have been doing something wrong so we should try again I guess :)

25 Oct 2012, 5:35 am

Great post. I think this should be added to unity manual. You might want some more refinements but it’s a good read in general.
Can you please describe about SerializedProptery and SerializedObject as well. What is their reliation to these.

25 Oct 2012, 5:53 pm

Thanks nice read. But I have a question in regards to serialization. Having a sequene of nested classes, it seems so that unity stops serialization after the 8th nested class in sequence. Is that a bug, an issue, a limitation… or something else?
Thanks in advance, and great read overall.

25 Oct 2012, 9:56 pm

Thanks for taking the time to explain serialization.
I would appreciate a lot more blog posts like this one.

Gennadiy Korol
27 Oct 2012, 8:33 am

Very good read. We have been conducting similar tests at some point before trying to understand it. This really lays it out in a very plain and understandable way. Thanks!

27 Oct 2012, 10:52 pm

Great stuff, Tim! This topic deserved to be covered in-depth.

ps I guess there will be additional questions, so consider a forum topic linked to this post?

28 Oct 2012, 3:06 pm

I have a few questions, would be great if you could explain it here:

1. How does this relate to keeping values of edited components using custom inspectors (inheriting not EditorWindow but Editor). Could you show the simplest example of keeping the string value living after the assembly reload.
2. How does it work with serializedObject and SerializedProperty?
3. How does it relate to “old recepy” of serializing values “by hand”, and reapplying them just after the play mode is stopped?

Thank you.

DynamicHead
7 Nov 2012, 5:16 am

Array Serialization.. –I love it. Thanks!

Andrej
28 Nov 2012, 7:41 pm

Am I crazy, but the first example didn’t work for me at all. I’m using Unity v.3.5.4f1. I created an empty Unity project and copied and pasted the “fixed/serializable” version of the first example. I had an error and only had to change the GetWindow() called to be GetWindow() to fix it.

But basically in Unity I open the EditorWindow, change values, click on the X to close it, then open it again and the values are reset. I didn’t even press Play. Am I missing something?

Thanks for your help.

XYZ
29 Nov 2012, 12:43 am

Andrej i am also getting same error GetWindow()……the first set of code
changed getwindow() to what?

Andrej Gombar
29 Nov 2012, 11:32 am

@XYZ darn, it looks like my tags were stripped. Maybe this will work. GetWindow() should be GetWindow<MyWindow>

If that doesn’t show, I’ll spell it out. GetWindow lessthansign MyWindow greaterthansign

Or this should work: GetWindow(typeof(MyWindow))

XYZ
29 Nov 2012, 8:40 pm

hi XYZ
thankyou so much for the reply

XYZ
29 Nov 2012, 8:41 pm

hi Andrej XYZ here tat is wat i meant :)

XYZ
29 Nov 2012, 9:01 pm

Andrej another question when i execute the first code and the serialized code the data in the window should be retained in later case but nothing as such is happening here :(

XYZ
29 Nov 2012, 9:04 pm

how do i test it plz explain step by step thank u so much

30 Nov 2012, 12:57 am

It appears polymorphic types cannot be serialized within a MonoBehaviour. Using ScriptableObject, as described in this post, will not survive the transition over to being in a prefab. If you’re reading this post, you’re probably running into serialization issues. Please vote for this issue here: http://feedback.unity3d.com/unity/all-categories/1/hot/active/serialization-of-polymorphic-dat

15 Dec 2012, 11:49 am

i really want to learn c++ but it looks too confusing

12 Jan 2013, 2:28 pm

I’ve tried this with both Unity 3.5.5 and Unity 4.0.0 and no luck in either case.

I’ve done everything stated here and no properties are being serialized.

Steps:
1) Open the Editor Window
2) Modify properites
3) Close window
3) Open window again, all properties are default values.

I’ve even tried making all of the properties public but that didn’t make any difference.

I am thoroughly confused why this isn’t working!? Is there a bug in Unity that doesn’t allow serialization of Editor classes?

12 Jan 2013, 2:31 pm

I’ve tried in both JavaScript and C#. in both cases none of the properties (or properties of the referenced classes) are being saved after the window is destroyed.

12 Jan 2013, 2:52 pm

The only solution I found was to use EditorPrefs to manually save and load any settings.
Is there anyway to save these class properties in an asset file in the ProjectSettings folder?

Leave a Reply

Comments are closed.