Search Unity

This post discusses state synchronization in networked Unity games. We begin with a summary of how this is done in the existing (legacy) Unity networking system, and move on to how it will function in the new UNET networking system and the decision making process that lead to that new design.

Background and Requirements

As a little bit of background. It is common for networked games to have a server that owns objects, and clients that need to be told when data in those objects change. For example in a combat game, the health of the players needs to visible to all players. This requires a member variable on a script class that is sent to all of the clients when it changes on the server. Below is a simple combat class:

When a player on the server takes damage, all of the players in the game need to be told about the new health value for that player.

This seems simple, but the challenge here is to make the system invisible to developers writing the code, efficient in terms of CPU, memory and bandwidth usage, and flexible enough to support all the types the developer wants to use. So some concrete goals for this system would be:

1.      Minimize memory usage by not keeping shadow copies of variables

2.      Minimize bandwidth usage by only sending states that have changed (incremental updates)

3.      Minimize CPU usage by not constantly checking to see if a state has changed

4.      Minimize protocol and serialization mismatch issues, by not requiring developers to hand-code serialization functions

5.      Don’t require developers to explicitly set variables as dirty

6.      Work with all supported Unity scripting languages

7.      Don’t disrupt developer workflow

8.      Don’t introduce manual steps that developers need to perform to use the system

9.      Allow the system to be driven by meta-data (custom attributes)

10.   Handle both simple and complex types

11.   Avoid reflection at runtime

This is an ambitious list of requirements!

Legacy Networking System

The existing Unity networking system has a “ReliableDeltaCompressed” type of synchronization that performs state synchronization by providing an OnSerializeNetworkView() hook function. This function is invoked on objects with the NetworkView component, and the serialization code written by the developer writes to (or reads from) the byte stream provided. The contents of this byte stream are then cached by the engine, and if the next time the function is called the result doesn’t match the cached version the object is considered dirty and the state is sent to clients. To take an example, a serialization function could look like this:

This approach meets some of the requirements listed above, but not all of them. It is automatic at runtime, since OnSerializeNetworkView() is invoked by the engine at the network send rate, and the developer doesn’t need to set variables as dirty. It also doesn’t add any extra steps to the build process or disrupt the developer workflow.

But, its performance is not great – especially when there are many networked objects. CPU time is spent on comparisons, and memory is used for caching copies of byte streams.  It is also susceptible to mismatch errors in the serialization function because it has to be updated by hand when new member variables are added that need to be synchronized. It is also not driven by metadata, so the editor and other tools cannot be aware of what variables are synchronized.

Code Generation for SyncVars

As the UNET team worked on the new state synchronization system, the solution we came up with was a code generator driven by custom attributes. In user code, this looks like:

This new custom attribute tells the system that the Health  and Alive member variables need to be synchronized. Now, the developer doesn’t need to write a serialization function, since the code generator has the custom attribute data and it can generate perfect serialization and unserialization functions with the right ordering and types. This generated function looks something like this:

Since this overrides a virtual function on the UNetBehaviour base class, when the game object is serialized, the script variables will also be automatically serialized. Then, they will be unpacked at the other end with a matching code-generated unserialization function. So there is no chance of mismatches, and the code updates automatically when a new [SyncVar] variable is added.

This data is now available to the editor, so the inspector window can show more detail like this:

But there are still some issues here. This function sends all the state all the time – it is not incremental; so if a single variable on an object changes, the entire object state would be sent. Also, how do we know when this serialization function should be called? It is not efficient to send states when nothing has changed.

We wrestled with using properties and dirty flags to do this. It seemed natural that a property could wrap each [SyncVar] variable and set dirty flags when something changes. This approach was partially successful. Having a bitmask of dirty flags lets the code generator make code to do incremental updates. That generated code would look something like this:

In this way, the unserialization function can read the dirty flags mask and only unserialize variables written to the stream. This makes for efficient bandwidth usage, and lets us know when the object is dirty. Plus, it’s still all automatic for the user. But how do these properties work?

Say we try wrapping the [SyncVar] member variable:

This does the job but it has the wrong name. The TakeDamage() function from above uses Health not HealthSync, so it bypasses the property.  The user can’t even use the HealthSync property directly since it doesn’t even exist until code generation happens.  It could be made into a two-phase process where the code generation step happens, then the user updates their code – but this is fragile. It is prone to compilation errors that can’t be fixed without undoing large chunks of code.

Another approach would be to require developers to write the above property code for each [SyncVar] variable.  But that is work for developers, and potentially error prone.  The bitmasks in user-written and generated code would have to match up exactly for this to work, so adding and removing [SyncVar] variables would be delicate.

Enter Mono Cecil

So we need to be able to generate wrapper properties and make existing code use them even if that code isn’t even aware of their existence. Well, fortunately there is a tool for Mono called Cecil which does exactly this. Cecil is able to load Mono assemblies in the ECMA CIL format, modify them and write them back out.

This is where is gets a little crazy. The UNET code generator creates the wrapper properties, then it finds all of the code sites where the original member variables were accessed. It then replaces the references to the member variables with references to the wrapper properties and Voila! Now the user code is calling through the newly created properties without any work from the user.

Since Cecil operates at the CIL level, it has the added advantage of working with all languages since they all compile down to the same instruction format.

The generated CIL for a final serialization function that gets injected into the script assembly now looks like this:

Luckily ILSpy can convert between CIL and C# in both directions, so in this case it allows us view the generated CIL code as C#. ILSpy is a great tool for working with Mono/.Net assemblies. The C# looks like:

So let’s see how this meets our requirements:

1.      No shadow copies of variables

2.      Incremental updates

3.      No comparison checks for state changes

4.      No hand-coded serialization functions

5.      No explicit dirty calls

6.      Works with all supported Unity scripting languages

7.      No workflow changes for the developer

8.      No manual steps for the developer to perform

9.      Driven by meta-data

10.   Handles all types (with new UWriter/UReader serializers)

11.   No reflection at runtime

Looks like we have them all covered.  This system will be efficient and friendly to developers. Hopefully it will help make developing multiplayer games with Unity easier for everyone.

We also use Cecil for RPC call implementations to avoid looking up functions by name with reflection. More on that in a later blog post.

49 Comments

Subscribe to comments

Comments are closed.

  1. This will only work when the calling site uses the property setter. It does not work when you e.g. do things like this:

    class Foo { [SyncVar]public List MyList; }

    foo.MyList.Add(23);

    This will not detect the changed list, am I right? (Same goes for changes within classes returned by a getter)

  2. Unknown Coder

    June 14, 2014 at 6:46 pm

    Will there be a method for the receiving client to check if a syncvar has changed and compare against the previous value? This is extremely useful feature that allows clients to run script code based off a variable change rather than receiving (typically) more expensive RPC’s.

  3. All this is only about collecting and sending data over the network. But game networking have more challenging and critical things to do, like dead reckoning, client side prediction. This require physics roll back on clients for rigidbodies.
    How UNET will handle this ?

  4. 31 fields in a class is not enough? Read the Clean code book by Robert C. Martin because you’re doing something horribly wrong.

  5. @MADMAN,
    I bet you’ve never made a solo player game.

  6. Its funny I bet most commenters here have never made a multplayer game.

  7. Robert Cummings

    June 3, 2014 at 1:41 am

    Excellent, I’m psyched for UNET. I really cannot wait to put this in all of our games (they’re all multiplayer).

  8. @GAVALAKIS,
    Same here, I actually like the high level API idea but it’s too much voodoo (to many “unknown” things happening in the background) and low level API it’s too much “low”. Something in-between (middle ground) would be awesome. I suggested more customized [SyncVar] a la UE4 macro decorators, which still pretty high level but you have more control on how and when things syncs/updates.

  9. Gavalakis Vaggelis

    June 2, 2014 at 3:05 pm

    I like voodoo when I also get the doll.

  10. @ALAN
    Is not about turning on/off Reliability but rather make [SyncVar] a bit more customizable with attribute properties such as “reliable”, “synctime”, “syncfrequency”, etc.
    Having just [SyncVar] it’s quite too much black magic inside. I know it’s a high level API but to me it’s quite too ultra high level without a bit of flexibility.

  11. I prefer rolling my own when it comes to networking systems. I just need something that can serialize a group of variables and send. Any other code, I will write myself.

  12. @CODEMONKY
    They already mentioned the ability to turn on/off reliable sending.

  13. @LARUS, OLAFSSON,
    Will there be more attribues (or attributes with properties or params) than just [SyncVar], for example:
    [SyncVar(“reliable”, 1f, )]//make it reliable, update each 1 sec, etc
    public float myHealth;

  14. Glad the networking is finally getting an upgrade.

  15. SeanRiley interesting post. I like stuff that don’t require writing boilerplate code by the developer if it’s not needed. HOWEVER, i also like dplitting the API to 2 versions – the automatic one and a manual one, for advanced users.

    Also, injecting stuff at the IL level is cool (and is used quire heavily by Unity as i see in your build process tools), however it has the downside of losing the ability to properly debug the code, since the actual compiled code and tge source now differ.

    All in all, interesting as to where this is going! Keep postung more :)

  16. For those that come from web this is a very common paradigm (http://en.wikipedia.org/wiki/Aspect-oriented_programming). But use it in unity3d is a little scary :P

  17. Lárus Ólafsson

    May 30, 2014 at 3:46 pm

    @Kryptos, @Terry, and others, currently we only support 32 syncvars, it could be extended but we do have other methods of synchronizing more complex/lengthy structures, more on that later.
    @Jes, we do have different levels of QoS, like unreliable/reliable/etc
    @Beejay, this is inside the Unity build pipeline, so not affected by your choice of IDE.
    @Sunny Davis, arrays will be supported but for more complex types we have other methods.
    @codemonkey, built-in method for special handling transform synchronization is on the roadmap yes (hopefully makes phase 1).

  18. Tristan Bellman-Greenwood

    May 30, 2014 at 10:47 am

    @YASEENELTII New networking will be part of the 5.x dev cycle. See the blog post called ‘Announcing UNET – New Unity Multiplayer Technology’:
    http://blogs.unity3d.com/2014/05/12/announcing-unet-new-unity-multiplayer-technology/

    This might be part of Phase I?

  19. Tristan Bellman-Greenwood

    May 30, 2014 at 10:40 am

    Cool but will we be able to sync strings without an RPC?

    Can’t wait for the RPC blog post!

  20. Got a question, will UNET support client interpolation and predictability? If so on version 1 (or futures version of UNET)?
    This is important to know, we are shopping a new engine with good networks capabilities right now but we can give UNET a try if it have such support.

  21. I have to plug my own networking solution, which I finally launched today. Which supports all of the features in this article, plus a ton more: http://www.boltengine.com/

  22. Patrick O'Day

    May 30, 2014 at 6:13 am

    Could we pass options into the syncvar attribute? Say I wanted to write my own generated property setter instead of using the one autogenerated by cecil? Could I write [syncvar(setter=customSetter)] or some variation? The variable value and the dirty bit mask would need to be passed in but I could think of some edge cases that would be incredibly useful.

    And of course exposing the cecil build hooks so people could make their own custom cecil build plugins would be nice as well.

  23. How about arrays and complex types?

  24. In my current workflow I like to keep my script code in an external library project so that I can use the newest version of Xamarin Studio (5.0 atm) to edit my source.

    Will there be a plugin available to make the magic work in the latest versions of the IDE, or will I need to stick with the IDE provided by Unity?

    Oh, and the magic does not bother me as long as someone has explained what the magic does. Keep up the good work, and I am looking forward to using the new networking API almost as much as I am the new GUI!

  25. @FHOLM: Once again, not the biggest caveat any engine has dealt with. You already have to deal with that with a bunch of Unity code, like assigning transform position or rigidbody velocity.
    It’s not exactly a hardship to treat your syncvars as if they are properties. Think of them as syntactic sugar.

  26. This would also break this implementation:

    [SyncVar]
    Vector3 pos;

    // this will explode if pos turns into a property
    pos.x += 1;

  27. @UnityUser: Given that there will also be a low-level API, why wouldn’t it be good?

  28. Can’t think that all this magic will be a good thing in the long term.

  29. Marc Schärer

    May 29, 2014 at 9:02 pm

    Very interesting Solution but i would like to +1 above question regarding the 31 fields.
    If that’s really the limit it would not work reasonably. This has been one of torque networks major issues already half a decade ago as it had exactly this limitation which is absolutely insufficient for action games or data heavy use cases with more complex data sets. It would force us devs to store it into containers that would use the delta sync benefit or a 2 tier sync solution of some kind.

    I would prefer a flexible length bitstream for the dirty flag handling that’s defined at compile time (I assume that this is already the case though as the way Cecil is used would allow this).

  30. @Madman:

    UNET is built with a low level and high level library. The low level library lets you send packets with a number of different quality of service modes. And you are in full control you can select which connection to send messages to. There is also a simple & fast serialization library that you can use from code. (the high level library & code generation is built on it)

    If you want to only use the low level library thats fine, it is fully exposed. And the high level library builds on top of it. The high level library is written in C# and very extendable.

    See here for an overview:

    http://blogs.unity3d.com/2014/05/12/announcing-unet-new-unity-multiplayer-technology/

  31. @Madman
    They mentioned in previous articles that they intend to give low level access to advanced developers. Being user friendly for non network developers doesn’t mean you won’t get a low level access for advanced scenarios.

  32. Madman: OK, more power to you. No solution can satisfy everyone.
    I’m just surprised that after revealing *one* feature you’ve decided it does not fit your needs. It’s hard to tell if you actually feel that way, or are just trolling.

  33. More black magic great this is not what we need! The current networking system is a burden for complex games. What we really need is a low level networking API as a foundation.

    Honestly after reading this it’s so far from my requirements that I will probably switch to Lidgren. It almost feels like it would be a waste of time explaining why.

    This just a another noob friendly solution.

  34. GREAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAT ;)

  35. @Alan Stagner I can’t read your comments without Morgan Freeman’s voice in my head. I’m excited to see how UNET pans out!

  36. This is cool, but I have to assume that this has to be limited to 32 member variables per class?

  37. @Kryptos on amount of syncvars: The 64-bit engine wouldn’t change anything of course, it’s just down to whether they use an int or a long for their bitmask.
    That said, I have a hard time believing one object is going to exceed 32 serialized fields without having some kind of code smell about it.

  38. @FHOLM IMHO this is actually a pretty smart implementation. If you think about it, avoiding refs for sync vars is not one of the biggest caveats in any game engine ;)
    Actually, I bet it would be possible for the UNET to detect situations where you pass a syncvar by reference, and generate a warning.

  39. @FHOLM “this would for-example break if you try to pass a field by reference to an method, and you guys then re-write it to a property = boom”

    1. IMHO using reference on class fields is bad practice.
    2. There are workarounds and maybe Cecil can generate them itself :

    // original code
    SomeMethode(ref health);

    // modified code
    var copy = HealthSync;
    SomeMethode(ref copy);
    HealthSync = copy;

  40. I can have tons of data in my scripts that do not need delivery guarantee.
    How UNET will work with it.

  41. SyncVars are used for script data, not transform synchronization – as people point out, the requirements for transform data and script data are often quite different. UNET has a built-in component for optimized, configurable synchronization of transform data that does not use SyncVars.

  42. @Emil: Thanks :) But can I disable delivery guarantee for my Position filed, if packed with new position is lost or was delivered after packed with more recent position I dont’ want you system resend/use it, system must just ignore it.

  43. I really wish this was not being hidden so much from the developer. I’ve had need for property change notifications/dirty tracking for a long time, and constantly have to roll my own suboptimal solutions. It would be great to hook into a common design rather than bolting my own solutions on top of this.

  44. ImaginaryHuman

    May 29, 2014 at 4:11 pm

    I haven’t done any network programming as such but this new solution does seem to make things simpler and easier. I presume though it is still necessary somewhere somehow to define the network topology or to create connections and find players and so on … I hope those parts will be very easy also.

  45. Emil "AngryAnt" Johansen

    May 29, 2014 at 4:09 pm

  46. This looks great, but what happens if I need more than 31 fields to be synchronized on a single object? Will the 64bit version of the engine allow up to 63 fields?
    Or can I serialize structs so that some fields might be grouped together and hence allow an pseudo-infinite number of fields (but still limited to 31 structs) ?

    Note that this question is only theoretical.

    Keep up the good work :)

  47. Honesty I don’t like this solution because fields and properties are not 100% identical in .NET, this would for-example break if you try to pass a field by reference to an method, and you guys then re-write it to a property = boom.

    There’s just too much magic in it for me.

  48. What protocol you system use? TCP or UDP?

    How you recognize two types of data:
    – important (with delivery guarantee)
    – not important (for example position of character don’t need delivery guarantee because next packed deliver more actual position )

  49. When Will The UNET System Be released ? :)