This post discusses state synchronization in networked Unity games. We begin with a summary of how this is done in the existing (legacy) Unity networking system, and move on to how it will function in the new UNET networking system and the decision making process that lead to that new design.
Background and Requirements
As a little bit of background. It is common for networked games to have a server that owns objects, and clients that need to be told when data in those objects change. For example in a combat game, the health of the players needs to visible to all players. This requires a member variable on a script class that is sent to all of the clients when it changes on the server. Below is a simple combat class:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
class Combat : MonoBehaviour { public int Health; public bool Alive; public void TakeDamage(int amount) { if (amount >= Health) { Alive = false; Health = 0; } else { Health -= amount; } } } |
When a player on the server takes damage, all of the players in the game need to be told about the new health value for that player.
This seems simple, but the challenge here is to make the system invisible to developers writing the code, efficient in terms of CPU, memory and bandwidth usage, and flexible enough to support all the types the developer wants to use. So some concrete goals for this system would be:
1. Minimize memory usage by not keeping shadow copies of variables
2. Minimize bandwidth usage by only sending states that have changed (incremental updates)
3. Minimize CPU usage by not constantly checking to see if a state has changed
4. Minimize protocol and serialization mismatch issues, by not requiring developers to hand-code serialization functions
5. Don’t require developers to explicitly set variables as dirty
6. Work with all supported Unity scripting languages
7. Don’t disrupt developer workflow
8. Don’t introduce manual steps that developers need to perform to use the system
9. Allow the system to be driven by meta-data (custom attributes)
10. Handle both simple and complex types
11. Avoid reflection at runtime
This is an ambitious list of requirements!
Legacy Networking System
The existing Unity networking system has a “ReliableDeltaCompressed” type of synchronization that performs state synchronization by providing an OnSerializeNetworkView() hook function. This function is invoked on objects with the NetworkView component, and the serialization code written by the developer writes to (or reads from) the byte stream provided. The contents of this byte stream are then cached by the engine, and if the next time the function is called the result doesn’t match the cached version the object is considered dirty and the state is sent to clients. To take an example, a serialization function could look like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
void OnSerializeNetworkView (Bitstream stream, NetworkMessageInfo info) { float horizontalInput = 0.0f; if (stream.isWriting) { // Sending horizontalInput = Input.GetAxis ("Horizontal"); stream.Serialize (horizontalInput); } else { // Receiving stream.Serialize (horizontalInput); // ... do something meaningful with the received variable } } |
This approach meets some of the requirements listed above, but not all of them. It is automatic at runtime, since OnSerializeNetworkView() is invoked by the engine at the network send rate, and the developer doesn’t need to set variables as dirty. It also doesn’t add any extra steps to the build process or disrupt the developer workflow.
But, its performance is not great – especially when there are many networked objects. CPU time is spent on comparisons, and memory is used for caching copies of byte streams. It is also susceptible to mismatch errors in the serialization function because it has to be updated by hand when new member variables are added that need to be synchronized. It is also not driven by metadata, so the editor and other tools cannot be aware of what variables are synchronized.
Code Generation for SyncVars
As the UNET team worked on the new state synchronization system, the solution we came up with was a code generator driven by custom attributes. In user code, this looks like:
1 2 3 4 5 6 7 8 9 |
using UnityEngine.UNetwork; class Combat : UNetBehaviour { [SyncVar] public int Health; [SyncVar] public bool Alive; } |
This new custom attribute tells the system that the Health and Alive member variables need to be synchronized. Now, the developer doesn’t need to write a serialization function, since the code generator has the custom attribute data and it can generate perfect serialization and unserialization functions with the right ordering and types. This generated function looks something like this:
1 2 3 4 5 |
public override void UNetSerializeVars(UWriter writer) { writer.WriteInt(Health); writer.WriteBool(Alive); } |
Since this overrides a virtual function on the UNetBehaviour base class, when the game object is serialized, the script variables will also be automatically serialized. Then, they will be unpacked at the other end with a matching code-generated unserialization function. So there is no chance of mismatches, and the code updates automatically when a new [SyncVar] variable is added.
This data is now available to the editor, so the inspector window can show more detail like this:
But there are still some issues here. This function sends all the state all the time – it is not incremental; so if a single variable on an object changes, the entire object state would be sent. Also, how do we know when this serialization function should be called? It is not efficient to send states when nothing has changed.
We wrestled with using properties and dirty flags to do this. It seemed natural that a property could wrap each [SyncVar] variable and set dirty flags when something changes. This approach was partially successful. Having a bitmask of dirty flags lets the code generator make code to do incremental updates. That generated code would look something like this:
1 2 3 4 5 6 7 |
public override void UNetSerializeVars(UWriter writer) { Writer.Write(m_DirtyFlags) if (m_DirtyFlags & 0x01) { writer.WriteInt(Health); } if (m_DirtyFlags & 0x02) { writer.WriteBool(Alive); } m_DirtyFlags = 0; } |
In this way, the unserialization function can read the dirty flags mask and only unserialize variables written to the stream. This makes for efficient bandwidth usage, and lets us know when the object is dirty. Plus, it’s still all automatic for the user. But how do these properties work?
Say we try wrapping the [SyncVar] member variable:
1 2 3 4 5 6 7 8 9 10 11 12 |
using UnityEngine.UNetwork; class Combat : UNetBehaviour { [SyncVar] public int Health; // generated property public int HealthSync { get { return Health; } set { m_dirtyFlags |= 0x01; Health = value; } } } |
This does the job but it has the wrong name. The TakeDamage() function from above uses Health not HealthSync, so it bypasses the property. The user can’t even use the HealthSync property directly since it doesn’t even exist until code generation happens. It could be made into a two-phase process where the code generation step happens, then the user updates their code – but this is fragile. It is prone to compilation errors that can’t be fixed without undoing large chunks of code.
Another approach would be to require developers to write the above property code for each [SyncVar] variable. But that is work for developers, and potentially error prone. The bitmasks in user-written and generated code would have to match up exactly for this to work, so adding and removing [SyncVar] variables would be delicate.
Enter Mono Cecil
So we need to be able to generate wrapper properties and make existing code use them even if that code isn’t even aware of their existence. Well, fortunately there is a tool for Mono called Cecil which does exactly this. Cecil is able to load Mono assemblies in the ECMA CIL format, modify them and write them back out.
This is where is gets a little crazy. The UNET code generator creates the wrapper properties, then it finds all of the code sites where the original member variables were accessed. It then replaces the references to the member variables with references to the wrapper properties and Voila! Now the user code is calling through the newly created properties without any work from the user.
Since Cecil operates at the CIL level, it has the added advantage of working with all languages since they all compile down to the same instruction format.
The generated CIL for a final serialization function that gets injected into the script assembly now looks like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
IL_0000: ldarg.2 IL_0001: brfalse IL_000d IL_0006: ldarg.0 IL_0007: ldc.i4.m1 IL_0008: stfld uint32 [UnityEngine]UnityEngine.UNetBehaviour::m_DirtyBits IL_000d: nop IL_000e: ldarg.1 IL_000f: ldarg.0 IL_0010: ldfld uint32 [UnityEngine]UnityEngine.UNetBehaviour::m_DirtyBits IL_0015: callvirt instance void [UnityEngine]UnityEngine.UNetwork.UWriter::UWriteUInt32(uint32) IL_001a: ldarg.0 IL_001b: ldfld uint32 [UnityEngine]UnityEngine.UNetBehaviour::m_DirtyBits IL_0020: ldc.i4 1 IL_0025: and IL_0026: brfalse IL_0037 IL_002b: ldarg.1 IL_002c: ldarg.0 IL_002d: ldfld valuetype Buf/BufType Powerup::mbuf IL_0032: callvirt instance void [mscorlib]System.IO.BinaryWriter::Write(int32) IL_0037: nop IL_0038: ldarg.0 IL_0039: ldc.i4.0 IL_003a: stfld uint32 [UnityEngine]UnityEngine.UNetBehaviour::m_DirtyBits IL_003f: ret |
Luckily ILSpy can convert between CIL and C# in both directions, so in this case it allows us view the generated CIL code as C#. ILSpy is a great tool for working with Mono/.Net assemblies. The C# looks like:
1 2 3 4 5 6 7 8 9 10 11 12 13 |
<span style="color: #0000ff;">public</span> <span style="color: #993300;">override</span> <span style="color: #ff0000;">void</span> <strong><span style="color: #000080;">UNetSerializeVars</span></strong>(UWriter writer, <span style="color: #ff0000;">bool</span> forceAll) { <span style="color: #0000ff;">if</span> (forceAll) { <strong>this</strong>.m_DirtyBits = <span style="color: #000080;">4294967295</span>u; } writer.<strong><span style="color: #000080;">UWriteUInt32</span></strong>(<strong>this</strong>.m_DirtyBits); <span style="color: #0000ff;">if</span> ((<strong>this</strong>.m_DirtyBits & 1u) != 0u) { writer.<strong><span style="color: #000080;">Write</span></strong>((<strong><span style="color: #ff0000;">int</span></strong>)<strong>this</strong>.mbuf); } <strong>this</strong>.m_DirtyBits = 0u; } |
So let’s see how this meets our requirements:
1. No shadow copies of variables
2. Incremental updates
3. No comparison checks for state changes
4. No hand-coded serialization functions
5. No explicit dirty calls
6. Works with all supported Unity scripting languages
7. No workflow changes for the developer
8. No manual steps for the developer to perform
9. Driven by meta-data
10. Handles all types (with new UWriter/UReader serializers)
11. No reflection at runtime
Looks like we have them all covered. This system will be efficient and friendly to developers. Hopefully it will help make developing multiplayer games with Unity easier for everyone.
We also use Cecil for RPC call implementations to avoid looking up functions by name with reflection. More on that in a later blog post.
Imi
Июнь 23, 2014 в 8:02 ппThis will only work when the calling site uses the property setter. It does not work when you e.g. do things like this:
class Foo { [SyncVar]public List MyList; }
…
foo.MyList.Add(23);
This will not detect the changed list, am I right? (Same goes for changes within classes returned by a getter)
Unknown Coder
Июнь 14, 2014 в 6:46 ппWill there be a method for the receiving client to check if a syncvar has changed and compare against the previous value? This is extremely useful feature that allows clients to run script code based off a variable change rather than receiving (typically) more expensive RPC’s.
Acropole
Июнь 11, 2014 в 3:27 дпAll this is only about collecting and sending data over the network. But game networking have more challenging and critical things to do, like dead reckoning, client side prediction. This require physics roll back on clients for rigidbodies.
How UNET will handle this ?
Laxika
Июнь 5, 2014 в 10:59 дп31 fields in a class is not enough? Read the Clean code book by Robert C. Martin because you’re doing something horribly wrong.
codemonkey
Июнь 3, 2014 в 4:13 дп@MADMAN,
I bet you’ve never made a solo player game.
MadMan
Июнь 3, 2014 в 3:18 дпIts funny I bet most commenters here have never made a multplayer game.
Robert Cummings
Июнь 3, 2014 в 1:41 дпExcellent, I’m psyched for UNET. I really cannot wait to put this in all of our games (they’re all multiplayer).
codemonkey
Июнь 2, 2014 в 11:52 пп@GAVALAKIS,
Same here, I actually like the high level API idea but it’s too much voodoo (to many «unknown» things happening in the background) and low level API it’s too much «low». Something in-between (middle ground) would be awesome. I suggested more customized [SyncVar] a la UE4 macro decorators, which still pretty high level but you have more control on how and when things syncs/updates.
Gavalakis Vaggelis
Июнь 2, 2014 в 3:05 ппI like voodoo when I also get the doll.
codemonkey
Май 31, 2014 в 6:42 дп@ALAN
Is not about turning on/off Reliability but rather make [SyncVar] a bit more customizable with attribute properties such as «reliable», «synctime», «syncfrequency», etc.
Having just [SyncVar] it’s quite too much black magic inside. I know it’s a high level API but to me it’s quite too ultra high level without a bit of flexibility.
Andy Martin
Май 31, 2014 в 3:23 дпI prefer rolling my own when it comes to networking systems. I just need something that can serialize a group of variables and send. Any other code, I will write myself.
Alan Stagner
Май 31, 2014 в 1:37 дп@CODEMONKY
They already mentioned the ability to turn on/off reliable sending.
codemonkey
Май 30, 2014 в 8:32 пп@LARUS, OLAFSSON,
Will there be more attribues (or attributes with properties or params) than just [SyncVar], for example:
[SyncVar(«reliable», 1f, )]//make it reliable, update each 1 sec, etc
public float myHealth;
MSylvia
Май 30, 2014 в 7:38 ппGlad the networking is finally getting an upgrade.
Lior Tal
Май 30, 2014 в 4:30 ппSeanRiley interesting post. I like stuff that don’t require writing boilerplate code by the developer if it’s not needed. HOWEVER, i also like dplitting the API to 2 versions — the automatic one and a manual one, for advanced users.
Also, injecting stuff at the IL level is cool (and is used quire heavily by Unity as i see in your build process tools), however it has the downside of losing the ability to properly debug the code, since the actual compiled code and tge source now differ.
All in all, interesting as to where this is going! Keep postung more :)
Sisso
Май 30, 2014 в 3:46 ппFor those that come from web this is a very common paradigm (http://en.wikipedia.org/wiki/Aspect-oriented_programming). But use it in unity3d is a little scary :P
Lárus Ólafsson
Май 30, 2014 в 3:46 пп@Kryptos, @Terry, and others, currently we only support 32 syncvars, it could be extended but we do have other methods of synchronizing more complex/lengthy structures, more on that later.
@Jes, we do have different levels of QoS, like unreliable/reliable/etc
@Beejay, this is inside the Unity build pipeline, so not affected by your choice of IDE.
@Sunny Davis, arrays will be supported but for more complex types we have other methods.
@codemonkey, built-in method for special handling transform synchronization is on the roadmap yes (hopefully makes phase 1).
Tristan Bellman-Greenwood
Май 30, 2014 в 10:47 дп@YASEENELTII New networking will be part of the 5.x dev cycle. See the blog post called ‘Announcing UNET – New Unity Multiplayer Technology’:
http://blogs.unity3d.com/2014/05/12/announcing-unet-new-unity-multiplayer-technology/
This might be part of Phase I?
Tristan Bellman-Greenwood
Май 30, 2014 в 10:40 дпCool but will we be able to sync strings without an RPC?
Can’t wait for the RPC blog post!
codemonkey
Май 30, 2014 в 9:52 дпGot a question, will UNET support client interpolation and predictability? If so on version 1 (or futures version of UNET)?
This is important to know, we are shopping a new engine with good networks capabilities right now but we can give UNET a try if it have such support.
fholm
Май 30, 2014 в 8:43 дпI have to plug my own networking solution, which I finally launched today. Which supports all of the features in this article, plus a ton more: http://www.boltengine.com/
Patrick O'Day
Май 30, 2014 в 6:13 дпCould we pass options into the syncvar attribute? Say I wanted to write my own generated property setter instead of using the one autogenerated by cecil? Could I write [syncvar(setter=customSetter)] or some variation? The variable value and the dirty bit mask would need to be passed in but I could think of some edge cases that would be incredibly useful.
And of course exposing the cecil build hooks so people could make their own custom cecil build plugins would be nice as well.
Sunny Davis
Май 30, 2014 в 4:33 дпHow about arrays and complex types?
Beejay
Май 30, 2014 в 12:32 дпIn my current workflow I like to keep my script code in an external library project so that I can use the newest version of Xamarin Studio (5.0 atm) to edit my source.
Will there be a plugin available to make the magic work in the latest versions of the IDE, or will I need to stick with the IDE provided by Unity?
Oh, and the magic does not bother me as long as someone has explained what the magic does. Keep up the good work, and I am looking forward to using the new networking API almost as much as I am the new GUI!
Alan Stagner
Май 29, 2014 в 10:19 пп@FHOLM: Once again, not the biggest caveat any engine has dealt with. You already have to deal with that with a bunch of Unity code, like assigning transform position or rigidbody velocity.
It’s not exactly a hardship to treat your syncvars as if they are properties. Think of them as syntactic sugar.
fholm
Май 29, 2014 в 10:10 ппThis would also break this implementation:
[SyncVar]
Vector3 pos;
// this will explode if pos turns into a property
pos.x += 1;
Alan Stagner
Май 29, 2014 в 9:48 пп@UnityUser: Given that there will also be a low-level API, why wouldn’t it be good?
unityuser
Май 29, 2014 в 9:12 ппCan’t think that all this magic will be a good thing in the long term.
Marc Schärer
Май 29, 2014 в 9:02 ппVery interesting Solution but i would like to +1 above question regarding the 31 fields.
If that’s really the limit it would not work reasonably. This has been one of torque networks major issues already half a decade ago as it had exactly this limitation which is absolutely insufficient for action games or data heavy use cases with more complex data sets. It would force us devs to store it into containers that would use the delta sync benefit or a 2 tier sync solution of some kind.
I would prefer a flexible length bitstream for the dirty flag handling that’s defined at compile time (I assume that this is already the case though as the way Cecil is used would allow this).
Joachim
Май 29, 2014 в 8:15 пп@Madman:
UNET is built with a low level and high level library. The low level library lets you send packets with a number of different quality of service modes. And you are in full control you can select which connection to send messages to. There is also a simple & fast serialization library that you can use from code. (the high level library & code generation is built on it)
If you want to only use the low level library thats fine, it is fully exposed. And the high level library builds on top of it. The high level library is written in C# and very extendable.
See here for an overview:
http://blogs.unity3d.com/2014/05/12/announcing-unet-new-unity-multiplayer-technology/
Ana
Май 29, 2014 в 8:12 пп@Madman
They mentioned in previous articles that they intend to give low level access to advanced developers. Being user friendly for non network developers doesn’t mean you won’t get a low level access for advanced scenarios.
Alan Stagner
Май 29, 2014 в 8:10 ппMadman: OK, more power to you. No solution can satisfy everyone.
I’m just surprised that after revealing *one* feature you’ve decided it does not fit your needs. It’s hard to tell if you actually feel that way, or are just trolling.
madman
Май 29, 2014 в 7:35 ппMore black magic great this is not what we need! The current networking system is a burden for complex games. What we really need is a low level networking API as a foundation.
Honestly after reading this it’s so far from my requirements that I will probably switch to Lidgren. It almost feels like it would be a waste of time explaining why.
This just a another noob friendly solution.
bosnian
Май 29, 2014 в 7:17 ппGREAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAT ;)
Nick Metnik
Май 29, 2014 в 5:40 пп@Alan Stagner I can’t read your comments without Morgan Freeman’s voice in my head. I’m excited to see how UNET pans out!
Terry
Май 29, 2014 в 5:08 ппThis is cool, but I have to assume that this has to be limited to 32 member variables per class?
Alan Stagner
Май 29, 2014 в 5:05 пп@Kryptos on amount of syncvars: The 64-bit engine wouldn’t change anything of course, it’s just down to whether they use an int or a long for their bitmask.
That said, I have a hard time believing one object is going to exceed 32 serialized fields without having some kind of code smell about it.
Alan Stagner
Май 29, 2014 в 4:52 пп@FHOLM IMHO this is actually a pretty smart implementation. If you think about it, avoiding refs for sync vars is not one of the biggest caveats in any game engine ;)
Actually, I bet it would be possible for the UNET to detect situations where you pass a syncvar by reference, and generate a warning.
Kryptos
Май 29, 2014 в 4:50 пп@FHOLM «this would for-example break if you try to pass a field by reference to an method, and you guys then re-write it to a property = boom»
1. IMHO using reference on class fields is bad practice.
2. There are workarounds and maybe Cecil can generate them itself :
// original code
SomeMethode(ref health);
// modified code
var copy = HealthSync;
SomeMethode(ref copy);
HealthSync = copy;
Jes
Май 29, 2014 в 4:45 ппI can have tons of data in my scripts that do not need delivery guarantee.
How UNET will work with it.
Sean Riley
Май 29, 2014 в 4:28 ппSyncVars are used for script data, not transform synchronization — as people point out, the requirements for transform data and script data are often quite different. UNET has a built-in component for optimized, configurable synchronization of transform data that does not use SyncVars.
Jes
Май 29, 2014 в 4:21 пп@Emil: Thanks :) But can I disable delivery guarantee for my Position filed, if packed with new position is lost or was delivered after packed with more recent position I dont’ want you system resend/use it, system must just ignore it.
Duke
Май 29, 2014 в 4:16 ппI really wish this was not being hidden so much from the developer. I’ve had need for property change notifications/dirty tracking for a long time, and constantly have to roll my own suboptimal solutions. It would be great to hook into a common design rather than bolting my own solutions on top of this.
ImaginaryHuman
Май 29, 2014 в 4:11 ппI haven’t done any network programming as such but this new solution does seem to make things simpler and easier. I presume though it is still necessary somewhere somehow to define the network topology or to create connections and find players and so on … I hope those parts will be very easy also.
Emil "AngryAnt" Johansen
Май 29, 2014 в 4:09 ппVery nice bit of Cecil juggling. Congrats :)
@Jes: http://blogs.unity3d.com/2014/05/12/announcing-unet-new-unity-multiplayer-technology/
Kryptos
Май 29, 2014 в 4:07 ппThis looks great, but what happens if I need more than 31 fields to be synchronized on a single object? Will the 64bit version of the engine allow up to 63 fields?
Or can I serialize structs so that some fields might be grouped together and hence allow an pseudo-infinite number of fields (but still limited to 31 structs) ?
Note that this question is only theoretical.
Keep up the good work :)
fholm
Май 29, 2014 в 4:04 ппHonesty I don’t like this solution because fields and properties are not 100% identical in .NET, this would for-example break if you try to pass a field by reference to an method, and you guys then re-write it to a property = boom.
There’s just too much magic in it for me.
Jes
Май 29, 2014 в 4:01 ппWhat protocol you system use? TCP or UDP?
How you recognize two types of data:
— important (with delivery guarantee)
— not important (for example position of character don’t need delivery guarantee because next packed deliver more actual position )
YaseenEltii
Май 29, 2014 в 3:50 ппWhen Will The UNET System Be released ? :)