Unity 5: API changes & automatic script updating

June 23, 2014 in Technology

As the breadth of the Unity feature set keeps on growing, and as we build it with more and more engineers at the same time, the necessity for being able to split up the product in separate modules becomes bigger and bigger. Obviously because it is just sound engineering practice. (It’s nice that when you change something in the physics code, that your audio doesn’t break). It also gets us closer to a situation where we could ship updates to different modules at separate timelines. And it’s nice that it gets easier to work on the different components with different teams. There’s one additional reason that is much more user visible: the ability to ship a smaller game. Don’t use particle systems? Don’t pay for them. Don’t use 2d rendering, Don’t pay for it. If you take a look at the WebGL demos that we posted, you can see that the entire unity engine that is converted to javascript (not manually thankfully!) clocks in at about 3mb of compressed javascript. That’s already pretty good, but we know there’s more juice to squeeze out of that. The web is still a platform where size matters a lot, and we want to bring the minimal size of a Unity published WebGL project down further. This is also true for other platforms, but at the moment, WebGL is definitely one of our more size-conscious platforms. So, with many good reasons to modularise the way we write code, test code, and ship code, we’ve started work down that path. There is still a very very long way to go, but if you don’t start you’re not going to get there either, so we started. While doing mostly non-user-visible work on this, we hit some dependencies in our API that need to be cut in order for us to proceed further in our efforts. Let’s look at that API problem: Many moons ago, when we started building our API, the C# code required to do something with a renderer component looked like this

void Start()
{
  Rigidbody r = (Rigidbody)GetComponent(typeof(Rigidbody));
  r.velocity = Vector3.up;
}

We disliked the verbosity of that so much, that (in addition to making a new .NET language unityscript-aka-javascript with much less boilerplate), we also added quick component property getters for commonly used components:

void Start()
{
  rigidBody.velocity = Vector3.up; //"rigidBody" calls: UnityEngine.Rigidbody UnityEngine.Component.get_rigidBody()
}

Over time, we’ve been inconsistent in wether or not we added these quick accessors to the API for new components. For some we did, for others we didn’t, and by now it’s not clear why some components have them and others not. These accesors are also not nice for modularisation. When you modularise, you want to make sure that all the dependencies “point in one direction”. It’s okay for the physics module to talk to things in the core module, but it’s not okay for the core module to talk to the physics module directly, because it might not be there. Turns out that is exactly what we have here. We have the MonoBehaviour class, which is something that we want to put in a “core” module. but it has a property getter implementation for .rigidBody that returns a RigidBody object, which should obviously live in the physics module, that the core module shouldn’t be able to reference. For Unity5, we decided to bite the bullet, and remove all the quick component property getters with the exception of .transform (since that one belongs in the core module, there’s no need for it to go, and it’s also by far the most used one) [1][2]

As I wrote in a previous post, one of the hardest things of making Unity is finding the balance between “making things better” and “not breaking projects that already work”. Since we feel that modularisation is very important in the long run, we decided to remove the .rigidBody style accessors, but we were not comfortable with the amount of pain that would bring our users. So we embarked on a project to reduce the user pain introduced by this work: Automatic Script Updating

Unity5 will ship with (unless something really bad happens) the ability to detect when your scripts use old API, and is able to, after asking for your permission, rewrite that code to use the new API. So if you have a script that does this:

void Start()
{
  rigidBody.velocity = Vector3.up;
}

We will convert that for you to:

void Start()
{
  GetComponent<Rigidbody>().velocity = Vector3.up;
}

The script updaters work for C#, UnityScript-aka-Javascript, Boo, and prebuilt .dll assemblies. (We actually read the IL code inside the prebuilt assemblies using the wonderful Cecil library by Jb Evain of UnityVS fame, detect usages of deprecated API, and then rewrite the IL code to use the new API instead). Perhaps unsurprisingly the feedback from early Unity5 testers was “hey, there is a script updater now, and it fixed all my quick property accessors, but there are more API changes you guys did that it did not fix”, and we find ourselves in the kind of good problem to have situation where it looks like the updaters work well enough that it makes sense to look at the more minor API changes that we’ve done and see which ones of those we can update reliably, and which ones we cannot. Maybe one day we could even open this up to asset store vendors, and allow them to provide “rewrite rules”, so that they also gain increased flexibility in improving API, fixing old mistakes while keeping user pain at a minimum. Should you have become accustomed to the component accessors to the point that you really really want to keep them anyway, it’s trivial to write a MyMonoBehaviour class with all these properties in it, that you make all your scripts derive from. We hope this approach strikes a good balance between not putting too much work on the plate of our users, while still being able to improve API, fix mistakes, and make Unity better.

Bye, Lucas (@lucasmeijer)

 

[1] An alternative approach would have been to introduce extension methods. This is basically the physics module being able to add additional methods to the MonoBehaviour class. This would mean user code would have to change from

.rigidBody

to

.RigidBody()

We felt that was hard that was not a very attractive alternative. Hopefully one day C# gets extension properties. Should those have existed, we would have been able to keep source-level compatibility and still move the rigidBody code out into its own module.

[2] in Unity5 we also cache the transform component on the c# side, so there should no longer be a performance reason to cache the transform component yourself.

Comments (58)

Subscribe to comments
  1. http://glucreditshack.wordpress.com

    August 21, 2014 at 6:52 am / 

    If some one wishes expert view on the topic of running a blog
    after that i advise him/her to go to see this web site, Keep up the fastidious work.

    Here is my web site … glu credits hack (http://glucreditshack.wordpress.com)

  2. ducks unlimited

    July 19, 2014 at 1:54 pm / 

    Wow, fantastic weblog layout! How long have you been blogging for?
    you make blogging look easy. The total look of your site is wonderful, let alone
    the content material!

  3. ashlyn broke

    June 26, 2014 at 9:24 pm / 

    Download Free Jurassic Park Builder Hack Tool v2.4 , Get Free coins,money and meat in Jurassic Park Builder , Jurassic Park Builder Hack For Facebook and Android and IOS

  4. Matt B

    June 26, 2014 at 11:33 am / 

    @LucasMeijer YAY for data refactoring changes, and yes an attribute would be one approach. Any chance that attribute might be able to specify a different component on the same object that used to hold the value? An example for where this would be useful is when one component needs refactoring into two separate, more focused ones.

  5. Sympathiser

    June 26, 2014 at 10:39 am / 

    I agree on virtual methods on MonoBehaviour.

  6. dusho

    June 26, 2014 at 12:35 am / 

    Nice step.
    Regarding clarity and clean code. What about methods in Monobehavior. Would it be possible to have methods like Start(), Awake(), OnTriggerEnter(), OnTriggerEnter2D(), … marked as virtual ? So intellisense can give you option to override it, and actually give you exact name of method and it’s parameters? I saw those methods being misspelled so many times and figuring out what the problem is..

  7. Laurent

    June 25, 2014 at 9:11 pm / 

    Two comments:
    1- Components which are not in the core module don’t get shortcuts, but how do I recognize Rigidbody is in a separate module? Does it have a different name space?
    I presume that your code would in fact look like this:
    GetComponent().velocity = Vector3.up; but that you had a using UnityPhysics in your header. Right?
    2- To offset the added verbosity, do we get the new version of Mono, I’m eying transform.position.x=3, current c# needs 2 lines.

  8. Lucas Meijer

    June 25, 2014 at 3:10 pm / 

    @benoit: I completely agree! data refactoring is a big pain that I would really like to fix. for unity4.6 we’ll add a attribute you can put on a field that says [OldNameWas(“bla”)] (I can’t remember the exact attributename we chose). it only supports renaming of fields, not changing things from int to float etc, but it’s a good first step. I’d love to get more solid support for more complex scenarios though.

  9. Benoit Fouletier

    June 25, 2014 at 2:29 pm / 

    Thanks for the post Lucas!
    Regarding the “make an extensible API for refactoring Asset Store packages’ API”… as much fun as it sounds, personally I would put a (much) higher priority on providing a “data refactoring API” than an “API refactoring API” ;) !

  10. Richard Fine

    June 24, 2014 at 10:46 pm / 

    @CAYL: It’s the former. Though I guess maybe you *could* keep writing old-style code and running the upgrader over and over? Your code wouldn’t stay old-style, though, and once you’re used to reading the new approach you’d probably be writing it too.

  11. Cayl

    June 24, 2014 at 6:07 pm / 

    Modularisation is the right direction, and so is automatic script updater, but I didn’t quite understand the purpose of the latter, is it just a transitional tool to help port deprecated API in old projects up to date, or is it intended as a “auto-correcter” so to speak to allow devs the luxury to write quick getters like before?

  12. Sympathiser

    June 24, 2014 at 4:35 pm / 

    I also plus 1 MATTB and BryanLivingston’s on .NET.
    Right now there is no chance to be consistent on naming, you either break Unity’s or .NET’s. Brings severe pain to perfectionists :)

  13. Matt B

    June 24, 2014 at 4:24 pm / 

    @LucasMeijer I think it’s AWESOME that Unity is going through this “growing up” phase :)

    I second @Byron’s request for some kind of library support. Perhaps a good way to handle this would be to allow the Visual Studio/Mono solutions to be safely editable. So Unity only updates it’s own libraries, but users can add library projects from outside of the Assets folder. This would allow shared code to be included in libraries, but allow working on their code in the same solution as the main project.

    I’d also like to second @BryanLivingston’s request for proper MS .Net naming conventions. Yes, this is a major difficulty to change, but that also highlights something that is generally difficult in Unity-refactoring of any kind. Perhaps some kind of reliable field renaming tool, that would update preefab and scene files appropriately in a project when renaming fields, would also help achieve a switch to correct conventions.

  14. Sympathiser

    June 24, 2014 at 3:27 pm / 

    Same consistency request would go to which class actually has the GetComponent method. Why the Component class even has it? You’re not retrieving a component of the component, are you? I can see especially how much confusion it brings to novice developers. They attach components to gameobjects but then access them for some magical reason on a completely different instance of another components.

    I personally always write gameObject.GetComponent().Foo()

    I almost never use GetComponent on components themselves and I almost never used thee shortcuts too anyway. But I suggest if you go that way — go all the way clean.

  15. Sympathiser

    June 24, 2014 at 3:18 pm / 

    Since you Unity guys decided to go that way and rething/cleanup the API, you might as well bring more consistency to other aspects of the API.

    Here is my Feedback: http://feedback.unity3d.com/suggestions/bring-consistency-to-naming
    The general idea is to not use synonyms (like “motion” property instead of “animationClip” for an animator controller’s state) and not use the same word for multiple things (Layers vs. Sorting Layers).

    You’d also want to bring some more thinking into naming things in general.
    Like the “samples” property of the AudioClip class. Such name would suggest it’s an arrays of some Sample class instances. But it’s actually an integer number showing the amount of samples. Why not calling it samplesCount or numSamples?

    Or (speaking of the AudioClip again) having a method GetData() and SetData() on a class. Everything is data! You don’t call methods of variables data. If you’re retrieving samples, then call it GetSamples.

    Etc. etc. etc. etc.

    The entire API is a stylistic synonymous ambiguous pile of mess! Involve an English speaking architect or something. And read The Clean Code and The Clean Coder.

  16. nikescar

    June 24, 2014 at 11:47 am / 

    @BYRON
    I haven’t had the need to try this, but couldn’t you use symlinks to link folders in all of your projects to your shared code?

  17. Lucas Meijer

    June 24, 2014 at 8:04 am / 

    @bryan: we’ve discussed this internally quite some times in the past, but never felt the upsides were strong enough over the downsides. Maybe we should revisit this one now that we have the script updaters. If we could start over, we would definitely go for upper case properties.

    @galen/lucas: a partial class cannot span multiple assemblies.

    @david: as part of this work we have also made GetComponent faster. Wether or not the boilerplate of caching is worth the performance gain it can bring depends on a case by case basis. I usually cache when I’ll be calling something on the componnent every frame.

    @tswalk: the plan so far is that you use your source control system to realize which changes have been made. also, you should never ever ever do a major version upgrade of a project without making a backup. source files are not the only thing that get changed, all your data files get changed as well.

  18. Lucas Vasconcelos

    June 24, 2014 at 5:38 am / 

    another possibility beyond Extension Methods would be transforming MonoBehaviour into a partial class and moving the rigidBody property definition to physics library.

    public partial class MonoBehaviour {

    public RigidBody rigidBody { get …; set …; }

    }

  19. tswalk

    June 24, 2014 at 4:37 am / 

    I have no issue with this at all as long as I have a way to review all changes.. perhaps a post process change log?

  20. Mark

    June 24, 2014 at 4:31 am / 

    Great update! Trimming Monobehaviour down goes a long way towards having a better separation of concerns.

  21. David

    June 24, 2014 at 3:31 am / 

    I thought it was bad to call get component like that without caching it inside start or awake…..is this wrong?

  22. Peter Dwyer

    June 24, 2014 at 3:26 am / 

    Well I hope that pay for it statement was just about performance and memory, though the wording of the article i.e. mentioning the performance angle later on as a separate item kind of made me think otherwise.

  23. Galen Ryder

    June 24, 2014 at 2:46 am / 

    This change seems fine but if you marked MonoBehaviour as partial you could include a partial implementation of MonoBehaviour with each Module to include Module-specific properties. But you can’t reference between the members of partial implementations and it could potentially make things more complicated for Plugin Creators when they can’t depend on a property as available in every project.

  24. Bryan Livingston

    June 24, 2014 at 2:16 am / 

    Since you’re pulling off the impossible here, why not go further and change to the standard .net capitalization conventions?

    Rename MonoBehavior.transform to MonoBehavior.Transform.

    It would make sense since C# has won out over UnityScript anyways and the JS style naming doesn’t have any use any more.

  25. larsbertram

    June 24, 2014 at 12:26 am / 

    break whatever you have to break to make unity5 a valid competitor to the other major engines around there. we are definitely leaving the comfort zone these days and it is all about making the right decisions for the next 2 or 3 years: multi platform support will not be such an outstanding selling point nor will be the ease of use: more advanced features and performance will be ranked higher than the ease of use – that might follow as udk shows these days.

    lars

  26. Lucas Meijer

    June 23, 2014 at 10:58 pm / 

    @James: if you compile outside of unity in MD or visual studio, these things show up as normal deprecation errors, which you can doubleclick and be taken to the right line etc.
    we’ve not seen problems with debug information yet (allthough often I see asset store vendors not ship .mdb files, something that they’d probably want to do actually).

  27. Emil "AngryAnt" Johansen

    June 23, 2014 at 10:05 pm / 

    @Nicholas Ventimiglia:
    At which point they’re likely quite far in the IL2CPP integration, meaning such an upgrade would not be a lot easier.

  28. James Hillery

    June 23, 2014 at 10:00 pm / 

    This seems like it will save everyone a lot of time! However, for larger unity projects that exclusively use assemblies generated outside of unity, it sounds like they are going to hit this automatic script update prompt every time an assembly is recompiled until they manually fix up their code. Having a generated report of what is being flagged as deprecated would be useful for engineers in this situation. Or at the very least, are the existing console log warnings about deprecated code going to remain?

    Also, I’m curious how this process of patching pre-built assemblies with Cecil affects debugging in terms of line numbers, call stacks, etc?

  29. Lucas Meijer

    June 23, 2014 at 9:51 pm / 

    @YUKICHU: You could defeinitely do that. I think there is a very low limit to the amount of code they like our automatic upgrading tool to write for them. In fact, I suggest that you always look at a diff of what the upgrader did, understand the changes, and then figure out that you maybe want to cache some of them. We just were not happy with a situation where all hundrerds of thousands of developers (yes, you are with that many), would have to fix hundrerds of compiler errors, just because we want to make some steps towards modularizing our codebase. something that also does not come with anything very user visible thing in return people can be happy with :)

  30. Thomas

    June 23, 2014 at 9:19 pm / 

    @YUKICHU
    you could do this:

    private Rigidbody _rigidbody = null;
    Rigidbody rigidbody {
    get {
    if (_rigidbody == null) {
    _rigidbody = GetComponent();
    }
    return _rigidbody;
    }
    }

  31. Yukichu

    June 23, 2014 at 9:11 pm / 

    Automatic Script Updating – could it instead create a private variable in that script, assign reference in Awake(), and then use it? I guess most people already realized to cache things anyhow, so… I guess it doesn’t matter really.

    Pretty cool overall.

  32. Thomas

    June 23, 2014 at 9:07 pm / 

    Sounds awesome can’t wait.. :)

  33. Lucas Meijer

    June 23, 2014 at 8:45 pm / 

    @Nicolas: the difference between rigidbody and rigidbody2d is more than two letters :). they use completely different systems, and internally completely different. shoehorning them into a single component wouldn’t make much sense. we usually like our components kind of the unix way, do one thing, do it well,

    @Nicolas: sharp! indeed, we have the same problem with the WWW class. we haven’t tackled this one yet.

    @anamolousunderdog: when we feel there’s a change interesting enough to blog about. I have a non related long post coming up about the inner and outter workings of the serialization system which I’m looking forward to share though :)

    @lior: for c# we use NRefactory. for UnityScript we use the parser inside the unityscript compiler. for Boo we use the parser in the boo compiler. for .Net assemblies, we use Mono.Cecil

  34. Nicolas Borromeo

    June 23, 2014 at 7:21 pm / 

    I really dont like that approach, it would be nicer some factory method that receive the bytes, but i understand that you do that in order to keep it simple for new programmers.

  35. Jes

    June 23, 2014 at 7:04 pm / 

    @NICOLAS BORROMEO: they can become extension methods -> AudioClip(), Texture()

  36. Nicholas Ventimiglia

    June 23, 2014 at 6:51 pm / 

    > Hopefully one day C# gets extension properties

    I found this funny. Even if C# adds extension properties in vs 6… UT will need to most likely adopt C# 4 and 5 first (async, await and Tasks would be splendid btw.)

  37. Radna

    June 23, 2014 at 6:18 pm / 

    we need you keep old scrip details just add new code in different color to documentation :)

  38. Nicolas Borromeo

    June 23, 2014 at 5:31 pm / 

    Modularization will change also the class WWW right? if you separate rendering from audio, the property audio clip and texture must dissapear

  39. Vectrex

    June 23, 2014 at 5:25 pm / 

    They’re talking SIZE cost.
    ps virtual OnStart etc could be auto-updated and finally implemented. More C# events to subscribe to aswell.

  40. Anomalous Underdog

    June 23, 2014 at 5:24 pm / 

    Very nice. Will there be more blog posts on other API changes for Unity 5?

  41. Storm

    June 23, 2014 at 5:22 pm / 

    Thanks for clearing that up guys. Now I’m thrilled.

  42. Lucas Meijer

    June 23, 2014 at 5:20 pm / 

    @Byron: cross-project code and asset sharing is something that we would really like to make easier. we’re brainstorming concepts around shared libraries. nothing shipping any time soon, but wanted to let you know that we realize that this is a pain point

    @Tijmen: icecream for you!

    @everyone: when I say dont pay for it, I naturally mean pay in terms of bytes/size!!

    @liortal: we update .cs/.js/.boo on the sourcelevel, and assemblies on the IL level. as an assetstore vendor today you do not have great options of doing the same kind of thing when you want to change API of your package.

  43. Lior Tal

    June 23, 2014 at 5:18 pm / 

    @Richard – i missed the part that said it updates the code :)

    Can you elaborate on how this is performed? Are you using any open-source library for parsing code? or is that a simple text replace?

  44. Nicolas Borromeo

    June 23, 2014 at 5:11 pm / 

    I like the idea of modularization, its gonna be great to remove the api i dont need, but i also see that you have to do that because you starting duplicating scripts for everything new you had added to unity, i.e.: now i have all the physics component duplicated for 2d, now i have sprite renderer, now with the new gui i have another transform, and probably another renderer!

    I understand the reason you do that, but i like when one component allows me to do as much as that one can, and having a sprite renderer when its just a meshrenderer of a plane (probably its not exactly as that im saying, but you got the idea) its not somewthing that sound nice to me, going that way mean creating duplicated for components for every new kind of rendering way, that seems to have minor changes between each other.

    Just my humble opinion, anyway i think you are doing a great work!
    Greetings!

  45. Novack

    June 23, 2014 at 5:06 pm / 

    Excellently done! Good choice to get rid of old things.

    Sometimes previous things need to be breaked in order to evolve. Dont be so afraid of people having to refactor code, thats what developers do when required!

  46. Jes

    June 23, 2014 at 5:02 pm / 

    “Don’t use particle systems? Don’t pay for them. Don’t use 2d rendering, Don’t pay for it.”

    I think it’s about performance/memory pay, not money!

  47. Ryan

    June 23, 2014 at 5:02 pm / 

    @Peter @Storm – they meant ‘pay for it’ in the sense of binary size and performance, not actual monetary cost.

  48. Storm

    June 23, 2014 at 4:52 pm / 

    “Don’t use particle systems? Don’t pay for them. Don’t use 2d rendering, Don’t pay for it.”

    Does this mean the pricing model will change or become more complicated? Or by “pay” are you referring to the paid cost in performance?

  49. Peter Dwyer

    June 23, 2014 at 4:50 pm / 

    “Don’t use particle systems? Don’t pay for them. Don’t use 2d rendering, Don’t pay for it.”

    Ok that rang loud alarm bells for me. It already costs way too much for each platform module for Unity so it sounds like a new excuse to charge for stuff that should be included anyway. I mean that statement can easily be reversed to be

    Use a particle system? Pay for it! Use 2D rendering? Pay for it! etc. etc.

    Where the heck does it end?

  50. Richard Fine

    June 23, 2014 at 4:44 pm / 

    @LIOR TAL: No, it’s performed at both levels: source level for script files in your project, and IL-level for managed assemblies (that you maybe don’t have the source for).

  51. Louard

    June 23, 2014 at 4:38 pm / 

    It was also pointed out to me that since it is recommended to only use GetComponent in initialization functions rather than runtime this change will could help users optimize their code as it will no longer be hiding invisible GetComponents throughout your otherwise nice looking scripts.

  52. Lior Tal

    June 23, 2014 at 4:32 pm / 

    It’s kind of misleading calling this “source update” as the update is performed (as you explain) at the IL-level.

    Nevertheless, this achieves the result.

    You mention that “one day you may open this up to the asset store… ” – what’s keeping us TODAY from injecting IL code/replacing code in the assemblies that unity compiles? Can’t i hook up into the build process somehow, modify the assemblies and store them back to disk?

  53. Jes

    June 23, 2014 at 4:29 pm / 

    Modularisation is very good thing, thanks :)
    Do you think about modularisation of Developers code and AssetStore packages, because of the same reason as you mentioned in first sentence.

  54. TJHeuvel

    June 23, 2014 at 4:24 pm / 

  55. Antao Almada

    June 23, 2014 at 4:23 pm / 

    I knew the field shortcuts to GetComponent would bite your ancles some day. ;)

  56. pahe

    June 23, 2014 at 4:21 pm / 

    Glad to see some more insights and that former concepts are reviewed (and thrown out if they don’t fit anymore)! Makes me very happy and I’m looking forward to the new API changes!

  57. Byron

    June 23, 2014 at 4:14 pm / 

    I’d like to see directory support so that we can have code modules that are shared across projects. Currently I have to copy or branch code per project when I would prefer that all projects share exactly the same code so that bug fixes are propagated without manual intervention.

Comments are closed.