Search Unity

Cloud Build for Unity Teams is a continuous integration service for games and apps built in Unity. Cloud Build makes it simple and easy to create and share builds of your game, letting you and your team iterate faster and always be on the same page. Setting up Cloud Build takes seconds and works with your existing source control repository.

Cloud Build now supports Large File Storage in Git source control repos, also known as Git LFS for short.  So now, you can create larger builds with Cloud Build when using it with Git!  Read on to learn more.

What is Git LFS?

Git Large File Storage (LFS) is a technology that allows providers to keep large assets/files outside of a Git repository, keeping total filesize down while still maintaining important base assets alongside the project. When a file is added to a traditional Git repository, Git maintains a separate copy of every version of that file. This is great for files containing code, or very small binary/image files. However, this becomes a problem with large files, and providers may warn or prevent you from uploading your changes. Git LFS allows providers and users alike to maintain leaner repositories.

How do I use Git LFS with Cloud Build?

There are no special steps for Cloud Build to recognize LFS; it just works!  As with all Cloud Build projects using git, simply add the SSH key provided in your project’s Source Control Settings area to a user on GitHub with access to the repo. Cloud Build will recognize the need to use LFS and download your files accordingly before a build begins. Need a walkthrough? Check out this video, showing how to setup Cloud Build.

How does Git LFS work?

The Git LFS client is a command line utility that sits next to the existing git command line executable and interoperates with it. It allows a developer to track certain files within their repository, which marks them for management by the LFS subsystem. These files are replaced with text-based pointer files in the repository, and the files they represent are uploaded and stored separately outside of the repository. Upon checkout, Cloud Build uses the Git LFS client to determine where these files are and pull them down before your build.

How do I use Git LFS?

Download and installation instructions for Git end users can be found on the Git LFS website, and can be used with any Gitbased source control where LFS has been implemented. Once installed, the git lfs track command is used to specify which files should be handled by the LFS system:

This example command will generate or add to a .gitattributes file, specifying that all WAV files should be tracked by LFS. That’s it! All WAV files will be managed via Git LFS. Additional file names or wildcards can be specified with the git lfs track command. Note: Please be sure to include the .gitattributes file when you commit changes.

Git LFS Provider Support

GitHub is the source of the Git LFS project, and as such is the most prominent server implementation at this time. There are other services that also offer Git LFS support, however they do not currently support using LFS with SSH keys. In order to use a Git LFS host with Unity Cloud Build, they must allow authorization for it via SSH. Some other notes should be taken into consideration:

  • GitHub deployment keys are not supported: SSH keys are handled in various ways within GitHub, and usage of LFS via deployment key is not currently supported. An alternative approach is the use of machine users, or separate user accounts that the SSH key can be used with.
  • Bandwidth and storage are likely measured: Git LFS is a bandwidth and storage intensive service, and as such, there are likely charges associated with both when heavily utilizing it. In GitHub’s case, free usage is capped at 1GB per month for bandwidth and 1GB of storage. In addition to any caps, it’s important to take any additional hosting costs into account.

Conclusion

In conclusion, Git LFS is a great option to consider when utilizing Git as your method of source control while using Unity and Unity’s services. If using Git LFS seems complicated, but your project works with large files, try out Collaborate for Unity Teams.  It handles large files without any extra effort, and setting it up to work with Cloud Build is super simple.  No bandwidth limits to worry about either; just storage.

The Cloud Build team works hard to support the needs of Unity developers that wish to take advantage of continuous integration in their development pipelines. We look forward to improving the service even more as time goes on. Happy building!

Cloud Build is built right into Unity. Click here to get started!

Comments are closed.

  1. This is great, but please work towards adding support for https as well. Bitbucket only supports LFS over https.

    1. Hi Rick,

      Your best bet in order to get this feature on the radar would be to request it at https://feedback.unity3d.com – Currently, we only support SSH in order to align with our Git support that was already in place. For now, any provider where you plan to use LFS will have to support authentication over SSH. Here is a feedback entry for support of Bitbucket app passwords if you agree with that approach:

      https://feedback.unity3d.com/suggestions/support-bitbucket-app-passwords

  2. I tried Cloud Build with a LFS project a few months ago and it seemed to use a huge amount of bandwidth, much more so than using a local Jenkins-based continuous integration server. I had to switch it off as it was costing too much. Has that changed – is there some caching solution in place now? It felt before like it was pulling the whole repo each time, rather than just the changed files.

    1. Hi Richard,

      This is actually a function of how Cloud Build checks out your project, which requires a full checkout in order to have everything needed to begin the build. However, if you have large assets in your project you may want to look into creating a custom LFS config file that excludes specific directories. Say I have a ‘BaseAssets’ directory with huge raw WAV files for a soundtrack, but I’m using the mixed-down MP3 versions as the actual asset source. In this case, setting lfs.fetchexclude to exclude BaseAssets should prevent Cloud Build from pulling down those resources:

      $ git config -f .lfsconfig lfs.fetchexclude “BaseAssets, SomeOtherDirectory”
      $ git add .lfsconfig

      NOTE: Adding actual game assets in-use to the lfs.fetchexclude list will absolutely make your builds fail. This is only useful in a situation where the project is not dependent on the excluded files, and those files are *not* in your Assets directory. Otherwise, asset import will attempt to treat the original pointer files as audio, textures, etc.

      Hope this helps, please also visit our feedback site if you would like to create a feature request that the community can upvote: https://feedback.unity3d.com

  3. … I’ve been using GIT LFS with cloud build for like 6 months. Not sure what I did but it’s been working for a while now. I’ve used it on both GitHub repos and personally hosted GitLab repos.

    1. That’s great to hear. LFS compatibility has in fact been available on Cloud Build for awhile, but most recently improvements were integrated that allow for batch downloads instead of one-per-file calls. This makes for a faster, more reliable experience when using Large File Storage. Thanks for building with us!

  4. Speaking of Git LFS, do you have any dos & don’ts in regards to what file types to track with LFS? How about prefabs or scene files?

    1. The best use cases related to Unity will generally be large base art assets (think hi-resolution PSD files for textures, large WAV audio files for soundtracks) as they are binary files. Git does not like to store large binary files, as patching is not available so an entire copy of the changed asset needs to be retained. With Large File Support, instead these large files are stored outside the repository and replaced inline with text pointers to the actual asset, which keeps your repository lean.

      In terms of Scenes, it is advisable to store scenes as text instead of the default binary format for source control purposes. You can see the following link in order to change this setting:

      https://docs.unity3d.com/Manual/TextSceneFormat.html

      With text based scene files, there shouldn’t be a need to store them using LFS. However, teams should be careful to avoid conflicts when multiple users are working on a project. My personal preference as a Unity developer is to restrict my changes to a “clean room” scene when adding new functionality, which can then be ported to a main gameplay scene in a controlled fashion. Your mileage may vary, etc. Feel free to share your own best practices here. :-)