Search Unity

Tips for working more effectively with the Asset Database

October 6, 2020 in Engine & platform | 10 min. read
Share

Is this article helpful for you?

Thank you for your feedback!

The Asset Import Pipeline v2 is the new default asset pipeline in Unity, replacing the Asset Import Pipeline v1 with a rewritten asset database to enable fast platform switching. It lays the foundations for scalable asset imports with robust dependency tracking. Read on to explore how the Asset Database works and discover some time-saving tips.

Since the release of Unity 2019.3, the new Asset Import Pipeline has been the default pipeline for new projects. Combined with many other improvements, this lays the foundation for a more reliable, performant and scalable Asset Import Pipeline. This rewrite changed the way the Library folder works in order to support new workflows.

Let’s have a look under the hood at a number of situations you may encounter and how to manage them efficiently. You’ll learn how to spot and address some of the bottlenecks that cost you time and project performance.

The tips you’ll find here apply to Unity 2019.3 and later versions. Remember, if your project is in production beyond prototyping, for maximum stability we recommend you use the latest Long-Term Support (LTS) version of Unity, Unity 2019 LTS.

Explore import times as a starting point for optimization

First, let’s talk about some of the things that happen when you’re creating a new project or opening a project where your asset Library isn’t already present.

If you look at the Editor.log file, you’ll notice a lot of lines that look like:

  • Start Importing …
  • Done Importing …

This is the way the Asset Import Pipeline leaves a trail of its operations so that they can be inspected at a later point in time.

You can use this information to figure out a certain type of bottleneck, namely Asset Import Times.

When you look at the output from each line, you can extract the following pieces data:

  • Asset path
  • Import duration
  • File extension

Parsing that data, we can then categorize by extension. Once you know which importer has imported which asset, you can aggregate that data into a pie chart showing you which types of asset take the longest to import. For example:

This data can give you a clearer picture of where the bottlenecks for your project are.

In this particular example, Textures, Models and Prefabs are the biggest time sinks, providing a starting point for investigating which assets could be optimized.

The SimpleEditorLogParser

Download this SimpleEditorLogParser sample project and use it as a base for your own parser.

Being able to see which asset category takes the longest to import can help you plan where to direct your optimization efforts. If Texture Importing is the category that takes the longest, then examine the textures that take the longest to import and consider removing textures that aren’t supposed to end up in the final build.

This will not only speed up your import times, but also improve the performance of your Continuous Integration Pipeline if you’re doing clean nightly builds or something similar.

How to profile startup times to track down bottlenecks

Being able to open your projects quickly is important. The minutes it takes to restart the Editor or open various projects throughout the day can add up to valuable time lost.

As a project becomes more complex and uses more features, it takes longer to open. This can be due to a large number of factors, and prior to Unity 2019.3, there was no way to get the profiler up and running while the Editor launched.

A command line argument to profile at launch

Among the several command line arguments you can supply upon opening Unity, the -profiler-enable command line argument allows you to profile the Editor during launch.

This command tells the Editor to begin recording profiling data for the first frame of the application, which is all the code that is executed until the Editor is visible. Using this argument can help you see what happens during startup and what takes time. You can see whether it’s a system in Unity, an Asset Store package, or code specific to your project. This can help you figure out what to do next.

In this capture, you can see that opening this particular project takes ~50 s.

At first glance, it appears to take 43 s to load the AssetDatabase. Upon further inspection, it’s clear that 14 s are spent on calls to OnPostProcessAllAssets. Further down, the code that executes during RegisterScriptsAndTryLoadingExistingUserAssemblies adds up to 10 s, and 5.7s of that is loading the Domain, which is further slowed down by calls to Scripts that have the [InitializeOnLoad] attribute.

This startup data can help you track down performance bottlenecks, and see whether the code is in your project or in Unity itself.

How to find the import result on disk

The Library folder can now contain multiple import results for the same asset, so projects that use the new Asset Import Pipeline no longer have a “simple” GUID (global unique identifier) to folder mapping in the Library folder. 

Files in the Library folder are based on the hash of their contents plus a number of static and dynamic dependencies. This allows Unity to have features like fast platform switching, asset de-duplication, and skipping an import if the hash of an asset is already present in the Library folder.

So this means that it is no longer trivial to find import results in the Library folder. Here is an example that shows you where you can find the import result of “Assets/Prefabs/MyPrefab.prefab”:

public class LibraryPathsForAsset
{
    [MenuItem("AssetDatabase/OutputLibraryPathsForAsset")]
    public static void OutputLibraryPathsForAsset()
    {
        var assetPath = "Assets/Prefabs/MyPrefab.prefab";

        StringBuilder assetPathInfo = new StringBuilder();

        var guidString = AssetDatabase.AssetPathToGUID(assetPath);
        //The ArtifactKey is needed here as there are plans to
        //allow importing for different platforms without switching
        //platform, thus ArtifactKeys will be parametrized in the future
        var artifactKey = new ArtifactKey(new GUID(guidString));
        var artifactID = AssetDatabaseExperimental.LookupArtifact(artifactKey);

       //Its possible for an Asset to have multiple import results,
       //if, for example, Sub-assets are present, so we need to iterate
        //over all the artifacts paths
        AssetDatabaseExperimental.GetArtifactPaths(artifactID, out var paths);

        assetPathInfo.Append($"Files associated with {assetPath}");
        assetPathInfo.AppendLine();

        foreach (var curVirtualPath in paths)
        {
            //The virtual path redirects somewhere, so we get the
            //actual path on disk (or on the in memory database, accordingly)
            var curPath = Path.GetFullPath(curVirtualPath);
            assetPathInfo.Append("\t" + curPath);
            assetPathInfo.AppendLine();
        }

        Debug.Log("Path info for asset:\n"+assetPathInfo.ToString());
    }
}

Get example GitHub gists

Here are two gists for different versions of Unity: 

The samples are different because as the implementation of the Asset Import Pipeline has matured, a number of APIs have been moved from the Experimental namespace to the AssetDatabase’s own namespace.

How to find assets faster

Often, you may want to find a particular asset in your project and do something with the result. You may even want to do this multiple times when you’re running Editor code.

The brute-force approach

Calling AssetDatabase.FindAssets will traverse the entire project to match the query you’ve given to it. As projects get bigger, this can become a performance bottleneck as the time taken to search through different size projects grows linearly.

As expected, the more assets a project has, the greater the time to search through them. Fortunately, the time to find each asset remains somewhat steady over time.

As you can see, if your project has hundreds of thousands of assets, searching for assets can lead to a noticeable development slowdown. At 200,000 assets, there is already a 200 ms delay for a simple search query.

Using the brute-force approach produces this common usage pattern:

string[] assets = AssetDatabase.FindAssets ("t:texture2D");

if(assets.Length > 0)
{
    string path = AssetDatabase.GUIDToAssetPath (guids[0]);
    if (!string.IsNullOrEmpty (path))
    {
        Texture tex = AssetDatabase.LoadAssetAtPath<Texture>(path);
        //Do something with this texture
    }
}

Essentially, this code is traversing the entire project to find one texture and then do something with it.

Scalable lookup

The AssetDatabase provides a way to lookup an asset’s path by GUID. You can think of it as looking up something in a Dictionary by key instead of by iterating an array for a match.

The benefit of using this approach over brute force is that the AssetDatabase doesn’t need to look through the entire project to find the asset. It can just use the GUID as a database index and follow that path to load the asset into memory.

How can I find the GUID?

You can find the GUID in an asset’s corresponding .meta file, for example:

fileFormatVersion: 2
guid: 9fc0d4010bbf28b4594072e72b8655ab
DefaultImporter:
  externalObjects: {}
  userData:
  assetBundleName:
  assetBundleVariant:

In this case, the GUID for this asset is 9fc0d4010bbf28b4594072e72b8655ab.

Given that information, you can do the following:

var path = AssetDatabase.GUIDToAssetPath("9fc0d4010bbf28b4594072e72b8655ab");
var asset = AssetDatabase.LoadAssetAtPath<Texture>(path);

Now your asset is ready to be used.

Speeding up search with Quick Search and TypeCache

As a side note, if you’re interested in speeding up searching within the Editor, you should also be aware of the following tools:

  1. Quick Search Package which allows you to search over multiple areas of Unity.
  2. TypeCache to search for Scripts derived from a type you know.

What to do if the GUID changes

Normally, the GUID of an asset is constant.

However, in certain situations, an asset and its .meta file are duplicated, causing a conflict which the AssetDatabase resolves in the following ways:

  1. If you duplicate a folder inside a project to another place in the same project
    1. All the .meta files in that folder will also be duplicated, all of which contain a GUID corresponding to the asset on the same folder
    2. The AssetDatabase detects this as a conflict and gives priority to the original GUID, rather than the GUID for the asset which it is currently importing, and assigns a new GUID to the conflicting .meta file
  2. Importing an Asset Store package multiple times or copying a folder from another project multiple times into your project
    1. This has the same effect as #1 and is also quite common
    2. This has the added problem that an asset refers to the original GUID instead of the new GUID, and references are not updated, which can lead to unexpected behavior

Things that lead to slow Refresh

When AssetDatabase.Refresh executes, a lot of systems need to work together to present your project in a valid state. In my Unite Copenhagen talk, I detail the various steps of what happens when Refresh is called.

This content is hosted by a third party provider that does not allow video views without acceptance of Targeting Cookies. Please set your cookie preferences for Targeting Cookies to yes if you wish to view videos from these providers.

Particular callbacks executed during a Refresh interact with code. This can affect how long it takes for the Refresh operation to complete.

Pay attention to callbacks that happen during Domain Reload

The more code that gets executed during a Domain Reload, the slower your Editor experience will be. To stay in the flow when iterating on code, think carefully about when code should be executed and whether it can be deferred to later in the project.

During a Domain Reload, your code will be executed if it contains any of the following methods:

  1. Awake
  2. OnEnable
  3. OnValidate

Your code in those methods should ideally be very fast or should be deferred to run at another time (not during a Refresh, for example). This is because these callbacks are supposed to help you restore certain states, but since there are no restrictions on what can be done within these calls, then any code that doesn’t scale (i.e., anything that traverses the entire project) slows down your iteration velocity for Scripts while the Editor is open. 

Another approach is to use EditorApplication.delayCall, where your code gets executed on the next Editor tick after the AssetDatabase has had a chance to detect and import all changes on disk.

Follow this thread on the Unity Forums to stay up to date with news about improvements in this area.

Conclusion

I hope you’ve found these tips useful. Let us know what other things you’d like to know about the Asset Import Pipeline or what your pain points are. We are actively working on improving the Asset Import Pipeline, and we want to make your iteration as close to instant as possible, so you can be more productive when working with the Editor and changing assets and/or scripts.

October 6, 2020 in Engine & platform | 10 min. read

Is this article helpful for you?

Thank you for your feedback!