Search Unity

The heat is on!

Last year, Unity Analytics released our beta version of the Heatmap system. It arose from a hack week project, which itself came from a customer simply asking me to build it (see what you get when you ask nicely?). As it happens, I write this blog post on my way to another hack week, and here we are getting ready to release a major update to our Heatmaps plugin. This update is a big leap forward on several levels, focusing on both speed and usability. The beta was pretty good. I’m proud of what we released. But we listened very carefully to user feedback, finding several things we could do better. For example:

  • The first-time user experience, frankly, sucked.
  • It took way too long to get at your data.
  • There were many problems seeing the data quite the way you wanted it.
  • And, of course, there were some bugs.

The new release addresses all these issues, speeding you up both as a first-time heatmapper, and as you progress. We may never address everyone’s ideas about the “right” ways to look at the data, but I’m convinced we’ve added many excellent options that will please many.

This post won’t be a tutorial. That sort of thing is covered in the documentation. Rather, I’ll focus on explaining the new features, and why we picked each one:

Installing Heatmaps

As part of improving your first time experience with Heatmaps, we’ve finally pushed it to the Unity Asset Store. You can still download the installer from the Bitbucket repo, of course. Double-click the installer and BOOM! you have Heatmaps in your project, at least to a degree sufficient to immediately take it out for a test drive.

With Heatmaps installed, you’ll see not one, but two new Inspectors under Windows > Unity Analytics: Heatmapper and Raw Data.

 

image04

Let’s start with Raw Data.

Raw Data – at near real-time speed

Hiding under Heatmaps from the beginning has been a secret-not-so-secret service called “raw data”. All this means is that we store each event as it arrives at the Unity Analytics server, then make it available for you to retrieve. Heatmaps fetches that raw data and aggregates it inside Unity to display your results.

The official version of raw data has just been released and here’s the headline: it operates at near-real-time speed. How ‘near’? Let’s call it in the range of seconds to minutes. We think this a vast improvement over the previous system that forced you to wait hours before you could get a glance at your data.

All users with Pro licenses get access to raw data…which means Heatmaps will shift into the ranks of a Pro Analytics tool (though all existing projects will be grandfathered into the Pro tier).

image05

We’ve given Raw Data its own Inspector. By doing this we’re signaling an essential and important idea:

Raw Data is separate from Heatmaps.

While we use raw data to populate heatmaps, Heatmaps are just one of innumerable potential applications of raw data. So put on your thinking caps, and ponder this question: how else could you leverage raw data by displaying it inside Unity? Let us know in the comments below, or better yet: build it and share!

Use Test Data to learn how the Heatmapper works

image07

 

The new Raw Data Inspector fetches raw data. But it can also generate test data that stays right on your computer. Again, this is part of improving the first time user experience. It’s now possible to create lots of data in order to see how the Heatmapper (or any other raw data tool) works. None of this data ever leaves your local drive…it just sits there, ready for you to consume it.

What can you generate? Here’s a list:

  • Random heatmap events
  • “Demo” heatmap events that help explain Heatmap features/scenarios
  • Freeform (non-Heatmap) data so you can experiment with ideas of your own

With the data generation feature, you can instantly create thousands of points, load them into the Heatmapper, and understand the many capabilities that Heatmaps provides.

The Heatmapper itself – superior usability

image06

The other Inspector is, of course, the Heatmapper. Our update brings no less than a dozen improvements to this tool. All of these improvements focus around superior usability.

Custom data location
Your raw data is still stored in Application.persistentDataPath by default, but you can now choose any arbitrary location. This is especially helpful when playing with test data.

Remap color to visualize areas of importance

Color in a Heatmap usually refers to density. That is, if an event happens a lot in one x/y/z location, we make that location appear “hotter” than a physical location where it happened less frequently. This is still the default behavior, but you can now specify any parameter to override density.

Why would you do this? Imagine you suspect that the FPS in your game is low in places. Send Heatmap events with fps as a parameter. When you build your Heatmap, you can now choose to remap color to the fps parameter and further select a calculation to perform on that parameter. By doing so, you can see “hot” colors where FPS is low, helping you isolate where these slowdowns occur.

image00

Remap operations to dig deeper

In addition to simply being able to remap color to a parameter, you might want to operate on the result of that parameter. Take the prior case of mapping FPS. You’re probably most interested in seeing the lowest recorded values (or something on a lower end of the spectrum), to find any places where there are failures. If you’re looking at hit points remaining, the mean or median value might be more your thing. Maybe you want to see the first point, or the last. We’ve got you covered for each of these cases by allowing you to perform operations on points that occur in the same space. Here’s the complete list of supported operations:

  • Display earliest point
  • Display latest point
  • Display highest value
  • Display lowest value
  • Incremental (adds 1 for each instance)
  • Additive (adds all values together)
  • Average
  • Percentile (0-100, Median = 50)

Separate to distill groups of data

This is actually several features that all do variations of the same trick (so credit me if you’re keeping score): separate your results into distinct groups. Want to break apart Heatmaps by level? We can do that. See an individual play of the game? Sure! Separate your iOS users from Android? No problem. Each list appears separately under options, allowing you to break apart your Heatmap data in various useful ways. Here’s the complete list of ways you can separate your data:

  • By user (unique device)
  • By unique play session (roughly equivalent to individual plays)
  • By platform
  • Separate debug builds from non-debug
  • By one or more arbitrary fields (such as ‘level’)

image01

Render real gradients for optimal customization

The original Heatmapper used three color swatches to represeAnt low/medium/high color densities. No one liked that. We now use a true gradient to display your map.

image03

And you can save gradients you want to have available for later.

Mask data to zero in on what matters

Nothing but a handy way to trim your data by location. When you have a lot of points in 3D, it can be nice to hide some of it.

image08

 

Point-to-point rendering for enhanced spatial context

There are times when understanding more than one point can be helpful. Consider an event in a first-person shooter called “LookAt”. You want to know not just where the player was when she looked at something, but what she looked at. A change to the HeatmapEvent now allows you to send two positions, not just one. And you can use point-to-point rendering to display the two points and their relative orientation.

image10

Hot tips tell you what’s behind the data

Sometimes you look at an individual point and you really want to know exactly what that point means. It’s kinda red, and is approximately near some point in your game, but what is it exactly? Hot tips is a tooltip-style addition that lets you mouse over any point and know the precise data that created it.

image02

What will you map today?

We’ve given you what we hope is a powerful upgrade to your analytics tool belt. Let us know what you think. Let us know how you use it. We’re always hoping to build the tools that serve you best, so don’t be shy. Tell us what you need next.

16 Comments

Subscribe to comments

Comments are closed.

  1. James Murchison

    July 11, 2016 at 8:41 am

    Can you render the heatmap data inside a build or do these visualization tools only work within the editor?

    1. Marc Tanenbaum

      July 11, 2016 at 4:33 pm

      Mostly heatmaps is intended for editor-time analysis, but there are instructions (and an example) in the docs for rendering at runtime. Since it’s not considered a primary use case, I’ve not explored it deeply, but you should imagine that rendering large numbers of points at runtime might be performance-intensive.

  2. Thanks for the catchy headline.

    Now I have to watch Beverly Hills Cop all over again.
    https://youtu.be/Fb6e2iBsRjc?t=27s

  3. Is it possible to create heatmaps for 2D setups? Would be fun to know where players touch and hover on Mobile.

    1. Marc Tanenbaum

      June 30, 2016 at 6:43 pm

      Yes, Heatmaps works in 2D. Note, however, that screen real estate is a little different from game space. Game space is a constant, whereas screen space can vary by device. So you might need to provide a discriminator so you can separate out one screen size from another.

  4. 1. Will it work with new Unity Plus – ?
    2. Can it run on WebGL – ?

    1. Marc Tanenbaum

      June 30, 2016 at 6:45 pm

      2. Yes.
      1. Heatmaps relies on Raw Data, which is a Pro feature. So just now it won’t work for Personal or Plus. We’re not actually all that happy with that answer, though, so we’re looking into a way to keep Raw Data for Pro, but create an exception for Heatmaps.

  5. Interesting stuff.
    One idea for example could be marking scene icons with active plaeyrs in them in DAU/WAU to know which levels got traction.
    Or maybe mark weapon prefabs with their amount of usage and …

    Btw do you have any relations with the great computer scientist Tanenbaum?

    1. Marc Tanenbaum

      June 30, 2016 at 6:48 pm

      I get asked about my relation to Andrew T. fairly often, but no, not to the best of my knowledge. :)

      Your idea’s a good one. We’ve been investigating something similar.

  6. Tyler Tomaseski

    June 30, 2016 at 3:21 am

    No worries ^_^

    I’m super excited for the changes, addressed all my problems with the previous version. I might look into it myself, definitely the pros of opening the source!

  7. Tyler Tomaseski

    June 30, 2016 at 1:59 am

    Where would be best to submit this bug?

    “No headers found. The likeliest cause of this is that you have no custom_headers.gz file. Perhaps you need to download a raw data job?”

    Received from generated data. I can’t seem to get it working on anything generated by me or demo data generated in the inspector.

    Would seem the field is returning -1 for the custom fields entry.

    1. Marc Tanenbaum

      June 30, 2016 at 2:28 am

      I got one other bug report like this on the forum. We’re investigating. Would you mind telling me your Unity version and platform?

      1. Tyler Tomaseski

        June 30, 2016 at 2:44 am

        Windows, 5.3.2f1

        1. Marc Tanenbaum

          June 30, 2016 at 2:49 am

          OK. Apologies for the inconvenience. We’ll post when we’ve isolated the issue. (Note that we’re not yet repro’ing in-house, so we’re still struggling with where things are going wrong.)

      2. Getting the same issue as Tyler. I am on Windows 10 64bit, Unity 5.3.4f1

    2. Marc Tanenbaum

      June 30, 2016 at 8:15 pm

      OK. We’ve isolated the issue. I’ll be validating a fix, but here’s the line that’ll do the job:

      HeatmapAggregator.cs, line 188:

      Change:
      tsv = tsv.Replace(System.Environment.NewLine, “”);
      To:
      tsv = tsv.Replace(“\n”, “”);

      The character in question will always be a UNIX line ending, not a system-dependent one. Sorry for the issue, and thanks again for helping to expose it!