Game of Sites: Unity Between Five Kingdoms
In the beginning, there were many houses. Though they began life at the same time, they were in different parts of the land, and thus had different ideas about what was best. As these great houses often do, they grew apart, and tensions heightened. When the king decreed “there must be consistent banners across the houses,” they all decided what that meant on their own, and were satisfied that the banners were consistent and true. They were not.
Though they had all risen their castles from the same Angular stones, and had used the same engine to power their forges, they had developed differences over time.
And so, a handful of brave knights were tasked with the impossible: uniting the houses. This is not their story; it is the story of how the great houses became greater together.
The First Quest: Unite The Four
Gather round the campfire for a rousing tale! Allow us to regale you with a story of victory. A story in which the wicked wizard Bad User Experience and his underling The Tech Debt Collector were vanquished by uniting different sites under a common goal. Are you interested in web development? Do you use our Developer Dashboard? Do you like stories about deleting way more lines of code than writing them? Then join us on this adventure!
So when this whole thing started, the Cloud Services team was responsible for owning and maintaining four separate nginx sites in one location and a node.js-based one in a second location. Though hosted in multiple regions, the teams managing them were separated between Seattle and Austin. They covered core things like managing your project settings, users and access levels between the two, and four services: Unity Multiplayer, Performance Reporting, Collaborate and Cloud Build. The other services — Ads and Analytics– were hosted on their own platforms in other locations, so we’ll just focus on those first five pieces.
It was hard to keep things organized across five separate repos. As a result, the user experience was similar… but not consistent. Buttons looked slightly different and were in different spots for different products. Fonts were the same family but sizes varied greatly from place to place. There was no consistent information architecture: no agreement on what level of headers and body content were appropriate. This was bad for us, obviously, but it’s also really bad for our users.
Multiple things had been tried: a trove of shared components that each could draw from; a singular definitive styleset that all sites used by importing a large compiled stylesheet as well as one set of visual assets. But no matter how diligent the teams were, inevitably someone would be forced to copy code from one repo to another. Sometimes, they’d need to tweak two components such as a site header so that they looked the same, even though their markup and styles were drastically different. The goal was to combine all five of them into a single codebase. The design team for Cloud Build had been pushing for a very long time to get all the sites — including ones not mentioned here — into a consistent visual language and was relieved to hear there were finally others wanting to do the same.
Unfortunately, the timing didn’t work out as well as expected. The Seattle group had just finished a major support push of Collaborate and had some time to really focus on their web experience. At the same time, the Cloud Build team had promised a lot of great new features on one of their products and didn’t have the cycles to try and convert their more complex node site into the Seattle nginx sites. It was decided that the four sites already owned by Seattle would be combined.
Each of these four sites was already an Angular 1.x site, so the hard part was just finding a way to make sure that all of the pages lined up somewhat well, and that duplicate styles were simplified. Combining sites that have existed for a range of one to three years ended up being more complex than originally anticipated, especially in a large single-page app. For example, the latest version of Collaborate included brand new features that other sites were missing.
One of the biggest challenges was combining the different ways views are resolved. Some of the views were put together something like this:
While some of them were more like this:
This was complicated by the fact our sites were on different minor versions of Angular. And in order to combine with Cloud Build in the future, we’d need to convert from the default path router to the new and shiny UI-Router. This allowed us to have separate components for parts of the page without a million controllers. It also gave us the concept of states and state transitions so we could manage things like authorization much more easily. Of course, upgrading to UI-Router meant putting all our codebase on a single version of angular, and ALSO upgrading everything so it worked with the latest version of UI-Router. This was also taking into consideration the fact that we’d have to tweak our CORS headers so that the previous four sites were all served from one site but still hit the same internal subdomains for all our AJAX routes.
As is true in most forms of code, but especially web dev, we started solving one problem and ended up solving 12.
In the end, the way we solved this was with a healthy dose of fire. In this initial combination, we were able to remove about 1200 lines of redundant code, and orphaned another 2000 lines of abandoned routes and code paths that were no longer used. While we were at it, we made sure to improve our unit test coverage, and ended up going from around 34% coverage of legacy code to around 65% coverage of the new code. Security now was able to give us a pretty thorough review since everything was consolidated in one place, and we buttoned up a fair amount of glitches and holes that could have been exploited. While doing this, we also updated our bootstrap themes to match the new directives from the Unity brand: new logo, new colors, the whole nine yards.
We felt like we’d accomplished something great. But for better or worse, we weren’t done yet.
What were once four great houses was now one magnificent house. Knights from all over the kingdom came to inspect the mighty new castle. It was more sturdy than any of the four had been, and its doors were open to all knights of the realm. Still, they took clear note of the new parapets and murderholes above, cautioning all to mind their manners in this new home of the great unified house. Yet all assembled could not help but gaze to the South, where lied the House of Clouds. It was a proud castle built in the rolling hills and prairie, but it looked lonely there. They served all who came diligently and with great fervor, but they were frustrated the new great castle had been reinforced while they were busy serving the citizens of the realm. Were they ready to join the unified house? Only time would tell…
The Second Quest: Securing An Alliance
So after we got everything from our location combined, we came to Cloud Build to ask how we could help them combine our sites. At this point, anyone working on any of the four services mentioned above (including the new Collaborate website) needed to test their local changes by installing a local version of nginx, which then needed a set of config files not only to map the source of static files but also create local proxies to hit all the service endpoints in such a way the CORS endpoints could be resolved. Conversely, Cloud Build ran a node site which hosted files through express, so they just had to fire up their node process and everything would be configured. Of course, combining these two approaches would prove to be a complex burden of epic proportions, and a series of meetings indicated no one was quite sure the best way to approach it. So, of course, we had to try something.
We created a new git repo, set both repos as remotes, and started merging. This made sure the new repo would maintain the history of both sites, and we could collapse obvious redundancies. The only problem was that these two codebases were never made to be aware of one another; their file structures were very different, but often collided on certain names or pieces of code. With judicious use of git mv and namespaced folders, we were able to get it down to a mere.. 20 stages of conflicts.
It took us about three days, but by the end, we had created something. Some called it a Frankenstein. Since we used living sites and not a pile of corpses, I preferred Chimera. We tried to stay away from the centipede analogy.
But in the end, we were serving all of the routes of the individual sites through one single-page app, and had reduced somewhere around 3000 more lines of redundant code. In the process, we became much more familiar with how the Cloud Build code worked, and how they integrated their builds and support forms with the website for a pretty beautiful and seamless experience. This gave us the ability to provide a Collaborate support form that gave users direct access to our support team through the website. At the same time, Cloud Build was able to consolidate some of their older logic with the shared views, and rather than having five different ways to look at your project, we settled on just one component with access to each of the different services.
Quest To The Future: The Neverending Quest
Now that we have combined our powers, we have continued to iterate on a variety of new technologies. We’ve updated to the latest version of AngularJS (1.6), and we’ve converted large portions of old confusing nested controllers into simple and reusable components. We’ve also increased our test coverage, and have begun a series of steps to make our quality higher than ever. On top of that, we’ve created a common library that combines a common design language with thoroughly tested and documented components. Any team at Unity (who is using Angular) can potentially make use of these components, reducing duplicate effort and inconsistency across the entire company.
What does that mean for you?
Well, primarily it means that finding and fixing bugs has gotten significantly easier. What took days to debug now only takes hours. That doesn’t mean we won’t have problems, or end up somehow stumbling into confusing and not useful user interfaces on the dashboard. Our customers being people who make software for a living is a wonderful double-edged sword: they know exactly how hard it is to make everything perfect, but they know exactly how to find broken things and expect them to be fixed. This is great motivation for us, and we love hearing feedback from you on what works and doesn’t work on developer.cloud.unity3d.com.
Now that we’ve equipped ourselves with the tools to succeed, we can’t wait to show you a better experience every time you use the site — including an all-new layout designed to make use of our latest Unity branding and usability guidelines.
We hope you love using our products and services, and we really do care. And we know that you care too! So let us know via twitter, or in the comments below: what do you like? What don’t you like? Let’s make Unity better. After all, democratizing development means (if you’re doing it right) that everyone gets a say.