Search Unity

Accelerate and enhance creativity with an example-based workflow

, 四月 27, 2020

When rapid, truly disruptive progress occurs in an industry, instabilities are created – and unstable systems don’t last. We see many examples of this everywhere, but most of us rarely think about what it means and how this kind of instability might be taking place within the computer graphics industry. 

Why an example-based workflow?

Computer graphics is rapidly progressing, which leads to an unstable supply and demand market. The demand for 3D content to consume has long outpaced our ability to produce it. We see many symptoms of this inequality: exponentially growing team sizes, ballooning project budgets and durations, and dependence on distributed teams and outsourcing. These are cracks in the industry’s foundation. 

The rising cost of game development is exponential.

Thus, we’re making a case for a new paradigm in content creation, a shift in mindset as well as in the technology stack: the example-based workflow.

What is an example-based workflow?

The philosophy behind an example-based workflow is that we can’t keep painting and programming every pixel. Rather than starting from a blank canvas and building everything from scratch using a manual or procedural workflow, this workflow starts with an example of what’s needed, such as a scan, a photo, or an older asset from a previous generation of media. Starting with this raw example, the computer requires data, high-level instructions on what to do with that data, and then it can begin to automate an otherwise manual process.

The example-based workflow

To make things simple, here’s a list of the three ingredients that make up an example-based workflow:

  1. Start from seed data or an example of what an artist wants to create
  2. Define the intent
  3. Automate
  4. OPTIONAL: Refinement through human feedback, return to step 2

This might sound unconventional, but the industry is already using example-based workflows today. Although it’s a niche use case, photogrammetry, the conversion of 2D photos into textured 3D models, can be considered the first wave of example-based content creation because it leverages data to automate the 3D modeling process. Unity sees AI-Assisted Artistry as the second wave.

When Unity began to explore photogrammetry and how that would fit into their platform, it was a small team on a strict timeline for this project. This necessitated adopting the most efficient methods possible – brute force wasn’t an option. Most studios see scanning as a means to create highly photorealistic content very quickly.  Studios are investing in scanning equipment and associated software such as Reality Capture, which has gained an excellent reputation. That said, photogrammetry is not a full turnkey solution, since manual cleanup and tweaking are needed to go from raw scan to production-ready asset. This is where Unity had to develop and explore other solutions – leading Unity technical artist Cyril Jover to create the industry’s leading Delighting tool, designed specifically for photogrammetric scans. As part of this effort, the team adopted ArtEngine for automated seam removal and the general scan-cleaning process.

Microsoft technical artist Pete McNally is similarly charged with the task of building realistic environments with a small team. These constraints led Pete to become an industry pioneer in adopting and utilizing an example-based workflow. Pete shared his full thoughts on the topic at 80lvl:

“There are already some solutions for removing seams on materials but I haven’t found anything to match ArtEngine by Artomatix. It uses an example-based approach, rather than procedural to remove seams and can also mutate textures, creating new textures based on input from existing ones. For example, I used Artomatix in the Сraggy Cliffs:

Mutation and seam removal on photogrammetry scans courtesy of Pete McNally

It has given a new lease of life to some 3D scans I’ve done that were too problematic to be used in production, such as textures that were too blurry or geometry that was stretched or had holes in it. ArtEngine can fill in missing detail, allowing you to paint areas that you’d like to ignore and producing new viable textures.”

So that’s what we mean by an example-based workflow: the computer extrapolates and ideates upon example data, so the artist provides high-level sophisticated creativity and the computer contributes the low-level mechanistic creativity. It’s a perfect symbiosis between artist and tool, each empowering the other. This isn’t just a necessary evolution for the computer graphics industry, it’s an inevitable one.

How does it work? Enhance, expand, transmute

You’re probably wondering how you would use an example-based workflow in practice to streamline your creative productions. We’ve identified six pillars to the workflow, three on the data side and three that automate. While each of these pillars warrants their own blog post, I’d like to briefly touch on automation by highlighting the three tasks that should no longer require manual intervention, since this is likely foreign to many artists. 

Enhance

Enhance is a family of ArtEngine tools that help you to improve your visual creative assets. The industry is sitting on mountains of old, outdated 3D content. The standard resolution for textures doubles roughly every five years. At the same time, original master files are frequently lost, leaving behind only downressed and compressed production assets. Even assets produced by scanning today can suffer from blur and noise artifacts. ​Enhance takes assets and improves the quality in some way, extending their value and lifespan. One ArtEngine feature from the Enhance family is Up-Res, ​which can perform a ​2x ​or a​ 4x cartoon or photorealistic resolution improvement on any image or texture.

Expand

Going from a concept to filling an entire world is a long, tedious and largely repetitive process. Expansion was the first problem Artomatix set out to solve, and it’s still one of our core goals. We’re proud to support a number of expansion-based features in ArtEngine today. Seam Removal can make any texture or material tileable. Content Aware Fill can fill in any damaged or missing parts of a scan. Texture Mutation ​can grow a material out to a bigger size without creating repeats or generate infinite seamless variants from just a single example.​ Expansion brings the richness and variety of the real world into the digital realm.

Transmute

Often 3D content is unfit for the task at hand – not that there’s anything wrong with the content itself, it just doesn’t work with the rest of the project. Perhaps a material is in the wrong artistic style, or a PBR scan just doesn’t look right placed next to other scans due to scanner miscalibration. If putting together a 3D world is like assembling a puzzle, ​Transmutation ​is any function that helps the artist when those puzzle pieces don’t fit together other out of the box. Many real-world objects such as cloth and brick walls are “near-regular patterns,” this means they have a pattern, but it’s warped or inexact in some way and can’t be used as a tileable texture. The Pattern Unwarp ​ feature automatically detects the true pattern and unwarps it, transmuting ​it into a form that can be tiled. 

We’ve briefly introduced the three pillars of automation, or the three ways that artists can use an example-based workflow to transform their 3D content in the future.

Watch the Unity ArtEngine session

If you’re interested in learning more, watch Artomatix founder Eric Risser and Mike Geig, Global Head of Evangelism Content, show you ArtEngine’s potential for your projects. In this free one-hour online webinar, they demonstrate how to take full advantage of an example-based workflow with the help of ArtEngine’s features such as Texture Mutation, Seam Removal, Pattern Unwarp, Material Generation, Texture Upresing, Color Match, JPEG Artifact Removal, Texture in-painting, and more. Want to try Unity ArtEngine right away? Get started with our 30-day free trial.

 

Watch the Unity ArtEngine session

12 replies on “Accelerate and enhance creativity with an example-based workflow”

Not buying the claims here. The graphs don’t look exponential and there are no sources cited. In the left graph most of the data points are similar with only Grand Theft Auto V being an outlier. Sorry but that doesn’t indicate an exponential trend. In the other graph, the source of the data is important and isn’t cited. Are these means, medians? Where is the data from? This blog seems false and not fact checked by anyone.

If the cost of game dev is increasing exponentially and this data is from before 2015 then the costs between 2015-2020 would have grown much higher? It doesn’t make sense.

I think it’s funny this arguments about how exponential is the amount of artists are needed to produce games now a days, as if this was a bad thing… We live in times like never before, where you have some amazing indie artist producing great content in very small teams the AAA side of things is not the only viable side anymore.

On this blog from March 10, there’s a comment towards the top where a user says “will it be free for Unity users? Kinda like Unreal did with Quixel Mixer and Megascans?”

The answer was, “Now that ArtEngine is part of Unity, we are looking at how to make the key benefits of the software available to as many creators as possible. More on this to follow in the coming weeks. Watch this space.”

But I don’t see any other info.

https://blogs.unity3d.com/2020/03/10/artomatix-joins-unity-bringing-ai-assisted-artistry-to-material-creation-workflows/#comment-420913

Oh. Finally old good posts that are really interesting and useful.

It would be a cool idea to implement photogrammetry in Unity Remote, so the mobile app would use AR to scan geometry and send models and textures to Unity engine.

I found the blog interesting especially with Unity acquiring ArtEngine. I just curious how exactly it works and how it will be rolled out for Unity users. I see with MegaScanes going to Epic Game the focus is less on prepping Unity asset , getting a megascan asset into URP requires some work

We are looking at integration options, but in the short term ArtEngine will be a stand alone software. Stay tuned for more information on this in the future.

Are we ever going to see blogs that are not advertisements again, or is that phase of Unity Technologies over?

i whish they bring back some monthly tutorials, how new things in unity can be used. it feels like some sort of spam mail to see this here

Comments are closed.