The Next Leap: How A.I. will change the 3D industry - Andrew Price

so show a hand who here makes money from somehow inside the 3d industry they got a job at a studio yeah almost everyone who doesn't okay what are you guys doing it knocking so I love this job it's it's really it's fun and that that childlike wonder of like yeah bringing something from my head into life it's still that I just I really enjoy it but I want to continue doing this you know because it's it's a fun job but if you've paid any attention to the news and you've no doubt seen reports of automation and AI the doomsday theories that we're all gonna be out of jobs all sitting on the street nothing to do and I always thought like yeah but that doesn't apply to 3d because like what we're doing is art and computers could never replicate art but then you see a couple of stories like how machine learning was used to create high fidelity animations on the face of Thanos okay all right and then you see an algorithm that'll take the styles from famous paintings in history and apply it to a photo and then I'm really sitting up and paying attention because I'm like okay that I didn't think a computer could do that and so I wondered like what what are we looking at here like it it kind of made me wonder if this is you know the old days of the Disney 2d animators being replaced by the 3d animators are we gonna be replaced by server racks running AI and machine learning to auto-generate the movies so that is what I'm gonna discuss in this presentation how a I and automation might change the 3d industry quick sidenote I'm going to be using terms like AI and machine learning and some people are like no that's not AI you've got to call it this and for all intents and purposes it doesn't really matter for what I'm talking about at the end it's still the same result right it's a software doing something that an artist does so anyways there is one guy who's very good at making predictions for the future but yeah Masanga Jeff Bezos who's you know in hot water at the moment for his treatment of staff wasn't very good at predicting but for the last 20 years he has been basically ahead of the industry and so he said I frequently get asked what's gonna change in the next 10 years I almost never get asked what's not gonna change in the next 10 years and that's actually the more important of the two and he went on to explain that you know an Amazon customer of the future is never gonna say I wish the prices were higher or I love Amazon but I wish it took longer for my packages to get here right so those two variables he knows for sure are going to be desirable in the future so actually what's not gonna change is what you should be focusing on not worrying about hypotheticals and so I thought that's that's great it's a great exercise and applying that to 3d I think that any technology that makes things better faster or cheaper will eventually become standard it's inevitable things get rolled out once the studio execs realize that this is going to save them money they put it into practice so what is costing money at the moment games are costing money so Raph Koster plotted some games from 1985 to today and he found a disturbing trend that every ten years the cost of games go up ten times so this is a log scale it looks deceiving like it's not that big of an increase but every single one of those horizontal lines there represents a 10x increment on the one before it so basically every year it goes up 25% by 2020 the average triple-a game might cost 200 million dollars so they're already more expensive than feature films and it's getting unwieldly because they all have to spend more to try to out-compete the others and mobile gaming yeah it started out cheap but that's happening as well because it's now saturated space so the costs are way too high and they need to come down a big portion of this costs is assets assets are unreasonably expensive so let's say for example you're making a video game that involves the street you've got to have characters running down the street for whatever reason so you've got to model a building okay and it's a pretty detailed apartment so you work pretty hard you might be able to get it done in 12 hours okay then you got a texture at texturing often takes just as long as the modeling but let's say you get it done in 10 and then halfway through the workweek you've got it done in 22 hours which by the way is unreasonable because that assumes you're productive 100% of the time it's usually more like 50% of the time but let's say you got a 100% productive then if you're in the studio there's also a revision multiple of two to four times you know narrative changes maybe they want to make it take place in Paris instead of New York maybe they want to have a tunnel running through the building for whatever reason so changes have to happen repeatedly and they have to go back and revisit old assets so at a cost of $60 per hour the average wage this building ends up being about three thousand nine hundred dollars so when you look at a game like the division put price tags on everything and it's very easy to see why games are costing hundreds of millions of dollars it's and and this is kind of kind of dumb right because like we're working virtually none of these stuff exists in the real world like we should be able to approach this smart smart early because I think that a large portion of this cost comes to the fact that this is a static workflow you're getting a one-for-one input to output ratio if you want to make another building you often have to repeat the work and sure you might be able to you know start from the base of the other but you often have to do so many details in topology change and then you've got to read texture it so you're still getting a pretty similar similar output so I think that the first leap that's going to happen to the 3d industry in the next five to ten years is procedural work flow is becoming mainstream or the standard to say so approaching the problem of modeling a building to do it procedurally would look like this instead of modeling than building yourself you would set the parameters for what a building should look like within certain ranges so our building should be this high to this height it might have between five to ten windows on it two to three doors it could have this many floors in range and then basically you let that's off where self-generate and this is how you bring those costs down so this was actually a course taught by Anastasia opera I believe is actually Dutch could be wrong but she made a procedural lake village using Houdini and she went on to explain that she thought that procedural ISM would really remove the creative aspect of it but actually it forced her to understand like what makes something look good because you often don't know as an artist you just like doing things without thinking about it but when you had to put it into a program you had to explicitly define what that looks like and so that was a good exercise and then also what you got out of the machine at the other end was often ideas that you hadn't thought of before so it can actually be a huge advantage so this is procedural modeling obviously you've gotta then texture it in this example it's textured I'm not sure how she did a button texturing so I run polygon who he has heard of polygon alright my advertising worked so when we started polygon two years ago every single texture on the website was captured with a camera and we would find a floor like that and then we said great we need another floor we would have to find another wooden floor capture it make it seamless process it so it was that one-to-one input to output then we discovered substance designer whereby you would spend a lot longer generating a node set up making something 100% procedural meaning that no camera was involved whatsoever you would make it digitally but once you've made it once it's very easy to create variations from that and so immediately we realized the benefit of this not only could we capture things that are very difficult to capture like marble wood has you got to go into people's homes and capture their floors but also it was a huge cost saving so basically that's all we do now we we a lot of our materials are now made with substance designer say for photo scanning which is still best captured for four grounds but yeah we've doubled the size of our team and game studios have realized this as well which is why you look at job listings they all hiring substance designer artists then of course you've got to apply this to a model okay mortal will have you know creases and crevices and all sorts of occlusions that you've got to take into account for so algorithmic was very good by making his sister software substance painter which enables you to take the thing it'll bake all the maps for you and then it'll apply the smart materials that you got from substance designer smart masks and things to add in grunge in and whatnot so it's the number one texturing software in the world right now for this very reason it is saving studio's hundreds of thousands of dollars because it's procedural when you go back and you change something with the model if you've got the pipeline right it should auto update with the texture so very very cool and then finally you've got the level design so putting the assets into it so I had the D as the start and New York City example but let's say you're making a forest okay so far cry 5 they've made a lot of FARC rise five of them in fact but previously they went about you know like you would make your forest usually like you have particle systems and you place things by hand but the problem was was that as the story would change over the months the landscapes would move around as they had to have roads go through different places and every time they did that they had to update in like hand place and remove all of the trees this time they did it a different way they created an ecosystem so they set rules to define West certain trees and plants would live so big trees would grow towards the center of the forests small trees towards the edges underneath the trees there would be ecosystems of like smaller plants that were growing the shadows it was near a lake there was going to be different plants there it would automatically update according to the altitude so if tire up there would be smaller trees and then they are able to build in handy tools into that as well to quickly build roads and and buildings and the whole talk is definitely worth watching by the way if you get a chance so this I think is gonna be the future this workflow procedural modeling materials texturing and world building all rating materials and texturing are pretty well embedded into the industry now it's pretty well standard but I think modeling and well building are going to be taking the next spot at the moment Houdini seems to be really good at this but I'm really hoping that the jocks wherever he is yeah there he is hassle him about everything nodes because I really hope that this can come in to blend out so that was like number one was that leap number two machine creep okay so this is where it starts getting interesting so traditional software typically work like this you have an input so you've got a photograph and you bring it into Photoshop and then you want to apply a filter to it you apply the action to the photo and then you get one output it's very predictable you know what you're gonna get and also leads to obviously a lot of manual work tweaking things as you need machine learning is different you start with an input it might assess it you might apply an appropriate action and then most importantly compare it with others from its training data set then be able to judge whether or not it is good or bad sometimes a human has to do that step if it's not try again and this by the way is a very gross oversimplification of machine learning I'm not a computer scientist I didn't go to university but this is what's helped me as an artist to understand this this process and basically the result that you get out of it is usually multitudes multitudes better than traditional software so the key point here is that it has the ability to learn and to improve over time now what it needs in order for that to happen is huge datasets and fast hardware so I think like five years ago you started you know hearing the the train of news about machine learning taking all our jobs and everyone was hyped up and scared and then nothing really happened and I think a large part of that is because it's not a lot not enough data and also not fast enough hardware I think now we're reaching the tipping point and actually you're starting to see some consumer software that is making use of machine learning already which I'll show you in a second so one thing machine learning is very good at Dien noising okay so everybody's familiar with noise you render something that's grainy right and D noises typically will smooth it out a machine-learning denoiser will do it really really well so this is one which is owned by nvidia i assume it's the same one that is used for the RTF which is enabling them to do real-time ray tracing that is by the way how their r-tx is able to do that it is rendering one sample every single frame of the game and then it is applying adi noise on the same frame so it's doing this in real time and how on earth it's able to take that blue square that noisy blue square and read anything from it is a testament to how machine learning is is working like that's just absolutely insane that it's doing that so blenders cycles denoiser as far as I know has nothing to do with machine learning or AI and so it obviously would very much fail in this situation which is why I really think blender needs to get into the inter machine learning and AI Disney and Pixar also own one and they're trying to solve the problem of the frame flicker because that's a common problem with D noises but also building artists tools to do it Nvidia own most of the papers regarding it and I think that's because they realize there's a huge amount of money in it because not only every renderer will need to use their denoiser but also every camera manufacturer because if you've got low light sensitivity and there's all this grain well they can solve it with D noises so that's one thing that's good at another thing is appraising so you've got a small image small little JPEG you fine you like art but I need it to be double that size this is uprising so as a test I took this image which I'm making at the moment of a kitchen I rendered it at 50% and then using AI gigapixel from topes labs i upraised it by 200% so that it is now as if it was rendered at 100% and then I compared it to an actual render it 100% and if you look at it there's not a lot of difference there and obviously there's a little bit more detail in the one that was actually rendered 100% but not much and certainly not enough to qualify like a four times extra rendering so this was like really caught my eye I was like wow and this is consumer-level you can actually buy this get the trial or it's 99 dollars or something like that to purchase the full thing this is already out there and this is a start I think of where the industry is going and there's also other uses like for example motion capture so I think we're gonna be doing away very soon with the mocap suits and the expensive studios because now you can capture just from raw video so there's no censors this is not a special camera this is just simple video and the algorithm whatever it is I don't know what I'm talking about was able to figure out where the bodies are I'm most impressively it's able to guess the occluded part like what's on the other side of that arm because it can't see that but it's guessing and it's doing an amazing job at it so I think it's in the future they're not gonna have mocap suits I'll just film the actors doing whatever and that's gonna be it and then there's this other one which to be honest I don't know how it works but they filmed a dog in a mocap suit for an hour doing running and jumping and then using neural networks whatever that is they translated that into player motion in a game and what's most impressive is its transition from one behavior to the next going from running to walking to jumping is incredibly seamless there's also very little foot sliding and there's also another example with a human like and it adapts to the terrain and it jumps over things I think this is inevitable I think you're gonna start seeing it in games so that's machine lying basically we're just gonna start seeing it in our software in the future you'll start you know you'll have an action in Photoshop and you'll be like oh that looks really good you find out it was machine learning built into Photoshop Premiere we most Autodesk products I think it's really I think blender needs to really get into it I think it's the future and I think every every Silicon Valley company has realized it and yeah there's this quote by the guys who made the famous facial animation if you're not using machine learning in your software you're doing it wrong I think that's very true so elite number three is machine assisted creativity this is really fun because this is something that I always thought that you know computers would never be able to do like creativity like that's something that is very human and it's true like intent is very hard for a computer to but you would be surprised at what it can actually do to help you okay so as an example as I mentioned I'm working on this kitchen scene which I wished I could have got the tutorial app before I came to Amsterdam but I ran out of time so it's gonna take another month but I'm working on this scene and I'm you know whenever I'm making something I reach a certain stage like this where it's okay but I know it could be better if I tried more ideas so you just stop throwing stuff at the scene and seeing if it looks better than what it did previously so you might change the lighting add in some little blinds and might be a marble on the left you know and then you try it with like adding food to the table no food adding some boxes no boxes refrigerator instead and you just have to keep doing this and what I've realized that as my workflow this really eats up about 50 to 70 percent of the production time because and there's a bunch that I haven't shown you like 20 other renders where you just you've got to try this stuff and it takes so much time sometimes you have to model the thing like what's gonna look good with a wine bottle I have to model a wine bottle or you purchase it or whatever then you place it then you have to render it and that is a really really long process so if there was software that could do this for you without you having to go through all the effort to add it in there in order for you to make the decision if this is good or not that would be incredibly powerful right so there is hey so this is a really impressive paper you give it an outline of an object the photo that the outline was based on and then it will generate a bunch of ideas for you and it works really really really well so it actually helps to see it as it's generating so this is a building facade the only thing that was given was the top row and then this is the shoes and if you paused it at any point in this any one of these frames here could be a unique design so I think that this is kind of the future I think what we're going to see is you'll make a character for example and then you'll put it into the software and you'll try out a bunch of ideas different clothing different types this is one use for an environment different sceneries and what's crazy is that the input that's given to it is so simple and yet it's just churning through ideas and spitting it out and I think this is absolutely gonna be part of a lot of creative meetings in the future this is actually one you can try online if you go to that web address there it's based off a different paper technology but very similar thing so I drew this cat alright very simple looking cat and then I hit a button it reminds me of those things where it's like you know like the dad finishes the the children's drawings and he's like a really good artist and he just makes it look - ality obviously doesn't look very good that's cuz my drawing is crap but you could obviously see like this is it's all gonna be a finished product but this would definitely be a starting off point for a lot of concept artists so I think this is it's a sign of the times we're gonna start seeing this speaking of which does anyone recognize these celebrities does anyone know who they are you don't want to throw a name at me any name you think it is they look like you they look familiar right you're like music it's like a football pretty so I've seen her in something either she's a musician or an actor well I see their imaginary they don't exist they were generated this is a separate paper whereby you feed it a bunch of images and it spits out a result really really impressive so obviously it realistic examples you could get by you know if you've got like a game when you need to have 20 different NPCs and you don't know you just pull photos of people that match the region that you're creating the game for I need 20 NPC Mongolians you just downloaded a set of Mongolian faces and then you just got unique faces there's no like likeness infringement or whatever you've just got uniqueness but it also worked for other things I don't know how it did this because these photos have perspective in them but this was also generated these bedrooms there which is bananas that's exactly what I mean for my kitchen right if you could just put it in and then generate a bunch of things so I think that's what's gonna happen he's gonna have your mood board of all your ideas for your environments and it generate ideas oh and this is where it gets really freaky real spooky for Halloween what if you didn't even have to provide it with images what if you just typed out your idea like this bird is red and brown in color with stubby beak what that's real that's not it's not pulling from a stock library by the way that was generated so amazingly the way this works is that it was trained to recognize what the features are and what something is like red bird okay this is what a red bird looks like then when it noticed when you type that out it creates the blob of whatever that shape should look like and then there's a second pass where it adds in the detail to that and it's crazy like this is like the closest thing to like sorcery I don't understand it in fact like a lot of papers I've seen I'm like very skeptical I'm like they're cheating it's something they're not saying because this can't be true but if it is if it's as good as this is and it's like today I think this was actually from 2016 I think this is inevitable you're gonna have a creative meeting with a director who's like alright let's let's create some ideas ancient city roads through the middle Canyon in the background spit out like twenty different ideas and then one of those ideas you give to your one concept artist to flesh it out I don't know I think it's inevitable right and then another really interesting one is style transfers anyone's seen these by the way stall transfer basically you feed it a bunch of images that you like like I want this style from Claude Monet whatever and then you put an input in this is my photo I took on my holiday and then it spits it out which is crazy and it also works for video which is really cool as well if you look at the paper they filmed like in I don't know somewhere in Germany people walking down the street and it's a painting but every frame is moving and it's so weird and what's most crazy of all is that it actually fooled 39% of art historians into thinking that they were looking at something that was actually painted not come generated I couldn't think of anyone more wanting the machine to fail and an art historian but it worked right it worked so my prediction is that artists will use machine learning to explore new ideas I think that's just going to be the future it's gonna be part of our pipeline it'll first of all start going to studios and it's gonna be in the content of consumers and we're all just gonna go like yeah how I made this scene first of all I generated a bunch of ideas I think that's how it's gonna work so these are the expected changes that I predict will happen within the next five years I think procedural workflows will become standard I mean texturing is definitely already there with substance designer and substance painter I think it's gonna happen with that level design as well as modeling machine learning is just gonna slowly creep its way into all the software that we use and it's gonna do a lot of really tedious technical things and I think we're gonna get creative assistance from machines as well so I can feel the mood in the room it's a little like oh no what is this are we being replaced so people had this thought actually in 1997 when Kasparov was beaten by the IBM Deep machine that's what that's what it was called it was the first time that a computer had ever been a human and people thought like that's it for chess and it might be the end of humanity too it was depressing and they felt like chess is dead and then Kasparov realized that like you know if the computer has access to all you know potential moves of all the previous of hundreds of thousands of games and it's able to work from that it makes it should be fair if the human also has access to that so he created something called advanced chess whereby the human player has access to the same information the computer has and he will listen to the computer and occasionally override it just like you would a GPS navigation when you know something is wrong based on what the computer is giving you and today the best player is a human to machine counterpart a purely AI chess will win in a tournament in 142 games where as a human and machine 152 so there is there is a purpose to your life it's not just all machines and what's more so is that since then the number of grandmasters in the world has doubled so it wasn't the end of chess it was good for chess so I think it's the exact same thing is going to apply to our industry I don't think that these new technologies I don't think it's gonna it's not gonna make artists redundant because you know you give this to a monkey he's not gonna be able to make anything good out of it it's gonna be hit hit a few buttons and yeah it might generate some semblance of an image but it's doing is the human to put intent into it so I feel like the based on these these linings the most at-risk jobs are those that are labor intensive narrow skilled and repetitive the great jobs like mocap cleaner rotoscoping retopo mesh clean up make one hundreds of these jobs basically the kind of thing which is usually outsourced to a iron a floor of workers in India where they've been told to rotoscope these shots from the upcoming Hollywood film where you just try to just throw manual labor at it I think that's something which will definitely be replaced the safe jobs are those that involve critical thinking wide skilled tasks where you know how to use different softwares and different solutions in the right way and niche tasks so it's a 2 or are too small to bother automating so art-direction project management good generalists program is obviously to oversee a lot of the scripts and the programs people are using and freelancers because there will always be a market for people who don't want to use any software whatsoever and they want to pay you instead so really the the average jobs are the undesirable grunt tasks it's the stuff that nobody wants to do but we have to because it's part of the job the actual art I think that's safe and failing that if that itself you know wasn't enough to make you feel better the forecasts for the 3d industry is insane it is blowing up for almost every single one of these sectors you're expecting to see you of growth over the next five to ten years and that's not by the way even taking into account VR one report said that 3d rendering and visualization is expecting to see twenty five point five percent growth per year until 2025 compound annual growth or basically double by 2022 and quadruple by 2025 so the number of people at this conference in four years from now it could be double that by 2025 quadruple that I think the 3d industry is is poised to blow up so yeah maybe each studio there would be a reduction of the grunt work the the juniors that they're hiring to do the stuff that nobody actually enjoys anyway but in its place there might be five studios around it so that is it that was anticlimactic wasn't it yes thanks guys

Channel: Blender

Views: 934,142

Rating: 4.930243 out of 5

Keywords: blender, b3d, blender 3d

Id: FlgLxSLsYWQ

Length: 29min 54sec (1794 seconds)

Published: Mon Nov 05 2018

Reddit Comments

[u/drewdus42]

This was great I watched it the other day. Blender guru has great tutorials as well. The ai upscaling program he mentioned is amazing. In been using it a lot recently.

[u/CptAlbatross]

If there was ever a time to have a broad set of skills, the time is now. A sad as it is to see the culling of jobs for juniors or specialist, the cheaper more efficient method will always triumph. That's how the substance suite became king and what will probably put Quixel with their Mixer in the lime light again. As they say, work smart, not hard.

[u/vicabart]

That button on his shirt is driving me insane