A.I In Blender is Awesome!

hello ladies and gentlemen it's Mike here at game from scratch and welcome to 2023 been a little quiet the last couple weeks because well I legit took holidays this year but uh time to get back to doing videos like normal and today we're going to be discussing something that I think we're going to be talking about a lot in 2023 and that is AI specifically artificial intelligence in blender so you see in front of you this is blender 3.4 and we're going to be looking at something called dream textures now this is basically bringing stable diffusion to blender but at the end of the year it got a neat new superpower which are going to show in action in just a minute but first i'm going to Showcase what dreams is all about uh to start so first I have this uh default Cube and this time we're actually not going to sacrifice instead we're going to use it as our example we're going to switch over here into Shader mode I'm going to go ahead and we will just do a cube projection for this guy and now I'm going to go ahead and create a texture for it so instead of applying a texture map from you know the internet or me generating or whatever instead what we're going to use is AI to do so this is using something called stable diffusion we've talked about it a few times on the past in this Channel and basically what's dreams has done is integrated it entirely into the blender operating blender operating system well basically on blender application so you don't need to do anything really special to get this to work what I'm going to do is show you two ways of using it first we're going to go into the image editor this is where you would go ahead and load and open and handle textures like normal hit the toolbar and you're going to find dream there once you've actually installed it now you have to go ahead and install a model I have two models installed I'll show you why for that is later on we're going to go with diffusion 2.1 so this is using stable diffusion for creating the textures what you can do is some very neat stuff so let's say I needed to create a texture map here and what I could do is basically just give it a description a um an old Cobblestone Road all right so this is what we want to do is make this as a texture map so I want it to be seamless uh in the X and Y axis so I'm going to go ahead and say seamless I'm going to say both axis you can say individually just the x or just the Y and kind of that's about the extent of it the other thing you're going to notice here is it's going to create a history of everything that you did now when you're working with stable diffusion the max image resolution it will create is 512 by 512 but there are some tools in here for actually upscaling uh the size of the image that is created I do find once I start playing with these settings though that things get a little bit problematic so I'm not going to in this demonstration so what you're going to do you give the description you make it seamless or not and then you click generate and now this is going to do uh AI magic we're going to see in a second it should uh it should spit out an image right here it's not because I'm doing a demonstration oh there it goes all right so here we go now it is basically generating this image from uh basically going through this 25 iterations and there you see the end result of our Cobblestone Road texture now it is not by any definition of the word flawless and you have to run through this kind of a decent number of times until you get something that you actually like but for quick prototyping or coming up for textures Etc this is a very powerful Source you can also feed it raw images you can impaint or outpaint the end results so if you wanted to expand it in a certain direction or overwrite certain parts of an existing texture using AI you could do those things as well you've also got some fine-tuned controls over here you can actually provide the seed if you've got the seed as you can see right down here I can actually generate the exact same thing from the same phrase over and over again so we do have a history of things we've created if we don't particularly like this version hey just create another one now what you're going to notice is that original version though is still available so for each Cobblestone texture it generated for us we're actually getting a new uh image here so here we see this one is it's actually trash both of these have been pretty terrible so I'm going to generate one more time but I'll show you here as they're generating all of the ones that we've looked at here are avail available in images available there you'll notice it's overwriting it because it's rendering it as I go so let's say this third one is Flawless it's it's not but hey that's the nature of doing a demo I don't get amazing results now some of it also comes from the specificity of your prompt and I'm very bad at AI prompts for creating good results but you can just kind of iterate over and over and over again now Theory this should be seamless it's not always Flawless but it generally does a pretty good time so let's go ahead Shader editor over here we're going to go ahead and add our texture into the scene so texture image texture like so and now let's go ahead we'll find this guy so this is nine four something ends with eight so let's go here nine four something ends with eight that is our texture right there drop it into the color Channel and there you see the end result now it's not perfectly seamless but it's pretty close so what you're seeing is between these edges uh you want it to kind of continue and not not have a weird actually it's it's pretty seamless in this particular case uh and that is nice for tiling and repeatable textures now this again is not a Flawless texture uh but if you're doing quick prototyping you don't want to search for a texture you just want it to go ahead and generate something okay so yeah that's a little flawed at that corner right there again so pretty good across the various different seams in this texture and as you can see you can very very quickly generate textures now the cool thing here is there are a lot of options here so right now we're just doing with texture let's say I want to go into photography here and let's go up here and say um a scary Psych psychotic none all right so what I can do here is now basically specify how this Photograph was taken so instead of giving us you know texture style actually let's go and add one more keyword here a realistic all right so let's go here we do not want it to be an extreme close-up so let's do a medium shot from Custom position uh full color um long exposure shooting is Studio portrait photograph and we will do this in dramatic lighting all right this is not seamless anymore because we're trying to recreate a photograph in this case but you see the specificity you can actually give this for generating an image so it's not just for creating texture map stuff and now once you're happy with that just go ahead and click generate and now this is going to create a realistic scary psychotic nut let's see how good of a job it does right here so there she she comes uh so yeah that that is pretty freaky and again the lighting here is because I said dramatic I could also come in here we could have set this one up a completely different set so I could have gone uh black and white for example and then done um uh fisheye lens and we will have ambient lighting instead and we're gonna get a much different result so let's see what it generates with those parameters and we will use that previous example uh here okay so let's just let that run out over here we'll go ahead and pick there is our that is big of Terror all right so we'll wait for that one to finish so that is ends with 98. uh be this one right here and there you go so if you see if you need to add like a portrait or a photograph into your scene you can create some just absolutely terrifying things now when I started talking about this I said there was a neat new superpower uh that dream texture actually got so what you're seeing right now is the ability to basically use stable diffusion without having to go ahead and download any of that stuff you're gonna have to get a model and all that I'll show you how to do that in a sec but this makes it all nicely integrated and easy to work with the other cool thing that we can do let's go over here start a new scene now this time we are going to sacrifice a cube because well I do every video I'm going to go ahead and Import in uh an obj file so let's go find it uh obj where are you right there all right so this one is in my temp directory it's a simple house model doesn't really matter so basically giving some depth details for us to work with so this is a model uh off of sketchfab I think I grabbed it now the important thing here is going to be the angle or perspective I grab it from and now what I'm going to do is switch into edit mode like here we will grab all of the vertices by hitting a and then again let's go hit n and you're going to notice over here we have dream hooked up here as well now this is going to work a little bit different than the image setting we don't have nearly as many options and what you need to have is a depth based model so basically just go in here and find the depth version I'll show you how you can grab those again in just a second so now we're going to switch instead from uh texture we're going to switch this to concept art and now what we can do is set the environment that we're in so this is this is an environment we're working with it could be a character or vehicle or whatever and it's going to use the depth information that it captures from the model on the scene so it's going to know kind of where the texture based off of how it finds things and now let's show you what you can do with this so I'm going to do a fantasy drawing here and I'm going to say a haunted house in World of Warcraft style like so and now I can just go ahead and say okay dream this this exact same process otherwise the only real key difference here is you do have to set it up to um uh use that depth version I'm going to switch over here to render version you can see the texture being generated here and let's see the end result so boom there is our generated result not the most visible as yet let's crank this so there you see it has created a World of Warcraft style haunted house so let's do something completely different um 1950s house in the style of Norman Rock was it what Rockwell all right we'll go with that one uh and again I gotta come back here select all of the the polygons there now what you'll notice if you go look at the material of the object you have selected uh over uh I'm here I'll get back to that so sorry material over here we're going to notice is it actually generates the textures over there I'm going to go ahead and create a new one and basically create a new set of textures for us so you can switch between the different versions uh but it is creating the materials for us again the results are a little random in terms of what you get back out of this this is using again the depth of the scene so the way it is orientated in the scene is going to be relevant to how it goes ahead and renders this sometimes your results are very good sometimes the results are absolutely terrible the key thing is you're never going to use these in like a final render except perhaps if you're texturing things kind of like far off in the distance because you can get results that are definitely good enough so if you've got the general shape of a car you could say turn this into a car as long as you're going to be looking at it from a fairly fixed perspective you can use this easily for background materials you can also use for quick and rapid prototyping now the texture quality is nowhere near what a human would create but I do think that this is also kind of step one so we're going to be interesting to see where this ultimately leads so here is the end result of our Norman Rockwell style render so again as I'm up close to it so let me go out of that is not ideal uh we have some issues going on here but there well that doesn't look so bad at all so again you could use this for quick rendering of uh concept arts in the background uh it's using a combination of the depth of the object being passed in so it is aware of the various different areas uh but it is not a Flawless texturing Solution by any means yeah but you could use it again far off in the distance works great up close not so great and the other thing about this is it is very depth driven so you see if I go and look it's it's literally texturing it based off the angle that I passed in but still it is just the beginning and again if you're using this just for texturing things into a certain style far off in the background uh it works pretty darn well so this is the neat new feature it's got you can actually do this 3D modeling awareness here so if you want to go ahead and grab this guy well first you're going to want to go ahead and download it it is available on GitHub I do not want to make you my default browser thank you uh you see here it is updated uh pretty consistently the version I just demonstrated is the newest version released just kind of the end of 2022 that is what added the first generation of the uh 3D depth based rendering so you basically grab the release version from here so there are a variety of release versions there this does work with Mac by the way so you can use it uh windows I believe Linux and Mac as well yeah so you got all those different versions the Mac version is for both versions as well uh but the rendering only works on the M1 but you can also integrate with um their Cloud offering dream studio so if you have a key for dream Studio you can actually have it there so that it's doing all of the renders on the GPU in the cloud so if you have a subscription to dream Studio you can integrate that in regardless to which platform you work so really cool project again just grab this ZIP file and then what you want to do in blender is basically just go uh to your edit preferences and you go to add-ons and then you install it like so give it to the path to the zip that you just downloaded and then once it is installed it will be here as dream just go ahead and hit the check mark and then once it is installed there's a few things to be aware of with it so you have to go ahead and install a model it will automatically give you the prompt to download the one model you can download multiple models as well here you can see if you are implemented into if you have a token for dream studio and you want to use the online version you can enter here so let's say you need you do need the depth version to do the 3D version what we just showed right here so basically just come on in here search for depth like so you will find that model right there and then click right here and it will go ahead and download it otherwise when you first install this it will default to the 2.1 this version right here which is what I would recommend to use in the 2D mode so there are a couple of ways to use this guy you can integrate it directly into Cycles on a per frame basis uh we're not going to do that in this particular example but it can actually be hooked in to Cycles itself so that's kind of neat so dream texture stuff it's available right here again goes beyond the scope of what I I want to cover in this particular video but you can have it call per frame to generate some kind of neat compound scenes you also have it available in 3D version as long as you've got this depth one of the depth models available there so you have it integrated into your 3D View and as we saw earlier on it is also integrated into the image editor again all in the tools over here and here you actually get some more options available to you such as AI upscaling so if you want to move beyond that 512 by 512 resolution for your texturing you can do that right there so again it's not creating Flawless textures that scary none was pretty scary to be honest but uh and then when we go look at like this 3D view as I said earlier on you're not going to fool anyone it looks pretty awful at this level it looks like decent and at this level though so that is where I could see this really being used if you're doing quick prototyping or if you're coming to create again when things are in the background they don't need to have super precision and this you could spit out you know dozens upon dozens of different things just basically using the same prompt but a different seed and you'll have you can populate a very fast world without having to create all these different textures yourself and then of course if you need to create texture Maps you could be over here in the image editor and as you saw you could do fine tune control various different options we could do this as a photograph and have all that fine-tuned details over how it was done or we could create auto tiling textures Etc again a lot of what when you're dealing with stable diffusion in general it's iteration iteration iteration so you kind of just run through it until you get a result that you ultimately like but as you see this is very early on if we look back to here we're at version 0.09 and this is the very first version that added the depth functionality so that that 3D integration was just added into dream textures I could just imagine over time it's going to get more aware I'd like to see it actually have like a 360 knowledge of the thing that it is texturing so that it works from all angles and then we're just going to see over time it just gets Tighter and Tighter more aware of what it does it creates more realistic result but even at this beginner level this entry level that we've got here uh it's got a heck of a lot of functionality and we only covered certain things here so you've also got things like I mentioned earlier on uh in and out painting here uh so you can guide it uh have it kind of fill in the blanks or paint areas and textures kind of work with existing materials uh there's multiple render pass you can uh do AI upscaling of your image you can integrate it into your Cycles rendering Etc and the cool thing about it is you may find that setting up stable diffusion can be a little tricky this handles all of it for you you basically install the plug-in download the model and you are good to go uh so ladies and gentlemen that is dream textures for blender it gives you an idea of where the world of AI is going to start assisting developers going forward and I think it's a kind of cool world to be honest but I'm curious what you think of AI stable diffusion blender and all those things working together let me know comments down below and welcome to 2023 we'll have more regular updates going forward I hope you had a happy hour all day and I hope this next year is a wonderful year talk to you all later goodbye

Channel: Gamefromscratch

Views: 275,397

Keywords: Blender, AI, A.I, Artificial Intelligence, Stable Diffusion, Dream, Dream Texture, Texture Generation, GameDev, Game Development, Tutorial, Review, 3D

Id: aCVL9qz6IV0

Length: 16min 58sec (1018 seconds)

Published: Mon Jan 09 2023