SDXL IPAdapter is Amazing!

making images with Dary 3 via Bing image Creator is indeed fun as you can see it's great at interpreting your prompts but you're stuck with a square image of perhaps somewhat questionable quality I've already released the stable diffusion 1.5 power up workflows here you can see one in action we've got the original dar3 image there and the powered up one from stable diffusion as you can see there's quite a few more details in there even though it's gone a little bit uncanny valley and back to the original here's an example of one of those power up workflows available from the a very comfy nerd web page on GitHub you just drop your dar3 image in down there apply some shape and lighting there I'm using exactly the same shape and there I'm using a gradient going light down to dark IP adapter means the prompts can be kept very very simple indeed so that's pretty much all there is to it press Q prompt wait 20 minutes and eventually you get that reimagining you got the three different versions there and the final upscaled one which is in 4k needs a bit of playing with the colors but overall not too shabby you can batch up images as well for that instant Laura type effect but what about using sdxl cuz this is all stable diffusion 1.5 at the moment well for one sdxl doesn't have a tile control net yet which could be problematic as I'm using that for upscaling another thing is that the sdxl IP adapter plus model here we've got the sdxl ones this new one from about 10 days ago um actually still uses the old stable diffusion 1.5 clip Vision model so the the one from up here so it's using that one which is great for compatibility issues when you're mixing 1.5 and sdxl okay time for the thinking cap I figured given the IP adapter is somewhat similar to stability AI revisions workflows why not go back and take a look at these again as you can see here and may remember from my control Laura video revisions is pretty good with Styles and subjects but generated things that only are kind of sort of vaguely in the same ballpark as your input image interesting but I figured all right let's just start from the grand up with sdxl and battle through whatever problems I get thus the sdxl instant Laura version one workflow was born here I'm using the iterative upscaler provider there which seems to be okay even without the tile control net excellent so that's one issue down many more to go so sweet it kind of works as a basic sdxl instant Laura sure this one only goes up to 2K but it's you know it's all right that's the original input image there so it's got a sort of rodent with a curious machine and another little rodent on the side we've got the first upscale the lat up scale since we turned that rodent into the rest of the machine and then finally up into 2K there so it's okay it does its job all right some more testing because I like to do tests with things and let's give it a go with some faces now as you can see that one has upscaled nicely let's throw another one one in there and generate this there you go pretty similar obviously as you can probably tell this image is one that was created using style Gan so it's got all these little telltale signs on it which in the upscale have sort of changed into jewelry or something strange like that so that's fine but there you go it's it's upscaled it it's done it that's that's all right I'm okay with that let's double check it again with another face here we've got some rather vibrant red hair and blue eyes so fingers crossed that's what comes out in the upscale and it's pretty close it's all right it's not too bad we've got we've got red hair so that's all right there's the upscaler there I'm just doing a really really quick upscale you can put that up to two and maybe do four steps something like that for more detail perhaps a higher quality image as you can see the prompt is also very very simple it's just a woman you can change that but how much can you change that I mean we don't really want all these people do we we want cool images like I don't know something like a Roden because to do something like that we're going to have to change this IP adapter image weight down a bit let's try it at .5 is that enough uh okay cool image bro but needs a little bit more rodent all right let's put this value down even further let's go all the way down to 0.3 and cue that up see what comes out with that okay yeah that seems about right it's picking up the sort of style and stuff off that original image we got some redness at the top there but it's come out as a rodent excellent that means Styles should work as well so I can add watercolor onto the end there and do I get a watercolor image yes I do there he is sort of watercolor IP adapter influenced output very cool you've probably noticed I'm also using the sdxl prompt Styler here as well so if I take watercolor out maybe I just want to pick a style so let's go for something like flat papercraft and see what happens this time great that seems to have worked perfectly I have got my paper cut style rodent that has been influenced by the input image excellent everything seems to be working but you know what you should do if everything seems to be working if it's not broken tweak it this isn't even its final form but just for Giggles I added a couple of control Nets here you can see I've got depth and canny I'm also over here using that clip vision from the sdxl revisions workflow because way up here in the IP adapter I haven't actually even attached that to anything sure it still needs to load this old stable diffusion Vision thing but I'm not using it I'm using this clip Vision G instead all right so that's got rid of some of the stable diffusion 1.5 compatibility problems with you know when it comes up with errors and it's oh this thing doesn't match this thing and there's loads of numbers all over the place and there you go there's there's the upscale so that's almost similar to my original one that I showed at the beginning with stable diffusion 1.5 where you take your original darly 3 image and then upscale it to two or four or even 16k with that stable diffusion workflow I think that's not too bad there's the original image over there again the colors on that are quite nice I do like the colors but if you zoom in you can see it's not it's okay you've got some good detail on the fur little bit grainy but over here it's it's it's sort of more vibrant and I don't know it just seems like a nicer output you got the other option there the first upscaler too so that's sort of achieved in stable diffusion XL what I was doing in stable diffusion 1.5 brilliant as for daring 3 images with text well as you can see here it is a little bit hit and miss there's my original input image this rather cool rodent just see the news of course that aliens exist as we all know and trying to upscale things with text in sdxl is a little bit Yeah so okay so I've tried obviously using the canny control net there with various different weights some fonts are okay so if you've got an image with a fairly bold font you can probably get away with it you know play around and maybe you will get lucky thus onto its final iteration for now obviously each of these different version verions are available on the a very comfy nerd GitHub web page as mentioned so you can download any of them and see how they grew over time maybe help you gain an understanding of what plugs into what and where but for this one the first image here is the one that impacts the generation shape this one is tied into the Midas and canny control Nets you don't have to use those you can rightclick and set bypass group nodes and then it won't use the control net at all the other images beneath are the ones that sort of go into the IP adapter so these also help influence the style and content of the final output along with as we saw before the sdxl prompt Styler where you can put in your own prompt and even pick a style as well I've added absolutely loads of little dials and things down here so here we go we can just move that one out out a bit so we got the IP adapter Image Weight there so how much that's going to influence the final output I like it around 0.3 personally CU that gives you the ability to change all the other things as well prompt waiting I've given you two controls because why not personally I like sticking that on randomized because I don't necessarily want the output image to be exactly like that one there and what that one does it sort of randomizes how much your prompt weight is going to be and uh yeah you can get some interesting combinations of things out of that and there you have it instant Laura for sdxl which you can also use to power up any image you like such as those from darly 3 or just photographs and if you haven't seen any of my other comfy UI videos then do check this next one out

Channel: Nerdy Rodent

Views: 10,809

Keywords: nerdy rodent, ai, ai art, SDXL Instant LoRA, generative ai art, improve dalle 3 images, upscaling, stable diffusion, comfyUI, step-by-step guide to SDXL Instant LoRA, no training LoRA, IPAdapter, SDXL IPAdapter, SDXL IPAdapter workflow, no training lora, Ipadapter ComfyUI, ComfyUI ipadapter

Id: dGL02W4QatI

Length: 11min 3sec (663 seconds)

Published: Fri Oct 06 2023