ComfyUI IPAdapter V2 style transfer workflow automation #comfyui #controlnet #faceswap #reactor

hi today we will see how to transfer a style from one image to other images with the help of the IP adapter we will also use masks and so that we can get consistent results and a uniform style in a series of images we will also show how to automate the entire process so that in the end you can insert a single word and get the desired result let's start at the beginning and build the workflow together you can of course find the workflow and all the necessary links links in the description of the video we will work with the Juggernaut XL lightning model there is also a clip set last layer node to minus one positive prompt National Geographic photo of a lion City Lights in background high quality photorealistic sharp negative prompt empty latent width 126 height 832 and of course K sampler whose various values I chose according to what is recommended in the model card in civid Ai and so that we can check things properly we also select fix seed and press Q this is the picture we got a very good picture in my opinion considering the small number of steps now I want to have a more dramatic atmosphere and maybe also a little melancholic as exists in this picture in order to get this style we will use an IP adapter we are using the new version as of today V2 we will bring the IP adapter tiled node to which we will connect the IP adapter model because we are working with the sdxl model we have chosen the plus sdxl model note that although this a model that is compatible with sdxl in clip Vision we have to choose a 1.5 model it's very confusing but that's how it works we of course need our reference image and in order to have a good comparison I will simply copy the previous case sampler with the same seed and exactly the same values so that we can see the effect of the IP adapter on the final image and click on Q as you can see the IP adapter really changed the original image and it is very far from the desired result we will try to play a bit with the weights and see what comes out there is no doubt that now we get a lion but it is very different from what we got without the IP adapter and I want a composition more like the original let's try to change the weight type to style transfer it's already much closer but you can see the effect of the reference image on the lion and I want it mainly on the background so we'll try to raise the start at a bit this allows the model to start working without the effect of the IP adapter and it comes into action at a later stage depends on how much we raised it and you see that now the result is really similar to the original in terms of composition but you can clearly see the effect the reference has on the lion which comes out a little less contrasty and the color of the fur is also affected by the reference to prevent this we will have to use a mask I remind you that we want to automate this process therefore we need a tool that can create a mask without drawing it manually I will use batch clip seg this tool allows us to create a mask based on a keyw our word in this case will be lion we can connect the first workflow to the clip segment select the mask preview node and see the mask it produces for us it looks good enough in the attention mask because we only want to affect on the background and not on the animal we will apply the mask with the invert mask node and connect the inverted mask to the IP adapter and this is what we got now you can see that we have returned the color in contrast to the lion and we still get the haze and atmosphere from the reference image in the background let's see if it works for us with another animal I changed the lion to an owl both in the prompt and in the clip segment and it seems to work really well in order to make this workflow automatic so we can only choose the type of animal and everything will work we just need to find a way to replace the type of animal both in the clip segment and at the prompt we will start with the prompt we will search for replace and this is the node we need text find and replace we will connect this to text input we will copy the prompt we wrote to here and instead of a specific animal we'll just write animal this will be our code word we will change the replace to input and copy the text input node here we will write the type of animal we want to get we will connect everything to some show text node it doesn't matter which one and we'll see if everything works excellent now we will connect the type of the animal to the clip segment here too we will change the text to input and we will also o change our prompt to input let's check that everything is connected and working let's change the type of the animal to leopard and press q and that's the beauty of comfy you can automate almost any idea work technique and build many software that will produce the desired results so I hope you learned and we will meet in the next lessons you are of course welcome to ask questions subscribe to the channel and like if you liked it and most importantly have fun bye

Channel: PixelEasel

Views: 4,295

Keywords:

Id: AvAx4o5c5O4

Length: 5min 34sec (334 seconds)

Published: Mon Apr 01 2024