ComfyUI AI: IP adapter version 2, create artistic images with the Gestalt laws of perception

hello and welcome to my new video in this one Charlotte lends me her AI voice again to narrate my adventures with the IP adapter version 2 the Juggernaut XL lightning model and my idea this time to pack the gestal laws of perception and other principles of processing visual stimuli into the prompts the video should have been published a long time ago but then the upgrade to the IP adapters came out this second version not only replaces its predecessor but also requires all previously created workflows to be adjusted you can already see here that the jugut XL lightning model is ideal for generating great images quickly and efficiently and above all to make something out of my experimental prompt input the weaknesses of the model that only four to six steps are allowed for good results and the CFG value must be between 1 and two as well as the fact that it only runs with the dpmp sde sampler and the Schuler set to normal and that the only resolution that works well is 1216 to 832 are overshadowed by the obvious strengths Juggernaut is a fitting name these laws or principles of visual perception are law of closure law of proximity law of similarity law of symmetry and Order figure ground relationship methods of Image Design can be derived from these principles perception and depiction work together to create interesting effects such as color Illusions depth Illusions motion Illusions geometric illusions and the model can reproduce some of these effects with quite convincing results now that we have some suitable images we can proceed with the IP adapter embeds version 2 setup this requires the basic setup for the sdxl models two load image nodes the new IP adapter Advanced node the IP adapter model loader and the load clip Vision node and of course the prepare image for clip Vision nodes which convert every loaded image into a square shape and offer additional functions interpolation selection of the relevant part of the image and the option to resharpen the image we then connect the two images to the IP adapter Advanced node via the batch image node for the embeddings we still need the IP adapter encoder one of the new nodes we then only need to connect this to the IP adapter save embeds node and we are ready to go in the IP adapter Advance node you can already see the First new options however my tests have shown that in weight type linear and for combine embeds average are best suited for creating the embeds even if there were a few interesting Surprises with the other options you can watch the different results in the uncut version of this video if you like but let's get back to the IP adapters to test the changes to the IP adapters I created a new workflow that runs as many of the new functions as possible through the K sampler not all of them made it into the workflow simply because my Nvidia RTX 3060 has reached its limits despite the considerable 12 GB of vram but as you can see in the course of the video the new adapters helped me amazingly well to get a lot closer to the goal of exerting as much control as possible over the image to be generated part two I call it controlled mask madness here of course we also need the basic setup again and as the Juggernaut XL lightning model is to be used here again we need to set the steps in the K sampler to four the CFG value to one the scheduler to normal and the sampler to dpmp sde in the empty latent image Noe the resolution must be set to 1216 to 832 as previously mentioned to give us more control over the image generation we now create a mask for the left and right side of the image this requires a load image node two solid mask nodes the feather mask node The Mask composite node and an IP adapter advanc node as you can see in the video we need to tie these together and add a mask preview node for control purposes then we can select the whole thing with Control Plus left Mouse button press control plus C and insert it to the right with shift plus Control Plus alt plus v almost completely connected one of the new things is the IP adapter noise node which allows us to add more image noise to the IP adapters according to the motto the more image noise the better this is good because roughly speaking it gives the adapters greater scope for processing the input and the great thing about the noise node is that it also gives us more control over the image generation simply because we can set the type the strength and even add a blur and we can also add an image via an image loader all of this has quite an influence on the final image we can also connect the image noise to all IP adapter nodes used in the workflow via the IP adapter encoder node or the noise node all we have to do is connect the image output of the noise node to the image negative input of the adapter or the pulse embed output from the encoder to the neg embed input of the IP adapter embeds node this means that the image noise is omnipresent if we want to use multiple embeds we can connect the embeds to the adapter via the next new node the IP adapter combine embeds node and there are also several new functions for processing the embeds more on this later the workflow has not yet been fully set up for the next task we need an IP adapter encoder node an image loader and an IP adapter embeds node of course we send the loaded image through the prep image for clip Vision once we have tied everything together correctly we move on to the last point of the setup for this we need the known nodes for a control net in this case for an open pose sdxl model now connect the checkpoint with all the adapters in the correct order and the setup is finally complete the output images and the settings for the various nodes are still missing the IP adapter plus models are very effective so it is better to start with low values for weight for this let's call it a module is used in the workflow exclusively to transfer the style of the source image that is why the weight value is so high to influence the left side of the image we now need to set the mask to do this the operation add must be selected in The Mask composite node in feather mask we should set right to 350 relative to our preset image resolution the solid mask which is connected to destination specifies the overall resolution for the IP adapter so in our case we have to set it to 1216 to 832 however as we only need the resolution from it we must not forget to set its value to zero otherwise the mask will only remain a black area for the mask that is important for us we should set its width to a value relative to the overall resolution in order to influence the left side of the image the 6 108 pixels are exactly half the height can of course remain at 832 pixels the settings for the mask on the right side differ in that the black area on the xais must be shifted by 68 pixels and the value in the feather mask node for left should be set to 350 at least for the time being as far as the various settings in the new IP adapter are concerned I have tried them all in various test workflows in addition to this little insight here I'll put the entire recorded test Series in the uncut version it's definitely pretty cool what you can do with the second generation IP adapters but a word about the Juggernaut XL lightning model as it has its limitations due to the strict resolution specification I certainly tried to incorporate an upscaling method into the workflow but the methods based on models exceeded the performance limits of my computer the upscale image node worked without any problems while maintaining the image ratio have fun trying out the workflow if you found the video interesting and or helpful I would appreciate a like And subscribe otherwise have a great day

Channel: Show, don't tell!

Views: 2,188

Keywords:

Id: Is3QdwQVSbU

Length: 17min 2sec (1022 seconds)

Published: Thu Apr 04 2024