Create SeamlessTexture using ComfyUI

hello everyone I'm Jake the topic of AI generated seamless textures has been around since AI came out today I will share with you my operational experience in some projects all seamless Textures in the directory are generated using comfy UI and without going through other software in addition to the base color or diffuse color there are also normal height and curvature Maps the curvature map is often called an edge map today I will mainly introduce comfy UI and cover a 11111 and the material attribute conversion tool materialize let's first look at how to create seamless Textures in a111 the most critical point is the prompt I have listed a general rule for positive prompt divided into several parts first the green part is a directional description of the texture let's say it's organic attractive photorealistic or three-dimensional the Canan part requires that the generated graph be seamless or symmetrical if the tiling option is enabled this part can be omitted I will introduce the tiling option in a moment the yellow part is essential use pattern or Texture to specify that the generated image is planer rather than three-dimensional the purple part is the texture theme description such as brick grass stone Etc some modifiers can be added in front the next gray part is the optional elements description and the arrangement description of the elements for example the ones listed here include shapes natural variations artificial straightforward Arrangements Etc because what we want to generate is a texture in addition to the commonly used negative prompts we can also add words such as Shadow highlight reflection and shiny to reduce the interference of these elements in this case The Prompt is organic texture of flat weather dark Cliff Rock the checkpoint uses Juggernaut XEL the sampling method uses dpmp 2m caras and the resolution is 1,24 * 1024 please note that there is a tiling option in the Quick Settings bar in the latest a111 version this is a hidden option how to make it appear in the settings menu find the Quick Settings list in the user interface and enter tiling click info to see more setting parameters there is an extension when creating seamless textures called asymmetric tiling let's turn off the default tiling option first its Advantage is that it can control the X and Y directions separately this page has a detailed introduction to it after a period of waiting the texture has been generated I generated two textures one using default tiling and the other using asymmetric tiling extension the results make no difference the following focuses on the workflow in comfy UI let me first introduce a few tiling related custom nodes for me the first choice is the seamless tiling node the most complete and reasonable custom node there is also a model node in MTB nodes that can create seamless textures but some important functions are missing the following two custom nodes can also create seamless textures but they need to replace the default case sampler node which is not reasonable I'll open the workflow file there are several methods included in the tiling process let's take a look at them in turn the first is the default process without tiling enabled the second enables the model patch seamless node in the MTB custom node the third one is seamless tiling it provides multiple nodes such as the seamless tile model node which can control the XY Direct independently there is a circular VA decode node including a single node with integrated functionality and a separate Mak circular vae node that does not affect the default vae decode node using the set of nodes can add seamless functions to the original workflow and can be switched on and off at any time so it is my first choice as for the tiled K sampler custom node which integrates the tiling function into the default K sampler node if the corresponding function of comfy UI is upgraded it may not work properly in addition using it will cause the image generated by the default K sampler node to become seamless we'll see again later the last asymmetric tiling is also integrated with K sampler so it is not used the Juggernaut XL model and default sdxl vae were selected in the workflow the IP adapter process has also been added to this workflow to obtain results similar to the reference images there is also the Kaya upscale node it can directly output images with a resolution of 1024 or above after being enabled for the sd15 sd20 or sd21 models there are also sdxl lightning processes and image to image processes let's take a look at the default process first other settings are the same as in a111 this image is not seamless I used a repeat image batch to generate four identical images in a 2X two layout this is a quick way to check if the image is seamless let's try MTB tiling use the random seed number from the last generation seamless images can be created by simply embedding the model patch seamless node in the MTB node into the model flow but the XY Direction cannot be controlled independent ently note that there are three vad de code nodes here the first one is the default the second one is also the default but has tiled functionality the third circular vcode belongs to seamless tiling custom node and is an important node missing from the MTB custom node I use the image enhanced difference node for comparison the first is to compare the results of the two default VA decodes you can find slight differences between the two images in the comparison image when the generated resolution is not high such as within 1024 the adcode will be more efficient than VA tile decode and the result will be better because no cropping and tiling is performed so what does circular VA decode do as you can see from this picture there are obvious differences at the edges of the picture these differences are the key to whether the pictures can be perfectly and seamlessly tiled when zoomed in the picture appears seamless at first glance but irregular Jagged flaws appear at the picture edges using circular VA decode generates perfect seamless textures let's look at seamless tiling the usage method is the same as MTB node and it has the function of independent control control of XY Direction there are also three V code nodes here one of which is the default node this group is the default vcode node and the make circular VA node there is also a circular V code node that integrates the above two nodes we already know that the edges of the images generated using the default V decode node are defective the other two methods can deal with this flaw and the results are the same I prefer to use the make circular VA node to control it individually the two processes below are seamless only in the X Direction and seamless only in the y direction for tiled K sampler tiling and asymmetric tiling custom nodes although the same results can be achieved I do not recommend them because they replace the default K sampler tiled K sampler tiling custom node contains its own circular VA decode node let's look at its biggest issue we still remember that when using the default K sampler the generated images are not seamless but after using the tiled K sampler node to generate images let's try to use the default process change the number of steps and gener generate two images you can see that the images generated using the default case ampler will also become seamless it seems that a global tiling option is enabled we can only change the tiling value in tiled K sampler to zero and then use it to generate a picture to turn off the global tiling option I enabled the save workflow to save seamless maps at the same time the Deep bump node provided by the MTB custom node is used to convert the color map into a normal map a height map and an edge map in other words we can use comfy UI to generate seamless maps of the main material properties with one click please note that the normal map generated with deep bump is applied to Unity format when used in UI the green Channel needs to be flipped which will be introduced later next we try the IM image to image process we first use two methods to obtain the generated metadata of the image as a reference the wd14 tagger node can infer The Prompt based on the image you can use a custom node like the one I created to read the metadata directly for an image with metadata this metadata can be used as a reference for creating seamless textures unfortunately wd14 tagger inference of prompts for texture images is not ideal and has no reference value you can go to lex. Art to have a look users have shared a lot of textures created using AI you can refer to the prompts of the textures we reduced the Deno value and generate image next we try the IP adapter but be careful to close the image to image workflow first I used the same reference image suitable for SD xl's vth settings reduced the weight and endpoint value and generated the result compared with the imageo image process IP adapter generates images with a wider range of changes and may achieve unexpected results I explain further with this comparison chart I used an organic reference image which has nothing to do with rocks and listed the results generated by different weights and endpoint values with the same random seed number it can be seen that when setting the weight around 0.4 and the endpoint value around 0.2 the generated results are better next we try two texture checkpoints one is texture hell which is based on the SD 2.1 base model texture needs to be added to the prompt to trigger it the other is texture diffusion which is based on the SD 2.0 base model PBR needs to be added to the prompt to trigger it I changed the texture diffusion checkpoint enabled the default and MTB tiling processes enable the Kaya upscale workflow to directly generate 10 24 resolution images the first generated image looks like nothing I add PBR to the positive prompt and regenerate the result although very texture like is not ideal let's try texture hell the result is okay it looks like a rocky texture I think the texture held checkpoint could be an option then I tried sdxl lightning for quick generation select the Juggernaut XL model and enable sdxl lightning Laura the parameters of K sampler need to be adjusted accordingly overall sdxl Lightning's understanding of textures is still relatively lacking if you are interested you can try sdxl turbo and sdxl LCM however I would not use these quick generation methods to generate seamless textures I then observed and generated other material property maps and materialize click show full material to observe the material create a smoothness map or inverse roughness map create a ambient occlusion map then create material property maps in materialize and replace the maps generated in comfy UI save the textures generated in materialized to the directory and compare them with the textures generated by comfy UI there are differences in the directions of the two normal Maps which I will explain later the height normal and Edge Maps generated in comfy UI are more detailed than those generated in materialize please note that the style of the normal map is different in unity and UI or Maya and Max it can be seen from the comparison images on the right that the red channels of the two are the same and the green channels are inverted to each other before creating a normal map you need to create a depth map by default cyan faces the upper left and purple faces the lower left which is the normal map style of unity or Maya if you want to use it in UI or Max click flip normal y to get the normal map in the correct style or you can change the default normal map style to max style in the settings and materialize will automatically flip it I use renamer to organize the names of the textures first delete Del all characters then insert the subject name then add the serial number and finally insert the texture type as a suffix a tile Maps function in materialized can make maps seamless click show full material and you can see from the 2x two tiling result that this texture is not seamless the tile Maps function is based on height blending so we create a height map first adjust the parameters and see the results generate other material attribute maps and process them together with tile Maps let's take a look at how the textures look in UE the material has been set up and the roughness uses the inverse smoothness value the normal map used is generated in comfy UI note that the flip green channel has been turned on to ensure it can be used in uee compared with the normal map generated in materialize the style is consistent and the details are more the height map is used for the world space position offset of the model for easy observation the model uses 2x two tiles it's great thank you all for watching see you next time

Channel: Kefu Chai

Views: 3,123

Keywords:

Id: ExPUbiRkJo0

Length: 20min 24sec (1224 seconds)

Published: Wed Mar 13 2024