LORA Training - for HYPER Realistic Results

do your lauras look like they should be main actors in Borat movies something like this well maybe it is your face but probably it's just bad technique unless it is your phase in which case I would suggest just standing behind someone who looks better than you let's get started here so first I want to talk about the big elephant in the room there is no simple one-click solution to get a perfect Laura creating a really nice Laura for your purposes depends on what you expect as a result how you want to use the Laura and what kind of artistic style you want to use the Laura in because of course it's gonna be different from someone who creates an anime Laura so there is also no perfect recipe to get the result and that of course means the best results need testing not just for the Laura but also what goes into the Lauren afterwards what you do with the Laura and that means you want to test with the photos you're taking you want to test with the words you apply to these photos in the clip descriptions you want see what kind of models you use for the Laura training and afterwards you want to see what kind of lower weight you're using but also what kind of prompts and negative prompts you want to use to get the best results I know all of that sounds a little bit complex but believe me it is a lot of fun and once you figure it out it's very easy to do and I know some of you gonna say hey this guy actually wants us to do complex stuff not just click a button yeah that's kind of the thing if you want to have really good stuff it's a little bit of an effort to get there but don't worry I will help you most of the way there so the first thing we need of course to train a Laura is the images as an input now one thing I found through testing but also talking with other people online is that you don't want to have too many extreme expressions of the face in your Laura images that you use for the training because this is going to confuse stable diffusion and the AI on what you actually want but also what the face actually looks like so you want to have a rather stable face that either is neutral or has a bit of a smile or a bit of an expression in it but not too much a smile is good if it also has the teeth in it because then the AI can also learn what your teeth look like on top of just what your smile look like but nothing that covers the face or is too extreme or has strange angles it is really important also to understand the limitations of what any eye can do it needs to understand which of the parts of the image are part of your face and not part of your face so in the best case you want to have nothing on the face to begin with because then the lure can really understand the finer details of your face another thing that stable effusion is really not good at is different extreme angles of the face for example the face looking upwards to the sky looking downwards to the ground a face that is on an angle sidewards is really difficult for the AI to understand it will often not get a good result but at the same time it will also diminish the quality of the face another thing you want to avoid is images that are too noisy or Overexposed because this is also something that the Laura is going to learn in the process and when you have also noisy images there is no finer detail that the AI can pick up on for example in this case you can't even see the eyelashes so how is the AI expected to learn these details this also means don't use blurry images no low resolution images no images with compression artifacts which are these little squares in there or other defects in the image this will give the AI the Raw wrong impression and you will have a worse result from that another thing that might be important here is oily or reflective skin this of course is especially important in the Summer where the skin sweats so you want to get some skin powder to have it more matte so there's no reflection on your skin because afterwards this might be burned into the Laura and you might have a hard time removing that from your AI output now here you can see some examples of the images I have taken with a little bit of a smile and then with a bit of a smile we can also see the teeth different face expressions but all of them are very subtle at the same time we have a very even light we have a little bit of reflection on the skin but overall this is a very neutral picture that the AI can pick up on but still two things stick out here from that image there's a little bit of sweat on the skin because we didn't have skin powder to make it non-reflective but also there's a little bit of eye bags down here now one thing you want to tell your model before you take the photo is don't be sleepy don't be overworked have a refreshed face because this is something you cannot really get rid of in that moment unless maybe you have a specialist for makeup there in this case this might just be something that is part of the face and then that is completely okay but if it's not part of the face you just want to try to avoid that because again these subtle details are going to be learned by the AI and you don't want to have images afterwards where the person looks tired or exhausted in all of your AI images now another thing you want to do here is to crop the face out this is another advice from my friend Yuri thank you very much for that so when you have the bigger picture this is good so the AI can learn the face in relation to the body but then cropping it out also in a square format is good so that the AI can focus just on what the face looks like this of course afterwards can also help you when you're gonna do face in painting for example with a detailer another thing you want to do here is to have the person in different sizes so that you have the upper body than most of the body and then the complete body so that the AI can also see the head in relation to the body but also what the body shape looks like so that afterwards when you're gonna render the image the person is not just looking from the face like the person but also from the rest of the body there is some likeness to that after several test rounds I trained 10 different loras on this specific phase I came up with this selection of images so you can see most of them now are close ups there is very little rotation actually there's just one photo where she's looking up a little bit but only in a slight angle and then we have some other images with different clothing where she's standing a little bit further from the camera so we can also see the body and a little bit of the body Dynamic of course one element for this is better gear if you have a really good camera for example I'm using a Sony A7 III and also a sigma 1.4 DG DN 85 millimeter lens now this is pretty good professional equipment which is going to be in the thousands of dollars and if you can't afford that you could for example go to a photo store and say can I take some test photos and then snap snap simply make the photos of your model or maybe you have a friend who can borrow you or who couldn't come to the place to make the photos for you with this really good equipment because with a lens like this you get really really sharp up details on the face and that can very much improve your results on the Laura training of course if you don't have any of that here is a suggestion for minimal gear at least get you a couple of reflector discs these are pretty big ones and that is good because it gives you more of a reflection surface and that means softer light on your model so this can really help you and it's very very affordable and you can then still take the photos with your smartphone now let's talk for a second about the conditions of when to actually take these photos I would suggest to you to photograph on a sunny day but stand in the shadow so you don't have that really harsh light on the face of the person now a second choice here would be an overcast day which means you have a cloudy sky in which case you have a natural soft light that doesn't create harsh Shadows but of course you still want to look for for a day where it is overcast but still bright enough so that you can take sharp photos on that day and of course another Choice here is to go out on a sunny day but then use one of these reflectors one of them as you can see here is white which means the light is shining through it it doesn't actually reflect it you can also use it as reflection but this can be used as a giant softbox so in that case what you want to do is to have a second person hold that really big disk opposite to the person that you're taking a photo of so that the sunlight is softened through that reflector onto the face of your model in which case again you get a really nice soft light with no harsh shadows in there of course there's also the question of adjusting the images and I would suggest you to use Lightroom because this is by far the best software to easily adjust the images but of course if you're a little present who lives in your mama's basement you want to look for Snapseed which is a free app from Google and it's also very good at adjusting these images now this is an app on your phone which might be perfect if you take the photos on your phone in the first place also check out if your phone camera can do Raw photos because then you have a bigger range of how you can adjust these images so let's have a little bit of a look on what you actually want to adjust in these images now highlights are always a good idea because if you have a little bit of overexposure in there or some skin reflection or other highlights you want to bring them down a little bit for the Shadows you often have darker areas you want to have some medium look in there so the bright areas not too bright the dark areas not too dark so push the Shadows up a little bit afterwards of course the white balance so if your character has a gray shirt or a white shirt on you can tap on that shirt with your white balance to actually get the real colors from that photo situation Clarity can help you improve the image but use it very softly so this can bring out a bit more of the details if you push it too hard the image will look very harsh so only a little bit of clarity and then also sharpness applied a little bit to improve the sharpness sharpness is not meant to make blurry images look sharp it is meant to make Sharp Images more detailed so also apply that just a little bit and of course last but not least a very important question is here what model do you want to choose for your training if you want to go for photorealistic images like portraits Raw photos stuff like that as an output of your AI you might want to look into Photon I have a video about that here that shows you the usage of this model it's really great but it's also very good for Laura training if you want to be more purest about it you might want to use the stable diffusion 1.5 base model look at that on the internet I will also link it below my video this is a 7.5 gigabyte model but it's very nice for training and of course you can also look into realistic Vision which is a little bit harder to prompt for and reliberate which is from the same people who made the deliberate model so this is a little bit more artistic and easier to prompt for so experiment with these models or other models of your choosing to see what kind of results you get the keyboard here being experiment and see what you get so even if you get a nice result there might be a better result out there so train different loras with different models with different settings and test them to see what you get let's go into actually doing it so here I want to show you real quick in Lightroom classic we are are in the developer section up here I have an underexposed photo here because I exposed for the highlights and as you can see when we zoom in here it's very nice and sharp but it's just very dark at the same time you can't even see the eyes so what we're going to do here is on the right side here you have exposure so you can pull that up a little bit but of course you don't want to have things Overexposed so we can push down the highlights a little bit but you don't want the Shadows to be too dark so you can push up the Shadows a bit so you can see now we are arriving at a area where everything is kind of in the Middle with the eye shadows with the bright areas everything is balanced out now what you can also do here is for example to push up the clarity a little bit just a bit to get more detail in here and then maybe also want to use dehaze a little bit so so you get even nicer details in here another thing you can do up here is to reduce the wide values a little bit so you don't have any areas in your image that are completely white and what we can also see down here now is for example that we have very nice details here on the arms we can look into the Shadows here we can see the details and the fabric of the clothing so now after these changes we have an image with very quick adjustments that show all of the details we need for the training after you have adjusted and selected all of the images you want to use when you have for example Lightroom in the export you want to set the max width and height to 1600 which is a pretty nice high resolution for training of the images you can go even higher for the resolution or if you have an older GPU you might want to get lower with the resolution so that training is easier for you cheap you went also faster now in The Next Step what we need to do here is that you want to put all of the exported images into a folder and then you want to navigate to that folder inside of Koya SS you want to check out this video here on how to install Koya SS on your drive and here you can see I'm in the utilities Tab and I'm in vd14 captioning which is pretty nice for captioning this is usually used for anime captioning but I find that it generally works pretty well because it puts in a lot of interesting and very good words for the captioning of your image so navigate to your folder with this yellow button here in the middle when you have selected the button click here on caption images this might download a model for captioning the images first so in the first time this process might be slower for you and then next time it should be pretty quick and every image will have its own text file in there after you've done this you want to use a tool that is called the borrow data set tag manager this is a free tool I will link it below the video here you can see all of the images you have selected and then all of the texts that have been created inside of the text file you can also use here a preview that shows you a bigger view of the image but I would highly advise you that you put this preview somewhere on the site where it doesn't matter too much because this software sometimes gives you a pop-up where you have to click on yes for saving the changes and that might be behind the preview image and if that happens you have to alt for the software because there is no way to get to that pop-up otherwise and that's a bit of a pain now inside of that software you have three diff different columns here in the first one you have the image and the second one you have the keywords that are for the selected image or the selected images and on the right side you have the list for all of the images together so if you want to add or remove something for all of the images you want to do this with the tools here on the right so if you want to add something you click on the plus and here in the pop-up you can also select if you want to have this on the top Center down or custom so where the position of the tag is now if this tag is more important you want to put it on the top so it is at the beginning of the text file if it's less important you can put it at the end which means down here in this case now one thing you want to add at the start of every image is the calling word for your Laura in this case I'm using the name of the model so that when then you then write the prompt you can use the calling word but then also of course use the specific command for the Laura with the lower weight in there so I would suggest that you go through all of these images and you look at the different words and if they apply to the image if not you might want to remove them from the image now these words they work like variables so that means if you write a word you can change that afterwards if you don't write a word you can't change that afterwards so for example I would highly suggest that if there is clothing in there that you define the clothing but also the color of the clothing so you're able to change that also with the hair you want to define the hair cut and also the hair color in this case I have for example short hair but then also black and gray hair in there and of course these words that you define here are a big part of your testing of training the loras so if the Laura doesn't work correctly one part of that might be that your keywords are not good enough so you want to change that experiment with that what works for you what gives you the best results sometimes maybe bring in some additional description here that helps the AI to understand what you actually want to train and when you're finished with that you want to go in here file save all changes now for the number of images you want to use here I think there is no specific number that works best whatever gives you the best result works for you it might be only 15 images but at other times it might be 45 images whatever gives you the most precise results is the number that you need for your training so also experiment with that sometimes use more images sometimes use less images but of course make sure that all of these images have their text file with the description inside of the text file after that the after that the folder structure for a Laura is as follows first of all you have your folder where you train the Laura in and give it a descriptive name so that you define what kind of testing you're doing at that moment for example you can Define the model that you're using for the training the steps and epochs that you're using then also the dimensions and the alpha that you're using and other kinds of settings that you want to note down here in the file name and that is important because when you come back afterwards you still want to understand how you did that testing now let's go into that folder and you can see here we have a simple structure one fold is called images the other lock and then we have a model folder now inside of the images folder there are not the images inside there is another folder there this folder starts with a number at the beginning then a underscore and then the name of the Laura in here now the number defines the steps or repetition that you want to use for your Laura I mostly use 10 for my training but other people also use just five repetitions really depends on the process and what gives you the best results after all this is set up you want to go into Koya SS again and you want to use the dream Booth Laura tab up here now on the first step you have the source model in this case you want to decide what kind of model are you going to use and for that you have here this white icon you want to click on that and with that you can actually navigate to the folder in automatic 1111 where you have all of your models you don't need to copy them to any other place and you want to select the model you want to use for this for example let's say in this case we're going to use the photon model so now that we have this in here we're going to click on the second Tab and here you define the image folder the output folder which is the model folder and the log folder again click here on this yellow icon and then navigate to where your folder is now for the images folder it's important that you do not select the folder with the number you select the images folder here instead then you do the same thing for the model folder and you do the same thing for the log folder it is super important if you set the model name because this is not just a file name this is also how your Laura is going to be called inside of automatic 1111 with the lure commands so in this case let's call it bed cut test 10. now we go to the settings in here are a lot of different adjustments and you might want to go into different Discord groups and ask people what works for them what kind of adjustments they are using but usually what I'm using in here is for the training batch size that is how many images are going to be trained at the same time I'm using between one and four if your GPU is older just go with one here this trains one image at a time the epochs here is how many loras are going to be trained one after another as iterations now this is important because for example one Epoch with a thousand steps is not the same as 10 epochs with a hundred steps each even though you have a thousand steps at the second example too because every time an Epoch is finished this can then be used as a training example for the next Epoch for improvements which which cannot happen if you only train one EPO save every nth Epoch means if you want to save that Laura from the testing if you want to have 10 different lauras afterwards one for each Epoch you can of course put here one now for mixed precision and safe Precision I would suggest you experiment here with the settings I have seen some good results with bf16 but I more often have seen good results with fp16 so I usually leave both of them at fp16 now you will see a lot of discussion online for example about the learning rate and the warm-up steps that you want to use here you can experiment with that you can read into online discussions usually I leave them as they are and get still pretty good results what I found has more impact here and again thanks for Yuri for your advice on that is the net work is the network right is the network rank dimensions and the network Alpha if you set them higher you get a much bigger lower file if you set this here to 256 and this over here for Network Alpha to 128 you're gonna get a lower that is about 256 megabytes big so it kind of seems like you get as many megabytes as you as you set Dimensions here but you can also get much better results and a more consistent Laura in the different prompts you're using it with so again experiment with this setting what kind of value works best for you down here for the max resolution I would highly advise you to set this to 768 by 768 unless your GPU is too slow to be able to render that because I find that this gives you a much higher Precision in the details of your output but also more consistency when using the lower with different kinds of prompts after that we are pretty much set for training just click here on the training model button and I would very much suggest that you look here in your command window on what kind of information you get here and if the rendering is actually starting now it takes a little bit of time to load in all of the information and prepare the Laura training but after that you will see it go through the steps and it also will give you a time on how long it takes to train the Laura after the Laura training is finished in here you will find your Laura files now one of them has a very short name this is the finished Laura that has all of the epochs that you set as the Max epochs and then the other ones with the lower names these are the in between Laura epochs you can copy this over into the Laura folder inside of automatic 1111. now at this point of our Laura training we are ready for the testing so here the fun basically begins now here are several things you want to look out for but also methods that will give you better results and give you a more precise understanding if your Laura is actually good now of course one thing you want to do here is to test out different kind of scenes and you want to test out different kind of prompts in here that can also give you good understanding how stable is your Laura will the result look like the person in all of these different scenes or is the face changing every single time in which case you means that your Laura is not very stable now at that point of the process what you also want to do is to compare the face in the finer details so for example what we can see here is in the original photos the eyes are bending a little bit downwards in the middle they are not completely straight another thing we have here is that the lips are rather thin and specifically with this face and this made it pretty difficult to train this Laura is that the features have a certain male aspect to them it's a little bit of a harder face now stable effusion is mostly trained on very soft and very feminine faces so if you have a face with a bit more defined edges to them that could be a problem for the training and you need to be more precise and dive deeper into the process until you figure out the settings and the clip words and the method that works for you so in this case I'm also comparing the shape of the face the size of the collar bones up here for the cheeks the shape of the nose and also especially this middle nose Ridge here which often sticks further down in the creations of loras with the AI so in this case it doesn't stick down further it's rather flat with the nostrils on the side so you want to check for these little details in this case also we have thinner lips and stable diffusion loves to make bigger full lips for women so again you want to look into that if that is here for the training now in this case the lips are mostly there they are a little bit too puffy a little bit too big compared to the original lips which are nice and big in the middle but then very fast go very thin on the sides and have a rather pointy sides at the end so there could be even more Improvement to that but overall I think that the likeness is very good there and also the expression of the face the way the eyes look all of these smaller details are very much there now one thing you can see here and this is what I pointed out at the beginning is we have in my photos a little bit of sweat on the face and I might retouch that in the photos or we take the photos again so you can see in the results that now the AI has picked up on it and renders this into the images so also here we have a little bit of sweat on the face in the AI rendered image but overall I have to say that I'm very happy with this result and this comes very close to how the person actually looks now here we have a result from an earlier Laura and you can see right away it doesn't look at all like the person it comes a little bit close but stably Fusion had tremendous problems to recreate this specific face it took some time to figure out how to do this right now again also here you can see that the AI is pretty much also picking up on these eye bags under the eyes for example in this case it has nice thin lips but the face shape at all doesn't match the output here we have another example and that kind of again it comes pretty close you could think maybe I would recognize that person if you don't know that person too well and this is why it is important to compare your AI images with the actual photos so that you see the finer details because a lot of what is in the original phase is here in the AI phase but it is always a little bit off so the best thing you can do to actually figure out how good your Laura is is make a lot of different renders and compare it to the original phase and see for the finer details are they there how precise are there does it actually match what the person really looks like now to get these good results so you're able to compare these things here are the settings I want to suggest for you in this case I'm using Photon as a model for rendering and also using Photon as a model for the Laura training use restore phase and then you want to use high risk fix and in this case we are using high risk fix with the ultra sharp model or maybe if you want to with another model for example for example the r e s r again 4X model I'm gonna upscale this by two and because I'm using an upscaling model I'm setting the denoise rather low in this case to 0.2 in this case I also set the highest steps to 10. now here is the part that is even more important than that you want to scroll down here and you want to use a detailer for that this is an extension for automatic 11 11. you want to enable that and you want to put the Laura also in here that you are testing in the a detailer you can set the weight of the Lora higher than in the original prompt that you're using now what this is doing is that it is first rendering a low resolution image up here with your settings and then it is going to upscale the image to the higher resolution and then it is rendering the face into the image and in painting it with the higher resolution for that image image and this is also why in this case we have a very nice and detailed face that also resembles the character pretty well because the face itself is rendered on a higher resolution and this gives you a good testing if the lure works if you only do the low resolution rendering without the upscaling through high risk fix and without a detailer you might get a bad result even with a good Laura so it's really important to go through these steps for your testing I would highly suggest to you also to turn the high-res fix off and the a detailer off and set down here the batch count for example to 10 or 15 to render through a couple of low resolution images and then you want to copy over the seed of the images that work well for you you will find that down here copy that over into a text file and then test out these specific seeds with high-risk fix and with the a detailer to see how the face looks with these images that actually look like the images you want to have now after each of these tests after you've compared the output of the image to the actual photos that you're photographed you want to go back to the drawing board which means check again the selection of your pictures maybe you want to take some pictures out maybe you want to take some additional pictures because something is missing that could help with the Precision look into the text files with the borrow data set manager to check out the keywords that you're using here maybe you need to add some additional keywords maybe you need to take out some keywords that are confusing the AI then train again the model maybe with different settings maybe with another base model that you want to choose like realistic Vision a reliberate the photon model or the stable diffusion model to see what kind of diff different results you get also what you can do here is to train multiple lures but with different base models to compare them to each other to see what kind of results are you getting from that don't feel bad if this takes you a lot of different lauras for my training for this specific model I trained 10 different loras and I talked with people online to give me tips to give me feedbacks of what I can improve so that is also a very important step for you check out different Discord rooms for example I have a Discord where I have a specific room for model and Laura training you can ask the people there you can upload the images so they have a comparison between the photo and the end result to give you some tips and tricks on how to improve that process of course always take these tips and advices with a grain of salt so they might work for you they might not work for you but testing it out will improve the pro process and over time you will develop a training workflow that works specifically for you and the results you want to have so every time you train a Laura this process is getting easier faster and giving you much better results subscribe to the channel if you want to see more videos like that and thanks for watching bye oh you're still here so uh This is the End screen there's other stuff you can watch like this or that's really cool and yeah I hope I see you soon leave a like if you haven't yet and well um yeah

Channel: Olivio Sarikas

Views: 91,575

Keywords: oliviosarikas, olivio sarikas, olivio affinity, olivio tutorials, lora training, lightroom, adobe, adobe lightroom, lora best settings, lora best results, lora 2023, 2023, best ai tools, ai news, better lora training, lora tutorial, lora guide, lora training free, lora online course free, free lora training, best free lora traning, lora a1111, best a1111 settings, lora prompt guide, stable diffusion tutorial, stable diffusion, stable diffusion ai

Id: PadSEwA3xiU

Length: 36min 19sec (2179 seconds)

Published: Fri Jul 14 2023