Generate ENTIRE MOVIES with AI! Full Tutorial + FREE options

[Music] the center of the earth that's impossible not if we follow his Clues come on Axel we have to go to Iceland full AI filmmaking is finally here I found a way to combine different AI tools to create entire movies and it's all amazingly simple and pretty fast too just as I was finishing up my AI generator trailer for Jill Van's classic novel Journey to the Center of the Earth and YouTube popped up that allows for a much easier and better workflow for animating characters and also Landscapes and although I'm still in the process of reworking pretty much the entire trailer with this new technique I wanted to show to you right now it's just too exciting and I can't wait to see what you will create with it keeping characters consistent is a constant challenge with AI images there are tools to train models for faces of specific characters and tools to replace the faces of the characters after they are generated but I would like to achieve consistent characters in my journey only so the first step is to create a character sheet for all my characters let's start with a full body t-pose enter keywords like t-pose symmetrical full body arms outstretched front view and it should work and if it doesn't you can use an additional image prompt with any image of a character in a t-post I use the screenshot of this 3D model that I found online upload the screenshot to Discord and copy the URL paste it into your prompt and add the rest of the character descriptions and try to keep the image rate really low because we just want the pose and not like other features like colors for example so you can try to set it around 0.2 as a starting point now your character's pose should be way more consistent if there are still some kind of Parts you can bring them into the image using mid Journey's panning tool once you're happy with your character copy the image into a doc document and put the link next to it and this is just for organization purposes so we don't have to scroll through endless images later when we need them next we want to create a close-up of the face take a screenshot of the character's face and copy it into Discord and use the link as an image prompt I've also adjusted the prompt to generate a character's face and increase the image way to make the face as close to the one in the full body pose as possible and when you're done copy the image and Link into your character sheet now let's create the environments for our characters and to keep the look of the environments consistent I first created like a main reference image and to create new views from it I used the pan and zoom tool but you can also create entirely new shots in the same location by using a similar prompt structure and using the original image as an image prompt now the fun part is to bring everything together by merging our characters with the environment I use this prompt and this usually works very well the characters can still very slightly look a bit different but I just repeat the prompt once or twice and there's usually an image in there that's perfect using the close-up of the character's face usually produces also a close-up of the character in a neutral pose and you can use the zoom out or pan function to change the framing of your shot but of course we also want to be able to direct our characters and we do that by slightly changing the prompt he looks so Grim so let's make him happy or scared foreign but we are not limited to emotions let's make him play soccer for example first let's look for a reference image of a person in a cool soccer pose pose maniacs.com is a great resource for this just look for a post that you like use the 3D viewer to rotate it and take a screenshot now let's build our prompt I'll start with a link to my character's t-post followed by a medium shot that I've already generated by blending together the environment and the face I add the screenshot of the soccer pose as the third link and another image of the environment as the fourth I also add playing soccer to the text part of the prompt and now we have a character running around but is he playing soccer let's find out by using the zoom out tool and there it is the final image of our character playing soccer in some random Icelandic field if this was a bit too fast for you I highly recommend my new patreon page where you can get access to exclusive step-by-step workflow sheets with additional tips and tricks and copyable prompts for all my new videos and I will also try to help you wherever I can on our Discord Channel check it out under the link in the description the main ingredient for this workflow the tool that made it all possible is Runway gen 2's new feature to generate videos out of image prompts so let's take this shot for example I got it by blending these images then I went to runways Gen 2 and dragged my image into the image prompt window and hit generate now at this point using an additional text prompt doesn't really work so we'll just leave it blank I usually run the same prompt a few times because the motion it generates is pretty random sometimes the first try is amazing and matches the voice of a perfectly and sometimes it takes some more tries also sometimes the faces can come out pretty broken but that's actually okay we can fix that later just focus on finding the right movement of your character for your shot also check out Pika Labs another AI video generator that allows image prompts and this is now an open Beta the quality is often a bit worse than Gen 2 but you can try it for free on their Discord Channel right now and it works in a very similar way to mid-running and you should definitely give it a try once you've found your perfect shot it's time to add the facial animation I used 11 labs to generate my voices but of course you can use any other tool or even your own voice import the voice file and generate a video into Adobe Premiere but of course you can also use any other editing program like for example DaVinci Resolve which is free to use and match the audio to your video export the audio at the same length as the clip as a wave or MP3 file then go to the wave2lip Google call app a tool that generates really good lip Movement by uploading a video and a voice audio file click on copy to drive and run the first step the setup and this may take around a minute or something then scroll all the way down to lip sync to your video file and click the small play button in the Second Step you can now upload your video file once that's done click play on the next step and upload your audio file as well and make sure it's just the voice because here I accidentally uploaded the voice with the music underneath and this is what happened so once you've made sure you have the right clip and audio start the last step and after a minute or so you'll have a result that looks like this the center of the earth that's impossible amazing lip sync but really poor quality and the face is still kind of broken so let's fix that to fix the face and also bring back the original quality of the video import the original Gen 2 output and the wave to lip output into your video editing program of choice put the wave to lib file over the original video and create a rough mask around the mouth if necessary use some keyframes to make it move with the face and blur it so no seam is visible and then export the video as a PNG sequence now go to the extras tab in the automatic 1111 stable diffusion web UI and upload an image from your sequence preferably one from the middle of the sequence with a mouth open now we have the option to use two different tools to fix those broken AI phases gfpgan and code former they produce very different results so play around with the settings and mix and match them until you find a look that you like I usually set the gfp Gan pretty high and use just a Sprinkle of code former on top of it with reduced intensity when you find a look that you like go to the batch Tab and enter the path to your PNG sequence and an output folder click generate and the whole sequence should render pretty quickly and this is the result the center of the earth that's impossible pretty good right now if your result comes out flickery or it doesn't work at all or you want to enhance your result even further here are some quick tips and tricks wave tulip will Norm work if your character is too dark or contrasty but if you still want to use this shot you can you just have to increase the shadows and the general brightness before uploading to wave to lib and you can bring it back down once it's generated and the same applies to characters with non-human skin tones wave tulip is actually really good at recognizing phases with non-human skin tones but code former and gfp Gan won't work but you can use a hue shift effect to make the skin more orangey human-like before importing the sequence into stable diffusion to fix the faces later you can use another Hue shift with the opposite strength to restore the original colors if the face is generated by Gen 2 are too broken wave to lip might not be able to detect them in this case we can use the gfp Gan technique before uploading it to wave to lib and then it should work sometimes the video is generated by Gen 2 come out a bit flickery the same can happen in the last step when we try to fix the phase with the gfp Gan and code form and this can happen especially with characters with like lots of wrinkles and glasses generally a lot of detail in their face and so the Gans can get a bit confused where exactly to place the eyes but you can use DaVinci resolve's D flicker tool set to fluoride or the repair image node to make the footage look much smoother to bring your shot to the next level you can also add stock footage but not normal stock footage that you have to buy no AI generated stock footage for this shot for example I wanted to add a nasty abstract spider web effect so I use this prompt in Runway to generate them with a black background you can use the add blending mode and play around with the opacity and in this other test shot I used Gen 2 to create some flowery colored lens flares and I think having something in the foreground even in front of the facial animation really helps to tie everything together and sell these shots I'm still working on the full trailer as I'm basically redoing every second shot with this new technique but here's a first look at it so you can see it in action the center of the earth that's impossible not if we follow his Clues come on Axel we have to go to Iceland Hans bielker he is the toughest Hunter on the island he will be your guide what does he hunt excellent let's go I hope you enjoyed this tutorial and are inspired now to use this technique to create your own AI movies with it if you do please send me a link or tag me in your work I want to create a little Community playlist with all your projects soon and thank you to my patreon supporters who helped me create these videos if you want to have access to behind the scenes content exclusive workflow sheets with step-by-step instructions or just get feedback and share your work in our community Discord check it out under the link in the description thank you very much and see you next time

Channel: Mickmumpitz

Views: 295,768

Keywords:

Id: nTLuH-uRaZ8

Length: 11min 1sec (661 seconds)

Published: Thu Aug 03 2023