Create 3D animations of your AI-generated characters

life is good once you accept them embrace the monsters and you will find that they are not so terrifying after all so I'm currently working on an AI generated horror short film and I just found a way how to finally make AI generated character animation cinematic and even though the movie isn't finished and I'm going to spoil big parts of it I'm so excited I'm so happy with these shots I need someone to show it to right now but before we get into that make sure to subscribe for the final film and the full process video because what I'm about to show you is just one of many techniques that I'm going to use for the final film you've probably seen animations like this of AI generated talking characters and they all use the same super easy to use tool that we are also going to use as a base for our final effect and even though it's a great way to quickly bring your characters to life for me for for a film it's not enough because you just end up with Talking Heads in an otherwise stationary image and what's the first rule of filmmaking again chechi PT the number one rule of filmmaking is to show don't tell meaning filmmakers should use Visual and audio elements to convey information and create emotional impact instead of relying solely on dialogue or exposition to explain what's happening in a scene okay so how can we make this effect more cinematic We Lay the groundwork for this already during image generation think about your shot and how and when you want to share specific information with your audience for example this shot is one of the key moments of the AI horror film The Twist the reveal sorry for the spoiler this is more important in this scene we see our protagonist who we only heard talking until now for the first time and realize that she herself is turning into a monster even though it was previously implied that they were defeated and to make this moment as strong as possible I wanted to start with a close-up of her creepy face but I wanted to make this shot even more unsettling as it unfolds so as the camera pulls back and the background comes into Focus we see that she's also sharing her apartment with these other disgusting creatures so while generating your images think about framing camera movement composition lighting and potential Focus shifts to guide your viewers eyes through the scene I used mid-journey for this step and it took me a long time to get it right because mid Journey just doesn't allow for much control over the image generation process I would therefore recommend also checking out other AI tools like control net for stable diffusion that lets you control way more aspects of the shot generation process otherwise of course you can always use Photoshop to merge different images together to get the desired result also make sure that your main character's face is well lit and as human-like as possible for the next step to work though it will often also work with other types of creatures for the facial animation we are going to use did.com and this is probably the easiest step of the whole process just click on create video and here you can upload your own image it will then analyze your image and if something is wrong with it it will tell you for example I had to redo some images because they were just way too dark and it couldn't find any faces in it so the image is uploaded and I'm just copying the script over here and then I can choose one of the voices and it's also cool that for many of the voices they have different styles so I chose the whispering one because it sounds creepy life is good once you accept them but she could also be cheerful life is good once you accept them embrace the monsters and you will find that they are not so terrifying after all these voices are pretty great but if you want to generate them in another tool or use your own voice you can also just upload your own audio file right over here so once you're happy you can hit generate video and for this short video it only took a few seconds so you see this is an amazing easy to use tool to quickly bring your characters to life my only two points of critique would be that the final video resolution leaves something to be desired for it's not that great and you don't have any control over the final facial movement and expression so all the videos generated with it kind of look the same and a bit creepy but for my horror film this was actually exactly what I was going for in the next step we want to turn our sequence into a 3D sequence so we can add the camera moves and effects like depth of field to do this we'll use the depth maps for stable diffusion web UI extension which I already used in my last video to generate 3D and painted models for a short film about the city on Mars take a hike or take a ride on a martial Rover and immerse yourself in the natural beauty of this alien world be sure to check that one out if you haven't seen it yet in short this tool will generate an image for you that represents the depth information of the original image in black and white so white pixels are closer to the camera and black pixels are further away and we could now use the batch process function of the tool to generate such a depth map for each frame individually the problem is it will probably take a long time and flicker like crazy because the tool only looks at each individual frame but it doesn't look at the frames in context actually let me just try this I I haven't even tried it out yeah see what I mean if you were to use this to build our 3D sequence it would flicker like crazy especially in close-ups or if we add additional lights into the scene fortunately there's a very easy solution for this problem first we convert the video generated by the ID into an image sequence and then we generate a depth map only for the first frame then we'll use absinthe a program that analyzes the optical flow of the individual pixels of the video so we can stick images onto it and we're just going to use this to track the depth map we generated from the first frame onto the rest of the image sequence and since there's only so little movement in the videos generated by the ID this works very well with only that one keyframe and see how that compares the background stays completely stationary and only the face moves that's just what we want we can finally move over to blend and create the shot so in blender the first thing you want to do is create a new plane and then we're going to create a new material and open our image sequence in an image texture node and you could also use the import images as planes add-on for blender but that created some weird bugs for me with a displacement map so that's why I'm going to do it this way to get the right aspect ratio we just have to set X to 1.78 and now we have our 16x9 aspect ratio next I'm going to rotate our image so it's easier to work with bring it up a little bit and then we have to add some modifiers the first one is a subdivision surface modifier set to simple and cranked up to the maximum and actually I know that we're going to need another one so I'll add another one behind that and set that to two the next thing we're going to do is create a displacement modifier where we will click on new and open our absence generated Dev map sequence see we have our 3D image but it looks really weird so we're going to change a few things first for the displacement texture we have to select image sequence and then type in our number of frames that we use and hit Auto refresh we have to do the exact same for our file texture so we go to image sequence put in our number of frames and hit Auto refresh and now if we hit play we can see it works but it still looks really weird because this looks still very pixelated so behind the displacement modifier I'm going to add another subdivision modifier and now it starts to look correct for now I will remove the spatula and the roughness of the image in our bsdf Shader and plug the color into emission as well so we get the actual colors from the original image so the first thing I did for this shot was to add the backwards motion and I added some noise modifiers to the animation to create this handheld look and imply that maybe even the camera operator is scared and backtracking and slightly shaking as they move backwards the next thing I did was add the depth of field effect that starts really strong very close very personal shot and then we move backwards and the effect decreases and the background is revealed a very cool thing about this technique is that you can also add new lights to the scene so I added this flickering light to make it even more unnerving and again I just used noise modifier with another limit modifier to create this flickering effect I didn't do it for this shot but of course you can also create volumetric lights and atmosphere in your scene another cool trick to kind of fake more detail is to add a bump map and you can just do that by plugging the color of your image sequence into a bump node and then plug that into a normal and you can see how it it adds more depth more texture to it and this will actually interact with the light so that's a really cool effect especially for creepy shots like this also now you can bring back the spatula a little bit and the roughness and play around to find your desired look these are just a few options to play around with to enhance the Cinematic qualities of your image but keep in mind that you're still a bit limited with how you can move your camera but if you want more flexibility there's also a way to do that for example in this shot of this creepy talking head in the landscape I wanted the camera to move around a bit more so in Photoshop I separated the head from the background and then I just uploaded the background to the stable diffusion web UI Dev snap generator because there you can not only generate 3D depth maps you can also generate 3D in painted models check out the last video if you want a full explanation for that but then I brought in this in-painted 3D model of the background and used the same technique that I just showed you to add another plane with just the head in front as some optional final touches I also added some dust stock footage and the colors were also a bit on the pale side and I wanted a more intense vibrant look so I made the colors a bit more Punchy and DaVinci Resolve while also adding analog camera effects like hellish and vignette and chromatic aberration and here are the Final Shots again [Music] once you accept them embrace the monsters and you will find that they are not so terrifying after all okay now that I shared this I can finally go back to work on this short film if you like this video and want to see more AI filmmaking techniques make sure to subscribe and as always if you're using any of these techniques please send me a link to that work I would love to see it see you next time

Channel: Mickmumpitz

Views: 315,282

Keywords:

Id: 0tFe9dashgI

Length: 10min 43sec (643 seconds)

Published: Sat Feb 25 2023