Animate MidJourney Images - Full AI Animation Workflow.

hey welcome to this AI animation tutorial showing a new workflow that uses AI tools to bring life to characters and scenes cheers for watching oh and if you're on YouTube please press subscribe hey my name is John Draper and I'm an animator Studio owner and founder of AI animation.com a new site we've developed to try and showcase creatives using AI animation to create high quality professional animation it's early days there but uh please head over there and if you're interested sign up for free in this tutorial I'm going to share a workflow to bring AI generated images from sites like mid-journey stable diffusion leonardo.ai Dali whatever your chosen platform is and show how you can take that image and through various AI tools and some traditional digital animation techniques bring them to life adding some subtle 3D depth and animated head rig and then adding a few extra touches to really bring the environment to life the AI robots are coming but it's all going to be A-Okay because we can go to the beach there are some selected compromises we've taken with our workflow to be able to achieve really efficient production with a nice quality of output but there are some limitations for example we can't animate hands and arms or have the character walking around easily but as these AI tools mature over time and there's things like Wonder Studio from window Dynamics where you can use filmed footage of yourself or an actor and have it turned into a CGI character as well as export that mocap data or the 3D character so you can potentially combine that with some of our approach here to place your 3D characters into a scene from mid-journey or whatever AI tool you're using so with that in mind as we continue to evolve and test out different processes tools and workflows please if interested press subscribe um as we share more and more videos on our new YouTube channel oh and if it wasn't clear we're going to be making a video like that one here we go so to start with we're going to make a character and the scene using mids journey and if you're not familiar with it um if you head over to the mid-journey website um follow a link and it will send you over to Discord um as it's run on a Discord server so you actually have a conversation with the bot so press forward slash and write imagine press enter and then you write out your text problems I've got one here already made so um one realistic smiling but tired looking 3D cartoon CGI geeky child with oversized glasses staring directly into the camera standing Center of room next to desk in 70s Galactic spaceship interior messy artist bedroom neon lighting there's various prompts that you can use when chatting with mid-journey to get certain outcomes so here you put dash dash AR space 16 9 and that will help create a 16x9 image rather than a square so let's press enter and see what it churns out Okay so we've got some results here just make that full screen um so they're nice they're pretty cool I think we're going to try out uh the one here which is quite realistic less uh less Pixar and slightly more photo real so we're gonna jump out of there and we're gonna press U1 which will upscale number one and then we're going to press zoom out which is going to zoom the camera out and show us more of the scene okay so it's generated four images for us there and we can choose which of the four we prefer quite like this one with almost a sort of a sound mixer desk I think we're gonna go with that one so it's upscale number two there we go and press web and then press save to download the image to your computer now I've opened up the close-up image of our character on our scene from mid-journey in Photoshop and this is the new version of Photoshop currently in beta or beta um and we're going to use the Firefly powered AI tool that allows us to do generative fill and adapt the image through AI it's really quite incredible so we're going to use the Marquee tool draw around our character and we've got this little text prompt here and we're going to type in remove boy from scene and press generate and there we go that's actually incredibly well done better than average so we're just going to save that image as a clean background plate that we can use Save at then we're actually going to take that image and copy it open up the wider image and we're going to paste it in here and then I'm going to scale it down and just by eyeballing it replace the current scene we might do a little bit of um fading out choose the Eraser tool choose a soft brush just so it Blends and then we're going to save this as The Wider View and then we can close Photoshop now we're going to create a 3D depth map using a wider image of the scene without the character as we've painted them out in Photoshop so we're using a free tool that runs on huggingface.co called Midas huggingface.co is an amazing place where you can harness the power of various different gpus and computers to run different AI models and other bits of software but Midas works for free I'll include a link below the video description so on here you can just click and choose your image so we've got the wider image there without the character press open press submit it takes less than three seconds to create the step pass that we can then use to drive our 3D animation later on let's just press download we're now going to hop over to a bit of software from elevenlabs.io which allows you to create very lifelike AI generated voiceovers so we're going to use this to bring our short script to life that we can then use to drive the animation for our character so once logged in you can hop over to the voice library and try out different voice samples every two things others can do and will do blessed are the hearts that as we are liberated from our own fear we have committed the Golden Rule to memory let us now committed to life fantastic um so you can try any of these and then you can add them to your voice lab you can have more and more depending on how much you pay for your subscription with 11 laps um so I've got three here because I'm on the most basic tier and then go to speech synthesis choose your selected voiceover artist and try different settings as you go so we need a short little script so let's try out the AI robots are coming but it's all going to be A-Okay because we can go to the beach press generate the AI robots are coming but it's all going to be a okay because we can go to the beach so that's okay we can run it a few times and each time you press generate you'll get a slightly different outcome because it's based on the AI and it's not always the same but we can also include some pauses so the AI robots are coming dot dot dot jump down a few lines but it's all going to be a-okay full stop because we can go to the beach exclamation mark let's try that out the AI robots are coming but it's all going to be A-Okay because we can go to the beach there we go that's good enough for us we can press download and it downloads the audio file to our compute we're now going to hop over to a piece of software from Dash id.com did um that can create animated avatars from a photo or a stylized but quite realistic uh cartoon character just by providing a piece of audio or a script and it will create its own voiceover but the voiceovers aren't as good as those from 11 Labs so I'll quickly show you how we do that login press generate video and this is where you can add your script to upload a voiceover and also choose your presenter we're going to add a new image for our presenter so press plus upload our image with the close-up of our character press enter give it a second think about it and identify the character in the scene and then we're going to press upload your own voice the AI robots are coming and just press generate video and there we go just a few seconds later it's created the video Let's quickly press play to preview the AI robots are coming but it's all going to be A-Okay because we can go to the beach very happy with that so we're going to press download save it to the computer now we're going to use another amazing AI tool from runwayml which allows you to create all sorts of AI generated video content using text prompts or you can use video sources and image references to create unique videos and it's actually really a really really cool tool which will deal with in a separate tutorial but what we wanted for is to actually do some simple background removal so we're going to log in go to assets upload our the AI robots are coming our talking character from did and then we're going to press edit videos we're going to choose remove background then from our assets up here we're going to grab a character and we're just going to click once possibly twice done pretty well but it has added some of these materials so we up here in the right we can choose exclude and we just add a little dot over here add a few more of these red dots and there we go let's remove those and then press done masking and then press export 720p is fine for us MP4 press export video and then that will save it out to your asset so pick up the three bars on the top left press home go to assets and you can see the video being exported here then we can click on the video the AI robots are coming download to save it to the computer okay as a quick recap on what we've created so far we have our full scene created using mid-journey with the character removed using Photoshop we have our depth pass also created using that same image run through Midas on hugging face and we have our animated character brought to life using did with a voiceover made using 11 labs and the background removed using a Runway ml and now we get to do the fun bit and hop into Adobe After Effects bring those different assets together and bring it to life through some more traditional digital animation techniques grab our three files the depth files the background and our character with the background removed create a brand new composition let's do 1920x1080 and we can do a frame rate of 24 frames per second and we're going to grab those three assets bring them down to here um scale them up until our background fits nicely within the scene so we have our background our depth pass and our character we can ignore the character for now um so the depth pass we can bring that to Life by right clicking the effect choosing distort and displacement map choosing our depth fast and then changing the displacement to use the luminance channel for both the X and the Y so that's zero and then if you up these values you can see there's a cool 3d effect they've push too far just start to break the image but you can see how it does create this cool 3d effect for what is essentially a 2D image which is pretty cool and then we can grab our character bring him up and then choose keying key lights screen color and pick the green color and then if we hop back in here and actually if we import the wide view with the character still included as a reference match the scale of the other one so 132 and then reduce the opacity scale him down to match what we had in the original and using the mask tool just draw a mask at the bottom double tap M and press invert to invert the mask add a little feather of two pixels scale him just a tiny bit more composition length to five seconds A4 just so we know what we're dealing with and we've got our call uh background Distortion using the displacement map we're going to hide our character for now so we're not actually interested in him um and we're going to pre-comp all of this and call it uh background scene and then we're going to create a new camera let's stick with a one node camera open up the transform properties and we have a position and then we're going to open up our pre-comp so we've got both compositions here and we're going to grab our background and move it so we've got our background composition and our Master comp here with the camera we're going to link the displacement map values to our camera so we drop this down go down to displacement map horizontal and our vertical so we click our maximum horizontal displacement use the pick whip and select the x value on the position of the camera if you drop down here you can see the expression and we're actually going to we want it to stay at zero so we're going to link it to the camera but then we're going to type on the expression here at the end just put minus 960 which is the current value of the camera and then we're going to do the same for the vertical we're going to pick width choose the Y value position from the camera roll down to see the expression and then put minus 540. then close this background scene for now in our composition View Window we can press C to cycle through the camera controls we want this one with the panning tool we can pan left and right using the camera to affect the displacement map in the pre-comp give us that call 3D control next we can do some things to improve the scene even further so if we set some keyframes for the position of the camera double click on the camera and change it to a two node camera which will give you your point of Interest as you can see in the top view here and then with the point of Interest set up and push really far away you can then set keyframes as you pan or rotate around the scene we can also remove that white Edge to the scene by jumping into the pre-comp clicking our background and then ticking wrap pixels around on the displacement map effect another thing you can do is turn all these layers inside the pre-comp to 3D whilst you're here you can copy the character jump back into the main composition and on that background scene tick the constantly rasterize button now when you rotate around the scene you get a nice 3d effect but you can also zoom in with the camera and still have that 3d effect even when zoomed in now we're going to paste in our 3D character from the pre-comp that we copied earlier and as we pan across our scene using our keyframes you'll see that they're in the scene and they are moving with the camera so already a 3D layer but they're not in the right depth position for our scene if you're very lucky depending on your mid-journey artwork and the character position and your depth pass they might just be in the right position already but it's unlikely and so ideally we just grab hold of the said Direction and push them back in position but it won't work because technically they're on the same plane as our background image and our background image is just a flat image it's only that 3d effect we've got going on in the pre-comp that makes it seem 3D so the way around this is to create a new adjustment layer make that 3D and position it between the background and the character and that breaks the hierarchy of the 3D in the scene which is sometimes quite annoying but in this instance it's quite handy um so we can push our character back to where we think they should be and position them by eye and there we go we've added our character into our scene so now when we Pan the camera move around they're nicely situated exactly where we want them and it was at that annoying point that my computer decided to freeze up as the hard drive was nearly full and it turns out that my cloud storage wasn't syncing so I've had to jump through and redo all the steps to get to this point all exactly the same but the background is now slightly different from when I did the character removal in Photoshop and I've also been through and set some nice camera moves just panning left to right and there's jumps to a close-up jumps back to a wide now we could stop there and the video would look pretty good um but I want to make it a little bit fancier so I'm first of all I want to fix uh there's some blue spill in the hair from when we did the background removal using Runway which you can see up here isn't looking too hot so I'm going to duplicate our character layer draw another mask and get rid of the lower mask on that one and we're going to use our key light and we're going to duplicate it and grab some of this blue that's in there and then we can unsolo that character copy that mask onto the on beneath and set that to a subtract and then I don't know if you can see this on YouTube but there's a very small line there so on that mask we just duplicated we can just reduce the expansion down to five pixels and that's got rid of that line and there we go the hair is a lot better not perfect we could spend more time refining The Mask or picking some other colors from there but uh it's certainly better now to make the environment feel a bit more rich and Alive we're going to be adding some lens flares and some dust just floating around in the scene and the dust is nice and easy I've just grabbed a stock clip of dust particles from envato elements and they're just floating around they're moving slightly quickly so I'm just going to crudely slow it down using time stretch said it's 200 and then change the blending mode to screen so we've now got dust floating around in our scene and when we cut to the close-up camera I have separated the layer up there and made it slightly larger so the particles in a different position we can also make those layers 3D and position them so as the camera is moving the particles are moving in 3D space as well now to add some 3D lens flares to the scene we're going to use optical flares from Video Copilot which is a paid plugin now let's do this we're going to duplicate our background scene drop that down here call it light sample jump into here pre-comp the background layer and call this lighting dots and that won't affect our original background pre-comp but yes we have this local lighting dots I'm going to jump into here right click and turn that into a guide layer so it's not actually going to be seen in the final thing we're then going to use the shape layer to create some tiny circles which would be markers for worried like our lights and lens flares to appear in the scene another one I've just duplicated that but another one on the side of the Earth or the window in the background another one over on this Orange stick thing here I'm actually going to change the color for that one to Orange maybe the one by the Earth will make that slightly blue okay so we just hide that and now I don't know if you can see these on YouTube but we've got these tiny dots and you can play around with the scale to create bigger lens flares as well as the intensity of the opacity so let's jump all the way back to our main composition we're going to create a new layer be a solid or an adjustment layer call it lens flare and then go to effects and then we're going to right click choose Video Copilot optical flares and we're going to change the source type to luminance Source layer it's a light sample and we can see we have optical flares there already we can play around with the brightness scale you can set the center position for where the optical flares are pointing we can then go down to render mode and choose on transparent and now when we spin through the scene making sure that constantly rasterize is turned on for the light sample layer as we pan through the optical flares track with the scene and including when we jump to the close-up you can then play around with things like flicker to have some movement to the lighting play around with the settings in optical flares as well as the size and opacity of your shape layers for the lighting dots there we go that's all looking quite cool just going to add a black solid with a rounded rectangle tool mask double tap m invert the mask and add a simple black vignette to the side of the scene plus we can add an adjustment layer choose color correction right click and choose the effect color correction Lumi tree color which is the same coloring system built into Adobe Premiere if you drop down basic correction you can adjust exposure contrast and things like that also if you open up creative you can go through different looks and adjust the intensity of these which can quickly allow you to try out different things and see how they impact your image now you could evolve this project even further and spend more and more time you could add some subtle shoulder movement and head tilting into the character by using the puppet tool built into Adobe After Effects you could play around with the depth map by applying a brightness and contrast to that layer so you can shift the depth you could run some camera tracking on the scene directly in After Effects which would allow you to then place things in the 3D environment or drop lights in and Link lens flares to actual physical lights as well as Place 3D Motion Graphics into the environment with ease I've opened up the voiceover audio in Adobe Audition just to do a quick pitch shift so that it's slightly higher and Suits our character and this will look better so we select all the audio press FX climbing pitch pitch shifter and then we can adjust our semitones but it's all going to be A-Okay because we can go to the beach the AI robots are coming but it's all going to be there we go I think that's much better now I've dropped that pitch shifted voiceover track to the bottom of our composition and muted all the other channels so that's everything I'm going to cover in this tutorial and I'm going to render the scene out so you can watch it in all its Glory the AI robots are coming but it's all going to be A-Okay because we can go to the beach and that's a wrap hopefully this tutorial has been helpful and maybe introduce some new AI tools or shown some professional motion graphic animation techniques using Adobe After Effects all to help bring a static AI generated character and scene to life um if this is the sort of thing you're interested in please press subscribe as we'll continue to develop this workflow try out tools like Wonder studio and other AI animation tools and we'll be sharing what we discover on our new YouTube channel also if you are an active AI animator or creative using these different Tools in your production and you're producing high quality work please head over to AI animation.com it's our new site looking to Showcase creatives give them a platform show off their latest work and detail the skills and the production Styles they're producing um with a view to them finding clients paid clients and networking and everything like that alright that's it thanks very much have an awesome day cheers

Channel: AIAnimation

Views: 177,008

Keywords: ai animation, animate midjourney, stable diffusion, 3D ai animation, animate ai image, ai character animation, ai animation workflow, generate ai animation, bring ai image to life, stable diffusion animation, ai image animation, turn ai image into video, ai animation tutorial, midjourney animation tutorial, ai images in after effects, 3D depth animation, ai characters, artificial intelligence animation, ai animation news, generative ai animation, ai video, text to video

Id: t5Vq4ahmn74

Length: 24min 15sec (1455 seconds)

Published: Thu Jul 20 2023