AI Generated Animations Just Changed Forever | Dream Machine

Luma Labs recently released dream machine and some of the outputs coming out of this tools are really really amazing and in this video we're going to be exploring how we can use this for animations I'm going to show you a little clip I put together just to test out the tool and then I'm going to come back and break down exactly how I created that so that you can create the same in your project let's watch the clip together I am sure going to miss this place but it's time for me to head out to fulfill my dreams in Aurora we're dreams come alive wa Stark corporations I'm going to work there someday welcome to Aurora fresh vegetables no way hi I am Max and someday I am going to build a rocket to take us all to the sun they can laugh all they want but I think I got what it takes and it's all here in my book all I have to do is finish school and get a job at stack corporations yeah yeah yeah where have I heard that before I guess I do say that a lot I don't have it I [Music] promise they are after my notebook where are we going just that operation you said that was impossible that's when I thought you were pray easy I can't believe I have waited my whole life for this one that don't let achal cruises take it away from wow this is my city and Nobody messes with my [Music] friends so everything you watched including the music was generated using one AI tool or another and I don't know about you but I was very impressed by some of the animations I was getting out of dream machine of course it's not perfect there are tell tals here that you can tell that it's AI but I'm sure if I spent a little bit more time with the tool I can fix some of those issues here and there but some of this like this opening shot right here of the alarm clock I could use that as is in a professional project and a couple of all the other shots too some of them are actually pretty pretty pretty good this video is going to be split into two parts the first is just going to be an overview of dream machine how to get access and how to use it and in the second part of it I'm going to show you the workflow that I use to generate that demo so doing your lip sync generating your your images including generating the music using AI I'm going to show you how to put all of that together it's going to be on a high level because if I go into all the details then this video is going to get too long but I'm going to post a longer version of that in the member section of this channel so if you're a member and you joined as a member you can go and watch that it's going to be a long video and we're going to go step by step everything that I did if you want to watch that longer version you can check it out on the member section of this channel but let's get straight into the tutorial to access the tool simply head over to lumabs the good thing is that they actually have a free tier where you get about 30 Generations per month for free and this limitation is per account so this might not be a bad time for your mom to get an account if you know you know but you do burn through those credits pretty quickly especially when you're doing your first project experimenting so I had to move to one of the paid plans to finish that project that I was doing cuz I generated almost 100 generations to get the right shots another thing is that if you're on the free plan sometimes the generations are very slow because the tool is very popular right now so there's a long wait time in the queue one thing I noticed is that about 12 midnight eastern time it seems like the traffic is really low at that time so that's a good time to have your Generations done they move very fast so I have dream machine open here and the UI is very straightforward you can do text to video and you can also do image to video so let's try one of the example prompts they have here which is an extreme close-up footage of a young sailor woman with a concerned expression during a rain storm I'm going to leave enhanced prompt on what this does is that it tries to modify your prompt to make it more suited for the tool and let's hit generate and there you go there are the results quite impressive looks like some old theme um SD footage but looks quite convincing if you didn't think twice about it you might not have thought that this was air generated so this is quite impressive let's also now try some animation related prompt so right here I said an animated scene featuring a close-up of a for master and you can see that looks really really good so let's try another example this this one was a Pixy style animation of a girl playing in the playground and this one was a little bit me it's not that great um so I'm going to show you the trick towards getting this pixel style animations to look good and that's where we come to the image to video in my opinion this is where the tool really really shines where you generate your image in a separate image generation tool like dly or mid journey and bring it on here to have it animated for you so I have here one of the images I generated for Mid Journey for my project so I'm going to go back to dream machine Ma and click on the little image icon right here on the top and then upload this image and this time around I'm not going to type any prompt I'm actually going to give it the creative directive and just click on generate and see what it does with it and for this one this actually came out pretty good most of the times you get a lot of weirdness if you give it the complete creative directive so here's an example where I didn't have a prompt you can see there's a weird looking hand coming out of there and that's the same shot here where I gave it some prompt for what to do that's a more usable shot here's another example with no prompts you can see a guy moths a hand comes out very weird but now when I give it some prompts as to what exactly to do that's a much better usable shot so in my opinion the image plus describing what you want it to do gets you the best results from my many examples that I've tried and sometimes what I do is also take the exact prompt that I use in the image generator so like the prompt I used in mid Journey or Del and paste it here and here's an example of that where I attached this image and I gave it the exact same prompt that I used in mid Journey which is you know a close-up of a camera animated style character skating down and Ali and you can see this shot right here was very stable there was not weirdness going on it really adhere properly to this font and then what I do is that I then add some SLE animations that I would want to happen in addition to that like in this case I added that the boy is blinking his eyes and if you look through the end there you can see his eyes blinking a little bit so he did adhir to that prompt and it's quite good with that so in this example I did the same thing but I added that it's windy and the wind is moving his hair and as you can see it's doing the hair movement and the physics of that hair movement is really phenomenal the way it's getting that hair movement it's really blown away that blew me away so it does a very good job at that so in my opinion in general if you want to get the best of this generate your image from an image generation tool bring it on here use the exact same prompt or similar prompt that you use in the image generator on this and then add additional kind of animations that you would like to see the character doing moving their hands moving their heads and things like that have gotten the best result using those two combinations if you're enjoying this video so far kindly give it a like And subscribe for more tutorials like this one all right let's move on to the second half of this video where I show you the workflow that I used to generate the little trailer we did in the intro so my step was this create a consistent looking character and then generate all the images that I needed to use for the project I then took that to dream machine to animate them and then I used 11 labs to generate all my voice overs used another tool to do the lip syncing to fix any lip syncing issues and then took all of that that together put it together in a video editing tool put in sound effect music and all of that and the music was generated using a tool called udio and I'm going to show you all of that all right so let's get to the first step which is generating your character and generating it consistently because I was telling a coherent story it was important for me to have a character that kind of looks like Max across all sins I said kind of because if you look at some sins there it's not exactly the same person but you get the point that yeah this is the character we're talking about I used mid Journey for that uh but since mid Journey not free to use I'm going to show you how to do this using chat GPT for free so both if you're on the paid plan and the freed plan I'll show you a trick that you can use using chat GPT to generate your images consistently for free all right so I have chat GPT opened here and you can create an account completely free to use Char PT and as of two weeks ago gpts which are like apps in Char PT are now free to use which is what we're going to be using to generate a consistent character right here if you click on explore gpts type and search for one that is called consist consistent character GPT fast and high quality click on that and then start a chat now if you have a paid account with chpt you can go ahead and just follow all the prompts and create your characters with no issues but if you do not have a paid plan with chpt paste this additional extra prompt and I'll probably put that in the description and the prompt is this before we get started some additional instructions for you which is whenever you want to generate an image do not generate it but just give me the prompt that I can copy and paste in Del we're then going to use Microsoft co-pilot every time we need to generate an image to dump that there to generate it for free and let me show you how that works so I'm going to give you that instructions and you can uh copy that and paste it so first it wants to know what my character should be so I'm going to say a male character all right so it wants a name for the character so I'll just call him Max like I did in my intro and things like that so I have a prompt that I already kept here so I'm just going to dump that prompt and you can pause this video to see what the prompt is but the most important thing is that if you want your character to look like uh that 3D and animation style character just add the word pixel so if you put the word pixel in any image generator most of the times they'll give you a character looking like that 3D animation kind of characters so I'm going to hit enter so as you can see instead of generating the image it gave me the prompt so I'll copy that if you are on a paid plan and you didn't type that instruction on the top it will actually generate an image of Max but since I want to show you how to do this for free I'm going to copy that prompt that it gave I'm going to head over to Microsoft copilot that's Coop pilot. microsoft.com create an account for free and simply paste that prompt here and hit enter all right so you can see we have an image here of Max very similar to what you saw there in the trailer so I'm going to go back to D now and just keep prompting so if you want marks in different scenarios you can just keep asking it to do that and it's going to generate for you um a prompt for that so in this case I'll say create an image of Max in a rusty Warehouse with metallic equipment around Max is doing an experiment on a table so it's a random prompt so I'm going to click on that all right so I'm going to copy that prompt I'm going to head over to Copilot and I'm going to put that here and let's hit enter and let's see what that looks like so as you can see that looks very much still like our character and the results from this is actually pretty good uh almost on par with what I was getting from mid journey and if you wanted to change the orientation of this for a movie because this is all square just click on the little button right here and switch from Square to landscape and that's going to expand the image so that it looks a little bit more like what you could use in a video for Youtube and things like that so this was the exact step I took and I did the same for every single scene that I wanted in that trailer so I went ahead and kept prompting prompting several times until I got it looking right I think I created over 100 images and eventually picked like 15 of them so now after I did all of this I took these images and put them into dream machine to animate them using the way that I showed you in the beginning of this video for all my voiceovers I like to use 11 Labs I still find it to be one of the best voiceover tools out there generated by AI so that's what I used for the trailer and I have an entire video on 11 Labs so I'm not going to go into the details on how to use it I'll put a link to that video down in the description and the good thing is that 11 Labs also gives you a free tier you can use it for free uh might in that video a few people were pointing out the 11 Labs is not free they do have a generous freit here gives you about 10,000 tokens each month and again this limit is per account so this might be another instance where your mom wants to open an account you decide but yeah you can check that video out I'll put a link to that in the description all right next after the voiceovers have been generated the next step is to do all the lip syncing now dream machine would try to do some lip movement if it discovers that the mouth is open or maybe if you prompt it to but obviously that's not going to be in sync with your actual audio so to do that there is a tool I use it's called sync Labs now the tool is not free to use it is kind of free but on the free version you get this huge Watermark so if you're okay with having that watermark on your final video then it's fine you can go ahead and actually use this for free but there is another tool called uh pabs that also has water marks but these water marks are a little bit more manageable that you can use for free it's not as great as syn Labs but it also does decent job I'm going to show you the two and you decide which one you want to use so I have syn labs open here and I'm just going to click on a new project and what you simply need to do is upload your video so I have the video uploaded and then next I'm going to upload the audio so I have an audio right here and then you can choose from the different models I like this 1.6.1 but 1.6.0 also works pretty fine just depends on how high a resolution of an email a video you want either of them works pretty fine and then you simply just hit generate so this is before you said that was impossible you said that was impossible and now this is after when we actually have the lip sync working you said that was impossible so you see that's a lot better and with this weird artifact it does that sometimes when it's a silence at the end but this you can cut out when you're editing this on your video editor all right and on pabs once you go to p. add create an account you just upload your image your video sorry right here and then click on the lip sync option and then right here you can upload whatever audio file you have and then you say attach and continue where are we going that's how that sound it's not bad but you can see there's a lot of flickering and artifacts on the mouth so it's not necessarily 100% great but it's also usable and the water mark they have is just this little water mark right here which for me is a little bit more usable if you want to use this for free and once you pay the watermark goes away also all right finally let's talk about the music so I wanted to keep everything in the AI teamed space so I used a tool called yudo which generates really amazing music that sounds very realistic so I have yudo opened right right here and uh you can pretty much describe the kind of music you want here and put the lyrics and it will generate both the music instrumental and the vocals for the song So What I typically like to do is that sometimes I don't know how best to prompt this I come here under this cover and I try to find something that has been created in the same genre that I'm looking for and then that way I can just copy that prompt to get something similar so all right so for example let's say I wanted to do a hip-hop song I'm going to click on hip-hop and just just check out a few of these examples that they've done to see the one that's very close to what I want so for example uh let's try out this one [Music] here out in the pl Where Stars don't play nice two Captain SP sacrifice what I would simply do in a case like this if I like this one you could if it's shorter than 30 seconds you can remix it or if it's longer than 30 seconds then you can simply just copy the same hip hop style that they have here so you just copy that get over here to S paste that in and you can have it autogenerate random lyrics for you or you can come here and type whatever lyrics you have for your songs and you can do that and generate it and you get a full blown music for that so that's how to use this tool I can cover this tool in more details in another video if you're interested in that let me know in the comments and I can cover that so the final step in all of this is just to use a video editor to put all of this together I do normally use premier Pro but you could use a free tool like cop cut to just simply uh join all the clips together put your music put the lip synced version of all your videos and then add uh sound effects and overall I was really pleased with the outcome now the tool is not perfect but one thing you have to remember is that this is the worst this tool is going to be at I can imagine version two version three up to version five like two three years from now how well this tool is going to get so I really think that this is breaking the barrier to entry for a lot of people who have great stor they want to tell but they don't have enough resources or maybe the right skills to put those things to life this is really breaking that barrier to entry and I think we're going to see a lot more creators coming out creating really awesome content as this tools mature and don't get me wrong there's still Artistry that goes into this cuz a lot of people think that AI is coming and Artistry is dying of like I had to create over 200 images from my journey just to get like 15 of those even on dream machine itself I iterated back and forth and there's so much going to like the music the mood and there's still artistry that goes into it it's just that now the tools are much easier to use and the barrier to entry to learn these tools are lower so I'm really excited for this and excited to see where this technology evolves if you ever create anything using the concept I've taught you today you can tag me on Instagram or Twitter I'm going to put my handles down in the description I like to see what you've created and to also share it with others for them to see thank you for stopping by I'm going to catch you in the next one keep learning

Channel: Emmanuel Crown

Views: 114,013

Keywords: Luma Labs, Dream Machine, AI animation, AI-generated video, AI video tutorial, text-to-video AI, image-to-video AI, AI video workflow, mid journey, DALL-E, chat GPT, consistent character generation, 11 labs, AI voiceovers, Sync Labs, lip syncing AI, p-collapse, AI lip sync, Udio, AI music generation, AI music, AI lyrics, video editing, AI video tools, ai animation, dream machine ai, luma ai, luma dream machine

Id: Y40lrlaiwiI

Length: 16min 55sec (1015 seconds)

Published: Tue Jun 18 2024