Runway vs Pika Lip Sync - Compare the Best Way to Make Talking Characters for AI Movies (Tutorial)

these candles really cut down on my electric bill being able to have characters at talk is an important part of making AI movies P just released this a few weeks ago and today Runway released it as well for both Pika and Runway you have to start with an image or a video to sync things to you can't just do it from text to video and add someone talking and so what I did just to be just to be fair so we're not comparing the generators as well I decided to create all of the videos in stable video and I created one image in mid Journey as well so that way we're just comparing how well they synchronize the characters talking here's our cast of characters I chose this pilot because um he's wearing a helmet and sometimes when you do audio sinks these helmets and these mouthpiece kind of things get distorted and it it looks funny it doesn't look very real so I want to test that with both of them here's someone with kind of human features but it's more of a mythological looking character I wanted to do this one because it's a character who's kind of in profile you can see both of his eyes the whole time and his nose sometimes the profile shots don't do very well so I'm testing that this is the last one so one one thing I want to do with this one is also do a longer video so all of these you'll notice are 4 seconds long I wanted to do one that was a little bit longer to see how long we could push a talking character so if I start in Pika I'll go to my explore tab I'll click image or video down here at the bottom I'll choose the space pilot for the first one and then I'll click this button here that says lip sync so both paa and Runway have these options where you can choose voices but since I already have 11 Labs I've generated audio for each of them that way we're not comparing different voices we're comparing the same exact voice with the same exact video it it makes a better comparison that way so instead of choosing a voice from here I'm going to upload an audio file down here choose my pilot and then it gives me a little preview here I'll attach it and continue and this little bar here shows me that I can choose the whole the whole audio will be covered in this video so the audio is shorter than the video and then I'll just click generate all right and then I'll do the next one so I'll just um close so this is by the way this is another project that I'm working on for consistent characters so if you are interested in using mid Journey consistent characters and talking characters subscribe because I've got another video coming soon all right so we're going to go image and video again choose our Gorgon looking woman go in here choose hers select attach and continue and generate all right so those are generating so I'm going to do the other ones same thing for this monk we're going to go here then we're going to add that file attach and continue generate all right then for this last one I'm going to choose a longer audio file for this one so I'm going to do this with the 4 second video attaching continue and you'll see that it knows that the video isn't long enough to cover this audio that the video is only 4 seconds long and it tells me here your video is longer than your audio so I actually brought this into cap cut and I basically just took the video and then I added it again and reversed it and it matches with the audio really well too all right so what I'm going to do is add that longer video now so I'm going to go in here this is this longer video and I'm going to go to lip sync do the audio file and now it doesn't give me that warning I can do a 9-second clip and it's not going to cause any problems so let's generate the long one now let's take a look at what would happen if we decided to do the image version of that so I've got this image here um so this is just a regular still image and I wanted to lip sync this longer audio so the same audio that's 9 seconds long so even though it's an image I can't do it with these generic voices um with where it's just using the text audio um I can see like here it's like okay it's got 7 Seconds great I attach and continue but it's still limiting me to 3 seconds so if I choose a still image and a long audio I only get 3 seconds but if I upload a long video and a long audio I can do 9 seconds so that's just a little tip if you're trying to do if you're trying to do longer Clips this one's still going so I'm going to go to Runway right now and see how they do if you're on the the runway page you go down to generative audio and I I like this interface a little bit more one reason is that you can actually preview the voices as you're going through and then there's all of these um sort of templates that you can choose from they're fine we're not going to use the templates just cuz like we want to make our own stuff right so we have two different things this is where we will add our face and this is where we're going to add our voice um so let's just start by uploading the audio we're going to take this one start with our pilot again so I'm going to upload the face to the pilot it's going to detect it so it keeps the motion here which is really good um I like that blink that he's got and then we'll just this is the audio this is the video we'll generate it let's do another one all right so we've got our audio we've got our video do our monk generate this is where we upload our longer audio and our longer video so now we'll try and upload the longer audio and the shorter video so here it says video shorter than the audio will be modified to match the length of the audio um so in this case I think what's going to happen is it will abbreviate it so it's the same kind of thing um I actually like the Pea version a little better I like this part in Pika where you can actually see that it's going to get cut off there might be just one part of this that I'm more interested in I can adjust it on the Fly here and just generate the piece that I want so I I like this a lot more in Pika so there's not currently anything like that in Runway um I'm sure they'll come up with something right I'm going to download my Runway assets which I can do in bulk which is nice so in Peak I have to go and click the download button on each of the videos and download them that way all right here's the Pika pilot I'll fly anything except a Boeing 737 and here's the runway pilot I'll fly anything except a Boeing 737 I really think that the runway one is a lot better there's more facial expression in the cheeks I'll fly anything except a Boeing 737 the blink is really nice too I know the blink is in the original there's like a little bit of like jitteriness in the Pika mouth at the end there where it's trying to do something that you don't really get that here for both of them this was the test here to see how did it do with this kind of attachments around their faces so I think it did great like I didn't notice anything here there's something right up against the mouth and it there maybe a little bit it did a little bit of like you know cuz it's connected but that's not too bad although you know I think Pika didn't do that as much it so it separates so in Pika it does a better job of separating this out from the mouth let's try our Gorgon here next these candles really cut down on my electric bill it's pretty good no complaints let's see how it does in Pika these candles really cut down on my electric bill again there's the little bit of like weird lip quiver at the end this feels more like a ventriloquist mouth if you know what I mean I feel like the runway ones capture more of the facial expressions that happen when they're talking like you can see these these muscles moving as she's expressing things I don't think you get that here you just get the mouth moving so runways I think way better which should I do first do a lingo or so Runway wouldn't do this one at all and P did which is good but you can also see the quality like this is unusable at the same time so they did it but it's also not something that I would maybe use so maybe there's like some way that they know like all right if it's if we can't match it to this percentage um then don't do it and that's what's happening with Runway maybe whereas p is saying like let's just give them something and let them decide if it's worth using or not maybe that style would actually work for the type of character that I'm doing if it was like a horror movie or something all right so this is our longer clip let's see how this one did in Runway even before this whole thing started I always knew how it would end so before it ends I'm going to remember how it started again like I can't explain but it really makes it feel more real when you see all of this all of the facial movements while the character is talking um and when you're adding when we're adding these camera motions in you know you do a camera Motion in stable video or something um it really feels much more real let's see how P does even before this whole thing started I always knew how it would end so before it ends I'm going to remember how it started like do you see what I mean like when there's not that facial movement it's just the lips so it's it's literal lip syncing whereas I I feel like what Runway is doing is more like actually how people talk so in my mind like Runway here is uh way ahead of the game as usual I think there's a lot of like little micro Expressions that Runway captures that Pika doesn't that really helps it sell itself as a real person talking um we're still not totally there yet in terms of these two tools I feel like it's getting very realistic and on that note so I have another video coming out around consistent characters and having them talk and creating movies with that I think between talking characters in Runway and Pika and consistent characters in mid journey I think that the next few months are just going to be a an amazing time to be an AI filmmaker so so be sure you subscribe so you can get more tutorials like this thanks for watching

Channel: AI Video School

Views: 16,717

Keywords:

Id: Isw1rew8qrI

Length: 10min 27sec (627 seconds)

Published: Sat Mar 23 2024