Image-to-video comparison: Pika vs Runway (August 2023 - newer version in the description!)

I wasn't expecting to be excited about image to video but I guess I am I can really see how this will help AI filmmakers bring their imaginary worlds to life I tried out the same reference images with two different tools Pica labs and Runway to try and understand the difference spoiler Pico Labs gives you more control by letting you write text prompts to guide what animates in the image but Runway gives you much higher and sometimes stunning quality but with almost no control and if you're looking for the free option that would be Pica Labs alright let's take a look the first image here is a geodesic dome desert community and I wanted to have someone riding by on a horse with Pica Labs I was able to do exactly that by writing a prompt to go with the image it looks pretty good I mean it's kind of a weird looking horse but it's exactly what I wanted here's the runway version this is like completely stunning that I can take that image and create this video here they are side by side so the the runway footage is absolutely in my opinion the quality is much better but the Pico one is what I was actually looking for I didn't just want to animate the video I wanted to have something specific happen in the video so in the second example here I had someone with headphones on Staring at the window of a bus and so what I wanted to do was animate the traffic outside the window so you'd see the traffic go by so and I can do that I'm not sure why her hair is glowing maybe the window is down but you can definitely see that the traffic is going by things are passing by on the window with Runway we get some very realistic motion of her head while she's moving around a little bit but it's a stationary bus so it's kind of a trade-off it's a great looking shot very realistic but the trade-off is the motion is her and not the background and I wanted the background to be the thing that was in motion being able to add a a text prompt to the image allowed me to do that here's one where Runway clearly shines it's sort of an apocalyptic London in the future the whole thing is on fire and as much as I tried with Pika I just couldn't get the right amount of fire just as kind of all over the place and and not very realistic with Runway this is an amazing shot I think you get the like if look at the look at the way that the Flames reflect in the water too I think that that's amazing I think they do a great job so this is what Runway is awesome at you have the establishing shot you have sort of like your your Scenic portrait and it just creates an amazing animation from that all right backed off horses um I wanted to have this uh this cyber cowboy riding a horse through a town um and I just kind of pictured like maybe the horse and the rider are moving closer to the camera um I wanted to have dust in the background like the horses kicking up dust they're both moving in different ways but they're not really having the horse walk toward uh the camera at all and and I know that's a lot to ask for I totally get that but that's that's what I was hoping for with that one so I'd say they both kind of failed this test so this last one here there's this guy walking down a path in the Autumn I wanted to have him walking down the path but also a few leaves falling from the tree I didn't want a lot of leaves like I could picture just like leaves tumbling down I just wanted a few leaves here and there um so Pica Labs is pretty good there's a few there's like a lot of wind wrestling with Runway there's this one which is like the animation like there's the camera movement of the trees um the the movement like you can actually see he's taking steps forward and moving where the Pika Labs guy is almost like on a treadmill just kind of walking in place um which is fine because it looks like a natural Cadence of walking but he's not moving whereas the person in Runway is actually you know making strides and you can see on the leaves he's he's progressing on the sidewalk um which I think is kind of cool this brings up another thing with the runway um if you look at the reference image um it's really far away from the image that I gave it the Pika Labs one looks very similar it's a lot blurrier it's not as sharp but it's definitely the same backpack the same hat the same pose so I added a text prompt in Runway and it completely changed the look of the final video just to show you what this looks like in Runway when you add a text prompt so here's the picture of the woman sitting on the bus if I use the same exact prompt that created the image and mid-journey um this is what I end up getting and it looks nothing like the the image that I use as the reference here you know so I thought well maybe I'll just maybe I'm being too complicated let's just simplify it and tell it what we wanted to change in the video so I just said the city passes by out the window and again it's like now there's we're not even on a bus anymore so like I said stunning quality absolutely with Runway um but the the level of control that you might want it isn't there yet I'm sure it will be eventually but it's just not there right now you know with pico Labs you have more control because you can do the text prompts you can add horses walking through uh you know the domes you don't have to just zoom into the geodesic domes in the desert you can say I want something to be in that scene as well so it's not necessarily that one is better than the other it's just that they do two different things even though they're both image to video just one more example of what I mean with pico Labs is that you know here's a here's a portrait mode video of a DJ and it's just like okay the DJ is spinning records um and that looks kind of weird but it was like okay the DJ's hands are up in the air so you can get that flexibility to try different things and especially right now because it's free you can just have that ability to explore and bring your still images to life in different ways I think this is a really cool addition to an AI filmmaker's toolbox it's pretty fun I mean I can really see this being a game changer when it comes to establishing shots or setting the tone or the world that your video is going to be in so go play around have fun and uh let me know if you make something cool foreign [Music]

Channel: AI Video School

Views: 110,809

Keywords: pikalabs, pika ai, runwayml, runway gen2

Id: CB_Y-5yaQ-M

Length: 6min 23sec (383 seconds)

Published: Fri Aug 04 2023