Amazing AI Filmmaking is Here: Gen-2 The Ultimate Cinematic AI (Full Tutorial)

hey everyone so last week I did a video on how to take your mid-journey or AI generated images and turn them into cinematic presentation videos if you missed it it's linked below but today we're going into an even crazier Direction uh today we're going to take our images and turn them into actual movies buckle up this is wild So today we're going to take a look at Gen 2 by runwayml it's a text to video generator that honestly is nothing short of amazing we're gonna do a full walkthrough and go over some of the pros and cons of where the AI is right now plus I'm going to share with you some tips and tricks so that when you start using Gen 2 you'll be able to hit the ground running right away currently Gen 2 is Discord based which should make all of you mid Journey users feel quite at home generating videos is really really simple all you do is at the Gen 2 Bot and type in a prompt of whatever it is that you want to see in this case I did a man on a boat on the ocean near some islands and this is the video that it generated I mean that that's pretty crazy when you think about it that man doesn't exist that boat doesn't exist that moment in time doesn't exist and yet we have video of it currently the output from a prompt is about four seconds but I've got a couple of creative tricks that I'll show you later on in the video to extend that time out just a little bit overall I would say the rendering happens fairly quickly I think the longest that I've waited so far is about two minutes which is pretty remarkable when you think about what it's actually doing your video will output at a resolution of 768 by 448 in a 16 9 aspect ratio there currently isn't a way to change that aspect ratio to say like 916 for vertical videos or 2-1 for you know a big cinemascope kind of vibe to it yet I mean it's still early in so I presume that will happen at some point and it's gonna be awesome what it does now that said we do have commands to upscale our video to a higher resolution and we can also interpolate our footage via a command to kind of get rid of that choppiness we're going to take a look that in one second so let's take a look at how that looks with a New York City Scene so I generated this video using the prompt New York City street busy people walking and I utilize the upscale and interpolate commands again dash dash upscale and dash dash interpolate and it returned to me a video with the size 1536 by 896 which you know is a pretty decent size so let's take a look at that real quick yeah it's pretty remarkable so I did just want to briefly point out that a few weeks ago I was thinking about doing a video on text to video and the level of output that we were getting at that point in time via the various tools was this this is like literally nearly the same prompt it was New York City street and this was what we got so and and now we're you know and now we're here we've come a long way in a very very short amount of time so I can't even imagine where we're going to be you know two years from now so having done our establishing shot I wanted to start populating our city street a little bit so I prompted a businessman walking on a phone and this is the video that we got um that's pretty crazy remarkable um yeah there's some Jank in the hands but you know that's to be expected we're very early into this technology but still that's super super impressive and just to give you an idea of the difference between running upscale and interpolate on versus off here's the same prompt with those commands turned off so with upscale and interpolate turned off we get this which my God look at the size of that phone that's amazing I hope AI weirdness never completely goes away because I I love it um yeah I mean overall though you can see that there's a lot more choppiness between the frames and our resolution is definitely taking a hit so I continued on and generated a few more videos and strung the whole thing out and cut it together and this is what we got [Music] so yeah in about 20 minutes I had a video sequence of an alien invasion of New York City without ever leaving my desk and I gotta say for the most part despite the fact that there's a couple of you know weird AI things like this guy's got way too many buttons on his jacket and you know her hand is a little bit weird um those are things that I don't think you notice when things are in motion I mean there is the one issue of this girl in the background her eye gets a little bit wonky um but that's pretty nitpicky considering you know what is happening here that this is just a text input and then videos being outputted but now we're going to take a look at taking reference images and using those is part of your prompt to really hone in on the Cinematic Vision that you're looking for before we jump into the next section I would briefly like to invite you to hit the like And subscribe button if you have not had the chance to yet additionally I want to thank everybody that's donated uh to the mid-journey cheat sheets that have appeared in previous videos uh honestly your support means a lot to me I really truly do thank you from the bottom of my heart uh okay let's jump in bouncing over to mid-journey last week we created a sequence of about seven images to create kind of a Gothic Victorian spooky story we then took all of those images and brought them into wondershare filmora to do some parallaxing and masking to create a cinematic presentation the video ended up being about 30 seconds long let's take a look at that real quick [Music] foreign if you want to see the full process on how that video was made that link is below so taking our images that we generated in mid-journey and using them as reference images in Gen 2 is fairly simple but like anything else it kind of you know gets a little more complex as we go along so the Simplicity part is just hitting this plus button and then uploading your reference image as you would if you were using mid-journey and using an image prompt now you'll still need to add a text prompt and this is actually a pretty good tip that came out of the Gen 2 Discord discussions is you could just type in your own text prompt but you kind of get better results oddly enough if you take your image and run it over to clip interrogator I'll have a link to clip interrogator down below but basically you just drag your image in you hit submit and then it almost kind of functions as the describe feature in mid-journey it just analyzes through the image and then comes up with a prompt based off of it so my initial run without clip interrogator gave us this video which yeah it's not quite working right taking the clip interrogator prompt uh which came back with an old mansion with a dark cloud over it dark and spooky themes light green and Amber Coastal scenery it actually had more but I kind of whittled it down because there was a lot of sort of irrelevant stuff in there and then also playing around with a command called the CFG scale which is dash dash CFG underscore scale what you can think of that as is almost like the stylized command in mid-journey which is the higher you go the more it will look at your reference image but the more unstable your output is going to be the lower you go the less consideration it's going to take to your reference image but the more stable your video is going to be so running at a CFG scale of 20 got us this which is cool it's not quite what I was looking for but um yeah pretty neat a little super dark actually um but is definitely closer than that first one was and didn't have that weird ending so another quick note is that you can't just just link your reference image otherwise you end up with something completely insane you actually have to re-upload your reference image each time if you try to give them a link it basically gets super weird from that point so we had our house image that I just linked and then and then our prompt and then this was the output which is completely wrong hilarious but wrong ultimately once I figured that out and actually weirdly enough stopped playing with the CFG scale and left it at default I got something that I actually really liked so it goes to show sometimes default is best um so this was actually the output that we got and I was pretty happy with that so moving on to the second shot this one was nailed almost right away the prompt was a Woman in a Black Dress walks upstairs in the background storm clouds slowly move and the first output was this I did run it one more time just to see and actually ended up with something that I liked even more and we got this shot which I actually I like this a little bit more the angle is just a little bit more dramatic it just feels more cinematic um yeah I like this one that's another tip is that if you're trying to get shots like this if you put in cinematic action in the front of your prompt you're more apt to get kind of more cinematic looking things so our third image is where we started to run into some real trouble um you know we have a hand reaching out to a door and I think as we all know AI does not do hands well so um there was a lot of like David cronenberg-esque video outputs coming so let's take a look at a couple of them yeah there was this guy there was this one that just ignored the hand altogether we had this one which I don't even know what's happening here uh this guy which yeah again that's fairly horrific and this one was my favorite um that's just super surreal so the walking down the hallway shot was one that I was really curious to see how Genty would handle considering it's our our sort of our face reveal shot uh and Genji really didn't disappoint so this is the shot that it gave back um which is I think pretty good it really does kind of capture the essence of our character so let's take all of this string it all together and create a film version of our cinematic presentation from last week thank you so overall I think that's pretty amazing it actually reminds me a lot of experimental films that I saw when I was in college and maybe even made one or two experimental films like it when I was in college um but overall yeah it's not a one-to-one recreation of our original presentation but that's okay because we're essentially changing from one medium to another and more importantly it nailed the tone which I think has a large part to do with sort of storyboarding this thing beforehand in mid-journey and then ultimately for the problem shots like like the the door here in particular uh and for some pacing things I think that the best way to go is to actually use a combination of Gen 2 output and some of the diet show stuff from mid-journey uh like in this case just swapping out the um the door handle for our diet show version and uh you know just giving some pacing issues by using some of the mid-journey stuff I think creates an overall better aesthetic presentation as a final experiment I wanted to see how Gen 2 would handle animation I caught a couple of episodes of Samurai Jack recently again still holds up by the way so that was on my brain so I just went over to mid-journey and generated a Samurai Jack esque character and ran that with the prompt um Samurai walks Q camera Autumn Forest concept art Samurai Jack um and ended up getting this which it's okay um it's animated um it's not quite what I was looking for so close-ups it really wasn't handling very well um with the image reference so that was one and then here was another one but I did ultimately land on this and this which I thought were cool not you know in the Samurai Jack style but I had an aesthetic that I actually thought was pretty cool so I'm always pretty big on narrative when I'm doing like these little experiments it doesn't have to be anything Grand or anything it's just kind of a little short story and and so as I was generating out these ideas I came up with the story of a samurai coming across another Samurai and they duel it's very very simple so let's take a look at how It ultimately ended up looking [Music] foreign [Music] ultimately I think that came out kind of cool and sort of in that minimalist Samurai Jack style although it didn't ape the Aesthetics of it necessarily so one thing to note is that the sword fighting sequence was not actually gen 2. this was actually gen one the reason being is that Gen 2 apparently has not been trained on Samurai's sword fighting so uh you can't get that animation it just ends up being a character is just standing there but yeah overall uh I think it kind of works I mean I think maybe if I had spent some more time on it it could have been a little more dramatic and cool looking but it works personally I'm very excited about all of this this is kind of the moment that I've been waiting for since you know those original Dolly images first came out you know and I was thinking to myself man I can't wait until you can add you know motion to those pictures and you know here we are only much much sooner than I had anticipated so Gen 2 is currently in beta if you're looking to get access I would recommend joining the Discord uh the link to that is below and honestly just kind of hanging out and being nice usually about once a week they open up the door to let people in so you know if you're part of the community and you're not being a jerk you know you've got a pretty good chance of getting in other than that it probably won't be too long until Gen 2 is publicly released I want to say the beta for Gen 1 lasted about a month or a month and a half or so um so alternatively you could just wait it out in the meantime I do invite you to stick around and watch another video from the channel my name is Tim I thank you for watching [Music] [Music]

Channel: Theoretically Media

Views: 76,257

Keywords: ai video, ai art, artificial intelligence, ai filmmaking, midjourney, AI Movies, AI Video, Runway ML, gen-2 text to video, gen-2 video ai, gen-2 runaway, text to video, image to video ai, image to video movie maker, image to video maker

Id: PlZkWcnnRt4

Length: 15min 2sec (902 seconds)

Published: Wed Apr 26 2023