Create 4K AI Videos for Free + Big Runway Update

there's a new AI video tool that is better than some of the paid options Runway just announced a brand new motion brush feature and there's a tool that automatically adds sound effects to your videos this is your AI film news of the week I want to kick things off by talking about a brand new video generation tool that is completely free it's called pick verse and the results are actually pretty dang impressive the cool thing about this tool is it actually creates the videos in 30 frames per second and you can upres them to 4K without having to send them into an external tool like topaz video so let's take a look at some of these examples here so we have like this shot of a woman just looking at the camera this shot of a train coming into a station which frankly looks pretty dang realistic we have this shot of what looks like a spacecraft window and the cool thing is you can see there's that parallaxing with the clouds so there's many fantastic examples of pix ver that you can check out I want to show you how easy it is to actually use their program all you have to do is go to their website and click get started on the web and sign in to the account so they have an explore page right here where you can check out some of the videos that other people are generating we'll go to create so this is your creation window and like many of the other tools you have the ability to type in a prompt like we could say a giraffe at Sunset you can add in negative prompts so you know no morphing no Distortion no variations in lighting things like that and then there are a few styles that you can select from along with aspect ratio so I've actually already created a video of a giraffe at Sunset and I'm going to go ahead and play this for you here again this video was text to video so this was not using an input image so let's take a look at the example here so you can see that there's clearly something wrong with these giraffes and their legs are like morphing and generally speaking I'm not loving the results that we're getting from pixers using text to video at this point but I do love the image to video so let's take a look at that to do image to video you just go back to create and then go to image to video so now you have the opportunity to add in an image so I actually created a series of images inside of mid Journey downloaded the images and now I'm going to upload them inside of pixers so now that our image is uploaded we can type in a prompt I'll say a giraffe eating food and you have the ability to change the seed number and the strength of motion I like turning the strength of motion up to about 7 I find that that kind of gives you the best overall uh movement generally but of course it depends on what your shot specifically is and then all you have to do is hit create the cool thing is whenever you hit create uh you actually have the ability to render up to four videos at a time and again this is completely free there's not a paid version of this tools just yet so once you're done creating a video you'll see all of your video examples inside of your my videos page and you can simply click on any of the pieces of footage that were created we'll say this one and there's this button right here that says upscale if you click that upscale button it will actually interpret that footage which is natively in about a 600p format and it will upscale it to 4K automatically you don't have to download it and render it in a tool like topaz's video so I actually created a demo of the upscaled footage from pix verse again I didn't do any other post processing to this footage I added a quick little vo and some music let's take a look behold the giraffe a bold and majestic animal giraffes are the epitome of Grace with their oh God that one was missing ahead um okay I'm a little disheveled now giraffes love to eat uh never mind I'm too distracted now at least that one is cute oh great now there's a tiger maybe he's the culprit so as you can see pix firste is not perfect there of course is areas for the tool to improve but it's pretty darn good especially considering that it just dropped on the market I highly encourage you to check out pixers you'll find a link below this video and that brings us to our game of the week so I created three videos in three different tools and essentially I want you to tell me which one of these videos was created in P Labs which one was Runway and which one was pix verse the first person to get the answer right will win some merch from Curious Refuge so let's take a look at example number one example number two and example number three if you think you know the answer let us know in the comments also a big Shout shout out to Adena Trix for winning last week's competition there's also a new tool called Sonic Vision LM that basically allows you to automatically create sound effects for a video now it is a white paper and so in order to use this tool there's some code it's a bit Advanced to get it going on your machine but I think it really does highlight the future of basically being able to go into a tool upload your footage and get a rough soundscape for your film so there's some pretty cool examples inside of the demo like for example we have this guy hitting this Trader Joe's bag I mean it seems pretty darn realistic and there's actually some interesting examples from Real World films so we have first Leon the Professional and one from Titanic so there you go those are some pretty interesting results now obviously we are in the early stages of this tool but I can imagine that pretty soon you will be able to upload footage and get realistic sound effects that sound pretty darn good last week we also launched a brand new course at curious Refuge called AI advertising by Dave Clark Dave is an industry Legend and his work basically goes viral every single week he has Decades of experience in the Ad Agency World working with brands from Snapchat to pepsy and in this course Dave and I have teamed up to show you how to use AI to create advertisements and marketing campaigns using the latest tools the beta session will go on sale on January 31st and the course will officially start on March 5th now there are already hundreds of people interested in this course so it will likely sell out so just be be sure to sign up on the wait list to get notified when the course immediately goes on sale we also dropped a new podcast yesterday with Dave Clark where we talked about some of his viral successes and what his strategy is whenever he's working on an AI project it's a really great talk you can find a link below this video Runway also announced a brand new motion brush feature that allows you to have up to five different motion brushes inside of your scene there's some pretty cool examples online from people like Nicholas nert and Ane Akash let me show you how to use this new motion brush tool so all you have to do is go to start with image I will select an image from our assets but of course you can just drag and drop to upload an image I created this inside of mid Journey so now we can go in here to our motion brush and before all we could do is use one motion brush to basically change our scene now we can use up to five so it's really really cool so for this scene I think I want her hair to move so I'm just going to select her hair here and we'll turn up the ambient motion to maybe about five now we can go to motion brush number two and let's say we want her clothes to move a little bit so I'm just going to kind of outline her close here and we'll turn up the ambience just a little bit so maybe like two and now we can go to brush number three and brush the background so I'll just quickly brush the background here and we can turn the ambient motion to about five as well and so the cool thing is we can actually go in here and change the overall movement as well so we can go to our hair and let's say we want it to move to the left just a little bit just you know to have it breezing in the wind and we'll go ahead and hit save and we can add in a prompt here so we'll say a woman standing in slow motion abstract morphing background and of course we can go in and adjust camera controls I'm not going to do that for this example and we'll click generate okay so let's take a look at our result so we have this woman standing here looking at the camera and you know her hair is blowing in the wind the clothes are moving just a little bit the background has this abstract shape now the cool thing is we can go in and adjust this even further so if we don't like let's say the background moves too much you can turn down the ambient motion without adjusting the movement on the hair or the clothes so that overall movement will stay locked in but we can adjust our scene as we see fit so this tool is really powerful and I can't wait to see even more examples of it in action and speaking of Runway they will be hosting their second 48-hour film competition on February 5th the first one was absolutely incredible and they've up to the bar for this one the winning person or team will win $5,000 and there's also an audience favorite that Will Win $5,000 as well and the winners will also receive 1 million generation credits that they can use inside of Runway and on that note the Curious Refuge team will be launching our own AI film making competition very soon that will be our biggest one yet we can't wait to share the details with you be on the lookout for that very soon I'd also like to give a quick thank you to pabs for inviting me to do a quick Workshop inside of their Discord Channel they're hosting some live streams from time to time you can check out their Discord channel to see when the next live streams are happening Nick St Pierre also shared this last week a really interesting use of the tile function inside of mid Journey so basic basically mid Journey does have the ability to create equa rectangular photographs so that's a photograph that's used for 360 imagery and using the tile tag you can actually make it to where both ends perfectly aligned basically all you have to do is create an image in mid Journey upres it using a tool like magnific and there you go now you have a 360 image that you can preview let me show you how it's done we're going to hop over to Mid Journey do/ imagine and we'll say equ rectangular photograph of a modern apartment in Paris and we'll make sure that we have the aspect ratio set to 2x one and dash Das tile so that the edges line up with themselves so when you do tile the left and right side will perfectly line up which is very important so let's go ahead and create our result okay great so we have four examples here I'm just going to pick the one that makes the most sense I'll pick number one so we'll UPR number one and download it always when you're downloading open in a browser and then save to your computer now we'll go over to magnific and we will drop in our image and for this one I'm going to times for upscale we'll keep everything base for now and then click upscale and while we wait we can take a look at this image I uesed earlier using magnific basically magnific just goes in and adds in more details to make the overall image more realistic this is really important for making your seen believable and whenever you put on those VR goggles the images that you see are much more realistic if you use a tool like magnific now there are some problems that can happen in the edges whenever you use magnific to upres so that is one of the limitations of using this tool so if you ever run into an issue like that I do recommend going over to photoshop to clean up those edges Okay cool so now we have an image you can see this is our before and this is our after much more realistic so we'll go ahead and download that to our machine there's a link below this video to a tool that basically can allow you to preview 360 images inside of your browser I'll just grab our image that we upred and drop it in and so now we can explore our virtual world here so let's take a look around this apartment and you can see that yeah looks like a Parisian apartment it actually looks like a 360 image from those like home tour tools that are inside of Zillow and you know you can look around this apartment and check it out there's even a reflection in the mirror which is just absolutely crazy and yeah we have pillows on this couch now is it perfect of course not there's definitely room to improve but the fact that this 3D World never existed and now we have a high-res version of it is really interesting especially when you're thinking about the future of generative AI as it relates to VR entertainment now there's another tool called Skybox AI that we've showcased on this show that does a very similar result but I thought this specific iic workflow was really interesting and I'm sure we're going to see many more instances of 3D worlds generated from artificial intelligence this year Leonardo AI also came out with the ability to do realtime AI in painting and this is the first iteration of that specific feature that I've come across I think it's pretty darn cool let me show you how to use it so we'll go to Leonardo's website and we'll go to realtime canvas so I like that great couch in the Parisian apartment let me see if I can quickly put one together here I'll time lap so you don't have to follow me this entire time but let's just quickly create a rough sketch of a couch and now we have this couch here and the cool thing is we can now go to this in paint feature and now paint directly over the couch so at first all we have to do is go in here let's just say we want a yellow pillow on the couch I'll just go to Yellow here and we'll just paint in yellow and you can see that nothing is really happening well if we type in what we want to see then it will be in your scene so we'll say a yellow pillow and there you go the yellow pillow appears right on the couch we can do the same thing over here so let's add in another yellow pillow and now we have two yellow pillows inside our scene we can right click and save this image to our computer and use it on whatever project we want so I think this specific tool is going to be very helpful for concept artists in the future because obviously you can roughly sketch something and and it will create a highres version of your sketch but then having the ability to go in further and customize and really get the scene to be exactly what you want through prompting is a very unique and creative use of this technology and so the folks at Leonardo are doing some really cool things a team from Switzerland put together a really interesting demo of a tool they are working on in collaboration with Google where essentially you can add in 3D objects to a gaussian splattered scene so this is a scanned 3D environment and you can place a 3D object in that scene just by painting so this is like in painting inside of tools like M Journey or photoshop but in 3D and the cool thing is you can actually walk around that object inside of 3D space so that has huge implications for the future of AR and VR entertainment there's some really interesting demos over on their website where they basically composited in things like butterflies and Ottomans and cups into the overall scene now is it perfect absolutely not there's a long way to go but it is really darn impressive once you get some compositing passes onto these Generations the results are going to be absolutely breathtaking there's also a new tool that basically allows you to scan somebody and place them inside of an image with a lot more accuracy so it's really interesting and it's completely free basically is connected to stable diffusion and allows you to create your own Laura model almost automatically so all you have to do is drag and drop images you can use one image if you want but more images are better I uploaded just 10 images of my face that I just took little selfies very quickly and uploaded them to the tool so you can see we have this example that I generated earlier of me as a 3D animation character I think it looks kind of like me it's not exactly like me but I mean not bad for an AI tool so for this example I'm going to say a man with the IMG tag that will basically tell the tool to recall my face in the woods realistic shot on film basically we want it to be realistic and we can go in here and set specific Styles we'll say cinematic since we're trying to go for cinematic realism but there are many different styles that you can go in and mess with here fantasy art comic books things like that and we'll click submit Okay cool so the image was created and you can see we have basically it kind of looks like me in the woods wearing a hoodie here and we have another image here and my eyes are basically completely crossed but I do find it to be really interesting that you can generate an image with your specific face so easily so you're not having to get into stable diffusion and train all these custom models and do all that hyper technical stuff that can be quite challenging for day-to-day creatives to accomplish you can actually just use this specific tool to generate imagery with the subject that you need so you can play around with this tool by clicking the link below this video I would love to see some of the Clones of yourself that you're able to create a research team from South Korea came out with a new tool that basically allows you to do style transfers now this is different from other tools like Runway gen 1 that completely changes the style this one specifically allows your overall footage to remain the same it just changes the lighting and style of the footage to match a specific reference image that you input so in some of these examples they'll go in and change the color of flowers or the lighting of an overall scene so I think this is a very interesting and practical use case essentially you're able to relight and rest stylize your scene even further I imagine that whenever you integrate AI tools like this with traditional color grading software people inside a post-production workflow are going to have more control than ever before I also came across this really interesting example of a gaussian splatted scene that was generated from just three images it looks like a almost photorealistic apartment of course there's some distortion but the scan is really interesting and it got the team thinking a lot about what the future of recreating environments is going to look like especially when you consider historical events where maybe there was some photography but no footage where people could actually capture kind of what the overall essence of this space was like well it seems like in the future we might be able to walk around in Old environments using tools like this one there's also a motion capture tool that I want to show you it has a lot of really interesting features it's called mushan and it's been around for a while but they did release a few new features this week that I think you should know about so they kind of build themselves as this all-in-one tool that allows you to prompt in motion or upload footage and get Motion in return so for example if we typed in Reading we could actually get a 3D character reading a book with fairly realistic motion we can also do the same thing with a guy speaking at a seminar so we have this guy and he's just being really expressive with his body language and I came across this one I thought it was absolutely fantastic this is a person jumping out of an airplane it's so good it's a person being a plane but I'm not seeing a lot of jumping out of inside of that generation now they also have motion capture which is really interesting so in this example we have this woman who is dancing very typical AI demo right and then we have the video motion capture here and you can see that that video motion capture is really good so it does rival some of the best motion capture tools out there like move Ai and we also have some motion capture from some 3D animations so we have this character running here and then we have the motion captured result which looks basically just as good so it's really interesting to see the results here of course you can download the fbx files and take them into 3D applications which is cool and they're also beginning to experiment with actually prompt in a scene and having a 3D character interact with that 3D animation movement so we have this video here of Nicola Tesla dancing and you can see that he's just doing a perfect gangdom style here so you can see how this specific tool especially whenever the 3D model generation and the image bases get even better is going to be a huge part of animation in the future and again tools like mid Journey are looking to do 3D models very very soon so I imagine we will see many more instances of tools just like this one hitting the market in the coming months and that brings us to our AI films of the week so the first film I want to shout out is called Urban Melancholy by Julie designs Julie did a great job with the green color grading that kind of has that vintage aesthetic and we'll actually have Julie on the podcast very soon and we will more than likely ask her about this project the next film I want to highlight is called sister of battle created by Brett Stewart it really does feature some of the most sharp and photorealistic AI renders that I've seen up to this point and I was so inspired by the work that Brett did that I actually reached out to him and asked him some of his secrets and he was kind enough to let me share that with you with a video tutorial that we will drop next week so I can't wait to share how you can get really nice crisp highresolution AI video from AI generation tools the next film that I want to shout out out is called borrowing time and it was created by the instructor of our AI advertising course Dave Clark Dave did what I think is the best example of an AI generated film concept essentially it's this really interesting example of a Sci-Fi time travel film that also takes place in the segregation area and kind of follows the pattern of racism inside of the United States I think it's a really interesting concept for a film and I can totally see this project being green lit and created inside of Hollywood there are some really cool examples of visual effects inside of this project so I think Dave knocked it out of the park and our student project of the week comes to us from a student that goes by prompt try repeat which I think is a funny and very suiting name for someone who works in AI video and they did a really great job at creating an AI fashion film that features consistency and a good shot selection so fantastic job on this project thank you so much for watching this episode of AI film news you can of course get AI film news directly to your inbox by subscribing over at curious refuge.com and if you're interested in our new AI advertising course you can click the link below this video and sign up for the wait list that will drop on January 31st thank you so much for watching we'll see you in the next one

Channel: Curious Refuge

Views: 72,273

Keywords: AI News, AI Film News, Filmmaking News, AI Filmmaking, Artificial Intellegence, Midjourney, Pika Labs, ai filmmaking tools, ai filmmaking tutorial, ai film, ai film news, ai, ai video, ai news, midjourney ai, runway gen2, midjourney, ai art news, ai short film, ai tools, caleb ward, curious refuge, runway, runwayML, blackade, ai films, pivxverse, pix verse, midjourney news, ai video generator, magnific, magnific.ai, runway motion brush, motion brush, gen:48, ai film competition

Id: 0m2gYAbUi7M

Length: 23min 49sec (1429 seconds)

Published: Fri Jan 19 2024