These WILD AI Tools will Change the Film Industry!

there's a brand new tool that is going to change cinematography and VFX forever and a new tool that they are calling Sora for music is here and the results are absolutely incredible this is your AI film news of the week we're going to kick things off by talking about a brand new tool that kind of flew under the radar but I really do believe it has the power to transform the way that we create films the tool is called fizz Avatar and basically what happens is the tool allows you to record a scene with multiple cameras at multiple angles then you as the editor are able to go in and change the camera angle in post you can change the clothes of your subjects and you even have the ability to relight your scene essentially this converts a normal production into animation workflow this has huge implications because pretty soon there will be simply multiple cameras on set recording an environment and your actors will basically act out the scene as if it's similar to a play then having all of that coverage you can go into post and reframe your scene now is this tool screen ready in the current state no it is not but pretty soon as this technology progresses the line between animation and liveaction and gaming is going to become increasingly blurred we also came across an incredible AI music tool this last week that blows all other AI music tools out of the water it's called udio and it is making a splash so the tool really speaks for itself I want to share a few examples that I came across this first example is called Carolina o let's give it a quick listen sing Carolina oh it's a long and winding road but if you're ever [Music] lonely so the results from this tool are absolutely amazing that sounds like an incredibly well-crafted song from a legitimate real world artist so I want to show you another example it can do really incredible covers as well so it did a House of the Rising Sun cover in the style of gospel music let's take a quick listen there is a house way down in New Orleans they call the Rising Sun and it's been the ruin of many poor and finally I have to show you the the best song that's come from yudo it is Dune the Broadway musical I really think that they're on to something here maybe uh we'll have a dune musical in the near future uh so let's take a quick listen the greatest leader we've ever seen they say that he's a listen guy eyes bright blue and hair black you should see him ride on a sand worm's back up to Victory useful oh it's so good well uh Timothy shamain hopefully after they get done recording the new Dune movie uh we can do the uh musical version because I think that would be absolutely fantastic so as you can see udo's potential is absolutely mindblowing and the effects that this type of Technology will have on the music industry cannot be overstated but there's actually quite a few features that are specifically really interesting for filmmakers so let me show you how to use this tool basically you can go over to the udio website so yudo is currently in beta there are some bugs with the website so you may run into some issues but they give you a ton of generations for free you can click the link below this video to find the tool and to create a song all you have to do is go to the prompt box at the top and type in a prompt for our example we'll say a cinematic soundtrack that blends hope with Melancholy and they do allow you to click on certain tags here if you want to add more tags the cool thing is on their explore page you can actually click into any song and see the tags that they used which is really helpful it's very similar to the mid Journey explore page which is really really helpful so again you do have the ability to add in your own custom lyrics which is really really awesome I think the O Carolina song that we looked at had custom lyrics you actually have the ability to set tags for if you want the song to have a verse a pre chorus A Chorus and you can identify which parts of the songs can incorporate certain lyrics which makes it really helpful compared to sunno and of course they have an instrumental button which is really helpful if you're using music that will go into a film and all you have to do is go ahead and click create now the songs Take A few minutes to create sunno at this point is much faster than yudo but I highly recommend going to the udio website and clicking around and listening to some of the staff picks and trending songs from their website and on that note that brings us to our game of the week this week's game is super simple I'm going to play you two songs I want you to tell me which one was created in yudo and which one was created in sunno we will randomly select one of the winning answers from the comments of this video the winner will receive some free merch from the Curious Refuge merch store okay so let's listen to song number [Music] one tonight the candles it's true is Che to all right cheers to you Irvin that was song number one let's listen to song number [Music] two birthday [Music] [Applause] star cting oh oh raise a glass well cheers to you irin happy birthday I also came across a new white paper this last week that really does outline the future of Animation basically the idea is that you can take a gaussian Splat and animate the scene through text prompts so in their example they have a scene that they scanned and basically through typing in a prompt they are able to animate a vase and some flowers in that vase in a 3D World and this technology can do even more there's another example on the website where they have a Lego toy and they basically say they want to turn it into a pile of sand you type in the prompt and it basically inherits the physics and turns into sand they also have the ability to out paint and remove objects from a 3D World with just text so they have this mindblowing example on the website where they basically ask the tool to remove a chair but the ball still inherits all of the physics that would be present in a ball so the chair is removed and the ball starts bouncing and so that is going to have huge implications for the future of gaming and also for set design pretty soon similar to Minority Report which pops up again and again you will be able to look around and augmented reality experience and change certain components and the physics will actually be realistic now is this tool screen ready at this point I would say no but it's very easy to see that with the current rate of progression with artificial intelligence tools like this will more than likely be hitting the market in the next 1 to 2 years if you ever tried to create a time-lapse using AI tools you probably have been really disappointed AI is not really good at doing time-lapse for example earlier I typed in a time lapse of flowers growing inside of Runway way and we basically just have flowers waving in the breeze it looks amazing but it's not really a time lapse and there's even this example from Marquez brownley who basically asked Sora to create a time lapse of a 3D printer and again it doesn't really look like a time lapse it just looks like liveaction footage with the physics being all wrong so in light of all these problems some researchers put together a tool called Magic Time L timelapse and it basically allows you to create really interesting time lapses from a simple prompt there are some really interesting examples over on their website for example we have bean sprouts growing and maturing from seeds looks pretty realistic we have dough that swells and Browns in the oven looks uh pretty delicious we also have a bud transforming into a flower and it even inherits some anime styles for example we have an ice cube melting and a time lapse of a delicate pink plum blossom I didn't know a plum blossom look like that but there you go the reason why these results look so good is the researchers trained their data on time-lapse footage a lot of times these video generation models are trained on a ton of different data but if you want specific video results sometimes it's better to train a custom model so that the results are more in line with what you're looking for some researchers at Google came out with a new white paper that looks like it could change the way that we do video interpolation forever now don't get scared by the word interpolation all it really means is figuring out what happens between certain key frames it's where we get the term key frame and interpolation is basically the process of going from one frame to the other now interpolation is most prominent typically when you want to slow down footage let's say you want to take footage that was shot in 24 frames per second and you want to change it to 120 frames per second well in that case you want to use a tool like topaz video which handles interpolation between frames but let's say you only have a few images how would you create a moving video if you only have let's say three images to work with well that's where this tool really shines it's called video interpolation with diffusion models and basically all you have to do is upload a few images and it will create video that interpolates between those images there are a ton of really interesting examples on their website we have this video of a camel walking and again look at the reference images it's only a few reference images and it completely interprets between those images we have this video of a woman walking this video of a horse Galloping this video of a motorcycle and this guy playing tennis so as you can see this is in a simple research phase at this point but pretty soon you'll be able to upload a bunch of images and video will interpret between those images it's going to dramatically change the way that we recreate scenes and moments in time and of course this looks like it could have huge implications for documentary film making and speaking of Google they also came out this last week with the ability for their Enterprise customers to create text to live image essentially they're saying you can type in text and get a short AI video in return the generations are really Limited at this point they're only 24 frames per second at 360p for 4 seconds but it really does show that Google is really trying to be in competition with other AI video tools like Runway if you've ever played around with Korea's live AI image generator tool then you know it is super fun to play with well they actually came out with a new stylization feature this last week that is awesome let me show you how to use it to use the new feature just go over to the Korea website and go to realtime generation so you want to make sure you're in the HD mode here and we'll go ahead and clear our canvas and I'm going to quickly change the background we'll just change it to let's say a light gray here and we'll type in our prompt a 3D model of a dog so now now it's time to play around with the canvas it's super fun we'll just select kind of some like a darker brown color just making a dog here and uh it's so forgiving it's hilariously forgiving so we'll just like make a rough drawing of a dog click away and look at that we now have a dog now this is a shair dog let's say that I really wanted a fuzzy dog and I was really having trouble dialing it in well with this new styliz feature you actually can customize your realtime Generations really easy so I have this image of this kind of fuzzy creature this is just a mid Journey image and all you have to do is drag and drop the image into that image icon and we'll drop it in there and we can dial it in so we'll just dial it in about like that maybe to about we'll say eight and if we compare and contrast them to our original image this is is our original on the left this is our realtime generation on the right now the cool thing is we can now go in and completely change this drawing and it will still inherit the style so for example if we delete that generation and go in and adjust our brush size let's say that I want our dog to basically be a closeup we'll just design like this like big around like this snout here about like that and click away and oh no it looks like a a cursed Furby we'll uh we'll fix this no worries at all and uh oh it's really cute it's like uh an interesting like uh Pokemon character so we'll make sure we add the nose here so now we have our dog he's looking in the right direction I think we should uh give him some regular eyes here so we'll just like quickly draw some eyes like that and uh oh no he's cursed uh we'll give him some some pupils here here we go about like that oh no it's so funny oh my gosh actually you know what I love this I think uh I think he looks great so obviously you have the ability to go in you can finesse and fine-tune any way you want but actually I think we may have found just a new mascot for Curious Refuge pabs is also hosting a 72-hour film competition in collaboration with 11 Labs the contest is going to run from April 12th to April 14th and the winner will actually get $2,000 they have $500 prizes for runners up when it comes to visual sound narrative and experimentation and they said that the top winner gets a very special prize now they weren't specific so I can only guess that it is a diamond encrusted Pika character and it turns out that a pika is actually an animal and not just Pikachu and another cool thing Dave Clark the instructor of our AI advertising course is actually going to be one of the judges for this competition alongside Danny Bole who directed 28 Days Later and Slum Dog Millionaire you'll find a link below this video to check out that competition and speaking of competition our NAB party is right around the corner on Monday April 15th at 8:00 p.m. we are hosting an epic party at the hyperx arena in Las Vegas this event is being hosted in collaboration with a peer group from the TV Academy and we also have incredible sponsorships from Puget systems Fabric and devant now the event will start out as a party and transition into the world's first creative AI Esports tournament we have 16 competitors who will showcase their skills and compete for an apple Vision Pro we'll also have ai generated cocktails at the event along with an AI photo booth it's going to be really fun and I have a quick little bonus anybody that is watching this video who wants to attend the event if you use the code web show 50 at checkout you'll get 50% off your ticket you'll find a link below this video we can't wait to see you at the event our research team also came out with the ability AB ility to add style transfers to your footage this last week and the quality is much better than some of the other video style transfer tools that we've had up to this point so it's basically like Runway gen 1 but the style is much more consistent for example the other day I was working on the countdown timers for our AI creative competition and I wanted to change this countdown timer into a different style however when I ran the countdown timer through Runway it basically gave me this which is completely impossible to read and every now and then a frog pops up which is delightful but not really helpful the tool is free to use it is limited at this point but of course like so many other tools it is only going to get better from here now all you have to do is upload a video clip it is limited right now to 2 seconds so it's really just an experimental tool and after you upload your video it is then time to edit the image you can type in whatever prompt you want in this example we have a woman in the style of a Vincent Bango painting after you've edited the video to your liking you can go down to the video editing stage type in your prompt and click run video editing here's a quick example of our video in action you can see that it did a really good job of inheriting the style and the composition from the reference footage I think this tool has really big implications for the future of visual effects as well so for example we have this video clip of this couple and let's say that we actually want it to be snowing well all you have to do is add in a snow overlay or you could paint in the snow that you want to see in the scene and then you just go to the video editing stage type in your prompt and click run video editing and here's a quick example of that footage you can see that it's now snowing in the scene so as you can see this is laying a blueprint for the future of visual effects pretty soon you'll be able to drag and drop assets or images into your scene and AI will bring them to life and composite them all together I also came across a really interesting workflow example from OT Sharma this last week where basically he took a gausian Splat scanned scene and brought it into Unreal Engine this has huge implications because you can basically take that scan and create a 3D world not only for video games but also for virtual production screens and eventually liveaction footage a research team this last week also came out with the ability to stylize a 3D scanned world the results are really incredible basically whenever you take a scan you are able to type in a prompt and change the scene that the scan lives in so there are some really amazing examples over on their website we have a few examples of changing a scene from a daylight scene to a sunset scene we also have the ability to change it from summertime to a scene where it is snowing and so many more things in music news the team at Spotify came out with AI generated playlist now they're only available to people in Australia and the UK at this time but it does look really interesting now the thing that makes these playlists really unique is the fact that you can use natural language and emotion to create the playlist so instead of typing in I want some studying music for Focus you can be really specific with your life and your emotions and it will give you a playlist for that so for example you can say I want sad music for painting drying flowers or tracks for horse riding into a Sunset and the cool thing is it's a multimodal experience so as you are are using the AI generated playlist you can basically say I want more pop music or I want it to be less upbeat and the playlist will adjust as if you're actually talking to a DJ we also came across a really interesting experiment using Google Gemini 1.5 Pro this last week from Bela and it really blew our minds so basically Bela uploaded a video file that he was going to upload to his YouTube he then asked it to create the video chapters for his video and it did it automatically now if you share a lot of videos online whether it's on social media or YouTube or Vimeo this type of technology is an absolute GameChanger you can simply upload the video and have ai create the descriptions and chapters for your videos we also came across an interesting post this last week from Nick St Pierre where basically he was reflecting on the fact that mid Journey has been around for about 2 years now and he basically compared in contrast day one of mid Journey versus today we wanted to do the exact same thing first up we have a young Japanese woman smiling mid Journey 1 looks absolutely cursed mid Journey version 6 looks basically photorealistic we have a glamour Street medium format photography shot mid Journey version one looks terrible mid Journey 6 incredible we have this beautiful shot of a landscape look at the Reflections in the water it looks super realistic we have this cheeseburger and I think the mid Journey version one tried to create an onion based cheeseburger which does not sound very good but uh mid Journey 6 looks super realistic we have a woman's necklace as a sunflower a woman standing at the beach alone in the style of studio gibli and finally I had to do a double take on this one we we have a rabbit a porcupine two cats and a wizard having a tea party in a '90s animated TV series style aesthetic mid Journey version one created this uh beautiful abstract image that you know is kind of cool but not anything like the prompt and mid Journey 6 while it didn't include all of the characters the quality of the overall image had me doing a double take because it looks really amazing and that brings us to our AI films of the week we're going to kick things off with this Nike spec ad created by Adam Castle brand the spec ad feels like a real ad you can really see how Adam has video editing skills to help bring everything together the pacing is really nice and it's one of the better examples of an AI commercial that I've seen up to this point next up we have Warriors by Lei atah it's basically a short film that has humans against robots and he does an amazing job with World building some of the bfx shots inside of this film are really mind-blowing and it features a lot of interesting movement the sound design is really on point and it is one of the better AI films that we've seen over the last few weeks and our final film of the week comes from Dave Clark he created a spec trailer for a Gundam movie where he basically created this epic Hollywood trailer he actually was kind enough to break down how he put this trailer together in our recent office hours at curious Refuge so if you're a curious Refuge student be sure to log on and you can see how Dave put together this project thank you so much for watching this week's episode of AI film news of course if you want to get AI film news delivered directly to your inbox you can subscribe to our newsletter by visiting curiousreason like And subscribe and of course if you can make it out to our party at NAB we would love to say Hello thank you so much for watching this week's episode of AI film news we'll see you next week IC tea it's nice

Channel: Curious Refuge

Views: 35,958

Keywords: ai news, ai, ai film, ai film trailer, ai-generated video, pika labs, udio, ai music creator, ai song, ai music cover, ai music, ai music maker, best ai music generator, ai song generator, ai music generator online, suno, krea ai enhancer, krea, curious refuge, filmmaking news, google, google ai, google generative ai, gaussian splatting, unreal engine gaussian splatting plugin, google cloud, spotify, spotify ai playlist, spotify ai, spotify dj ai, gemini 1.5 pro, midjourney

Id: uZzvJukDIWU

Length: 26min 17sec (1577 seconds)

Published: Fri Apr 12 2024