Let's Explore Sora + New Runway Tools (Essential AI News)

we have some brand new features inside of Runway and P Labs we'll take a deep dive into Sora and Adobe just announced a new music tool that is probably going to change the way that we edit and create songs Forever this is your AI film news of the week first off I'd like to thank the TV Academy for inviting Shelby to come out and share the process of creating AI Productions along with our vision for creating a democratized storytelling future we'd also like to thank USC for inviting Shelby to come out and speak about the ethics of AI along with the Sun Valley film festival in Idaho for having her come out and talk about AI Productions Shelby has really been on the speaker circuit lately and that brings us to our first story and of course this is the most important story and I am talking about the Willy Wonka experience in Glasgow have you seen this it's absolutely hilarious basically some people put together a landing page with the help of AI that basically talks about this Willy Wonka experience and it has this beautiful enchanted garden and an imagination lab in a Twilight tunnel and then when people showed up they got this and it's basically a warehouse with some really cheap props and some tapestries on the wall and a bouncy house and the very best part was this Oompa Loompa in tool new Runway came out with a brand new feature inside of their motion brush tool it basically allows you to automatically select elements inside of your scene it's really easy to use so I'm inside of Runway Gen 2 all you have to do is go to the motion brush tool and make sure you have autod detect area selected so basically now we can go in and select different parts of our frame and you can see it creates an outline automatically which makes it really easy for selecting certain app aspects and the cool part is because it's a multi-motion brush tool you can select different brushes here and then we can go in and select different parts of our frame like this and go in and adjust the movements and animation and all of that now the thing that I should point out and I didn't really realized this at first when I started using this tool is you don't actually have to select every single Pixel of the part of the frame that you're trying to select if there's some edges here like with this stuffed animal character that are peeking through it's totally okay it still will animate it the brush is really an approximation and it doesn't have to be perfect pabs also came out with the ability to automatically add synced lips to your footage inside of pabs this works by uploading an image and adding in lips and also adding in video if you add in video and then sync the lips you can get longer Generations so I do recommend going with that route it's really easy to use let me show you how to use it to use the tool just just go to pabs and then go to the image or video button and select the video or image that you want to add sync lips to I have this animated shot that was actually animated inside of P Labs with a Runway image and I want to add lips onto this character and go ahead and click open so now all you have to do is go to the lip sync and we will upload our file I have a voice from 11 laps let's take a listen listen to me carefully I've made my decision tomorrow we attack the enemy castle at dawn Okay cool so we'll import that into P Labs attach and continue and then click generate okay the render took about a minute let's check out our final result listen to me carefully I've made my decision tomorrow we attack the enemy castle at dawn okay so this is one of the best Generations that I've seen using this tool it does a great job with animated styles now we also ran some experiments that were more grounded in reality or at an angle and the results were definitely mixed so it's not a perfect tool but it is incredibly easy to use you'll find a link below this video to check out Pika and we also have the assets that I just used in case you want to download them and play around with this tool there's also a new white paper that came out this last week called emo that basically converts your images into animated faces with just audio now we've seen tools like this in the past this one seems fairly realistic and the results are very impressive so we have this image of Audrey heern here and we add in some audio and here is the result never knew you were the someone waiting for me we were just kids when we love so as you can see the results are pretty realistic it has I blinks it has the eyebrows that raise the cheeks expand and I think that this underlying technology is actually going to push tools like the new Pika lip sync tool even further So within about 3 to 6 months we're going to have user facing lip sync tools that look absolutely incredible this really reminds me of a demo that we saw recently of flawless AI which is really branding itself as a very high-end industry tool for converting films into other languages is with lip sync it looks very realistic you can check out the link to Flawless below this video as well last week we talked about Sora and how some of the renders are taking a while for example a one minute clip can take up to an hour to render and some people mentioned that those render times are actually not that bad especially when you compare them to a modern VFX Pipeline and we totally agree so this week we wanted to talk about some awesome features inside of Sora that aren't getting as much attention but we think our are really big deals you'll find a link to a page below that talks about Sora being a world generator and that's what makes this so unique compared to other AI video tools it's actually creating a world and understanding how objects live so the first feature I want to talk about is image to video so Sora like so many video tools out there allow you to take an input image and animate it so we have this footage of this dog here he blinks the camera is a little handheld the parallaxing in the background looks realistic it also looks very interesting for motion design so we have this 2D image of these monsters here and then it animated the monsters and it looks pretty darn good it would have taken quite a while to rig all these characters in a animation tool like after effects it's also capable of doing visual effects so we have this Cloud image that says Sora and it animates with particle physics now obviously you may want to go in and clean up a few of those things using some rotoscoping techniques but it looks really good there was also this example of this wave inside of a museum and the Guy's surfing and he does turn into two Surfers there at the end but the water simulation is really really impressive it also has the ability to extend videos so you can make videos longer so there's an example here of this trolley car inside of San Francisco basically they added new intros to the beginning of each video example here and it's really interesting to imagine that in the future if you needed a longer take or perhaps you wanted to transition from some crazy sci-fi world into reality you'll be able to use tools like Sora to make that happen it also has the ability to create infinite Loops so there's this amazing example on their website of someone riding a bike and basically it Blended the video together to where they basically ride in a loop forever it also has the ability to stylize footage so think about a really advanced Runway gen 1 style experience we have this input footage of a car driving through a forest and then we can change it to a jungle and here are the results they look absolutely fantastic it also has the ability to combine two different videos together so there's this really great example of a butterfly basically flying underwater or through a Roman Coliseum that gets converted into this underwater scene there's this really weird like lizard mixed with a bird character and uh of course this lion running after a Jeep is just absolutely fantastic there's quite a few other features that we could talk about here now of course everyone's question is when can we use Sora and the truth is we don't know when we'll be able to use it and we also don't know what that user experience is going to be like but of course Sora is likely going to change the way that we do creative video production forever and we we can't wait to check out the tool and if anyone wants access to Sora we found this video on X that perfectly sums it up with some lip sying from pabs thinking about getting your hands on Sora forget it it's not available so there you go there was also a new tool announced this week called LTX Studio that seems really interesting and is really branding itself as an all-in-one platform for creating AI Stories the tool is on weit list right now so you don't actually have the ability to go in and test it yet but basically what we can tell from the marketing is you will be able to create storyboards with consistent characters all in one place art direct those storyboards and then animate them so it's an all-in-one experience that allows you to hypothetically create a rough AI film all in one place it's a really ingenious way of thinking about the process of putting an AI video together there are many different tools for performing different task in an AI I video production pipeline but having a tool that consolidates a lot of those tools together will likely save time and be an interesting alternative Adobe also announced that they are working on a brand new music tool that allows you to do a lot of really impressive things first off the song allows you to convert simple songs into Dynamic Styles and you also have the ability to customize the song so unlike tools like sunno where you basically have the ability to create a verse and then a chorus with the Adobe music tool you can go in and customize and say I want Chorus Verse then chorus or customize your song further you also have the ability to create Loops from text prompts so in the video example they had this song okay that's not bad I was curious because in the video they actually have their prompt groovy Rock in there so I basically went to suo typed in groovy Rock and then clicked instrumental version three and generated a song and this is the song quality that I got from [Music] there it basically sounds like some pretty basic stock music but not bad I think this Adobe music tool is a bigger deal than we realize because this will really Elevate some of these AI workflows from just prev visualization and creating experimental Concepts into legitimate post-production pip lines that can be used on high-end projects I'd also like to extend a huge congratulations to project aeon for joining the Nvidia Inception program it was actually founded by a curious Refuge student named Dan and it creates automatic videos for Publishers to create videos of their content so articles or reviews into automatic videos it's really an incredible tool and we'll actually be doing a live stream with Dan inside of our program on March 7th so if you're student at curious Refuge feel free to hop in and you can ask Dan all the nerdy questions about his incredible tool Google also announced that it plans to relaunch Google Gemini soon they actually lost 70 billion in Market evaluation on the news that they had to take down their image generator and to create a little context for what was happening with Google's image generator basically they went in and added in mandatory University into their photos but that doesn't make sense when you prompt in certain things so for example people typed in founding fathers and there were founding fathers that were African-American and didn't really make any sense there were also people that prompted in Vikings and popes so this really does bring up a much larger conversation obviously you want AI models to not reflect the biases that are inherent inside of our data but you also want them to be factually accurate in his historically helpful whenever you are prompting in images so it's definitely a back and forth and I'm really surprised that Google missed the mark so big with the launch of their image tool and speaking of Google we actually came across an interesting conversation point this last week where you know we really feel like chat gbt and Google they kind of battle it out but when you actually use Google Trends to see what is being searched for more it's not even close on its best day Google Gemini is 11 times less popular than chat GPT so chat GPT still seems to be the language model of choice for the world and in so many tests whether we're talking about custom gpts or just simply the knowledge base and its ability to be more creative with less limits Chad GPT is still our preferred language model this last week we also dropped a podcast episode with Julie Wheeland who goes by Julie W design online she shared a lot of good insights for how AI is shaping graphic design she also shared some Pro tips on working on AI projects specifically how she gets such high quality from her work and she also went into a lot more detail about how she's able to make more money as a graphic designer using the power of artificial intelligence and while we're talking about AI design I want to point out that enrollment is open for our AI advertising course this is our very first session ever and we will likely sell out yet again we have people from some of the largest Brands and ad agencies inside of this program and we can't wait for you to learn alongside them we also have enrollment open for our AI film making course if you want to learn how to use AI to assist with every aspect of the Film Production pipeline you'll find a link below this video this is already our biggest session ever so congratulations to everyone who was able to snag a spot and that brings us to our AI films of the week I want to kick things off with a project from Nicholas Newbert Nicholas works at Runway and is basically one of the best AI art directors in the world he did this really interesting project called not a dystopia and it features some really good curation and it kind of has this kylo Ren meets High fashion Vibe which I think looks very good so great job on that Nicholas next up we have this project called Earth home turkey farm from siki Wang who actually created this during the MIT hack on recently that uh our instructor Dave Clark actually spoke at and the project is really funny it's kind of like a travel video for Earth after humans have basically just decimated the planet and it's a really good use of motion design and sound design to create a cinematic atmosphere so I recommend checking it out you'll find a link below this video and our final film of the week comes from Justin Kaplan who put together this AI music video concept I think that it has some really good consistent shots some of the simulations especially with the water look absolutely fantastic and I think it really does showcase the potential of using AI for music videos and finally a research team from UC San Francisco basically said that AI can now predict Alzheimer's disease up to 7 years in advance by diagnosing the disease earlier they're better able to treat the patients and increase their quality of life thank you so much for watching this episode of AI film news of course like always you can subscribe over on Curious Refuge to get AI film news directly to your inbox every single week and if you want to subscribe here on YouTube you will be the first to get AI film news tutorials and the latest AI content thank you so much for being an awesome member of the Curious Refuge Community we'll see you in the next one

Channel: Curious Refuge

Views: 27,346

Keywords: ai, ai film, ai films, ai film news, ai news, artificial intelligence, runway, runwayml, pika, pika labs, sora, open ai sora, gemini, google gemini, google ai, ai short film, ai filmmaking, ai video, ai filmmaker, ai advertising, open ai, ai tools, ai tutorial, ai art, ai image generator, curious refuge, curious refuge trailer, caleb ward, shelby ward, pika labs lip sync, LTX Studio, Flawless AI, ai filmmaking tutorial, sora openai, Sora, AI news, Ai Film News

Id: EdT-f9yy7IQ

Length: 16min 53sec (1013 seconds)

Published: Fri Mar 01 2024