EASILY Translate Your Film into Any Language + AI Costume Design

we have an automatic AI translation tool that is absolutely amazing there's a brand new AI music tool that sounds fantastic and AI costume design is here and it is breathtaking what' you think of that TI AI wow that was quite the intro let's see here the delivery sounded really confident and engaging you've got that Showmanship down I like how you highlighted all the AI Tech with enthus I ASM the only thing I noticed was a slight hint of confusion or frustration at the end there when you addressed me as tick AI overall great job I am impressed by your energy well thank you so much this is your AI film news of the week let's get into it then what's the scoop this week in the world of AI and film I am all ears hit me with the latest well it's been a big week so let's take a look this week we're going to kick things off by talking about haen motion 1.0 now you probably know about haen because we've covered quite a few of their features on this show up to this point in time but this week they came out with the ability to upload video of your subject actually walking around and moving their face around and the lip sync is still almost perfect so here's some video that I recorded of me talking to the camera in this video we are going to test the limits of using Haun so instead of looking directly at the camera like this I want to begin to look around we'll look to different angles here and I'm so so traditional lip syncing tools would have a really hard time with that video clip because my mouth was moving in all sorts of different directions but with this new haen feature it actually was able to translate it very easily let me show you how to use it to use the new feature just go to haun's website go to video translation and now all you have to do is drag and drop your video it's super easy we'll click create new translation and I'll just go ahead and translate directly now you get to select your output language they have a ton of different options to pick from here I'll just pick Serbian it also has autod detect so it can detect the original language and it also can detect the number of speakers but it is a little bit better if you're able to clarify the number of speakers inside of your video and we go ahead and click confirm [Music] submission now the render takes about 5 minutes but let's take a look at our result wow that's absolutely amazing and of course there were a few parts where the lips got just a little messed up but it did an amazing job as my face was looking around it really did track the lips pretty well so obviously this technology has huge implications for the future of distribution by the end of this year I really do believe that most of the content that you consume online will be in your native tongue without having to read close captioning the folks at stability also came out with stable audio 2.0 so as a reminder stable audio is their text to music tool now it has a major upgrade with some features that you need to know about to use stable audio 2.0 just go to the stable audio website and click the try now button you actually get some free credits by default which is really awesome so just like every other prompting tool out there you can type in a prompt in the top box I'll say an emotional film score that embodies sadness with undertones of Hope and you can of course select certain presets if you want I won't click any presets here the model you'll want to make sure is the 2.0 model and what's really cool is you can actually go in and change the duration you can have songs up to 3 minutes long which is a really nice feature especially if you're trying to have the music fit a specific part of the film or video project that you are working on you also have the ability to upload audio that is kind of in the style of the music that you are trying to reference uh very similar to image referencing inside of other image generation tools and when you're ready go ahead and click generate I have the result from stable audio queued up here let's take a quick [Music] listen okay I actually think it did a pretty good job of embodying both sadness and that underlying idea of Hope now I also did the same thing in sunno so to use sunno all you have to do is go to the sunno website and I'll paste in the exact same prompt and make sure that that instrumental button is selected because if you don't it might put lyrics inside of your song which is not ideal if you're working working on a film score and go ahead and click create now I created this song earlier and here was the [Music] result okay so you get the idea the quality of the music coming from sunno is just a little bit better than stable audio in my opinion now I wanted to do a different test so I went over to stable audio and I said a modern indie rock song with driving electric guitar and this was the result that we [Music] got I'm not going to lie that one's kind of hard to listen to but I did the exact same thing over in sunno and this was the [Music] result that pretty good kind of sounds like Mayday Parade and that brings us to our game of the week this week I want you to identify a song that was created in stable audio sunno and one that I got from the stock Music Company art list so the prompt was a modern pop song with driving base here is our first [Music] result okay not bad let's take a listen to number [Music] two [Music] I just pictured like anime hedgehogs dancing around with that song and let's listen to number three I just hear the premium beat.com underneath that music it sounds very stock okay let me know which one you think was created in each tool in the comments below this video the winner will receive a free prize from Curious Refuge the team at pixers also came out with the ability to create consistent characters inside of their tools it's amazing how many of these tools have released consistent characters since mid Journey released their tool just a few weeks ago let me show you how to use it all you have to do is go to the Pix first website go ahead and click create and go ahead and click character now inside of the character window here I want you to go down to the character section and we'll click the plus icon to create a new character so all you have to do is drag and drop an image I have this image of me and I will just drag and drop the image into this section here and we'll give this image and name I'll say Caleb and we will create the character so now that we have the characters selected at the top here we can say whatever we want let's say Caleb writing a bike and we can also type in negative prompts if we want adjust our aspect ratio and go ahead and click create now I tested out this tool earlier with the exact same asset and I had a couple of results here so here's the like 14-year-old version of me with a mustache drinking coffee there's not a lot of movement I'd say it's pretty close to looking like me but it's not perfect but not a bad start and here's me riding a bicycle and uh again it really just wants to give me that baby face and my eye is like drifting so again this tool is not perfect at this time but having the ability to get character consistency again and again inside of AI tools is an essential step in the storytelling process and as we progress into the coming months I imagine that consistent character tools will continue to develop inside of AI applications I also came across an interesting use case for Korea this last week that I don't know how it has practical implications on a lot of projects but it is a lot of fun to play with let me show you what I'm talking about to use the feature just go to the Korea website and click on Real Time generation and go ahead and click on the camera icon so this is piping in a live feed from my camera kind of allows you to do real time photo booths and image changes which is really interesting so I'll go ahead and scale this up and uh there we go so we have in the prompt a man wearing a suit and you can see it's doing realtime conversions of the image which is really interesting and if we wanted to direct this further we could of course go into Korea here and we can use all of the same tools that you probably are used to so let's say that I want to wear a red hat you can just paint on a red hat here there we go and we'll say man wearing a suit and a red hat and there we go so now I am wearing a red hat and turn around there we go looks great and let's say that we actually wanted to upscale this image it's actually really easy all you have to do is Click enhance and upscale and it will actually send it to the AI enhancer that's built directly inside of Korea and you can go ahead and click enhance okay so this was our original image you can see it's pretty pixelated again it was generated in real time and then when we enhance it you can see that it did a pretty good job there's a lot more texture in the clothing and it looks like a pretty convincingly real image a team this last week also came out with a tool that looks like it could be very helpful in the process of AI costume design so the tool is called garment 3D genin and basically all you have to do is upload a two 2D image and it will create the 3D mesh of the clothing from the images now what's also really interesting is you have the ability to manipulate the clothing using virtual reality we actually chatted with a costume designer inside of the office hours in our AI film making course it was really fun to see how modern costume designers are using AI to enhance their creative output there's also a really interesting tool that came out this last week that can basically judge the sentiment and the emotion behind your voice now this is really interesting especially when you begin to think that in the future audiences as they consume content will have direct feedback to a device that can tell a real-time algorithm What feelings the audience is having and it can change the content as the audience is consuming it which is absolutely mind-blowing let me show you how to use this tool the tool is called Hume all you have to do is click the link below this video to start up a conversation so let me go ahead and start the conversation hi I'm trying to show my audience your capabilities to judge the tonality and the emotion behind my voice do you mind doing a quick demo for me I'd be happy to show off my skills as the world's first empathic voice interface I'm all about reading between the lines let's see what I can pick up on here I sense you're pretty excited to Showcase my capabilities but also ATT curious and maybe even a little surprised I detect a hint of focus and determination with a touch of contemplation am I on the right track so far so tell me what would you like me to demo I am all yours I'm not sure is there something that's really interesting that you think a YouTube audience who's interested in film making would like to learn about your tool a interesting you sound a bit uncertain but I Ken told you really interested in showing off what I can do well how about about this let's dive into the Cool Tech behind my ability to read emotional Expressions your YouTube audience who's into film making would probably eat that up I use hume's Cutting Edge AI bottles to analyze the nuances of your tone facial expressions and body language that allows me to pick up on the subtle emotions you're conveying Beyond just the words you're saying for example I can tell you're feeling quite focused and interested right now with a touch of contemplation pretty neat right I'd be happy to walk through some more of the science and technology behind how I work or if there's something else you think would really resonate just let me know I am an open book now obviously this tool has huge implications not only for the future of film making but also for the future of therapy and conversations oh absolutely I can see where you're going with this no no no the implications of this kind of you're done got it I hear you no more specul on potential implications the team at free pick also came out with the ability to change the style of an image in real time it's really cool let me show you how to use it to use the feature just go to the free pick website go to tools and click on the reimagine button now all you have to do is drag and drop your images I have this image of me in a sweater so let's test the limits of this tool I'll go ahead and drag and drop it and what's really interesting is the free pick tool actually will analyze the image that you upload and it will actually give you different carrots that you can select to change the overall style so right now it says Caucasian male in his 30s with black hair and a mustache wearing glasses and a white sweater basically describing me but I can actually change this so let's say I actually want this person to be in his 50s and I want the background to be blue you can actually go in and change that and in real time it has converted my image into uh many different results now of course the images that you get from this tool is not perfect if you want it to be perfectly in line with the face that you upload you may want to use a face swapper like a cool but it is really interesting that you can change the Styles so quickly because all of these real-time image tools are really changing the way that we interface with AI this last week we also had Ali on the podcast to talk about ai's role in a marketing application now this episode actually gets into some specific nitty-gritty details about how to use AI in a professional marketing workflow Alli shared a ton of really helpful secrets so I definitely recommend checking out that episode and while we're talking about podcast Shelby and I were on the 01 cine podcast this last week you'll find a link below this video oder felco and Mauricio tonin interviewed us and it was really cool to chat with Mauricio who is actually a TA inside of the Curious Refuge program and speaking of our program I want to wish all of the students who joined us for the April session the best of luck you're going to create some amazing films and advertisements and the entire team is excited to see what you create this last week we also hosted a Meetup in Brooklyn which was absolutely amazing we want to host more meetups around the world so we actually just added a San Francisco Meetup which will be hosted on April 19th and while we're talking about events of course I have to let you know about our AI film making Mega party at NAB the party will be on April 15th and we have just confirmed the TV Academy as collaborators on the event this event is going to be incredibly fun we have a ton of AI Shenanigans up Our Sleeve so if you want to meet the larger AI film making Community I highly recommend snagging tickets for the event there's an awesome AI event happening in Amsterdam on June 20 first so if you happen to be in the area there's going to be an AI Film Festival Shelby and I will be presenting virtually at the event I highly recommend checking it out I wish I could be there but it looks amazing the team at X also announced this last week that grock 1.5 will actually have a context window of 128,000 tokens which basically means about 128,000 words now to put that in perspective the average script is anywhere from 22,000 to 30,000 words so basically you can upload an entire script worth of Text data and ask questions with grock back and forth now grock 1.5 is also much improved over the previous version of grock and its ability to compute information is very close to chat gp4 and Claude 3 Opus but the thing that makes grock really unique is the fact that it's going to be integrated directly inside of X I think integrating these types of tools directly in applications that the world uses day in and day out will Kickstart the adoption of AI in the larger cultural ecosystem and speaking of that the team at Apple came out with a research paper this last week that looks like it really is going to be a serious competitor to chat GPT in the coming months the paper is a bit complicated to understand but at its core apple is looking at the information on your phone and using that as context as you're having a conversation so for example if you're watching a video you could be like hey what's that uh pot and pan in the corner can you tell me what brand that is or if you're on a website you can say summarize that for me so this type of technology looks like it's going to have huge implications for Society at large and I wouldn't be surprised if we didn't see some AI announcements from Apple's development conference which is happening in just a couple of months the team at dolly also came out with the ability to inpaint and edit your images directly inside of chat GPT all you have to do is type in your prompt and you have the ability to inpaint and change the images underneath so if you use Dolly to create your images this is an amazing upgrade the team at open aai also came out with a new voice cloning tool that has some really practical applications on society now the first application is simple it's voice CL so they have some reference audio basically this tool only needs 15 seconds of reference audio to generate the results here is the original force is a push or pull that can make an object move stop or change direction and here is that same voice translated to different content some of the most amazing habitats on Earth are found in the rainforest a rainforest is a place with a lot of precipitation so that's pretty interesting but the applications that open AI outlined go Way Beyond Simple voice cloning they also talk about translating content which this tool has the ability to do so again here's the reference audio friendship is a Universal Treasure it brings joy support and laughter into our lives no matter where we are in the world and here's that same sentence translated into Spanish open a is also demoing this as a tool that can help people with different speech conditions to help them find their voice again we also heard some news this last week that mid Journey 7 will be here in the next 1 to 3 months so early summer which is really exciting mid Journey will actually come out with the ability to customize your own model So based on your ratings with the generations it will fine-tune a model that is specific to you now you have the ability to turn your custom model off and on but this is going to be very helpful especially if you go after certain Aesthetics again and again they also said that mid Journey video is going to come out later this year because they are continuing research and development on that feature the team at mustion AI came out with the ability to motion capture hands in a more realistic way let me show you how to use it to use the feature just go to the mus Discord Channel you'll find a link below this video and type in slash and we will do capture now all you have to do is drag and drop some video footage so I have some footage of this woman doing sign language here and usually finger tracking hand tracking very difficult with AI up to this point I'll go ahead and drag and drop that video into our frame here and for our character we can select any character we want I'll just select bot it's the default one and it's totally fine and then for our in place I'll select false because there is a little bit of movement and go ahead and click enter the render should take just a couple of minutes to finalize but I have an example from earlier so here is our result so you can see that the hands did a fairly decent job of tracking the fingers especially when we compare it to the original now is it perfect absolutely not but I imagine in one to two iterations of this tool fine motor skills will be captured at a semi-professional level we also came out with a new tutorial this last week on getting maximum cinematic quality from hyper again hyper is a tool that is entirely free so if you're looking for a way to create cinematic AI videos without any cost to you go ahead and check out that tutorial and that brings us to our AI films of the week the first film that I want to highlight is called Denny the shark it has this really interesting 1950s print aesthetic and it really comes across as a high art film the project was created by ethereal girl and I think they did a really good job on this demo film the next project that I want to highlight is absolutely hilarious it was created by Kevin Shaw who is a TA inside of the Curious Refuge program basically Kevin created a parody film that talks about a new feature that allows you to push a button and get a film it totally dunks on a lot of the tropes related to AI tools and I think it is one of the Funnier videos that I've seen in a long time and finally the last project that I want to highlight was created by Brett Stewart this project was really unique not only because I thought it was a really creative and awesome project but also because it was created for a professional client Brett basically used this tilt shift aesthetic and used AI tools to deliver a client project and I think in the coming weeks and months we're going to see many more commercial projects pop up around the web thank you so much for watching this week's episode of AI film news of course if you want to get AI film news delivered directly to your inbox you can subscribe by signing up over on Curious refuge and of course if you enjoyed this video like And subscribe to get the latest AI news and tutorials directly here on YouTube you got it like And subscribe to get the latest AI tutorials and news and we'll see you next time thanks for walking me through that you're welcome awesome I am glad I could help let me so if you need any other feedback on your outro

Channel: Curious Refuge

Views: 12,750

Keywords: ai, ai film, artificial intelligence, ai film tools, ai video, free ai video generator, curious refuge, ai short film, shelby ward, curious refuge podcast, caleb ward, ai filmmaking, ai filmmaking course, ai filmmaking tutorial, ai video editing, ai filmmaker, midjourney, pixverse, image to video ai, krea ai, freepik, hume ai, grok 1.5, ai generated video, ai video generator, alie jules, midjourney ai, heygen, heygen video translate, heygen video translation, ai lip sync

Id: L27Io8z2JTw

Length: 26min 52sec (1612 seconds)

Published: Fri Apr 05 2024