Google’s AI Video Generator is Insane + Runway Tricks

Google is now competing with p laabs and Runway there's a brand new tool that makes consistent characters easier than ever before we have some new tips and tricks with the new motion brush inside of Runway and 2024 is the end of stock libraries as we know it I'll explain why welcome to your AI film news of the week I want to kick things off by talking about the new Runway motion brush so last week we talked to you about the new tool that basically allows you to edit up to five different brushes and have customized control over the movement inside of your scene well we actually came across this last week a really interesting demo from David who basically tested the limits of the new motion brush parameters I think it's a really good example of what is possible with the new feature I came across an interesting demo from Runway where they actually created a really interesting Dolly Zoom effect using the new motion brush and I want to show you how you can do it for yourself so we're inside of Runway and we'll go to start with image and we will select from our assets you could also just drag and drop your file directly into the browser if you want and I have this image of this guy basically on a bus and we will go to motion brush so now here comes the fun part so the first thing I'm going to do is turn my motion brush up to Max size and let's paint the background so so painting on the background here and we'll change the proximity to Max now let's go to brush number two and this time we're going to paint our subject and now we'll change our proximity to about two doesn't have to be perfect and then we'll go over here to camera motion and change our Zoom you can do minus 1.5 -1 -2 they all create really interesting results and go ahead and hit save and then you hit generate so this is the result we got from Runway and it looks fantastic you can see we have that really interesting trumbone shot effect that looks pretty cool but it does have some of that flicker so what I always like to do with Runway footage is run it through a def flickering tool there's a plug-in called flicker-free that you can use inside of Adobe Premiere Pro or after effects or you can use Da Vinci resolves the flicker effect to get rid of some of that kind of exposure flicker that is pretty typical whenever you're working with Runway footage and and whenever we run it through the D flicker effect we see that we have this result that looks absolutely fantastic so hopefully you can test around with this effect and create some really interesting results and while we're talking about Runway I want to mention that they actually dropped a physical store that has a few different print magazines that you can purchase now their first magazine telescope is sold out at the moment but there's a few other iterations of their magazines that look pretty interesting and you can check them out by clicking the link below this video and we actually also talked with Ian from Runway who creates a lot of the demos that you see whenever you are on runway's Twitter account Ian is an incredibly talented video editor and we were able to ask him about Runway and the future of creativity will be dropping that podcast next week Google also announced Lumiere their video generation tool and it is absolutely amazing to see some of the results from their test now you don't have the ability to mess around with this tool just yet but you can see some of the results from their website so I wanted to take a look at a few of these so obviously they have text to video and what's really striking is how realistic a lot of these videos are whenever you compare them to some of the other video generation tools that are on the market now they of course do image to video as well and you can see some of the results in there obviously abstract and organic physical effects look better so really good wave distortions and lifelike movements when it comes to fire water snow things like that they also have the ability to do style Generations basically you upload an image and it will create a stylized video from that result so it's very similar to The Runway gen one effect that does the exact same thing but the results seem very interesting they also have the ability to do overall video stylizations that completely change your video into a different style and the results seem very very stable this is the most stable video stylization tool that I've seen up to this point similar to using the inpainting tool inside of pabs they also have the ability to select certain frames inside of your video and animate them the results look very realistic and they also have the ability to do video in painting where you literally can fill in certain areas of your video that never existed and the results are again the most realistic that I've seen up to this point there's some really interesting examples over on their website where they change the clothes that a woman is wearing they put a crown on an owl and of course a party hat on a chicken which is of course one of the most important things you can learn as a video editor I really think that this update really does highlight the fact that Google is looking to be a leader in the generative AI space and we can fully expect to see many more impressive announcements from them in the coming months over on X Martin Nong put together a demo that basically went from 3D generated asset to a highres render in just a few steps it's absolutely mind-blowing and I can't wait to show you the exact steps that you can take to get similar results so let's get started it all starts inside of luma ai's Genie tool so in our instance I'm going to say a mech suit with wheels and we have a few different instances of Mech suits here I'm going to pick the one I like the best I think the top left one so we have this very lowres render of this mech suit you can hit make highres if you want that does take a few minutes and we're not going to do that for this specific use case and I'm going to go down here to the format hit obj and hit download so now we're inside of Photoshop and I'm going to drag and drop that 3D obj file inside of my canvas and click okay so here you go you actually have this 3D model now directly inside of Photoshop we can move it around and just restructure our scene to put the model in the right spot I actually think I want to do a graphic design where the model is kind of over here on the side and you actually have the ability to adjust some of the lighting as well this is for a quick example you can of course use 3D modeling applications as well I think this looks pretty good so I'm going to go to file export quick export as a PNG and we'll just save this to our computer so now go to to Leonardo and you can go to the realtime canvas which is incredibly fun to use this is an example that we had from last week's AI film news and I'll just go ahead and delete this canvas so now go to the add image button and we will drag and drop our mech suit into the scene and we can go down here and say a futuristic mech suit with wheels and now as you scale up your image you will see that that image is actually rendered on the side over here and the cool thing is we can turn down the creativity strength which will make it much more in line with the 3D model that we uploaded so this looks much much better when we like what we're seeing all we have to do is go to rightclick save image as and we'll just save this as our mech suit now and you probably guessed it we're going to go to magnific to extrapolate even more information from that image so basically I'm going to grab that mexit drop it into magnific and let's go ahead and we will do a times 4 upu res and again I'll say a mech suit with wheels and we'll keep everything else at the default setting and click upscale so here we go here's our original image and we can slide over and see that we are getting all of that very crisp detail it looks fantastic and I'll go ahead and download it so now I have this very quick design inside of Adobe Photoshop and all I'm going to do is drag and drop our mech suit into the frame here we'll drop it behind our text and now we can just scale it up up as we see fit and there you go we have a highres 3D asset for a graphic design project and of course we can go in here and move the assets around and line them up to basically make the design as good as possible so really we were able to put this design together in just a couple of minutes and the assets look absolutely fantastic I highly recommend checking out Martin's demo over on X where he kind of did the same thing with a school bus and I was really impressed by the result results he was able to get one of the biggest challenges that we talk about on AI film news is getting consistent characters well many companies have been trying to address that problem and I have a really interesting tool that I want to show you today that basically allows you to generate consistent imagery with a subject in a more professional way than some of the previous examples that I've shown you on this show so it's called render net and you can find a link below this video from their website you can go to to face lock and now all you have to do is upload an image I have this really quick selfie that I took earlier today and I will drag and drop it into our scene so now that our image is inside of face lock we now can go in and type in a prompt so I'll keep this as default it's going to create this Cowboy scene and let's hit generate okay so let's take a look at our results so we have this image that really does look like me with a beard and it does look a lot more realistic than some of the other tools that I've seen that do consistent character faces in the past so what this tool is doing is basically allowing you to edit with some of these tools that are typically only available if you're running stable diffusion on a comfy UI and it's a whole technical setup they're basically making all of these tools available to you in a very easy to use platform and it doesn't stop there they actually take it one step further so they actually have the ability to use control net which can control the overall output that you get from from your scene so for example if we take this image that we used earlier and drag and drop it into our scene here we have the ability to go in and adjust some of these settings what I like to do is say that control net is more important for our prompt I have a mid shot of a man wearing a suit on a bus detailed clothes basically describing the overall image that we uploaded and I should note that there are a few other settings that you can go in and adjust so you have the ability to select certain models that you want to run so for our example we can say real viz XL you also have the ability to go in and adjust some of the settings if you want the default settings are really good I'm going to make sure that the aspect ratio is set to 3x two they don't have the ability to do 16 by9 just yet uh but I'm sure that will be an option very soon and you also have the ability to go in and select extra luras to kind of push your Generations in a certain direction I'm going to select the cinema Excel Laura model and you'll see that it also adjusts our prompt just a little bit I also like to make sure that in settings the batch size is more than one I'm going to say four for this generation and we will click generate so here's an image that I generated earlier from the reference using my face and you can see that it kind of looks like me and it also interpreted the composition from the reference image and created what you see here so it looks pretty darn good now I want to contrast this tool with my favorite face swapping tool Insight face swap which you can actually integrate inside of Discord which makes it very helpful whenever you are working on Mid Journey projects so I have this image from mid Journey I'm just going to bring it directly inside of this Discord channel here and all you have to do is train it on a face so you can go to forward slash and we will do save ID and all you do is drag and drop an image I have this image of me drag and drop and we'll set the ID name to Caleb there you go and now we can forward slash and say set ID and type in Caleb so now this is telling Insight face swap that it's going to swap my face onto any image that we select and we can go to this original image go to the three dots go to apps and go to ins swapper so there you go there's the result using Insight face swap is it perfect no but with a little tweaking especially if you were to take this image and go over to Adobe Photoshop and maybe use generative fill to kind of fill in some of the Scruff to make it a little more realistic or remove some of the smoothness in the forehead up here it could look pretty dang realistic I saw a really interesting example of using haen this last week basically they took a speech that Javier mle did at an economic forum and translated it into English and the result was very realistic it is in danger because those who are supposed to defend the values of the western world are co-opted by a world viw and I think it really does highlight two important points the first is obviously going into an election year misinformation there's a good chance that it could be spread like crazy so it's going to be up to a lot of these AI tool developers to put water marks inside of their Generations so that we can tell the difference the second is I really do believe that by the end of the year most of the content that you consume online will be in your native tongue AI will essentially translate all online video into the native language that you speak this will absolutely even the playing field as knowledge and information will be available in everyone's native tongue this also Echoes an announcement that came from Samsung this last week where they said that the Galaxy s24 will actually have the ability to translate live directly on their phone it doesn't have to send it to the cloud it translates languages on your phone and in real time to where you can basically have a call with somebody and listen to their voice in your native tongue so you essentially can talk to anyone in the world I also have an event hul right now and the atmosphere here is amazing a research team out of China put together a quick little demo that showcases basically turning gaussian Avatar so that's a gaussian Splat of of a human into an animatable object in real time so we have a subject that's basically doing some movements here and that movement is being directly translated onto a 3D avatar and in other research news a team at Tik Tok actually put together a really interesting white paper and demo that you can use that basically converts any image into 3D depth so think about lay aics but a little more realistic and you essentially have the ability to upload an image and get a very realistic depth map that utilizes a lot of the data that was ingested by the team so the results look really fantastic and I think it really does highlight the fact that the future of visual effects will include depth maps that are automatically generated as opposed to rotoscoped by hand the team at Leonardo is also giving away access to their new Alchemy versions completely for free until January 28th so you can play around with those tools over on their website now I did come across an interesting Tech demo this last week and I want to show you a very interesting use case utilizing one of the underutilized tools inside of Leonardo so I'm inside of their image generation tool here and let's say that we have a base image so I have this image here and let's say that we actually want to do a subtle variant of this image now assuming this was not created inside of mid Journey or any other image generation tool where you would have the ability to subtly change this image that could be a little challenging and here's what I mean if I upload this image to Discord grab the URL go down here to the chat SL imagine and paste and then describe our overall image I then have the option to do D- IW for image weight and you can type in a number between 0o and two two is the maximum weight wait to basically pull the style from your uploaded image so two is as close to the reference image as possible now let's go ahead and run that because really what I want is the core visuals to remain the exact same I don't want it to change the overall visuals that much but let's say I want the hair to be a little different and maybe the clothes to sit in a different way you can see here are the results from inside of mid journey and while it does look fairly close to the uploaded image it's still pretty dang different print and so this is not exactly what I'm looking for but if we go over to Leonardo and go to image guidance we can upload our reference image so I'm going to take our reference image drag and drop and we can turn up the style strength quite a bit I'm going to change the strength to 80 it goes a long way and I'm going to paste in the prompt that we used inside of mid journey to produce the exact same result and let's go ahead and generate so let's take a look at our images that were generated here we have these two images and you can see there are very subtle differences between the uploaded reference image and these results so you can go in and fine-tune that slider and change it however you want but it is a much better tool if you need it to stay true to the uploaded reference image in stock photography news I came across a tool that is very interesting but I think it really does signal the end of stock photography as we know it so it's called dwat mark and it does exactly what you think basically you can upload an image I have this image of Shelby and I it has water marks all over it you can see here and we can drag and drop it into the tool and within just a few seconds the water marks are removed and you're able to download it so obviously watermarking is the primary way in which stock libraries typically protect their images and now that is no longer the case however I did come across an article this last week from a stock photographer where they were talking about tools like mid journey and how they essentially can create images that are perfect and more customized than the photography they could shoot themselves for a stock library and really the takeaway was that stock photography as it relates to live events that are happening is not going anywhere it would be very uninteresting to have wedding photos that never existed or event photos that aren't quite true how however when it comes to using stock photography for marketing or advertising where there's a bit more flexibility AI is basically taking the entire industry by storm and while we're talking about graphic design I want to show you an interesting AI graphic design tool that I came across that is really interesting and it does some things that I've never seen before so the first thing I want to show you is the ability to actually change the color of the generations that are created so there's a lot of features I'm not able to get to right now but essentially you have the ability to create an image I'm going to say a vector image which means you can scale it up infinitely and we'll say a dog wearing a hat and we will click recraft so we have this fun image of a dog and the cool thing is you can double click on it and it gives you the color swatches so you can go down here to swatches and change them to your Brand's colors directly inside of this tool or you can select some of their presets here some of the presets don't make as much sense but it is interesting to see that you can basically upload your brand guidelines and generate artwork directly using your color palette the tool also has the ability to turn raster images or images that use pixels into Vector images that you can scale up infinitely so for example I have this image of our logo here and we see that we have our Chester mascot and let's say that we want it to be in a vector format well it's super easy all you have to do is select the image and go to to vectorize so now we have the ability to scale it up infinitely and we're not going to see any pixels obviously that is very helpful if you work on Advertising projects and while we're talking about advertising I want to let you know that our courses will go on sale on January 31st at 1100 a.m. Pacific time this is the first session where we are offering AI advertising taught by the one and only Dave Clark it's going to be an incred incredible course but we still are offering our AI film making course that is updated to use the latest tools and Technologies to tell your stories using artificial intelligence our training has helped many students land jobs even this last week and we'd love that we've been able to train artists at the biggest Studios and companies in the world we'd love for you to join us on this new creative Journey the tmat 11 Labs also updated their interface it's a lot more sleek and easier to navigate navigate you have access to the dubbing Studio the project studio all of that good stuff and 11 labs they were actually sponsors of our Halloween competition which was incredibly fun we'll have to do that again this year and I just wanted to note that they actually raised $80 million this last week and are valued at over $1 billion so the team at 11 Labs is absolutely slaying it in addition to the lip debing features that we talked about earlier the team at haen actually put together an interesting demo where they had their CEO talk as if he was a chatbot so essentially you're able to have a conversation with it like it's chat gbt for example but the subject actually talks back to you in a realistic way it's incredibly interesting and I think it does highlight the future of creative interactivity as a real-time Avatar I can discuss a wide range of topics such as technology entertainment and general knowledge the Apple Vision Pro is a powerful system on a chip s so designed for computer VIs vision and machine learning tasks and now that brings us to our AI films of the week I want to kick things off with this film called tainted by Tristan Reddit and I think that it is a really interesting artistic piece that shows a lot of good curation skills it feels like it belongs inside an art gallery and I think Tristan did an amazing job on this project the next film I want to highlight is called Silent Shout by Bill marxy and it's more of like a mood film there's not as much of a story but there's some great shots that utilize fire and some really Dynamic camera movements that I haven't seen on too many AI projects up to this point so fantastic job on this project our next film comes from Ashley mccauly who put together what she calls a mood film that was created for a Beach Club Hotel competition by hi 3 they host some really cool video competitions you can check them out she utilized runway in her project to basically create this fictitious hotel scene that has some really interesting curation and shots so I think this project is awesome and our student project of the week comes from a Discord user that goes by the name of Bronco they put this together for their week three assignment which is basically to put together a poem or a travel film and I think some of the imagery that they curated is absolutely stunning I love the shot curation it looks like a travel film and many of these shots have this kind of ethereal ambient that really evokes kind of a a sense of awe inside of the film I definitely recommend checking it out thank you so much for watching this episode of AI film news of course you can subscribe to get AI film news directly to your inbox just go over to curious Refuge click the start here button and you can join us for weekly AI inspiration and of course you can join us for our February session en rollment opens on January 31st AI film making starts the first week of February and a advertising starts the first week of March thank you so much if you enjoyed this video be sure to subscribe we'll see you in the next one

Channel: Curious Refuge

Views: 73,114

Keywords: AI Film News, AI Video, Google AI, Runway, Pika Labs, Elevenlabs, Runway Tutorial, Dolly Zoom, Dolly Zoom Runway, Consistent Characters, AI Graphic Design, AI Filmmaking, AI Advertising, AI Course, AI News, AI Art News, Caleb Ward, Curious Refuge, gaussian splatting, HeyGen, heygen dubbing, Motion Brush, AI Films, AI Depth tool, AI Magazine, AI Tutorial, Fun AI Tutorial, AI Video Generator, Best AI Tools, Google AI Video, Google Lumiere, Google Video Generator

Id: lxAkt89WQtg

Length: 25min 6sec (1506 seconds)

Published: Sun Jan 28 2024