Adobe Supercharges Premier with AI Tools (Powered by Sora)

check this out guys generative AI tools powered by the new Adobe Firefly video model are coming soon to Premiere Pro take a sneak peek at object Edition object removal and generative extend among other things this is a big deal for a couple of reasons we've seen a lot of generative AI video tools so far made by small companies small startups trying to gather a new user base Premiere Pro already has a massive user base so when to soe drops these tools they kind of set the stage for the rest of the ecosystem but before we dive into this I do have a quick word from our sponsor as someone who is constantly investigating and testing out the latest in AI technology online security is definitely a priority and who better to help me out with this than nordvpn today's sponsor nordvpn not only encrypts your internet connection but maintains top speed while doing so it's not just about security though it's also a great tool if you're into AI Tech because if you want to access geo-restricted content which unfortunately a lot of new AI tools are geo-restricted nordvpn has your back by allowing you to Tunnel through and connect to those countries where the AI is unblocked and here's the best part nordvpn is offering an exclusive deal just for the Matt vidpro Community you can head over to nordvpn.com vidpro Ai and you get a discount with four extra months for free it's even risk-free with a 30-day money back guarantee so the link for for that will be in the description below whether you're trying to get access to the latest in AI or you just want to keep your internet data secure nordvpn absolutely has your back grab this deal while you can and let's get into the rest of today's content let's jump in here and take a peek Adobe is using the power of generative AI to deliver the most advanced and precise editing tools ever in premere Pro let's take a look at some features coming soon powered by our new Adobe fire Fly video model that'll transform how editors work one common editing challenge is finding something in a shot you need to add or replace the object Edition feature in Premiere Pro combined with adobe's firefly video model lets you add or change objects in footage with a text prompt make a selection write a prompt and add anything you can imagine these diamonds were created by the Firefly video model currently in development and not only can you add objects but you can easily remove REM them too object removal with AI based smart masking makes selecting and removing objects across frames quick and precise here this distracting utility box is removed with firefly remove unwanted props crew or gear or copyrighted elements like brand logos and with Premiere Pro all your edits are non-destructive so you can always get back to your original footage when your clip is too short and you want to hold on a shot or a character for an extra beat generative extend intelligently adds frames with firefly extending your footage so you have the exact media you need like here where we extend this shot of the woman a few frames as we continue to innovate we're committed to transparency through the use of content credentials now coming to Premiere Pro so you always know whether AI was used in the creation of the media you're viewing we can't wait to bring these Firefly powered video editing workflows to Premiere Pro later this year and there is more we're excited to share our Explorations around giving editors the choice to use models that work best with their footage here are some early examples of how thirdparty generative AI models could look within Premiere Pro like Pika powering generative extend to make a scene flow better from open AI here's an example of their Sora model currently in early research generating b-roll for any scene through Simple Text prompts Sora creates variations for you to choose from here's the same use case now using Runway ai's video model generating a new video clip and adding it to the timeline in a snap and content credentials coming to Premiere Pro this year will always make transparent whether AI was used and what model was used in the creation of media to recap revolutionary features like object add remove and generative extend powered by the new Adobe Firefly video model are coming to Premiere Pro and we're excited to show early research Explorations with our friends at open AI Runway and paa so editors have the freedom to use the best model for their project it's Adobe premere Pro supercharged by a yeah good Lord okay we're going to go through this but I honestly don't know how Adobe has managed to do this so not only are you getting Tools in Premiere Pro that allow you to replace things magically or remove objects magically you get all of the models you get SORA which is not even released I mean people will jump on this just to try to get access to Sora Runway ml which is kind of the OG first legit video generation model and then pabs which in my opinion is a little bit better than Runway ml right now so you just get to decide which model you want to use for which task which will result in the best creation process without having to pick between all these little startup companies or these new models it's interesting it's an approach that I think works and I don't know how they've managed to coer everybody into allowing this adobe's Got Deep Pockets and a massive customer base so I do have lots of things to point out about this though first up let's talk about the integration directly in premere Pro I don't even use premere Pro but yeah for a feature like this I would consider getting it for sure and I'm not a huge fan of adobe to be honest especially those subscription plans they annoy me but you know this is kind of an unbeatable feature more or less where you can just sort of highlight something and then replace it with a prompt very quickly and this is the Adobe Firefly model by the way that's being used this is not soraa this is not Runway this is not Pika this is just their Firefly model which honestly at least in this example does a pretty impressive job at doing the diamonds there although it does just look like a static image that was placed inside if I'm going to be honest so I don't really know how their generative AI model works but in that use case it works very specific use case you'll take note here that this is exactly the same solution that they tried to do with Adobe generative AI in Photoshop and personally as a longtime user of that very feature the generative fill feature in Photoshop yeah it's a pretty good solution it works well I mostly use it just to extend images though for thumbnails of course because that's what I create but if I was using this for video Work short films Etc I absolutely would be doing some object replacement or background removal I grew up making budget short films so I can say that these use cases I mean they're real for sure there's a lot of use cases where you just don't have any props you have an image in your head of what something will look like and you just can't get it to that exact image and yeah this this is a game Cher for sure now I will say this object removal I think it's going to be a little bit more difficult than they make it appear in this ad see in this ad they have it just go down here click this magic little button and you can highlight any one of these objects and this object selection might work okay but let's say we want to click on this object and then remove it as they do in the ad now it does get removed but you'll not notice you can see where it was at first watch you might not really notice it but it definitely bothers me that these lines aren't very consistent and it's it's just kind of not perfect filmmakers even small ones are going to be a little bit peeved by that so the way that I see this feature is more or less as a last resort rather than something that you can use reliably all the time and sure in this situation yeah it works completely fine because we have a completely blank white wall but most of the time that's not the situation you're going to be dealing with because this is let's be honest not a very realistic scenario in reality you'd probably be able to remove this sign quite easily so unwanted prompts again this one you guys probably didn't even see because my head was blocking it but yeah they have this little you know cord hanging out in the background here that's a complete accident or that would be a complete accident in a real film set so they just remove it quick and easy solution this is more or less a Flawless implementation although maybe you can see a little bit of weirdness over here I say that's a that's a great use case this is okay again this is probably using their Adobe Firefly video model which appears to do like an image and then map it out into the scene but I I still wouldn't say that this is great because you can just tell it's an AI generated watch you wouldn't want to use that for a professional use case I'd still say this shot is better because well it just looks more like a real watch in this situation here they do something pretty awesome which is add this tie in and the time I got to say it looks pretty damn good but this is something that we saw from Adobe quite some time ago months and months ago this technology where you can add a tie in for example and that already looked good back then so that's just sort of us seeing something that we saw a few months ago but now we know where it's going to come in Premiere Pro so now they also talk about continuation of a shot for an extra beat and this again as someone who has had the experience of doing budget film making yeah this is something that you need sometimes it's nice to have and again I don't know how I feel about this because well yes you get to choose between any of the models Adobe Firefly Pika Runway ml even open AI Sora let's say even in this ideal scenario it sort of just holds it for a beat and like you can see the transition between the real footage and then the AI generated footage the models just aren't really good enough yet to sustain this sort of a workload let's say it's just it's very noticeable it's very noticeable maybe if you combine it with like a blur out effect or something you could get away with it but it's just not working for me and obviously right here is sort of the big reveal that they're going to be working with open AI to bring Sora into Premiere Pro and as you can see this model well it's quite a bit different than pabs or Runway ml or even the Adobe video model because it's just so much more there's so much more going on here and I mean this I would say is usable for a short film I think Sora is that high of a quality at least from what we've seen so far so this is very very much exciting probably the most exciting thing of them all just the the ability the chance to use this magic mystical model but that's how it feels right now and even I want to point out here they show off using runwayml to Addie clip in yeah they added into the timeline but let's be honest this just does not look quality for a short film maybe if you're doing a YouTube video right that makes sense you have some b-roll generated by Ai No One is really going to get fussy about that but if you're doing a short film you're telling a story that is distracting look at those little dials there those gauges for the car it looks it looks AI generated so again I don't know about adding entirely new shots it really seems like you should be using the AI at least in premere Pro right now for emergencies unless it's Sora where you're doing some object removal maybe in the right circumstances so yeah I think it's worth now talking and getting into what this actually means what is the TR true effect of Premier Pro getting all of these massive generative AI tools well first of all it tells us that open Ai runwayml and pabs see that their value does not come in the actual user interface that they've built for interacting with the AI tools because well Premiere Pro is a direct replacement for all of that they see that the value of what they have truly is inside of the models and while the models are not great yet they will get great I mean within one two 3 years they're going to be you know a couple fold better and they will be able to do things like continue your shot in a way that you can't tell it was continued or better object replication better object addition etc etc now the models aren't there yet so for the meantime the short-term micro distance I'd say that they're only useful in the emergency circumstances or for like I said making YouTube videos not necessarily the real deal short film use case however I think at least for now there is something that adobe is missing out on and that is from the groundup AI generative thinking they're taking their current tools and applying generative AI to them instead of creating a brand new set of tools from the ground up that is built around generative AI I am firmly in the camp that says hey we have new tools while there might be things we can learn from our old tools that never had the experience of being developed while generative AI was around it is crucial that we apply some outof the Box AI first thinking and really build some new tools for accomplishing tasks like movie creation I really don't know if Adobe knows that I mean I don't know if they're they could be working on some AI first video creation app but for now it seems like they're trying to just add the power of generative AI into their current day tools and this is where I want to bring things around contextually to LTX Studio not too long I did it video on this app right here LTX Studio this is an AI first rethink of how we create stories or short films and again the AI models are far from perfect so this is not usable in the same way that maybe some of those Premiere Pro Tools are today because the models aren't good enough for groundup creation but they have a head start in a different scenario for when those tools or models are good enough because this creates the story overview locks in our casting all AI generated characters that can be face swapped right away or clothing style changed right away it's going to go ahead and build all of our shots and our whole story for us with AI that can then be modified and edited by us we can change shot type on the Fly we can change how much motion there is on the Fly we can change the lighting per scene the weather per scene etc etc and you know this approach is quite a bit different it uses the capabilities of the AI in its design so this is a lot more AI first and thus when the models get better will be more user friendly and perhaps more effective than Premiere Pro in 1 2 3 years I really do think that adobe will be shooting themselves in the foot if they don't build something that is from the groundup AI first because theoretically right now they've taken away the competition of open AI Runway ML and pabs on that front so at least for now it's going to be Premier Pro with all of those tools in those traditional settings versus LTX Studio which is this from the groundup AI first creation process the models aren't good enough yet for LTX to dominate but like I said I really think we're only one two three years away from that let me know what you guys think about this again it really hits close to home for me and I really do care about this stuff because I love video creation no doubt this is why I'm here and I'll be very very intrigued and excited for when this comes out especially because I want to try Sora so bad oh yeah and by the way we don't really get a release date for this at all they just say hey take a sneak peek at all these things we're working on it's coming soon we don't know when I assume that means this year because we still have quite a long ways to go through 2024 so I'm expecting this year oh and I will mention the content credentials here I do want to make a note about that I think that for in the meantime that is a pretty decent solution for dealing with AI generated content people want to know when AI was used in the content they're watching so I think for now that's a decent solution but in the long term again I don't think it's going to work at a macro scale when these modelss get ridiculously good and even more powerful but thanks everyone for watching I'll see you in the next one more exciting AI news coming up shortly goodbye

Channel: MattVidPro AI

Views: 27,740

Keywords: mattvidpro, mattvidpro ai, adobe ai, premier pro ai, sora, sora ai, open ai sora

Id: qqcdo8MWAlU

Length: 16min 55sec (1015 seconds)

Published: Mon Apr 15 2024