SORA + Adobe = AI Video God Mode!

Adobe has announced its AI video project and it's also teased us with some of the generative AI features coming to Premier Pro this year there are three key features that adobe are adding to Premier Pro that are going to quite literally change the way we edit and create videos forever and the first is bringing in generative b-roll directly into Premier Pro and what that means is is that we'll be able to create extra and sequences inside of Premiere without having to go into another AI video tool they've specifically said that you're going to spend less time hunting for b-roll shots in stock footage and use Simple Text prompts to generate video clips that are just right for your story now you can create a shot that illustrat a difficult to shoot or Fantastical concept generate b-roll that's hard to Source or craft placeholders to assist with planning and pacing so essentially what they're talking about is being able to create any shot you like inside of premere now what's really exciting is they've not only talked about their own AI Firefly video model which they are developing but also bringing in thirdparty video generators into Premiere Pro and they have talked specifically about adding in Sora Runway and pea here is a screenshot from their Explorations where you can see that you're able to select a different model next up is generative extend which is if you have a need for more frames well no problem the generative extend tool can be used to make shots a little bit longer by generating entirely new frames at the beginning or end of a clip use the extra media to fine-tune edits hold on a shot for an extra beat or add a few frames to cover a transition just click and drag for the perfect edit so here they are describing the ability to extend any shot that you already have so that if it's not quite long enough for your desired purpose instead of having to slow it down or use other non-perfect tricks you'll actually just be able to add extra frames now this is a really useful tool that will give us a whole lot more freedom when it comes to editing our work now the next tool is particularly ex exciting and that's object addition and object removal what they said is new AI based smart masking and tracking tools make it easier and faster than ever to select moving objects over time combined with the object Edition and object removal tools editors can now replace moving objects in a shot remove unwanted items like boom mics Bing Bing Bing or stray coffee cops sometimes you might have noticed that my water bottle is inappropriately positioned within the shot here and I often berate myself for forgetting to ex exclude even my dearest water bottle pop him on the floor for professionalism but with the latest AI tools I won't have to worry about that anymore now we're very lucky because we actually have a demonstration from Adobe where it is showcasing how all of these may work so let's take a look at that together and break it down into more detail Adobe is using the power of generative AI to deliver the most advanced and precise editing tools ever in Premier Pro [Music] let's take a look at some features coming soon powered by our new Adobe Firefly video model that'll transform how editors work so one very interesting thing here is that adobe is generating its own AI video platform where you'll be able to generate videos inside of adobe and this can be interesting for a number of reasons firstly Adobe is actually putting itself out there for being more responsible in the space of AI but by being open and clear about how its platforms were trained and that's not something that we've seen very clearly with other AI video generators so far and my suspicions is that a lot of the existing AI video generators have been trained on YouTube One common editing challenge is finding something in a shot you need to add or replace the object Edition feature in Premiere Pro combined with adobe's firefly video model lets you add or change objects and footage with a text prompt so here we can see how you'll be able to mask out a section of a video and basically use a generative f for video style tool so you can see that you'll be able to select the model either Adobe Firefly Sora p runway or others and simply add in a text prompt in this example they're demonstrating a suitcase filled with jewels and here they're asking for a slightly different variation of what is already there so they've asked for a pile of large diamonds on Black Velvet cloth make a selection write a prompt and add anything you can now here you can see the options that are being displayed you have three variations of these jewels in the suitcase anything you can imagine and as you insert them you can see that it updates the content inside of the video now what's particularly exciting is that this stays coherent as the video progresses so as the camera angle zooms out it actually shows in perspective the jewels these diamonds were created by the Firefly video model currently in development and not only can you add objects but you can easily remove them too object removal with AI based smart masking makes selecting and removing objects across frames so here we can see how the AI is able to identify different elements in a scene so that you can quickly mask these and remove them from a scene you can just say adios seor you're fired quick and precise here this distracting utility box is removed with remove unwanted props crew or gear or copyrighted elements like brand logos they're demonstrating a couple of the use cases for this about maintaining adherence to any copyrighted material or even getting rid of more aesthetic stylistic elements and with Premier Pro all your edits are non-destructive so you can always get back to your original footage to elaborate on that point they say that it will be very easy to go back and make changes that it doesn't mean that if you've edited this video that you're suddenly stuck with this new version of the video that they will make it so that the process of being able to generate or regenerate elements inside of a VD video will be simple and nondestructive want to hold on a shot or a character for an extra beat generative extend intelligently adds frames with firefly so this is very beautiful here the ux that they have for simply adding frames to a shot will be a simp simple as dragging a clip and extending it in your timeline allowing you to seamlessly add extra elements extra shots to your clip it'll be interesting to see how that works with just a single frame adds frames with firefly extending your footage so you have the exact media you need like here where we extend this shot of the woman a few frames as we continue to innovate we're committed to transparency through the use of content credentials now coming to Premiere Pro so you always know whether AI was used in the the creation of the media you're viewing we can't wait to bring these Firefly power now this is something I'm a little bit more skeptical about which is their description of content credentials which aims to be more clear about when generative AI has been used to create a video but obviously I think that will only remain in the video format and actually there'll be many ways to remove these credentials easily from the video so it'll be interesting to see exactly how that n Works video editing workflows to Premier Pro later this year so this is particularly exciting is that we're expecting all of these features to arrive inside Premier Pro this year so let's stay tuned for the next six months there is more we're excited to share our Explorations around giving editors the choice to use models that work best with their footage here are some early examples of how thirdparty generative AI models could look within Premier Pro like Pika powering generative extend to make a scene flow better I I would like to particularly compliment Adobe on their commitment to integrating with thirdparty apps and I think that the AI video generators that they are describing Pika Runway and potentially Sora are all beautiful tools and it's very useful now to actually understand how well these work and their capabilities as well as getting your hand on them and if you're interested to have a closer in-depth look at how each of these tools Compares I recently did a comparison video exploring exactly the strengths and weaknesses of each of these AI video generators and you can check that out in the [Music] description from open AI here's an example of their Sora model currently in early research generating b-roll for any scene through Simple Text prompts Sora creates VAR so here we can see that once you've entered a simple text prompt for a piece of b-roll you can be offered three different shots that you can then select to insert into your project for you to choose [Music] from here we can see a beautiful video that is purported to have been created using open AI Sora and you can see that it has a high of realism and a very good level of coherence with the composition and also the perspective look at the way that this street naturally extends into the distance here's the same use case now using Runway ai's video model generating a new video clip and adding it to the timeline in a snap and content credentials coming to Premiere Pro this year will always make transparent whether AI was used and what model was used in the creation of media so they they're making a song and dance about these content credentials but I honestly am not hopeful for this being a clear and useful solution to labeling AI generated content revolutionary features like object add remove and generative extend powered by the new Adobe Firefly video model are coming to Premiere Pro and we're excited to show early research Explorations with our friends at open AI Runway and so editors have the freedom to use the best model for their project so there we have some very exciting developments coming to Adobe premere really taking the potential for AI video to the next level not simply creating small Clips in isolation but actually bringing in generative AI to the full film making process now this really offers us a huge increase in productivity and it's certainly going to reduce a lot of the editing tedium remove a lot of the boring and repetitive tasks that recur when editing video I believe that it will shift the focus to improving storytelling and allowing creatives like you and me to focus on what we enjoy most and really Elevate our ability to produce more at a higher quality and this is what I believe AI is destined to do for us all until the point that it takes over everything what now there are a number of very very useful AI features already available in Adobe Premier Pro that you might not be aware of and one of my favorites is text based editing where it automatically transcribes your entire video into text using Ai and then you can simply highlight pieces of text and delete them and it will make that cut in your video now what I particularly like about this textbase editing is it identifies when there are any long pauses and it automatically shows you where these are tells you how long they are and with a simple keystroke you can delete them and it speeds up one of the very repetitive tasks in video editing now there are other very small features that are beautifully interwoven into Premier Pro for example AI audio category tagging where it automatically tags what type of audio you're using now there is a speech enhancement tool that allows you to increase the quality of your audio even if it was not recorded in the most desirable settings so it can help remove background noise hump or uh reduce any annoying S's and PS and one of the other most beautiful and advanced features that Premier is already showcasing is auto reframe which uses AI to automatically track the most interesting part of a clip that is in landscape mode and turn it into a portrait video so that's really great for for taking your long form videos and repurposing them as shorts now another beautiful tool that they already have is morph cut which is the ability to easily cover cuts that you have in your video by blending two shots together now this works particularly well if you are removing awkward pauses in a talking head video like this so I've used it a number of times in this video and you might have not even noticed where perhaps I paused for just a moment too long and I wanted to bring in a tighter edit that I cut out the pores and then inserted a morph cut to cover that cut the AI Video Revolution is just getting started and if you want to follow along for the most exciting developments in AI then make sure to subscribe and stay updated with me AI Samsung thank you very much for being here thank you for watching if you have any feedback on this video do feel free to leave your suggestions in the comments and most of all I wish you a delightful day

Channel: AI Samson

Views: 13,215

Keywords: ai, samson vowles

Id: BcR6CbfQoz0

Length: 14min 32sec (872 seconds)

Published: Thu Apr 18 2024