Using AI to create VFX Environments ➡ After Effects Full Tutorial: Generative Fill with Moving Shot!

foreign [Music] what's up guys it's Brad from light architect in this video I'm going to show how you can use Photoshop generative fill in order to create distant set extensions for a simple moving shot instead of Adobe After Effects this is the second part of our generative build for video series we're coming out with three different parts initially the first one we have released already and that covers how you can generate an ad set extensions to locked off video or film clips instead of Adobe After Effects using generative fill instead of Photoshop in that first video we also show how you can expand that shot into a wider frame using generative fill to fill in the background and then clean up those filaments as well now in this second video we're going to take these set extensions one step further by showing how we can generate them and then track them into a moving shot and then finally in the last video for the series which we will release sometime this month we're going to show how we can use Photoshop generative fill combined with blender 3D camera projection so we can use these set extensions with even more complex moving shots where Parallax is a lot more prominent anyways Guys Without further Ado let's get started here we are inside of After Effects the first thing we're going to do is in import our footage into our composition so this is the shot we're going to be working with today we just have this moving shot of this woman looking out over a rooftop and it's pretty cinematic shot shot at Sunset you can see the background is fairly blurry here so we're gonna see how Photoshop generative fill deals with that and one thing to try to do is just add some taller skyscrapers in this deep background as well so somewhere to the right of our character here I'm going to try not to have our character include these objects so we don't have to Roto her out but anyways we're just going to add those buildings here we're also going to track them using a point of contrast on our footage here so anyways guys we'll go ahead and import this inside of our composition so just drag this over here and then we'll create a new composition with this footage and now we just want to scroll through our timeline and choose a frame that we want to add our matte painting to I think we probably want to use some of the first part of this footage I'll just trim our composite to here we don't want our perspective to change too much because this is a simple 2D track we're doing in this video other than a 3D track that we'll show in the next one we'll choose this Frame right here just so we have plenty of space to add the buildings and we'll export this Frame from our timeline and let's get to work inside of Photoshop so I'll go ahead and go to composition same frame as file and we'll make it a Photoshop file go to Output call this city set extension still okay it's going to be a Photoshop PSD file so we can open that within Photoshop and get to work go ahead and save that and then finally to actually export the file we'll go to render and now let's go ahead and open up Photoshop so we have our Adobe Photoshop beta which contains our generative fill for now we'll go to open and we'll find the still that we have just exported here go ahead and open this up now as you can see here this is our generative fill toolbar we can select this object remove the background and as I mentioned in the previous video if we want some more controls we can just make a selection using one of our select tools here wherever we want to do something and then some new tools will pop up on this toolbar as well so the first thing we're going to do is create some sort of Clean Plate just in case we do want to occlude our subject from the background so a Clean Plate is going to contain our image except for our character here in the foreground I'm just going to use the lasso tool and I'm just going to draw around our character very roughly just like that now I'll click on generate to fill again I'll do move character generate all right so now we have a nice little Clean Plate we've removed our character from the footage we have three different variations here to choose from I think probably the first one is my favorite and you can see we have this generative fill layer here that we can turn on and off or even see by itself so great we've removed our character now let's start adding some buildings so to make it a little bit easier to add the buildings in the background I'm just going to use the rectangular Marquee tool and I'm just going to select where I want to add the first building so I think probably right around here add a nice tall skyscraper in the background I'll click on generative fill type in add tall building just like that it's added a building to our scene now this one's not looking too good however for a few clicks it's not terrible let's look at a few different variations here this one's probably looking the best out of these three but I'm gonna try to get some more variations here because I'm not really liking any of these so we'll try and building with Windows go ahead and click on generate a lot of the time you just need to experiment with different phrases to see what the AI can give us actually uh not really liking any of these either to be honest the sky's not bad a little Tower we can actually keep that but I'm going to try to add some more buildings as well um rather than just selecting the area where I want the building to kind of come from I'll actually just select some of the area below the building as well just so we can use that data as well and try to combine it into our Base building plate as well so go ahead and click on generative fill say add building keep it simple all right so I'm actually liking this guy right here I feel like it's done a really nice job matching the blur on the photograph so that's quite nice I've got to go with this I might actually add some things behind our tree here and then composite our tree over our set extension so I'll just go here we'll actually make a larger selection this time we'll say add more buildings click on generate let's see what we get all right so now it's keeping that same architectural style I actually like this a fair bit this one matches very closely this guy so that could work as well we have three different options here I think this is pretty nice so we have before after maybe we could add some more buildings on this side as well to kind of fill in the frame now I know I'm not really being specific with the architecture type guys you can try to be more specific with it but um in my experience it's been hard to get very specific results that are actually good in this stage of development but we could of course tell the style of the building so we could maybe over here use generate to fill and then we could say add cyberpunk sci-fi building go ahead and generate so you can be more specific if you want to but it doesn't always dial in exactly what you want which is why this isn't the perfect tool but it can definitely help get a lot of quick things done so you can see it's gone over here and added some sci-fi Elements which is actually quite nice you know we can do something like this I actually kind of like this one you know we have like an older City here old architecture new architecture yeah I think it did a pretty good job matching the perspective could be rotated a bit so we can fix a little bit of that in the composition but uh yeah I'm actually going to go with this I'm going to roll with the punches here but just for the sake of this example you can see that we can actually change the style of our buildings with the prompt as well so we have a cyberpunk Sci-Fi building and we'll probably have to clean up these edges here in compositing since we see further to the left of frame in our actual video but I think it should be fine so finally I'll add one more We'll add one more set of sci-fi buildings right here maybe I'll try to get a higher one like an actual skyscraper so we'll do add sci-fi cyberpunk Skyscraper with lights on generate so this is kind of interesting the perspective isn't perfect um this one's a little bit nicer I do like the bokeh it's not exactly what I was imagining but it's not bad for a little Tower off in the distance you have like sci-fi on this side of the frame then we have our traditional buildings over here for the sake of this tutorial I'm gonna go with it now just like in our last video I'm going to export all of these different generative fill layers within our PSD file so that we can then use those layers individually within After Effects and tweak some things as we desire so you can see we have our elements to work with right here and if you want we can composite out you know the sky and everything to help everything blend a bit better feathers some edges Etc so we're just going to save our PSD file like this and then we're going to open up that PSD file within After Effects and track in our elements so we'll just go to file and save click on OK and now we'll open up after effects and start compositing these elements into our shot alright guys here we are inside of After Effects now we'll open up our PSD file so go to file import file and we can find our same PSD file that we have just saved go ahead and open this up now we do want to import as a composition with editable layer Styles so by doing that you'll have access to all the different layers we have inside of Photoshop within After Effects go ahead and open this and by doing that you'll have a composition with all of your different generative layers here like so but also you'll have access to all of your different layers right here so I'm actually just going to drag all these layers and put them on top of our composite like so and now we have access to all of our different elements here we can actually turn off a few you can see if we turn off these bottom two which include our original image as well as our Clean Plate this is what we get however of course we do have a problem because our matte painting isn't moving with our shot so let's get to tracking our footage so the first thing I want to do is Mark the position on our timeline so that we can come back to this as necessary since we did our matte painting to this Frame of our shot we're going to actually be able to access that again so to do that on this Frame we'll go to layer markers add marker and now you can see we have a nice marker that we can come back to where our matte painting will definitely line up with our shot so as you can see here if we play through our shot we get this mess like this and then finally it'll line up right there so now let's track our footage and get things to line up a bit better now I think probably since our elements are so deep in the background I can get away with just using a singular tracking point I'm not positive but we're going to try out a single tracking Point first but what I'm going to do is make that tracking Point as close to one of the buildings as possible so probably this first Cathedral that we added I'll add the tracking point near that so that very least the tracking point will work with this element so I'll go to our footage here and I will go to our tracker and we'll click on track motion and now we have a Tracker in our scene we'll zoom in here and if you don't know how tracking Works in After Effects essentially you're tracking the point within this middle square and then this outer square is your search area so you need to make this bigger if your shot is moving very quickly so to track this first point I think I'll just use the edge of this Cathedral here and I'll just track position and also do rotation I don't think we need to worry about it too much but might be a good thing to have so just so we have access to that data if we need it and track maybe this little dot on our Skyline all right so now go ahead and track forward here pretty good tracks are holding and this is the end of our work area so I'll end our tracking right there and actually so this is not confusing let's go to composition trim comp to work area just since uh this is the only area we're going to add that set extension to so now we've tracked forward from our timeline marker now we'll go back to that marker then we'll track backward and looks like the tracks are still holding fairly well here and yeah that's looking pretty good so now we want to add a null object for this first building I'll go to layer new null object we'll rename this Cathedral tracker and we'll go back to our footage double click so our tracker window pops up then we'll go to edit Target and we'll change the layer that we're adding this tracking data to to the cathedral tracker and I'll click on OK and then finally we'll apply this tracking data so as soon as you press apply this guy will pop up we want to apply the dimensions to the X and Y go ahead and click on OK and now if you kind of scroll through your scene here you'll notice that this null stays with our Cathedral so now we can do is go to the frame where we have added our map painting elements we'll go to the Cathedral first as well as this Tower I think would be fine as well we'll take both of these guys now just disable all the other ones for now just to take a look at these first two so we have our little radio tower and our Cathedral I'll select both of these and take our parent tool to the cathedral tracker now you can see they're parented to the cathedral tracker on this specific frame that we have exported and now you can see if you play through our scene here our building is tracked onto our footage now we actually have one more building on the far right of our camera that I think is close enough to this tracking point to work so again we'll just go to that same frame where we started our tracking data and that we've created our matte painting on and then we'll add this to our shot and I'll parent this to that Cathedral tracker as well and we'll also give this a shot really quick and yeah that looks like it's holding as well now obviously we're still seeing those weird edges but we'll clean those up in a second here all right so these two elements are tracked into our shot now let's add another tracking marker for the rest of our elements here our two sci-fi elements on the left of frame so I'll go back to our footage we'll go to our frame that we've created our matte painting for I'll go to our tracker tab track motion again and it's going to create a new trackpoint for us again we will track the rotation and position we'll roll in here make them a bit bigger and we just want to track some points near our sci-fi buildings this guy is a nice point of contrast right there as long as our character doesn't include it at all and then we'll do another one right here this should work okay we'll make this tracking point a bit closer since this is calculating the rotation you do want your tracking points to be fair bit apart but also near the element that you want to track all right so pretty successful track there now we'll go back to our keyframe reference and we'll track our shot backwards all right so these points are looking pretty good we can scroll through our timeline just in case but they're looking fairly decent here now once again we'll add a new null object we'll go to our reference frame here go to layer new null and we'll call this one sci-fi tracker okay then we'll go back to our footage double click edit Target and we'll choose our sci-fi tracker okay and then once again we'll apply this data to this tracking point okay and now we'll select our other two set extension elements and on our frame of reference we'll parent them to the Sci-Fi tracker and we'll go ahead and turn them on and let's see what we get our tracks are holding pretty well here obviously we have to do some cleanup we do have to Roto her out and overlay her on top of our footage but uh yeah at least our tracks are looking good and our map painting is looking solid as well so great now let's just do some basic cleanup work and we should have a pretty interesting final result the first thing I'm going to do is feather the edges of our different elements here so I'll first select this sci-fi Tower and I'll just go to our masking tool and I'll just make a very simple mask around our element here since we're going to feather the edges it shouldn't be too crazy just a simple garbage mat will do just fine I think go back to our composition we'll go to our mask settings and we'll just feather our picks a little bit just like that so 31 pixels let's take a look at our result so I'll just turn this guy as well as our sci-fi shot on turn off the masking settings now you can see that's blending into our shot much better obviously we still have to include our character that it's looking pretty clean in the background so great now we can do the same thing for our other elements do our next sci-fi element clearly see the edges on this one but that's no problem we can go in here with our pen tool just draw a rough little garbage mat just to feather in that Sky a bit go to our masking settings and again we'll just feather this 30 pixels so it's similar to the other one go back to our comp it's not quite there yet still a little uncanny you just feather this maybe 40 pixels now it's still not looking perfect I might actually just try to get rid of the sky here so what we can do is just go to effect keying then we'll use a extract tool and we'll just isolate this layer by itself really quick and I'll just take out the brighter portions of the image a bit and then feather that with this other tool so just like this this should help blend it into the background a bit so pretty much what we're doing here with this extract tool is we're bringing this over to extract only the highlights of the image which includes our sky here let's take a look with our footage and this is looking a bit better one more thing I'm noticing here is there's a little bit more grain in the sky so we can actually add some grain to our element as well to match that to our footage but yeah this is looking pretty good I'll do the same thing for our other elements do this guy next double click go to our masking tool quick little garbage mat here once again we'll feather this by around 30 pixels we'll go back and do our composite here and yeah that's feathered in quite nicely now obviously we have this tree here in the foreground which we'll deal with in a second all right so we have our next element here again we'll click on this so same thing guys just making a quick garbage mat around our element just to feather in those edges a bit so they're not too noticeable just like so now we'll feather this one by 30 as well we're almost there guys we have this last little Tower here with a quick garbage mat feather it by 31 go back to our comp and that's looking fine as well all right so now we've added all of our elements things are looking much smoother now let's add some grain to these elements to blend them into our live action plate a bit so I'll select our sci-fi element here we'll go to effect noise and grain We'll add some grain we'll zoom in here you can see we have some grain here but not on our element but now if we turn on Final output you can see we're adding some grain to our element we can bring down the intensity a bit maybe one is about right but the size may be 0.5 maybe the intensity could be 0.6 yeah you can see it's too much actually so 0.2 maybe 0.2 is looking pretty good that's matching to our live action plate pretty well maybe increase the softness a bit so we're matching our grain levels a bit more now we'll edit copy our grain and we'll select all of our other elements as well and then just paste this effect on top and now all of our elements will have a similar grain added to them as the live action shot so great we've matched our grain levels now finally for our last bit of cleanup we'll include our character here so I'll select our live action plate I'll press command D to duplicate then we'll add this on top of our composite and we'll re-label this one woman Roto okay and for rotor scoping characters I like to use the vertebrush tool so I'll select this double click our woman and on our first frame just drag and drop over our character here and we can start adding some rotor data to our scene and you can press option and select to remove rotor data or you can just drag to add something to your voter data and then you can press command and scroll to change the size of your Roto brush and this is looking pretty good one thing I want to do is use the refine Edge tool to get the detail of our hair so I'll select our rotor brush tool go to refine Edge tool and then I'll just go around our character's head here and as you can see here it's going to pick up that data as best it can go back to our standard rotor brush tool then I'll press Command right arrow key just scroll through our timeline probably go here like frame by frame here and letting the rotor brush do a thing if there are any errors I'll tweak the rotor brush data on that frame but it's looking pretty good so far foreign [Music] data is looking good we can freeze this data and work with it in the compositor so go ahead and click on freeze and it will go through and freeze all the frames that you have created with the rotor brush and now you can see here our woman is included nicely with this rotor brush data and you can see that even her hair has that fine detail in it if you do want to tweak this photo brush data you can play around with some settings right here you can use motion blur decontaminate the edge colors a bit if you're on a green screen for example there are a lot of different settings you can play with I think it's looking pretty good in this case but again feel free to play around with that for the sake of this tutorial because I think it's getting a bit long I'm going to skip rotoring out the palm tree here in the foreground essentially you just do the same thing we did for the woman but for the sake of the tutorial what I'm going to do is just extend the mask on our Cathedral here so it just covers up that tree and that shouldn't be too much of an issue so I'll just grab this guy extend it upward and we should be good to go and now this is looking pretty good we still have these trees blowing in the wind in the foreground so I'm going to go with this for our final result of course there are lots of different things we could add to this we could use our Spotify add-on for example to add some birds in the Deep background maybe add a spaceship coming in the foreground lots of different options for us but for the sake of the actual map painting in the Deep background I think Photoshop generative fill has done a pretty good job helping us out here so that's how you can use this to create these distant elements for your scenes I hope this video was helpful as always feel free to leave a comment if you have any questions or suggestions in the comment section below stay tuned for the third part of this video let us know what you'd like to learn next on the channel and I'll see you next time [Music] Fortune

Channel: LightArchitect

Views: 57,293

Keywords: photoshop generative fill, visual effects with AI, artificial intelligence and visual effects, using generative ai for matte painting, how to learn visual effects, lightarchitect viral youtube channel for vfx, filmmaking with blender, matte painting tutorial, vfx compositing tutorials, after effects tutorials, learning after effects fast, blender and after effects compositing workflow, vfx compositing tips, lazy tutorials, ian hubert, trending ai, is ai going to take our jobs

Id: uNGoBbqLrak

Length: 20min 56sec (1256 seconds)

Published: Tue Jul 25 2023