Using Generative Fill for Tracking in After Effects

now we can return back to the timeline press  spacebar and watch this magic this is the original   sign and here is the content aware field version  that we've managed to create using Photoshop   generative field amazing hello and welcome to  SternFX in this tutorial I'll show you how to   use content aware field using AI to create a Clean  Plate and replace a sign with the help of mocha Ae so we are going to achieve it using four steps  first we're going to track this shot using the   planner tracking built-in tool inside After  Effects which is mocha AE then we are going   to create a reference frame and we're going to use  Photoshop beta generative field to fill the sign   behind and create a Clean Plate next we are going  to feed these clean plates to content aware field   here inside after effects and finally we are going  to replace the sign with an animated version so   let me show you this animated version looks like  this and I'm going to break it down in a later   stage so stay tuned for that but now let's begin  with the tracking stage so I'll select this clip   and then go to the animation menu and choose track  in Boris FX mocha this will apply the plugin and   to get into it we'll need to click on Mocha over  here I'll grab the rectangle X spline layer tool   and I'll Marquee around the area that I want to  track and then I'll make sure the tracking points   are inside the surface that I want to analyze so  mocha is a planner tracker and it's very important   to define the correct plane to get a accurate  result if you want to check the results while you   are tracking you can show the planner surface and  you can also place it in the desired location and   this is what we are going to take away from mocha  so this is the data that we are going to use to   replace the sign so I'm just going to take my time  and place it at the edge of this surface and now   I'm ready to track I'll enable the perspective  option and click on the analyze forward button and that's it the tracking stage is behind  us if we want to check how this looks we can   go to the layer properties and over here  where it says insert clip we can use one   of the grids so I'll choose the 16x16  and then I'm just going to play this   and maybe also click to hide the surface just so I  can take a look at the result and I think it looks   amazing so I'll post the playback and I'm also  going to name layer one sign and save my progress   and Escape out of mocha next I'll open up the math  settings and I'll click create AE masks and this   is going to generate an animated mask which has  the same name of our tracking layer inside mocha   and you can see it follows the boundaries of the  sign now what we want to do here is change from   add to subtract because we want to fill in the  details here and over impose the new sign using   the tracking data that we've just collected now  there are a few steps we need to take to make it   happen first we need to generate new pixels and  to do this I'll go to the window menu and I'll   ask after effects to show us the content aware  field panel but the problem is that we don't have   any data for after effects to generate a field so  before clicking on generate free layer I actually   need to create a reference frame for it and I'm  going to use the new generative field in Photoshop   beta so if you want to follow along with your  files make sure to download the beta version from   the Creative Cloud app so to do this I'm going to  switch off momentarily The Mask here and then I'll   click on create reference frame and this in my  case is going to open up Photoshop beta and show   me this reference layer I'll zoom in and I'm going  to grab the lasso tool click and then hold down   the alt or option and Define the area that I want  Photoshop to fill so something along those lines   and I'm going to close it and once I have a  selection I can click on generative field I don't   need to place anything here in the prompt I'll ask  Photoshop to generate a new sign texture for us so these are the three initial results this  is the first one this is the second one and   this is the third I think this  is a winner but we can try again   so I'm just going to click generate and see  if Photoshop can come up with something better so again I've got three options to choose  from so the first the second and the third   so I think I'm going to stick with this one  and if I don't need any of the other files I   can just delete them and this is my recommended  workflow because otherwise the file on disk can   be very big so I don't need this version and  I don't think that I'm going to use this one   all right I'm going to just press command s here  on the Mac Ctrl s on the Windows side to update   this Frame and then I'll switch back to After  Effects and we can see that after effects now   sees the Photoshop reference frame now to make  it work I need to return to the sign mask and   change it back to subtract so other than the first  frame all the other frames are going to be empty   and then in the content aware field I'll  change the fill method from object to surface   this is going to generate the best results because  we just want to replace the surface texture so   this is the desired mode for it and then I'll  click on generate fill layer and let After   Effects do its magic so you'll need to be patient  you need to wait for the two stages the first one   is the analyzing part and then you'll see a second  progress bar which is going to render the result now we can return back to the timeline press  spacebar and watch this magic this is the original   sign and here is the content aware field version  that we've managed to create using Photoshop   generative field amazing and now we can composite  the new sign on top of it and of course apply   the tracking data to it so over here I have this  composition that I've created ahead of time I'll   double click just to show you its ingredients so  I'm using this free clip from Adobe stock and I've   applied the Minimax effect to it and the reason  is that I want to thicken the lines here so I set   the operation to minimum and change the radius to  1 and this is what this layer does and underneath   I'm exposing the sign which is a real sign from  our kitchen so just to show you what happens over   here I'll select the matte layer which is just an  animated mask that I'm using to expose the sign   underneath and of course the layer itself here  has three keyframes for the scale so it would   look like it is popping underneath and I have a  white solid which serves as a background because   I want to multiply those pixels above the sign  that I want to replace so let's return to the main   composition I also want to emphasize that the comp  here is the same dimensions as the original comp   as well as the same frame rate and this is very  important if you want to apply the mocha tracking   to it so I'm going to temporarily switch off the  visibility for the sign I'll go to the beginning   and I'm actually going to open up the tracking  data here and click on create track data I only   have one layer so I'll select it and click OK  and now we can see that we have keyframes for the   mocha effect now what I would like to do is apply  those keyframes the tracking data to the sign   layer so I'll set the export option from corner  pin to corner pin with support to motion blur I'll   make sure that the layer exports 2 is set to the  sign and then I can turn on the visibility switch   for the sign and finally click on apply export if  I'll press spacebar we can see how it is looking   so far which is not so bad but it's not believable  so I'll select the sign layer first I'll switch   the blending mode to multiply and this is going to  say goodbye to all the white pixels and now I can   stylize it so I'll add the turbulence displace  effect I'll change the size to 3 and this will   help to displace the pixels and make them look  a bit more connected to what we are seeing here   I'm also going to select the layer press T to see  the opacity and maybe lower it to 85 and then I'll   add a levels effect and I'll go to the green  Channel and play with the green output black   just to match the colors to what I'm seeing in the  scene underneath so 20 in this case I think will   work and I think that this is it so let's go to  the beginning and maximize the composition panel   and press spacebar to preview the result so my  guess is that soon enough we'll be able to do this   number inside After Effects without photoshop's  help but until then the future is already here and   you can use the generative AI to create a clean  plate in a fraction of the time that it used to   take before let me know what you think about  these AI Madness in the comments below [Music]

Channel: SternFX

Views: 5,625

Keywords:

Id: vD2BDbgv7x4

Length: 10min 48sec (648 seconds)

Published: Mon Jun 12 2023