Hi everyone, Harry here from Boris FX. In this tutorial, we'll dive a little bit deeper into SynthEyes 2024.1 and talk about lens distortion, how to calculate it, and how to bring that over to After Effects using the new advanced distortion plugin. So let's jump in and get started. Today we'll be using this drone shot to talk about some more advanced features found in the SynthEyes export workflow. Little shots like this are great for showing things like lens distortion. In fact, you can look at this corner of this building and you can see a little bit of curvature here due to the spherical nature of the lens. Now, with regard to tracking and solving, we want to address distortion because we don't, we're never really going to get a truly accurate solve. Also, if we want to composite elements into this scene, let's say I wanted to composite something into the side of the building here. If I don't compensate for this lens distortion, my edges are never ever going to line up. Now there's two possible workflows that we can do. And while I yap about this, I'm going to trim this shot down and do a quick track on this. So I'll trim this down to 400 and I'm going to click on auto and run an auto track. So like I was saying, with the curvature of the side of this building, if I was going to put elements in here and align it up with that surface, it's never ever going to line up if I don't compensate for the lens distortion. And there's two possible workflows I can use. One is I can simply remove the distortion and put undistorted elements in there. That's how a program like After Effects wants to work. It also wants me to save, so I'll click on save. So I could remove distortion and just work on a completely undistorted version of the footage. So that's kind of making an aesthetic call for the shot. Let's say we want to retain the curvature and overall look of the original shot, which is totally understandable. So for that, we would need to remove the distortion, add our elements, and then re-distort it to put everything back the way it was. Both of those workflows are completely reasonable, and I'll show you how to use both of those with SynthEyes and After Effects. So I've run a track here. We've got a pretty decent error rate. It's already below 1.0. The orientation is pretty decent, although it looks like my scene here is a little bit below the ground, but we'll do a little bit of refinement there in a bit. But right now, I want to talk about doing lens distortion calculation. And although that sounds pretty complicated, most of the time what you'll be able to do is simply go down here to calculate distortion, check this box, and then refine the solve. And what we'll see is that it brings the error rate down, and it's calculated a bit of a lens distortion coefficient here for us to use as our distortion setting. Now, I'm not seeing a huge number of trackers on the street here, and I'd kind of like to define where exactly street level is. I'm going to add some supervised trackers like I did before. I'm going to add a couple, so I'll jump to the trackers room. And I just want to kind of define where my bottom, my road should be. So I'd like to solve that with a couple more points in space just to kind of reference where the ground is. Being that this is San Francisco, it looks like the road kind of slopes down here and then changes off in the distance. So I'm just going to add a few trackers right around this area. We'll call this the bottom, the ground plane. So I want to make sure to be at frame zero, and I'll set a tracking point right here in the middle. I think this person right here is the drone operator, if I had to take my guess. So we'll do a nice tight track on here and size this down so it's not jumping around and tracking other parts. So I'll track there. I'll track this spot here and maybe a spot right here. And like I said, there's no tracking button, so I'm just going to hit play and it will start tracking these. And as they go off frame, if they do go off frame, it'll just turn them off, but it doesn't look like they do. Okay, so I've added a few more. I'll go to the solver and click on refine and it brings it down. Don't be alarmed if you run some supervised trackers and it actually increases your error rate. It doesn't always mean it's going to fix things. Plenty of times where I've run supervised trackers and it really doesn't help at all for the solve. In fact, sometimes it can make it worse if you've picked a spot that doesn't seem to help SynthEyes. So I've defined where these points are. I can do a quick defining of the coordinate system here. So I'll remove the coordinates, click on here, click on here, and then I'll click that one out there just to kind of set up what my basic grid system is. So I've defined my point of origin. I've defined some lens distortion and I can see that my scene is fairly aligned. As I look right here, we can see that this building is a little bit crooked. I think what the issue is is that the camera is actually a little bit angled, maybe looking a little bit more at this building. So what I want to do is rotate my scene to kind of compensate for that. Just want to show a little bit of a workflow tip here. So assuming that the points in 3D space on this building here are all coplanar or sharing the same 2D space, if I look down the front view, these should all be in a nice straight line. And right now they're a little bit scattered around. So with these selected, I'll select a couple more of these. I'm going to change the color of these so I can reference these a little more easily. So over here with these selected, I can click on this color and set these to something identifiable like a hot pink. So in my 3D room, I click on whole and in this view here, I'm going to watch that view and rotate everything along this X axis. What I'm looking for is those coplanar points to kind of squeeze together and align vertically. So what we can see is that assuming this building is relatively straight and flat, which relatively being the key word, the camera's at a little bit of an angle. And that seems to make sense of what we're seeing here in terms of our surfaces and our camera. So next I'm going to add a 2D plane and align it with the surface of this building. I want to be clear that I'm doing this for quick demonstration purposes. Match moving is such a delicate art and science that does take quite a bit of time, refinement and dedication to get things 100% accurate. So as I add this, please allow me a little bit of grace in terms of aligning the surface. I'm trying to try to do it as quickly and as accurately as I can, but hopefully we will be nicely aligned. So here in the 3D room, I'm going to select a different object here. Instead of drawing a box, I'm going to draw a plane and it's using this create object tool right here, that little magic wand, and I can draw in any of these views. I'm going to draw a plane to go on the surface of that. So in my left view, I can draw where those hot pink 3D points are, just kind of roughing it in. And obviously it's kind of in the middle of the road, so I need to move it over. So I will use this transformation tool here and I don't want to move the whole scene, I just want to move the object. So I can move this over and stick it right on the surface of that building. And I'm going to look at the various points here to try to see if it needs to be rotated or whatever. Now if I look in this view here, we can see that it's not quite 100% accurate in terms of the rotation. I'm going to just do a little bit of refinement, but I'm not going to spend a huge amount of time adjusting the overall rotation of this. I'm going to rotate it a little bit like that and I'm going to call that good. So from here, I can export this into After Effects. Now let's go back to that lens distortion discussion we were having. If I want to perhaps add something like an image to the surface of this building, and I want the edges to line up, I have to compensate for that distortion. And I mentioned two different workflows. Now there is a button literally called Lens Workflow. And you'll see that in the Lens Room. You'll also see it in the Solver Room. And I think you'll see it in the Summary Room, yeah. But I'll stay in the Solver Room and we've got our distortion calculated. Now I need to run the Lens Workflow and define how I'm going to approach the visual effects of this shot. Do I want to composite visual effects that are going to match the distortion? That would be redistorting it. Or do I want to add visual effects to a completely undistorted version of this footage? And again, both are correct. It's just a matter of the project that you're working on. I'll show you both and I'll start with "redistorted." This is actually going to be the more complicated of the two. So we'll do the more difficult one first and then the easier one second. So a redistributed workflow is going to do two things. It will undistort the footage, create a composition for us to work in an undistorted environment, and it will take all of that stuff and pre-compose it into another redistributed composition, matching the original distortion now that we have it calculated. So it's going to be two compositions and after effects that are created. An original undistorted one and then a redistributed composition. So if I run the workflow and I say, "OK," it's going to do some calculations and now it's applied the distortion. And you can see right here we are seeing an undistorted view. Now if we go to file, export to JavaScript, run the JavaScript file, and we're going to switch this back to run a new project and I can get rid of my old project there. I'm going to click on the shy switch to get rid of all those trackers, but I have my plane, I have my camera, and I have my footage. Now with my footage, you'll notice this is set to a guide layer. Anything that's set to a guide layer in a composition will be visible here as a reference, but it will not render it if I composite this or pre-compose it or nest it anywhere else. And that's important because all we want in this composition right here with the camera data are the elements that we're going to be compositing. We don't want the footage, we just want it to composite. The footage is just there as a reference. And the reason is, even though we are undistorting it and then we could possibly re-distort it later on, what we would be doing is twice processing the video unnecessarily because all we really need is the original footage. So anything that we add here is going to be added in an undistorted environment. Let's say I add some text. And I make this 3D. So I've got my text and I've got a layer here, and this is existing in a completely undistorted environment. Now this composition, Camera 01 3D, is nested inside another comp called Re-Camera 01, or the Re-Distorted comp. So this composition that has my elements, any motion graphics or any visual effects that I've added, these are applying the distortion again. So we're re-distorting our elements to match the original footage. So if I click this, we'll see that it shifts a little bit and matches up with the distortion of the original footage. And then also placed in this composition is the original footage. It has no filters on it because we just want the original undistorted footage. The best way to do that is to simply use the original footage. Hopefully that makes sense. The other workflow I mentioned is to simply undistort the footage. Now to do this, I need to go to Lens Workflow and then undo that. We've removed the distortion. We have our distortion coefficient back here. I can go to Lens Workflow and say, do an undistorted workflow. And you'll notice right next to these, it has a 1 and a 2. And what this refers to is the passes. We are doing a one pass undistorted workflow where we essentially remove the distortion. A re-distorted workflow undistorts it and then re-distorts it as well. So if I click on Undistort and I click on OK, and now I export this, which I can do by just running that, run the JavaScript export, we're going to run a new project. This is simply going to create one composition. So I'll click on Shy. And this is only using the undistorted footage. This is not nested or pre-composed in any way. So we are processing the footage and undistorting the footage like so. Now one other thing I want to point out, I'm going to go back to this Lens Workflow one more time and undo that last run. Now I've not run the Lens Workflow at all. And I can still export this. And I just want to point out, I've calculated the lens distortion, but I've not run the Lens Workflow. It will still let me run this, but it will give me a little warning once I run this to saying, hey, you've solved for distortion, but you have not run the Lens Workflow. You've not picked what your workflow is going to be. So it will let you do this, but this is not the optimal workflow. So it has run this for you. It still will apply it with the lens distortion, but your SynthEyes project isn't quite matching up with what you've got in After Effects. So we have not undistorted the footage or we've not re-distorted the footage here at all. That's what that error means, is that you really should run the Lens Workflow and determine what you want to do. So that is an overview of the new Lens Distortion Export Options found in SynthEyes 2024.1. I hope you found this lesson useful. Don't forget to like this video and subscribe to the channel so you can stay up to date. My name's Harry Frank for Boris Havex. We'll see you next time.
