Exporting Camera Tracking FBX from Fusion Studio 16 to Maya

okay in this video tutorial I'm gonna show you step by step how I bring in an image sequence which I've converted from a movie file track it export the FBX then bring it into Maya okay so that I can bring in the exact movement that I've tracked in fusion studio now once I've rendered this sequence then I bring it back into fusion again okay and I'll also be talking about another type of match moving soft alright so let's get started start a new project and then I'm going to bring in my sequence I was gonna click on the loader and go to the folder containing the sequence that I'm going to use so this sequence was shot on the black magic are so many camera and here's the sequence already converted into an image sequence which is the EXR so to do that you just need to load up the clip and then connect it to a saver and then save it as a EXR sequence so I'll not be demonstrating that so I already converted this into an image sequence so altogether there are 251 frames encoding frame 0 I'm gonna change the frame range of the time timeline here to match the clip length so I'm gonna double click this and change it to 5 0 so that a time ruler reflects 251 frames including frame 0 so the next time I want to do is to apply a little bit of lens and distort so press ctrl spacebar then search for lens then you find the lens distort hey if you have the loader the sequence sector it will automatically be linked to your node and then you will go to the inspector go over to the lens distortion model drop down and then we just want to adjust the then distortion a little bit I'm gonna scrub the timeline to find an area which I can find a straight line I think the beginning is just fine okay there's not much distortion on describ itself but we are still going to give it a little bit of understood so in order to preview the effect of the lenses dot no we need to click on the note itself and drag it to the second viewer so that we can see the effect of the key of the lens is not known all right so let me just adjust the values here again so you're gonna do a fine adjustment hold onto the control key left mouse click on the numbers and as you adjust you can control the amount of understood on the edge of the screen edge of the clip itself so I'm just going actually this one is too much I'm gonna go back to the default and I'm just basically going to distort this until I get this much get this edge as straight as possible so usually I will just give it a very small value yeah I don't want to overdo this otherwise later when we do the camera tracking it's gonna give introduce errors right so I think this looks good enough for me okay I'm just eyeballing this anomaly most of the lens right in major studios right will have the lens calibrated for the distortion okay so I'm now bringing the camera tracker so press ctrl spacebar bringing the camera tracker so type CT are so remember the camera tracker is only either using the DaVinci Resolve version of fusion the camera tracker is only available in the studio version now of course there's no free version of fusion 16 so a fusion student elite comes with the camera tracker so now we're gonna view what the camera is seeing so you can just click on a node and drag it to the viewer so now we're looking at what the camera tracker is seeing so we're just we're just gonna use the default two tractor features on this shot so I'm going to enable preview and just gonna scrub through the timeline just to make sure that there's a good spread of features that are being detected now you can play the detection threshold and minimum features separation okay so I think this this is a good spread in interest of time I'm not gonna use bi-directional and the gutter sighs I'm just gonna reduce it so that I have more features on the edges there being searched so for the new trendy force we're gonna use the optical flow and just way we're just gonna keep everything at default and then we're just gonna hit Auto track so when I shot this clip here I made sure that there's plenty of lighting and this video features on on the frame itself so that when I track when I'm able to track this in the tracker you'll be able to solve this quite easily now if there are people walking around or there are objects they are moving okay you will have to apply a mask to actually mask them out okay you can apply a polygon mask to mask it out and the mask can be actually connected over to the camera tracker here and of course you have any made the mask to block out any moving objects so you want to track features that are stationary they are not moving at all so that the the camera tracker is able to create that 3d space also if your camera has movement that's moving forward or sideways then they will have enough parallax information for the camera tracker to recreate the 3d space okay so now we're reaching 250 frames and then I can move on to the next stage I want to leave everything at default I actually used or some mini to shoot this on I'm going to keep it at this edit settings I'm not gonna touch the focal length so I'm gonna go to the soft straightaway and I'm just gonna hit solve so the tracker has detected about 1688 points so that's plenty of points to track so I think generally this show took about 13 seconds well yeah around thirteen seconds to stuff and then we get a soft error of 0.33 6 which is very very good all right so anything that's under 1 or anything under 0.7 will be very good so 0.33 60 pixels is pretty good okay so there are some yellow points no red points known great tracker points so I'm not I'm just gonna ignore this and I'm happy to tracker so I can now start to align the layout now the camera tracker for for fusion in the export has an export tab now I'm not gonna use the export tab I'm gonna use a slightly different workflow here I'm gonna change this to a single viewer and I'm going to create a 3d node so press ctrl spacebar create a merge 3d not selected much 3d which is the Shockers 3mg and then add it now now you notice that even though I select and I create the node it will not connect because they are two different type outputs from the camera tracker so you do not want to use the attract the image output here you want to use the scene output drag it and then connect it to the match 3d click on a much 3d node and then drag it up on top so that you will be you are seeing the output of the camera tracker wholly not control middle Mouse rollout to zoom out and you can see the camera and the point cloud or the locator cloud is created for you in 3d space and if you zoom in holding down control middle mouse rolling you can see the green a trajectory that is being created alright so you can see that the camera motion is replicated here right now were in a perspective 3d view in a much ready node so to to tumble tumble the view you hold onto alt + middle mouse click to tumble and you can see that the 3d space is generally created very well now the condition for this type of solve in order to see this clearly like right now I can see that this is the crown and these points are basically the tree the bushes the leaves and then the path of the of this walkway here it's many they put a tracker manage to replicate the 3d space very very well however you can see that the orientation is still not lined up yet so you can see that this is the if you look at the grid here this grid represents the 3d grid of t know what the fusion fusion compositor alright so we need to get these group of tracker points you can see here if you look if I look at the side when I tumbled the view to look at the side you'll notice that they all form a flat plane so we are pretty sure I'm pretty sure that these are actually the ground plane okay so let's take a look at through the camera so to look through the camera you move your cursor over to the lower left hand side corner now to pan you minimott just middle mouse click and drag you see the lower right here that's perspective if you right mouse click on this symbol here can change the view to the camera so right now we are viewing through the camera now you notice that the locators are way too big so to reduce the size of the locator we're gonna click on the camera tracker node go over to the fourth tab and reduce the locator size just bring it down until they become like dots all right so now you can see very clearly you can see that the main the wall grid is obviously not lined up with the floor grid so we need to do a little bit of orientation okay so to help the camera tracker to reorient the camera we can click back on the camera tracker and this time go over to the fourth export tab now we are not going to explore anything from here because we're going to change the what rolls a bit so go to the 3d scene transform we are going to do the line up so that the these group of points right will be lined up with the grid points so currently we have not done the linemen yet the default setting is always set to a line so we're going to switch this off to underline all right so now we're going to tell the camera tracker that these points are on a flat plane so you can hold on to the shift key and you can select the points that you know definitely on the flat plane so I'm selecting all these locator points so I think all these locate points and then I'm going to the orientation and then I'm going to specify by clicking on this button set from section meaning that these locate the points which are selected now you want to add two directions you just hold on to shift and then drag a box to select them so the default is XZ plane okay we know that XZ is the X orientation and the Z orientation Y is always pointing up to the sky so it is oriented to the ground so now we go and click on set from selection and click well nothing has happened yet because we need to switch us back to a line right but that's fine and all now we need to select one point to specify the origin now before we do that let's try switching this back to a line and let's see what happens I'm gonna click on aligned all right and now you see that the grid is has started or seems to be lined up with the ground plane all right order the ground in the plate itself in the video play so I'm going to switch over to the perspective view and I'm going to zoom out holding down to control middle mouse Rolla and holding out the alt middle mouse click and drag to tumble to pan and then control middle mouse roll in to zoom in okay right now I need to zoom into the shot they may just reset view of it again okay so you can see that although the the ground plane which you can see from these dots here are now lined up flat but it is not sticking on to the grid that's because we have not set our origin point yet so let's go back to underline again and we're gonna switch back to our camera view so right mouse click on this icon and go to the camera view and I'm going to click on one point or any other locators here so I'm gonna maybe specify this locator to be the origin all right then I'm gonna go to the origin set from selection option and click on set from selection okay and then I'll go back to aligned again and this time you notice that the ground level is actually lined up so we can verify this by switching to another view we can go to the left view and then you can see the ground but it says are now lined up okay and well it's very hard to see well okay you can see that this is the base grid line I'm gonna switch over to the perspective view again just to confirm that the vertices are actually lined up all right so let me it's a little bit difficult for me to hand because it's very very sensitive right now okay but we can see that yeah the these points are now lined up with the grid okay so let's go back to the camera view again so right mouse click here and go back to the camera view the only thing that you will notice there is a bit off right now is that I'll prefer that the green the red arrow which represent the x-axis this line here is lined up along this line here okay so this is what I'm gonna do I'm gonna go back to the camera tracker I'm gonna switch you back to unaligned again okay and I'm going to turn on the orientation click and drag and then just manually rotate this until I get the X says to line up parallel to this line and then the z-axis to line up parallel along this line okay so that looks about right I'm gonna switch it back to a line again and the grid should slowly start to come into place and right now I can see that it is a little bit tilted okay so that means I have to rotate along the z-axis so that this line is line up here so I'm gonna switch back to underline again the z-axis and just rotate this upwards and I'll switch you back to line again so I might need to do this back and forth several times okay so the Z needs to go pop again go back to a line again and I think I get a pretty close line up okay so I think this is good enough for me okay and yeah the lineup looks good okay you can also test it out by creating a 3d shape right so I'm gonna create a 3d shape just by clicking here shake 3d so it's not it's not connected to the much 3d yet so I'm gonna click on the output and drop it onto my 3d and the shape instead of a flat plane I'm gonna change it into a cube and then I'm gonna move this cube up they go over to a translate tab here and I'm just going to translate zero point five units so that is line up on the ground and then the pivot I'm just gonna lower it down - your point five so that when I scale it will be scaling down and lining up to the floor so I'm gonna reduce it to zero point two okay the purpose of me putting this cube yeah I'm gonna turn on 3d options showing a lighting so I can see some shading yeah and then I'm just gonna test out and see that the line up is working okay right now my playback is a little bit slow I'm gonna uncheck the high quality and motion blur I'm gonna turn down proxy and Auto proxy then just gonna play back so that my computer can render this a little bit faster okay so one thing to check is to make sure that the cue doesn't slide because we got a very low area of 0.3 so I think this is a very good almost a rock-solid solve we can you don't see that the cue is sliding anyway okay it's not sinking IDing anything like that so it's very important that the bottom of the cube right is line up with the base of this screen so once you're certain that your lineup is good and the next step we want to do is to export this line up and then bring it into Maya so for the output we're gonna connect it to a FBX export so click on ctrl spacebar type FBX fix water and then we're gonna save it I'll give it a name but I'll call it what way okay number three and then the type will be FBX and then save okay so once we've done this once you confirm that you are exporting it out okay just to be sure you have to make sure that you name your file f PX 2px this is just to be certain that your file be exported as an FBX file so we're not exporting any lights so we can uncheck that we are going to export out the camera we're gonna reduce constant keys and then the geometry I'm just gonna export the queue now from my experience I realized that the scale whenever export to the FBX projects water is very small so what I'm gonna do is I'm gonna create a transform 3d a transform 3d node and then I want to put it just after the lens distort and or actually just after the camera tracker okay so shift and put it after the camera tracker and then the transform I'm gonna give it a value of maybe 20 in the scale okay so when I increase the scale you know notice everything just shrinks down okay you try to worry about it because the track information is still intact okay just that everything will become very small okay and then for the cube itself I think this time I can just increase the size to maybe ten times maybe this is a little bit too big so I'll bring it down to five okay now I'm ready to do the FBX export okay so the reason why again why I put this transform treaty is from my experience if I don't put the transform treaty the resulting scale of the export of the FBX is very small so this is one way to get around that that problem okay so I'm gonna explain it now so go to click on the render button and then start render so it's gonna go through the entire sequence and it's gonna create the FBX file for us and then we can open up Maya we can import in the FBX and then from then on we can start to create our 3d models so this is a shot that I've already solved before so I'm going to just going through the whole process again so that you guys can try to follow along so the whole name is going to be what we track 3 dot DX and we are done so quick okay go over to Maya I'm gonna start a new scene then I'm gonna import and then this is how we are going to change our settings to bring in the FBX so I'm gonna bring in export FBX number three and if you zoom out okay I'm gonna zoom out and you should see the camera and then the cube and if I scrub the timeline you can see that the camera has a movement and he also has T okay the frame that actually contains the the image sequence alright but this is not texture to the image sequence so actually for this portion here we can actually get rid of it right so I'm gonna delete this and can you see that the skill is much bigger now because if I go to wireframe pressing 4 and if you zoom in I think the grid gateway is a great let me see okay the great is there okay this is that my agree so this scale is actually quite okay right so right now let's look true look through what the camera is see so click on panels perspective and then camera tracker okay and then we are going to bring in the image sequence so click on View image plane import image and we are going over to the folder that contains the image sequence okay so I'm gonna open up the first frame and then I'm gonna click on use image sequence you notice that I'm not seeing anything I'm not seeing the image sequence okay that is because when you import in the camera so let me just select the camera quickly we import in the camera the clipping distance is very very short the factory distance is very short so we need to increase this far clip plane you need to go to attribute editor of your camera and just increase it to a ridiculous value mark where 10,000 so now we can actually see the the first frame right so I already activated the image sequence and everything still seems to be moving along just fine yeah you notice that the image appears a little bit of we're exposed so I'm gonna go back to view image and then image plane attributes we really start to use image plane and right now we need to just change the color space from raw to srgb okay so that we get back the look of our image and also because we shot at 16 by 9 aspect ratio we also need to activate the camera tools oh sorry the camera settings and then change turn on over scan and we get back the I'm gonna press number 6 to turn on the shading again and this cube right I go over to the material properties the ambient color is set to a high value so if I reduce the ambient color I'll be able to see the shading so now at this stage right you can bring in your training models you can bring in any 3d model and put it there and then basically you can render out your 3d models so once everything is rendered out you and you render it out through this render camera it should have the exact same motion okay so this is basically how you bring in the the file itself so I'm gonna open up the previous file which I done so basically that's how I set it up so in in this sequence is basically the same sequence but I brought in my 3d model and also I created a ground plane and this ground plane I assigned it with a eye shadow map material so that is able to capture the shadow so if I click on play for the uh no viewport you'll notice that I've set it in such a way that the image plane is is invisible and also the the global light which I'm using the 360 what I call that the sky dome light which I'll use the HDR image it's also hidden as well so I'm gonna show you how to hide how to prepare to render your file or render your 3d object so that it will produce an image looking like this all right so the resulting image that I've got when I render looks like this all right I've got my directional light to cast a shadow and then if I click on the Alpha so this is what God I've got the the Shadow Man material on the ground plane to capture the shadow alright and then you notice that the the background is not visible because I'm not rendering the background I need to keep this as a separate element okay so I'm gonna show you how I turn that off when I click stop on this first so if I click on View image plane image plane properties which is the same image sequence under your image plane attributes before you start to do your rendering under the display mode you or rather the in the display mode you click on none so that the image plane will be invisible when you're about to render and what about the sky dome light okay for the sky dome you have to go over to the visibility under the camera option yet scrap it down to zero before you do your rendering okay so for the SkyDome itself if you just create a SkyDome line you need to bring in the image in this case I've took a HDR or 360-degree image right specifically for this area so it was used to light up my 3d object okay so this is what the image looks like okay so this this was the image that was being used okay so this was the walkway this is the area where that those video clips were shot okay so it was used to illuminate and like this okay so once you're done you can render because I'm using this this is a student version I in order to render you have to go to rendering and you have to click on render sequence okay you cannot use the batch render you used to render sequence option make sure that you are rendering out to the camera tracker and then make sure that your render settings is set at our narendra now for this demonstration i have to speed up the rendering so under the under our no renderer tab I changed default from 3 to 2 2 2 2 2 2 1 1 1 so this will just beat up the rendering alright so if you've got a lot of time or if you have a very fast computer you can just leave it at default and of course you also want to make sure you're rendering it as a sequence okay from frame 0 tutorial 50 and then we're gonna render full HD which is the original resolution of the backplate so already rendered the sequence and I'm gonna show you what it looks like so let me just go back to the original this is the original and I actually brought in the 3d model itself from here so let me just remove the model first so this was actually the soft track it's actually very similar to the demonstration that I did okay this time I want to bring in the 3d rendered sequence so I'm gonna click on that loader I already have the 3d elements rendered and I'm just gonna show you what they look like so this is the render sequence yeah you can see it's a little bit dark but no problem we can later in a color corrector to it or a brightness and contrast adjustment so now we have the 3d element we just want to put it on top of our plate so just click drag this output and then click on the output of the original clip and then we have merge down I'm gonna disconnect it and then I'm gonna look through this much now this is a different type much this is a 2d merge and this is a pretty much so I want to view the results of the merge so I'm gonna drag it and drop it over here okay I'm also press number to give you the same result so you can see that now the because of the lineup everything is matching properly of course our plane is looking a little bit dark so we can fix that by applying a color corrector or I'm just gonna put a brightness and contrast between the way so go between the 3d element which in this case is this vertical take-off helicopter and now I'm just gonna bring up the gain bring out the brightness okay the brightness is affecting the pack so I want to do that okay so I'm gonna and label this pretty by post if I not multiply so it doesn't affect the background okay so now you can see that the acro is a little bit brighter and that is how you bring in a 3d element from Maya with the exported 3d match information okay so once we are ready we're just gonna put this aside and basically we can just output this into a render you're gonna put a saver and I'm just gonna save it as a QuickTime movie just name a movie safe all right so I'm gonna connect this and then let's take a look okay so this will be the output result so once you're ready you can just click on save so I'm just gonna check the settings very quickly just make sure that I'm rendering out at h.264 all right and then then you can hit render so this is just a very simple composite so because a rendering of the 3d elements from Maya is gonna take a long time so I'm not for the sake of time I'm not going to demonstrate that in this tutorial so right now you can see that on our end combining the 3d elements with the original plate now of course the shadows needs to be worked a little bit better because if you have more time you can see create a shape to match this this brick section here so that the shadows were actually form over whoa I should deform correctly or project correctly onto this surface okay so once is done I should be able to see the movie out okay so we're here and then let's check out the movie there we go so there is the entire process of creating a match move and then bring in exporting it to Maya and then from Maya rendering it out and then bringing out the EXR render elements back into fusion again okay so yeah that's the entire sequence hope you can follow this and do try this out on your own now before I end this video I just want to talk about another type of 3d tracking saw which is the nodal soft so for this particular shot here which are already soft now the camera tracker in fusion right is specifically I'll classify this as a very general tracker because you will try to track anything and everything okay and he's trying to get his best guess so it's not like a dedicated tracker like sin x or PF track or 3d equalizer where you can actually tell that the tracker itself during the soft that okay this is a tripod shot okay when I took this shot this is actually a shot on a tripod okay so he actually took some time or some tweaks right in order to get this to work okay because in my first software the camera is like moving over the place and if you zoom in closer to the camera the trajectory you can see that the camera is actually moving quite a bit as well okay so it's still quite okay the end result of this solve the error is let's see it's about your my four nine which is actually not too bad as well however for noodle sauce okay so this is more for my class students for notice off you have to be aware that these dots right do not represent the treat the true 3d space right it doesn't represent the soft 3d space because for nodal cameras or for panning cameras like that it is impossible to tell the depth of of the shot okay so what do you basically get is like a bunch of points right that's projected on the inner side of a sphere if you think of it that way okay so you'll be if you put if you're solve is good right if you put anything this in front of a camera generally you will still stay quite locked okay or another another way is if you can put the object as close to the tracker as possible because the tracker points the soft points or the locators are pretty much locked now okay so I will I will create possibly create an attitude tutorial show you guys how I solve this okay but meanwhile okay if you can try to do this do this on your own okay we find this video useful and give this a shot right

Channel: weelian soh

Views: 4,802

Keywords:

Id: DFGb3Xnkk9w

Length: 38min 15sec (2295 seconds)

Published: Tue Jun 23 2020