DEE 22 June 2020 - 3D Matchmoving with Fusion

okay so the recording is started so for today's lesson right we're going to be using DaVinci Resolve to do your very first track so let me just start topically about this first now all these notes and everything they look very scary and if you don't know what you're doing right this can appear to be very complex but if you understand the steps right it's actually quite straightforward and quite I wouldn't say easy that I just need practice alright so so this is a shot that I did at at bookie alright so you know this famous statue at the it is Babel a roll statute of this factor according to the fat butt so basically I took a shot of this and I went around it and then after that I brought in the shot to to extract the 3d motion of the camera everything is running a little bit slow now let me just see what I can go over to 3d view yeah so basically let me just show you the original footage first maybe they'll play that a little bit better because right now is loading everything into the memory you just go over to the original clip itself okay so this is where I want a clip looks like you just try to go over to the original clip now what I did here was I converted this clip into an image sequence that's why it appears a little bit choppy alright I took the original movie file and I converted it into a image sequence okay this is something that I will show you guys how to do it within resolve now previously right in you can also do it in nucleus well in previously I thought the other batch of students to use another software free and open software there is a blender here to convert an image sequence in fact this is the sequence that you guys will be working on right so while it was this is blender it is the free software that you can actually bring in and convert this into image sequence the first thing we need to do is to comfort the movie into an image sequence now is it necessary to do this all the time no I mean if you are short on time and if you've got a fast enough computer you can use the movie file straight away to do the tracking but the industry standard right where I come from right we almost never use in fact we never use the movie files itself to track directly okay what what happens is that we convert it into like a slightly lower resolution file like for feature films right the resolution minimum right the feature film resolution is like 4k 4k resolution and what they do is that they down scale it they reduce the size of the image to a much more manageable resolution like usually HD resolution is good enough for me or sometimes 2k resolution right and also depending on your hardware because you can imagine if you are trying to process a super high resolution image right everything just slows down okay so if you don't have a fast enough for computer then when you're doing this match moving right you're just adding another layer of processing for the software okay so for example if you are doing you are working on a movie file the movie file right itself get let me just go through to show you the contents of the movie file and I think on all the computers that you guys have right you should have the you should have the media media information software install alright let's see where is my file okay there you go so this is the file that I shot using the Blackmagic camera now on your computer's later you guys were will have to go and copy this file hit so this file right is labeled as VFX footage in sia d de z drive folder for students folder and then there's a folder called VFX footage and then within the VFX footage there's a folder called BM or some mini or Blackmagic Ursa mini okay so you want to go inside and look for the folder that says car pop okay so what I want all of you to do later on is to copy the files you can copy any of the three files in fact are to just keep in consistent to keep it consistent right we are going to use the very first file the ak-47 this file yeah okay so just copy this file into your data drive cage will create your own personal folder which I have a computer you pick to work on please go ahead create your own personal folder on the data drive please do not create any work folders on your C Drive or on the desktop okay please take note this is very very important do not create any work folders on the desktop or the C Drive please remember that because the school lack computers i installed with deep freeze okay so what this program does is that every time you hit restart the computer everything in C Drive will be reset will be wiped out so if you do not want that to happen please save all your files in the data drive which is usually the drive again the Eco Drive or the elephant drive but depending on the lab okay the data drive might be another letter so please ask her a mr. Chechen to verify withdraw okay so remember this do not save your work files on the desktop or the C Drive please make sure you save them all in the data track create your own personal folder okay just for your IC number underscore followed by your name go inside create another subfolder maybe de e and then put all your work files there keep everything organized alright so you know where to find them okay so copy this file and then this is the file that we are going to work on now as I mentioned if you were to right mouse click on the file all of all of the computers I believe in the school Rack will be installed with this little program here called media info alright media info so if you right mouse click on it and click on media info now I'm not sure where the media enjoys it available for mac computer but this is a free and open software you can even install it onto your computer so this complete this offer is very useful it tells you a lot of information about the clip itself it tells you things like what camera is being used to shoot this ok this is very very useful especially when you're doing mesh moving you want to know what device is being used and you will find out information like what is the bitrate Gateway's the data rate that's being used to and cook this file alright things like aspect ratio a lot of information you can see right the type of encoding for me either the picture format for tutu encoding which is a pretty high quality all right frame rate okay another important thing this is the frame rate 25 fps in skin type right progressive okay is it interlaced or is it progressive this is a very important information okay so all this information these will be helpful when you need to do things like doing your match moving anybody even tells you I think that shutters be everything now the reason why we have so much information on on this particular file on this video file is because it is shot on a professional Blackmagic Design camera it's a these cameras are actually available for you alright so if you guys need to look up these cameras right you can actually book out the camera and then shoot footage for your projects but at this point in time I think I will be providing you with the footages okay so that you guys can do the practice alright so not once you've got the file brought into the scene okay so and converted it into image sequence let me just walk through you through the process then you bring the clip right into the Vinci resolve now the Vinci resolve as I've shown you guys last week right you have to bring in the clip itself and then you get drunk and have to drag it into the cut then so it gives in the timeline then drag the playhead over the clip then you go over to the fusion then you'll be able to see the clip itself okay so okay so all of this looks very very scary and confusing but actually listen once you've done it a few times it's quite quite easy so you can see here what I have here is that the the motion of the camera right being replicated in this senior alright okay so I can see the trajectory and then the flaw which I created I think I've accidentally moved this thing several times so it didn't be very careful when you are many bleeding around the view clear let me just turn on mine shortcut key display so there you guys know why I'm pressing there's no holding out and then with the mouse click case to revolve around in 3d mode now we haven't really explored much on the 3d mode so later I will just go do a quick introduction into it okay so let me just go through each of this flow right this one Chris now when you when you first bring in the footage right this is what it looks like let me just go to the view so this is the raw footage and these has been come this has been converted into an image sequence already right then after that I'll go into a no-call lens distort okay so let's take a look at the lens stop if i press number two if I toggle between this two and one okay you can see the difference now I'm going to toggle between the lens to stop and turn it on and off if you look over to the dent lens to start inspector there's a red button here which you can activate and deactivate the distortion alright if I click on it you can see the sights of the decides frame you can see that it is being stretched up a little bit okay so what I did here when I brought in the clip is that when I shot this I shot this on my DSLR my camera has a saw sort of a circular Distortion if you look at his side right it's a very slight Bend there's a very slight Bend to all the straight lines okay so you can see that there's a straight line here but then it seems to curl upwards like that it just very subtle and you can't see it very well unless you have a black and white grid place in front of your camera so you need to run a under start okay you can need to turn on the lens to start and then enable it and then you need to straighten everything else so what we understood here I had to manually do this okay because uh when I shot this with the lens settings right I did not put it in front of a grid so normally when you do it and start right you need to film a grid okay let me just quickly show you what it's like okay so this is a Z why do we need a lens distortion to it so for VFX one set so this is this guy will explain very well why you need to have a grid so usually for big visual effects productions right like like Lucasfilm every lens right will be calibrated here with a landscape that looks something like this let me just stop there like this one so usually position the camera in front of the grid and then you keep on tweaking right from the distortion like for example here can see that when you place the camera in front of the grid you can see the distortion it's very very clearly then when you bring in this original footage into the software you basically you undistorted you just twist it back and for every setting of the lens right for example when you adjust the lens that lets say the lessons of zoom lens if you zoom it to from 32 millimeters right the dystocia will be different like if you zoom in there will be a slight change in the distortion again so over here they are using the nuke to do the lens distortion or under stop okay so let's see how to do it so this one right you they actually draw the lines onto it then after they run through the under stop you can see that it is completely flat enough now why do we need to go through this lens the stop process especially for match moving now the the match move software in fact most much more software will will do the song much better if it is flat if everything is on a flat plane if everything is undistorted on a flat plane now if you refer to your notes right they are referred to your notes I talked about the different types of lenses right that can actually cause this kind of distortion okay there's a spherical lens okay an aspherical lens okay we just try to bring up the images first okay these are all in your notes on so you can just refer to it now a spherical lens right will creep usually on this type of distortion the image didn't load up maybe I'll just use this as an example okay now if you look at a spherical lens and when you try to focus something on a flat plane a spherical lens will usually cause a kind of distortion whereas aspheric lens okay is specifically ground in a certain way so it has this lens profile so when you focus right everything right all the distortion right would will not appear and disturb a spherical lens focus better on a flat plane now most of the lenses in fact our human eye lens is a spherical lens as well because it is meant to be focused on a curved plane now when it's not the back of our eyeball is curved so when we look at images with a spirit lens right of course everything doesn't appear distorted a straight line is a straight line however if it is focused on a flat plane then the straight line will become curved okay there is a reason why there is this significant difference between spherical lens and s very planes okay but for most movie cameras right they are mostly a combination of spherical and sometimes as very lenses so you need to be able to understand what count distortion right these kind lenses are produced okay so the gist of this is that we need to understand okay whatever footages that we seem you need to understand now for a lot of matchmove software let me just show you a very powerful man shmoo software in the industry cause in ties it is my own personal copy I think and get it for power for 500 400 plus US for the pro version okay I'm not okay they took this photo already so anyway this software right is designed specifically for match moving and the speed in which works right is incredibly fast okay somehow I just managed to crash it okay I'll just open up it bring this example so you can see this is this is a professional match mood software that's dedicated just for the match movie now other software out there like tweet equalizer this one is like industrial-strength when you select by Allah be show effects videos then another one is PF track okay so this synth is considered the more affordable version for like independent of joy texts videos but I love movie studios visual effects producers also use the software because it is extremely fast and it's very very efficient ok so you can do your tracking and then you can export now the good thing about this is that you can actually do your distortion right you can actually do your lens distortion calculation you can give it to the software and the software will try to do the lens distortion automatically by itself okay so that is syntax now another software that I want to show you that is capable doing match moving is of course blender alright I've showed you guys this before let me just see what I can load the movie file so if you go and load the movie file in blender so again this one I had not I can bring in the sequence okay and then I can run the sequence and then I can do the tracking so for this one it's going to be a little bit more labor-intensive because there is no that well there's an auto function but the the other function is not as refined and well developed as a professional tracking software I just wanted to let you guys know that if you want to do tracking you can also you want to do you want to use the software that is free and it's available blender can do the same thing as well however it's just going to take you a longer time but it can deliver very very good results if you know what you're doing okay so same thing it also comes with its own lens Distortion alright for example if under the solve panel there is a there is a a what it called a lens lens and distort or if a lens a soft compensation for the footage alright so you guys just need to know that you need to do lens distortion correction okay and but then sometimes if you look at the footage frame and if the by for example this footage you can see the there is very very minor lens distortion you can see that the edge of this row here is very very strict okay and even when I bring it up to here right I I can only see very very little lens distortion going on here okay so but then again when you are solving a difficult motion okay for example if the camera is moving in a very very or the features right there is insufficient for you to solve okay then you might want to throw in things like lens distortion correction right to improve yourself okay so let's go back to that sequence again so lens to spot okay so now I think I'd talk quite a bit about lens to start and then we come to the next curve color curves alright calico is basically a color corrector which I've added in right just to boost up the details of the shot okay so for example the shot it came in this is a original shot it's quite bright then after you go through the correction right I just did the contrast and sometimes you might want to add in another note right so do to sharpen okay or it will increase the detail okay so that the the camera tracker itself is able to extract in more details all right so when you have more details there's more things to track and then you will be able to solve the motion okay so this one already run the tracker now because this is a very long sequence ISM almost 300 over frames I will not use this as an example and and frankly this shot is also very difficult to solve okay so this this shot itself took me quite a while now if I run it through the professional solver I can solve this in like in a few minutes but in in the the solver in the Tahitian resource to do because this this is actually not a dedicated standalone solver but it is a capable solver and it's fairly easy to use okay so if you understand that the steps you should be able to get very good results but again I have to emphasize garbage in garbage out if you do not manage the floor the information properly right you will not be able to get effective soft like that Hey so these are the trackers that I got gotten out of it and then I went to the solve stage so after I solved it right okay so the camera that I used was my Canon Canon find the SLR when I shot this footage so I've entered the current information and also I have the lens information which is 28 millimeters this where I set it at so that's why all this information is very important when you bring your footage to solve for camera movement okay this is this is why I emphasize so much fun you need to know what is your focal length hey what is your focal length and what is the sensor size which is the film gate that you use now because this is a full frame this is a frame camera in fact most of the footages that you guys will be using right is going to be full-frame maybe perhaps not the the Blackmagic camera however the Blackmagic cameras information right the lens information everything right well we're just gonna let the camera solve it for us okay so anyway I got information I placed it in and then you'll just increase the chances of solving and then the dot tag yeah this is the soft okay I managed to get on soft error under one pixel you remember the value always has to be one and the resulting soft camera instead of 26 is actually twenty five point three all right and it took about three minutes 49 seconds to solve on my computer alright so because this is a very very long shot now for most matchmove shots right you won't you'll have to worry about solving such a long sequence in fact next time if you guys have been doing this in the industry right most of the shots I would say will probably maximum will be about 100 plus frames okay or sometimes even shorter alright so this one right I just want to check out how robust this software is and I'm managed to get this result I think it is fine okay and I'm managed to get everything to lock now okay so later once you solve ready you can do things that bring in the 3d objects you can bring in kind of objects which are going to show you how to do it okay so once you've done it you have to export up the motion hey this is where the you bring in things like the 3d objects and if I go over to the camera view right so at this point right you can bring in 3d objects you can bring in 3d shapes for example if you look everything properly right you can even replace like the buildings in the background you can bring in 2d cards okay or 2d elements which you can paint place in the background so that's why mesh moving is very very critical in fact mesh moving for a lot of productions is the first first part of a production line so this the mesh move is first part of the layout and then once you've done a matchbook then you pass the information over down to animation then what animators will do whatever animation they need to do then after days pass over to the the set set designers they will dress up the whole place the sets now after they pass over to the lighting people they what's up to the light then after that the lighting is done really then they pass over the compositors will start to composite all the elements together and before it's finally rendered out this is typically how it works in a production pipeline okay so next week right basically for this week we just focus on match moving but next week right which is technically the tenth week right you will need to understand about the production pipeline there why is much more theory in fact I want you guys to spend more time to get more hands-on experience on understanding and getting the match move right first okay let's see okay currently the lessons give me just a monitor my okay my video is still being recorded okay so I'm just gonna continue so I wanna start start a brand new project and I'll just go through the processes right so I will just go over to new project and record my first match mo lang crypt okay so once you create a new project everything of the empty there'll be no clicks right then that's where you need to bring in the clip alright so I'm gonna go over to navigate to the folder and then bring in the clip so I want you guys to open up the copper grip I think this one will be very easy to solve and I want to teach you guys how to convert that clip into a image sequence but now I'm gonna be using this clip alright so I'm just going to drag this clip into the media pool and then when you drag it into the media pool ride the clip you're gonna get this message say that the different it has a different frame rate than the current project settings so we're gonna change it okay because the clip itself contains is 25 frames per second so we want everything to match the frame rate okay and if you look over at the metadata right you can even show you like things like 25 frames a second the resolution and so on the length of the clip alright so next we go about to cut alright and then we're gonna drag the clip right into the cut timeline okay so now we have the click within the timeline and so now of course this clip runs very well in resolve because this was shot on a Blackmagic Design camera so it's designed to work very well now I'm gonna leave this choice to y'all but I'm going to demonstrate how to convert this into an image sequence first I'm gonna convert into an X our sequence then I'm gonna bring back the sequence as a clip and then we are going to solve that one instead alright so next we're gonna go over to our edit bring the playhead over the clip then go over to your fusion okay so now we're gonna convert this into any sequence alright so click on the clip let me just maybe I didn't explain playing clearly now and why do I need to convert it into image sequence now for some video clips alright I'm noting all of them but for some video or movie clips Rydia encoded in some weird codec or some codec based on Stan or even worse they are heavily compressed maybe a h.265 in the h.264 which is an impact codec okay for every movie Farah India correct to compress it now again depending on how fast your computer is if your computer is very fast is able to decode okay decode the video from uncompressed into a frame rate that you can scrub very quickly alright then that's not much of an issue but if you've got a slower computer that means that you have to decode you need to have to decode the video into a image before you can process it so you have to go through these different processes and I'm trying to explain at the hardware process that the computer is going through so to skip that process we convert it into image sequence now when you have image sequence right an image tend to open up faster than a compressed video because from a video you need to strip it up into an image and playback okay that means the computer has to do double work all the time whereas the image right is I read the image straightaway okay that's why we prefer using image sequences when we are doing our match move and this is the standard when we are doing it in the industry so what I'm gonna do now is I'm gonna convert this into an image sequence so I'm gonna select the media in first medium note and I'm going to bring in a note that is called a saber note okay the say but not now if you're using fusion standalone right the saber note is where you export up the image sequence which I have demonstrated to your before so I'm gonna click on media in and I'm gonna press shift or ctrl spacebar okay I don't know where that yeah hold on a second let me bring in the yeah this one okay I'm gonna press ctrl spacebar make sure you know no editor one contro spacebar and then look for saver and then add a save a note okay click on the media in so there is highlighted click on the media in so that's highlighted then holding down to your shitty left mouse key and drag the saver until you change it into this blue blue line and release okay so now it is in the pipeline okay or you can manually just drag it in okay so remember the yellow is always the input yellow or yellow is the background going to the saber so go inside the saber and look at the what are the properties okay so the saber right is basically telling you where do you want to save this sequence so I'm gonna click on browse and I'm gonna browse over to a folder here I'm gonna create a new folder I'm gonna call it cut out then underscore sequence and let's go seq 'la so that I know there is a common sequence so I'm gonna go inside then it will only say for this resolved the only image format that it will output is from fusion is EXR okay so I'm gonna save this as Kappa underscore sequence and then I'll just click on save now nothing will happen okay so if you want the sequence right to be exported out you have to run the saver okay you had run the saver know it's different then over here this is the export all right or deliver so let me go back to fusion again so go over to the fusion tab and then go over the fusion and there's this button below fusion called render all savers now if you are using the fusion standalone there will be a green render button there which you can click and then it will do the same thing all right so I'm gonna click on render all savers so right now it's going to render the entire sequence into an image sequence alright do an e XR image sequence okay so next okay while waiting for this to download we will need to have an image player to actually see what we have rendered out so there is this there is this very good program called djb is a open source software let me see what I can go to the desktop yeah unfortunately I couldn't go elsewhere for now we're just gonna wait for this to finish now the EXR file right is is a very very correct it's quite a versatile file is able to contain many layers or many render passes HR PC so you need to have a specialized image viewer to view it now Photoshop can do it okay but if you want to view the sequence that means you want to view the entire sequence you need to use a specialized viewer now when we when when I was working at Ireland they have in-house developer or rather this guy used to work for Ireland he developed this viewer very very powerful viewer that can view very large image sequences even a full 4k or 6k resolution alright so anyway he left ILM and they started a company to make that software and now Eila island pays him a lot money just reuse it often but just to let you know there is a free alternative and it actually works pretty well and I just only discovered it earlier this year and it can view the EXR sequences very well especially for the 3d students we need to render turntables or image sequences so you can use that to view other sequences as well ok so now we are done we are done rendering converting this into an image sequence okay before we bring it in again let me just show you the software that I'm talking about the software is called djv ext it is a free software I really asked mr. Chen to go and download the software now the website is here DJ stands for the author ok Darby Johnston alright Darby Johnston video viewer ok so the latest version is 2 point zero eight you can download it free okay and so once you download it it just gonna show you what the program looks like once you downloaded it and you open it up DJB this latest version now I open it up right you'll find a hey there's no executional file you need to go into the bin folder and you need to locate the djv DXE so this this is this is why you do you have to double-click to run the software so I'm gonna double click to run the software and this is what the software looks like remember this is a free way you can go just go and download it and basically just unzip it and they can use it already I don't think you even need to install it alright so just go and open up the sequence or you can navigate to the folder where we exported out the sequence ok I'm gonna grab the first file and then just gonna drop it drop it in so the software should be smart enough to recognize it as a image sequence and try one more time ok yeah and there you go so you can't mouse wheel up and down or you can just zoom to fit and you can drag the timeline I can see that you can play back the frames very well and because these frames are quite big there's about six six megabytes in size so you can click on the playback right just to load up the memory first yeah I think my RAM allocation is a bit low so let me just keep it a bit more rent ya don't locate - my shrimp we otherwise our crash my computer because I've got limited randhir's only 24 gigs ran here now there's the system that you guys are using have 32 gigs of ram so you have more than enough RAM you can click on the RAM icon here to allocate more RAM to it ok I think everything is slowing down because I'm doing a lot of things right now I'm doing recording and resolved so everything appears a little bit choppy so but if if I'm not doing all this live streaming and recording right I think everything will run much much smoother okay so I just want to let you know that you can use this software to review your image sequence okay so this is the EXR that was exported up so this is very high quality so I'm just gonna close the program and return back to having to resolve okay so I've exported out the sequence already so I don't need this anymore I can delete this I can select the node I can delete this and let me just go back to the timeline okay I can also DV that way to click on the timeline I don't need it anymore so the next thing I need to do is I need to bring in the image sequence into resolve all right so let me just go back and then I'm gonna navigate to the folder containing the clip comeback sequence you can see resolve will recognize this as a sequence straightway then I'm just gonna drag it and drop it into my pin okay so this is my common sequence then I'm gonna go to my cut drag it down to my cut timeline and then go over to the Edit and then you can see I can now play back you can see that it's playing quite fairly smoothly I mean considering that I'm I'm streaming everything right now this is actually not too bad for image sequence then I'll go over to the fusion and then that's where we're going to start to do our first tracking okay so let's go through this step-by-step right now now I deliberately shot this Kappa okay for reason because I know that this is going to be very easy to solve there's a lot of detail there's a lot of sharp contrast points right on the ground for me to track okay so what you are doing you're tracking right if you're doing like for visual effects you're shooting in a green screen room when there's hardly any details that's why you see for some green screen Studios right when they are doing visual effects we'll stick the love markers right all over the place okay but you had to be very careful remember the planar tracking exercise that you're did for the then with the moving logo all right so if you put too many markers right that means you had to spend a lot of time cleaning those markers up and not every okay not every shot right you can pick a not four I'm not trying to say here not every shot right will allow you to conveniently just clean up all the markers at one point all right so if you if you have a free camera right that means the camera is moving over the place all right and then you literally have to track the marker points and then you have to manually go and paint those trackers out frame by frame okay so a different situation right requires different methods all right so in other words what I'm trying to say is that if ever the next time you're doing a green screen production and you need to put markers make sure you put sufficient matters so that your camera or I can detect a parallax okay so what do I mean by parallax information okay so you can see there's a lot of information in the distance and there's a lot of information that is close to the camera okay so things that are far away whenever the camera is moving all right let's say the camera is moving sideways okay things that are far away right will tend to move much slower but things which are right in front of the camera will move much more rapidly okay and in the combination of this shot right the camera not only is it moving laterally is also painting that means it is tilting left and right as well so the software will have to think about ways to solve this as well okay so let me just go through the process okay so media in right now and the image looks bright enough I don't think I need to adjust too much and also if i zoom in holding down control and middle mask in and see the detail of this image even though it's HD right it's more than sufficient for me to solve so I don't think I need to run a color correction or even a soft or Toyota or even a sharpening filter on it okay so I'm going to apply the protocol it may be a nd stock so lens is starting to look for lens it's not add a lens to start and then click on the media in holding down the shift left mouse click and then drop it in front here and I'm gonna use under start then the lens distortion model right I'm just going to drag this and I'm gonna use the default setting here a the camera is just nice we are using the Blackmagic camera so I'm just gonna drag this up slightly a little bit no not that much you can see it just a little bit just to straighten out the outer corners just a little bit hey maybe 0.016 I think this is good enough okay then with the lens to start up light already now as I mentioned earlier the correct process would be to aim the camera right with a distortion grid and then then later right bring that bring that shot with the grid and then you do the distortion calculation then we know the exact values the in industry we call this the lens lens calibration all right distortion calibration okay so once we've done that the next step we need to do is we apply the you can straight we do the cannot track them so the bringing the camera tracker you press ctrl spacebar and you type CT our camera tracker alright and then head okay camera tracker is added to the media outages incorrect and you want to put a camera tracker after the lens to start so click on the lens distort no holding down the shift let must be and drag and drop into it okay all let me just show you in another way that you can connect if you bring in the camera tracker remember an output from the lens remember our put right is always this gray box here the gray square box is the output of each of the note and you drag the output a gray box and connect it into input of their camera tracker alright so notice once is connected right so I'm only starting to notice this when the box is connected it turns white okay when the box is not connected the output of the box is not connected right it is gray color okay so remember yellow yellow is the background yellow is the input now this will through some of the especially the progression students write EF exponents because you are so used to using new if you move you're not around right you'll notice that the box right will also move around now in new write new output basically it just stays put it doesn't move at all so if you are if you're coming in from you this can throw you out I mean I've used new and then I come to this and this is confusing the very first time so just know that when I move this around move the node around write the output know also shift around it will shift automatically yeah so that's why you'll need to remember the colors all right so blue as I mentioned before is the masking okay the mask input if you don't know what they are you can hover over it and then you thought tells you what each of these input stands for all right okay so now I brought it into the camera tracker so I need to view it through the camera tracker now it by default right there is a viewer one there's a viewer - currently the tractor you can see although we've selected the camera tracker node we are not viewing it yet because bureau 1 and viewer - is not activated so I'm gonna activate viewer one right you can click here and then now we can see viewer 1 and then we can start to do our tracking and we are just want to try to use the auto track first don't worry about all these symbols and everything for now okay so we brought in the lens to start we did a little bit of lens correction or district correction now we are in the camera tracker and then we are just going to look at the inspector of the camera tracker that means the properties or this camera track that now for tracking right you have to go through every single step that means so you need to track then after that you need to in the camera information then after that you do the song okay so let's do the track first let's go through this one by one okay now if I click on preview nothing happens now in order to preview right we need to scrub the timeline forwards and backwards okay when you start the turning forwards and backwards right you start to see all these green dots out to appear okay that doesn't mean the a already start has started the track what this green dot means is that is giving you a preview basically the program is telling you that oh I detected these areas and I think these are good points for tracking so why is the rule here you want to have a good spread of points okay you want to have a good spread of points or points that are far away points that are closer to you so that when there is camera movement right the software will be able to use these parallaxes or differences of distances right to solve the position of the camera okay so remember when you guys are doing your photo shot line up okay you have to manually move it until it matches the perspective you can think of it like the software is doing the perspective lining up right every single frame but in order to help it right in minimum right it needs about two or three frames where the camera is moving differently for example it needs to have a difference between this frame and this frame so that it can calculate the movement where the camera has shifted due to the perspective shift all right so that is how it calculates the movement of the camera so now with the preview on and if you want to see the green dots you have to scrub the timeline so basically it's telling us okay these are good points okay we have some good points here and then we can track and notice for this particular clip here I decided to film it in a place where there's not much traffic movement or where there's not a lot of people walking around now when if you want to get a good track one the most important things you need to know is that you do not want to track things that have movement inside okay there's a lot of things that are moving alright that will mess up yourself okay that will give you the false false information basically garbage information so we have a good spread now if your spread is no good now we move on to the next step of the properties the detection threshold now if you scrub the detection threshold the green dots would disappear for a short while and then you will you can see how much whether they are bunching up together or whether they are spreading up okay then the minimum features if you reduce this okay you can increase the number of features that you want to solve now some of you might think okay good the more features the more points again trick definitely I'll be able to get a better information true however more information means that the computer has to work extra hard to keep track of all these points so if you have a slow computer you want to try to avoid having so many points bunched up like that because from my experience using this it's going to slow down a computer tremendously so I think the default settings is actually good enough yeah what I use the default settings now if you feel that you got way too many points you can then slide the slider up okay of course the less points you have the trick the faster the tracker will work however that means if it tracks to lead to few points right it might not have enough information to solve the shot okay so for detection threshold I think basically the detection threshold is how sensitive to you won the tractor to to detect points okay so basically this is these are these are white means alright then we move on to the track channel luma okay that all red green and blue luma means the almost like the brightness and ductless areas so it will track based on contrast objects okay then Rick it means it will track anything right there has a red channel in it green green Channel so you can actually preview by pressing G this is the green channel rest are by clicking hard is the rich channel be this the blue Channel okay if you press B again he will jump back to the RGB okay and since this one don't have l5 request a right there's no alpha alpha is a transparency Channel I press a again okay we'll go back to the original colors you can also do the same thing here in the drop down can switch between all the channels of this image okay alright so we're gonna track the luma because luma boots generally was pretty well and basically I've adjusted the minimum features until it is quite to you so that I think I can track this quite quite fast alright and I'm gonna leave the track range is render that means the entire range if the render range is basically these blue yellow lines here if you hold onto control key if you hold on a control here I can set which is the area that you want to track okay let's say if I hold on control I'll try from 0 and then a lab mouse click and drag this yellow point basically I'm tracking this entire range so this is what the render stands for okay you can click down and you can see other that there you can specify the number of frames the frame range that you want to track you're gonna give it a Brenda now bi-directional tracking what does it mean usually I will turn this on okay I prefer to have bi-directional tracking reason is because once you track one round you want the tracker to go back again to track B to verify the track the point positions so I usually enable bi-directional tracking okay then the gutter size wise this gutter size data size is like the edge of the frame here now if you reduce the cutter size to like 0.5 the minimum the tracker points will actually start to track all the points nearer to the edge okay this is what I got the size means if you increase it right then you will actually I need to scrub this you can see the decibel let me just ink the threshold just give it a yeah exaggerate the amount of points for so I can see so again see the gutter size you can see that there's again the gutter Arai is much much bigger so forgot the size right I will usually use the minimum because I wanted to track as much as I can here closest to the edge get the value for this one for some reasons I cannot bring it down to zero that means I cannot track it like literally touching the edge here so not a big deal right now my God my meaningful minimum features way too much I'm gonna bring it back to this value now and then yeah this range is fine okay then now the track defaults now for my setup right I prefer to use you can use optical flow and then there is the fusion tractor and then there is the cleaner alright so there are three different track methods now optical flow right it's just an algorithm of detecting features right within image sequence alright they will try to basically try to look for features in between the frames as the frames progress through all right so optical flow usually the fastest okay but in when I was working in the industry right the time that we use a lot is the the planar type however this one is much slower and the pattern right we use this a fine th RS is basically it look for trackers with transformation are is transformation rotation then transformation rotation and scale alright that means that there are changes in order K fine and perspective right these right are the slowest but it might give you very good and accurate results if you were to choose these now from my experience right if I'm is it's like like the best or the most precise however the drawback is it's very very slow okay especially at least for this program okay so for this exercise we're not going to use this we are going to switch back from planar to optical now fusion has its own tracker you can also try it out just know that there are different options for tracking okay the tracking methodologies there's available for the camera tracker so I'm gonna use optical zoom because it's very fast and it works very well so once you're ready you can start to do auto track okay so I'm going to click on auto track so you can see the blue points here are the track points that are being detected and you can see the trail lines behind them alright now track points right I'm just gonna come up here and disappear so on my system this is considered fast ready so if you use the optical flow sorry if you're using the planar a fine track method right this could slow down maybe three to four times okay if we treat the four times slower than this alright so we'll just let it track first and I'll chop through this so there's a reason why we do not want to have things that are moving around when you have things moving around right you will add in a lot of false data false information and it will give you a wrong song in other words the thing is you want to keep on tracking the same point right as long as possible in other that it will just go back to tracking the attack clip again so this state is 301 frames so it's not a shot clicked at all so is fairly long as well it's about 12 seconds I would say a compared to fully manual trackers right this style tracker is much easier to use but in the industry right I would say like for double- from my colleagues who used to work there and ex-colleagues and then my current colleagues right we almost never use auto track at all in fact we we never use the auto tracker oh so all the points that you see in your right we have to manually track and in solving a camera Bookman right minimum ok minimum you need about six to seven tracks six to seven good tracks if you can track across the whole shot it means if the track disappears right you have to quickly go and find another track to continue tracking or we call the handover track so you need to track as many or other continuous number of points and minimum about seven to eight tracks or six to seven seven to eight tracks right just to give you enough information for the software to solve the position okay so we are almost done and we can turn off the preview right now otherwise right the preview will always keep appearing the preview of the track and then you can scrub the timeline back and forth now if your timeline is moving a little bit slow okay is responding a little bit slow okay let me tell you how to increase the speed a little bit so move your cursor over here right mouse click here uncheck high quality and check motion blur and turn on auto proxy okay so this will speed up your interactivity a little bit all right I think on your computers right it might be a little bit better speaking of which okay I forgot to tell you I start to increase the performance of your table entry resolve so let me just save this project first ok because on the dubby to resolve any click on preferences and if you click on memory and GPU all right there is a option for you guys to turn on OpenCL my graphics card right dozen yes I think I'm gonna turn on not maybe not quema for your graphics card I suggest you turn on OpenCL no my graphics card is a fairly old graphics card so on your side right if you go to memory and GPU you're going click on open CL and then you hit C alright so after you saved already you have to close da vinci resolve and reopen it again yeah otherwise it will not have this settings apparently from where I've read and and from experiments are done in school OpenCL seems to perform better alright so if you can turn enable open CL on your computer let me see what I can turn on you can see you and then save okay I'm not gonna restart my computer I'm just gonna continue in my the CUDA drivers still works fine so a tip for those of you who are considering buying a new computer so please make sure you buy a computer or laptop right there is equipped with an Nvidia graphics card and make sure that is a GTX type or if you can afford it the our TX series cards okay so that you have a much better experience when using the software because this software is very memory hungry and it uses very high spec requirements okay so now I can see all the tracker points have been tracked already we're gonna move on to the next stage which is the soft stage right so I'm gonna click on camera alright so I'm gonna use the default settings 25 and then the film Gate is actually the Blackmagic camera so Blackmagic 4k 16 by 16 4 16 by 9 alright so I'm not gonna change anything so I'm gonna leave it at that so if you know the exact camera model right can click on the drop down and then you can look for it so all these are the what is called at the most popular movie camera settings or backplates and even DSLRs a lot of a lot of amateurs right like to use the Canon Canon DSLRs to shoot the footage including me I'm using the 5d so the settings are all day alright so you can I think for most of us right we will be either choosing the HT 16 point by 9 or D for the footages then passing to you guys the most likely the Blackmagic settings so basically I don't have to do anything here for focal length right I'm just gonna let the software solve it for me so now I can go over to the soft this is the sub panel so right now we have not solved anything yet so what does that what's off but I saw him but does it solve do when you hit on solve right it will take all these points and it will start to figure out where the camera is based on these points movement and then it will try to calculate the 3d location of these points okay based on the movement and movement of these points okay so everything here you see all the solve options I'm going to leave it at default first and if you look over at the properties here they are photo 1631 trackers they are being detected right there is a lot of checkpoints okay so we can go ahead and hit song okay so this one can take a while because it is how long is this this is 300 frames okay I think from my experience right this is probably gonna take about 20 seconds or 30 seconds to solve okay which I think in current standards right this is actually not bad already now if you're talking about what almost almost 20 years ago now the very first only solve software like Google right buzu is one of the earliest tracking software where I understand if you want to solve a shot like that right you probably take you maybe 12 minutes or even worse half an hour just to try this tough shot so the the solver brain don't think I because on the old hardware it takes a long time to solve and let's take a look at the result and we get a very very good result here let's take a look at it's all timing here you don't have to do much the song error is 0.5 - which is fantastic alright anything that is below 1 is good alright 0.7 is good enough right if you if you have a very difficult shot and get 0.9 and there is two possible all right but anything there is a perfect one okay that is not a good song it also managed to solve the camera for the length for me so it managed to detect the focal length as twenty five point two one I think this is good enough then it tells me it tooks about thirteen point eight four and then I'm happy in fact I can start with this shot straight away however I'm gonna show you how to refine the shot I can even get this off error down even more okay so if you scrub through the timeline you will notice that there are some points that are written in color and if you hover over these trackers it will tell you the error the soft weights are information like the track error information and you can actually manually go and select order it points you can scrub through and then select order it points yourself and you can press delete delete delete them even those which are orange okay basically how to tell if it's green there are good points it is red there are bad points so I'm going to delete way those are great points and orange points this weekend manually select and delete it like that oh there's a faster way which is to use the filter the track filter okay so for example you can select points which only appear for a short while so a minimum track length it has to be six frame so you can increase the number let's say I want to increase it like I want minimum right to be eight frames it means right I need to see the each of the track points at least eight frames right before I will consider them for the song and then the minim maximum track error and a minimum track sorry maximum track error okay let's say I want to make it even stricter and the maximum solve error you can drag the slider until you can see more the yellow side up here okay so now I've selected quite a lot of points you have to be very careful here you can see right the selected tracks here along the tracks based on this error filters right about 615 out of 1492 sorry 95 tracks I selected I can delete them against you delete them because I still have plenty of checkpoints so you just need to scrub the timeline just make sure that you have a good spread of green points all along the solver alright this is very important you need to have a good spread okay I think I had some good points in the distance some input points in front I'm happy with this I think I can delia way all these other points which don't satisfy my conditions then I can click on the button here delete okay so now I'm left with 80 points do a lot okay then I can hit solve again hits off and I expect my error to go down a bit more so instead of 0.5 okay okay see it's a little bit faster right now because we have less points so remember the last point you have the faster the song now you can see the soft error has reduced down to this ridiculous level now if you get this value zero point one nine this is like almost perfect already okay generally it is impossible to get a soft area of zero okay we get a software of zero is like you're tracking a computer camera movement so a soft area of this value right is extremely house a very very good already this is a very very good soft so now I know that I have my camera on lockdown we have an extremely good song the next step is to export export this into a 3d space now we haven't really talked much about 50 phase so this is a good opportunity for you guys to learn about it so far whatever we are working on everything is in 2d okay the all the frames you see I just still images flat 2d now now that we gotta solve we can export all this information into 3d space so if I go over to the the next tab which is this one the export tab it's gonna take the soft information that we have and it's going to recreate the 3d space for us alright so we now have the tracker points based on the trackers and from the video itself is very clear to us that these are the ground this is the flat ground so we okay let's let's take a look at this Expo here now I can click export now and then you see everything but that is just cool trick to gain through this 3d scene export now this this is an area which I'm afraid a lot of you might get a little bit confused so I'm gonna go through this slowly ok so what's gonna happen is that when I hit export right it's gonna generate a ground plane for us okay and but we need to tell we need to tell the solver where is the ground plane because right now the software is not so smart enough to recognize okay this is the ground I will orient the camera nicely to the ground okay I like the professional software like the what I call it the synth I so it's inside actually smart enough to actually figure out ways to graph but this one is not okay so we have to tell it we have to tell the software where is the ground plane so I'm going to select yeah I'm gonna select the points which i think is gonna be the ground plane and then go over to the 3d scene transformed now the 3d scene transfer right now because we have not explored anything yet we can't change anything but however we can tell the exporter ways the ground first so click on the align and change it to underline and you will see a bunch of buttons right now start to appear all right so we have scale okay scale is like to one generic one Union so this scale is kind of useful for example let's say if you are on set okay if you're on set and you took out a tape measure and you measure the distance between this tractor and distractor let's say for example you actually knocked out this tracker and you say that this tracker is 1 meters all right then you can use the scale function to select these two points right and then you can say select from selection ok and then you can enter the reward distance as one meter between the two points okay then you click OK and then now you automatically calculate the scale for you all right so there is skill all right so now how about the ground on the ground plane so I need to select the ground plane for Brantly you need at least three points okay for example you need to have three points right that you know that falls on the same plane ok one two and three these three points must be on the same plane but if you want it to be even more aligned I'll suggest you try to select more points that you know for sure yeah I know for sure that all these points that I see here they be careful to select points that are not in the same plane okay what do I mean by same plane that means right they are they must all be lined up on the same flat surface the same plane okay that's why I mean you can select as much as you want to inspect all these points maybe those there on the grass maybe not okay I just want to select those what do you not control to deselect yeah I just want to select those that are sort of line up along the ground okay like these these are the concrete points so all these points are I'm not selected so now I go to orientation for the orientation the xz-plane it xz-plane rise basically okay now in 3d space Y is always pointing to the sky and X and Z is the ground all right so this is always defaulted to XZ crane to the ground unless you want to like align this to the wall all right so exci played around and then we disliked it click on the button set selection and you see the numbers automatically change then finally the origin Kato origin is basically the zero zero zero coordinates of 3d space okay if you learn a little bit about 3d already origin is where you see the center of the grid where you create object so now we want to position the center of the grid so I'm going to select this point here you just need one point and then I'm just going to click on set selection and then this will become my center point okay so I'm gonna scrub to the end okay you can see the points online disappear and then reappear again alright so I think this point is good enough I want to use that point as my origin then with all this done routine architects for now the ground plane if I click on the ground plane options here in the inspector the default color of the ground plane is going to be this purple color so I'm gonna change it to another color maybe on a change to a bright red color so that we can see it a little bit better and everything also gonna leave it I said before then and now I can hit export alright if you look at D the node editors right now is very empty so later when I click on this and a hit export it's gonna be populated with the 3d objects all right so I'm gonna click on export and now with all these notes selected I'm gonna pull this now you can see that the export comes up from here and then right now is how do we see the export yes right now we are still looking at a camera tracker okay in fact let me just fit into two window views hide the media pool so this V 1 this is V o 2 I'm gonna click on you can see a new much this is called a much 3d now you guys are familiar with the much to normal much the normal much is only for 2d objects okay right now because we're working in 3d space we have a 3d much alright so click on 3d much and we want to take a look at what the 3d much looks like here so press number 2 and you can now see all right the K the aligned order camera with the movement of the object alright so if I click on playback you can all rather I scrub the timeline I think it's much better that way you can see the camera is moving along the flat plane but you notice something this is my 3d grid but this is the exporter grid just now the ground plane but it is not lined up is not lined up to the world right here right so how do I get it to line up so we have to go back to camera tractor remember we set this to underline when we hit export we can still fix this so click on aligned all right click on a line then after that you click on update previous export and then the exported version will now look like that ok hold onto alt + middle mouse click and then you can see the camera moving accordingly to the position all right okay it's in the lead shift at the camera just now so I'm just gonna do a bit okay so make sure is aligned and right now because we are looking at both you cannot track the end it pretty much is just appears a little bit very confusing so I'm going to click on much the exported version and then this is what it looks like so right now we are in the 3d space okay and how do we change the views if you look over to the lower left hand corner of the viewport you see this button here alright or rather this icon here you can see that we are not in this perspective you let me just realign everything so you can see this is the x-axis this is the y-axis it shows you in the icon here so if I want to see what the camera is seeing I need to change to the camera view so the change in the camera view you have to right mouse click ok on this symbol here and then change the camera view and now we can see okay then the camera is lined up or at least track to the ground now you can start I should put in things and then you can start to a composite stuff however you notice that the ground grey drive is not really lined up okay to the throg rid of this of this copper okay what I want to do is I need to rotate this until it matches the the ground level now some of you might start to actually select in the ground plane okay in this case the ground plane is this red one red ground plane and then can click on the transform node alright and then you can start to rotate it okay you can do that okay but the the origin or the world grid itself a like this and right now I try to rotate this to match now the world grid itself will not be lined up okay so my preference is that I will try to rotate it before I export it at all rotate it and re-export it again to get my lineup to match to match also right now you can see wow there's a lot of this point crap all over the place and it just makes everything very hard to see okay so I'm going to show you how to turn the thing off alright so let's go back to the camera tracker again caracals evil ways you see all the points are all over the place so you click on a camera tracker click on the options the locator sighs okay just bring it down to in fact bring it down to zero until you don't see it anymore so the other point cloud that you see is actually from the point heart that we exported okay so you can go over to that point cloud by clicking on the point cloud that we exported and you can reduce the size so there is not so bring it down to a smaller size so that we we want to see the point crop out we do not want to see the crosses so big okay also the grid itself our floor grid is way too big we want a for every right to actually be within this size here so we can go back to our camera tracker go back to our camera tracker then go to the export settings scroll down and reduce the size I'm gonna reduce the size Antilles about maybe two goes right now two units something is I'm gonna leave it at 10 then I say update previous export and then I've done smaller now okay this clock is very annoying so this point well I think for my export options I'm gonna disable my point cloud and then I'm gonna update my previous export again and I'm going to delete the way this point cloud yeah I don't need a pointer out now if you need the pointer again you always come back here and check on the point cloud here okay so now everything is cleaner and we are looking at the camera tracker and I see the grid is much smaller now so now what I want to do this I want to rotate yeah I want to rotate my view before I export it out again and I need to switch it back to underlined okay this like online thing I think it will confuse a lot of you so meanwhile just watch what I'm doing first alright so you can see the grid get the floor great everything is not lining up and this is not why what and also this giant great right now is also very distracting I want to hide this great so I go over 3d view right mouse click here 3d options and then click on grid to hide it first hideaway I just only want to see this frog plain okay so now I'm gonna find a angle which I can start to rotate the alignment okay so I will go to unaligned here we go on a line right you can see all these what I call it the D manipulators are visible so this one's gonna be a bit tricky we are going to be playing with the the orientation okay so the first thing I want to do is I want to rotate this so that this what if I be the manipulator right we'll start to especially the the this is a ZX noise yeah the z-axis for AK is line up with this line so okay let me just draw this out so you can see right now the Z line is pointing this angle like this okay I want the Z line to be rotated until it line up with this straight edge here okay so I'm going to rotate not going to rotate here I'm going to rotate along the y axis so if I rotate the Y axis by dragging this you can see the symbol position sorry rotation if I rotate you can see that it is rotating and I want to rotate it until you can see that it is parallel along here now everything here is not a line because we need to click on this button here update previous export okay so now I can see it is rotated okay but I think I rotated too much so I'm gonna rotate it back a little bit more so this one right you have to do it several times until you get the rotation to line up all right so right now the line at least seems to be okay so let me just for a few more frames just to confirm that the perspective again is lined up correctly in fact I think it looks pretty okay looks pretty alright okay maybe this one is a bit off let me just go to the very first frame okay so for this one I will have to rotate maybe in the z-axis downwards a little bit now if you if you click on this roller button right and it is too sensitive one way to rotate at a much what I call it a finer finer pace would be the fallout control left mouse click on this last condition you can see is not rotating much less basically you just want to rotate this until the alignment right matches the alignment of what you see in the the free okay and then right now my rotation for that y-axis that just want to rotate it slightly you can see it moving a little bit then you need to update the export okay now you can see that for this part here I need to rotate the Z downwards here a little bit more until you can see the perspective lines or lined up and then we can confirm by just moving along the time line and again should see that all these right should line up okay and then now you can see that this thing is a little bit off in the Y rotation so wholly now control I'm just gonna rotate this a little bit more to the left here update the previous export a little bit more okay and at the same time boob and tracking is a bit too much right so now I have to rotate this in the x-axis now it's a bit just to check at another angle okay we can spend some time right to actually rotate this until we are happy but I think I just want to I'm just going to stop here all right because we're going to spend a lot of time trying to get this true we'll take until it is precisely accurate right I'm gonna fit this view now okay right now we're looking through the camera view and there's no way for me to zoom in and zoom out okay because we are fixed and the camera view right so they we cannot zoom in and tomorrow okay so the lineup is good okay so now we are going to kill me just resize this all right so I can see the grid is now perfectly lined up on this and we can start to bring in our 3d objects all right or even bring in a 2d plane all right so let's bring in our our astronaut character okay so how do we bring him in okay we have to bring in bring him in as a FBX okay so we had to go to Maya and then explored him as a fps right so let's go and open up Maya let me see whether anything else opened and let's hope that I don't know no memory and crash this thing and I'll say this here we wait for my aunt open up then I'm going to load up my my astronaut file later on but while while waiting for Maya to Dora I'm going to just bringing out a 3d object first all right now any 3d object in infusion right can only be connected to the you'll come back to this later can only be connected to connected to the much 3d all right to the merge 3d all right so I'm gonna create a 3d object a 3d shape right now and created to create a 3d shape the fastest way will be to click on this button here so click on it and then you have a shake 3d all right and then you can connect this shape to the match 3d now let's take a look at this shape 3d first the default shape is if you click on it and impress one it's a cube all right or rather a flat plane now if you hear a flat 2d plane all right so if you go over to the inspector you can change it into a tube or into a sphere all right but currently there is no material you can give it a color but you can see that there is no shading of thing so if you want to see the object okay let's turn let's change it to a a cylinder instead okay let's give you a cylinder and then I'm going to give it a solver a shader material so that I can see when I bring it into this 3d much 3d view all right so I'm going to assign a material to it so the material is a Blinn material so press it one more time control spacebar then apply a Blinn material click click on the blend material and then can give it material currently you can't see any 3d shape object because we need to enable lighting ok and we need to enable shadows all right so right must be on the viewport 3d options to turn on these things that I can see the object itself so right now we are ready to bring in this 3d shape into this world so drag the output of the shape 3d and then drop it into the match 3d and then if you go to the much 3d right we should be able to see the object okay so let's click on the shape 3d and we have to go to 3d perspective view because I know in a camera view I don't see the this object appearing so I'm gonna go over to perspective view and I'm trying to see where is the cylinder in the cylinder is actually running fine in fact the cylinder is so big that it actually encompasses everything that's why ok let's go to the top view so right mouse click on this icon go to top view and we can't find where is the cylinder it click on here we can move the cylinder until is right in front of the camera here so now we right mouse click go to camera view know you have it this is the cylinder okay right now our view is not aligned because we didn't export it up after we so we need to put it back to aligned and then update previous export ok very good yeah I forgot to do that step earlier on after we aligned this screen you need to change it back to a line and then update previous export all right so now we also can see the lighting effects here so to see the lighting effects in this viewport hit the right mouse click go to 3d options turn on lighting then you can see the effect right so now I can switch to the my camera view in much 3d then I can select the shake treaty and then I can move into place now the rig grid here which we exported from our camera tracker represents the ground the ground plane so we need to move our cylinder up until the bottom leg is line up with the promptly so if you want to see a little bit more accurately we can change how much 3d view the first view as well and then change this view right mouse click to the left view okay order front view okay I'm gonna change the front view okay now with the cylinder selected let me just rename this press f2 and course cylinder we can't move the cylinder up and down translate often down in fact I think I just enter the value point and they be perfectly lined up with the ground plane right now the cylinder the diameter is a little bit on the large side let me see what I can reduce the radius okay so now I planted a cylinder and then you can just redo through the camera 3d there is a note that is visible here this note is what it's going to be rendered out this is the renderer so if I click on the camera tracker renderer and press number two okay this is what you will see all right and right now on the camera renderer if you look at the inspector the lighting is not enabled so we need to turn on lighting and shadows and right now there is no lighting because we did not bring in any lights at all so let's just create a single light go do control spacebar just start light and then we create a point light and then we plug the point light into the scene and then we're gonna move the point light up then now you can see the point light lighting at the cylinder of course if you want the light to be much more realistic right you will have to use another light source in a point light is just acting more like a a lamp all right so I think a suitable light right would probably be a directional light me just add a directional light oops you make sure you don't select anything close otherwise it will try too much into the last thing that you have selected so control spacebar we're gonna add a directional light and then connect it to the senior they're gonna move my directional light away and then select the angle and rotate the directional light until it matches the lighting conditions of the the battery all right so intensity and the color I'm going to use a slightly warm afternoon sign color right and yeah use the intensity a little bit okay then you can see the final rendered result and if you don't need the the ground plane anymore we can unplug it so we don't see the ground plane anymore now of course this is just a very simplified example for for today all right if I play back right now okay we'll start to render every single frame and then basically you can see that the cylinder is now locked up now of course we can play this as smoothly right now because we turn on proxy instead and see whether it will render to the proxy so we need to render this in the memory before we can play back again again like I said I'm streaming and recording so everything is running very very slowly right now on your system it might be a little bit better okay okay so let's try to bring in other elements I'm not bringing two more elements which is the 3d model from Maya and then I'm gonna bring in a flat 2d plane and then I will conclude this lesson then after that I really want you guys to try out I know this lesson is very long and it sounds a bit complicated but if you practice it a few times then you'll be quite familiar in fact since we got many hours in the coming weeks right I will be using other examples and then we will just keep on practicing until you get a hang of it all right so this this will not be the only time that we are doing this all right I'm going to bring in the 3d model so let me just open up the let's say import open up the okay so we got this guy here so I'm gonna export him and um I'm gonna bring him into fusion okay future I can actually accept 3d models so I'm gonna exploit this guy export him as a FBX alright so you say obj I'm gonna explore a matter FBX export ok so right now I already have one that's being exported so I'm gonna react spot again I'm just not clean closer to what we are doing an expo selection okay so take note that when you export as a DX right not only are you exported in the model you're also exporting the the material information all right so all this will be brought in a special separate nodes in fusion so let's go back to fusion and we're going to import in the astronaut okay so you can go over to Fusion import FBX scene okay and we're going to locate the folder containing the astronaut okay so now I gotta do this twice okay I might have to use another method import try I had to use a loader instead right rather than port so I'm gonna use a loader and I'm going to see whether I can find this yeah and let me see what it is working okay for some reason I got this problem before I am able to bring in the FBX within the standalone fusion but for this one I seem to have problems bringing it in okay I'm gonna try exporting it again right but I'm gonna change the settings you're gonna export file export from I'm gonna use export all instead of export selection and going to use FBX export and then export all okay am i end up bringing other stuff but I'm just gonna try this I'm gonna overwrite the astronaut file again okay so sometimes you're the we export used another method okay the loader didn't work okay so I'm gonna try one more time I'm gonna use that diffusion import and big scene FBX and hopefully this one works this one fails again okay so anyway I'm not able to bring in the object but I'm able to bring it for some strange reason I'm able to bring it in in school okay so I will try again and then once it works and now I probably show you to you guys again tomorrow another method is to use in okay I think I did I'm a mistake I should not have import I should not be export importing as an FBX scene alright this is my fault alright I should be importing as shapes all right there's a reason why they don't work I think let's see quick oh wow and let's try this ah that's a work let's see let me try blending seeing instead okay I'm gonna try using a lambic seen it maybe that's the one that I've exported let's see export all okay I don't remember exporting it as a lambic so you see Oh DJ again try it will be J and export all get obj my work and then I bring it in as a shipping stead import shapes let's try obj still doesn't work okay I'm gonna have to go back to school again and then figure out why is the method I use but I remember I imported in as a FBX and actually works so for some reason I'm not sure why it didn't work this time let's see it try this one import media you try and import media instead J does it work miss Amelia pool okay we have the FBX 3d mesh let's see what it is works nah it doesn't work okay all right anyway I will try to figure this out okay I'm gonna so far we only managed to bring in a 3d object that is created within fusion itself there's another object which is a 2d plane which you can actually link in with image texture which I'm going to show you how to do that all right so right now I'm gonna select one point we did the much-much 3d space the sauce space which other we did and I'm just going to react spot the point ecology cannot go to the camera tractor then go over to the export settings point cloud and I'm going to update my previous export again unfortunately I do need to rate the point cloud so I don't have it already okay doesn't matter okay I will show you in another example again maybe I should just disable the point tracker the point cloud rather so anyway I'm gonna create a 2d plane object this image plane 3d alright and then I'm gonna hook it up in a much 3d and now I can see this image plane treaty which I can go to the transform tab and rotate right and move it up y-axis one better so much needs to be 30.5 let's see yeah then I can move it in front in the z-axis okay so what can you do with this plane object here now this for this plane object right you can actually attach a Sawbuck on all right you can put a image plane with a transparency on it I'm gonna bring in my favorite character now there's a lot of images that you can find on on the web I'm gonna use Google and I'm going to search for SpongeBob PNG alpha yeah you want or PNG transparent then click on images you look for an image right there you can see this checker pattern next to it but some of it might be fake okay so but anyway I want to use this one and just want to make sure that this is a PNG maybe this one no if some of it might be fake so he had to be very careful otherwise you to create the transparency yourself okay maybe this one alright so I'm going to open up this one open image in new tab and I'm gonna save this image save image task right mouse click save image best I'm going to rename this is really lonely another foil Sponge Bob and I'm gonna drop it into my work folder and I'm gonna bring him in as a media clip all right so I'm gonna file import file import media and bring him in okay I think that's an error there I'm not even able to read this image so let's see okay so this this image is corrupted okay I think this it could be reason why I cannot read save this image or look for another one instead okay hopefully this won't work I'm gonna replace it with this one okay and let's bring in the image again hopefully this time it doesn't get corrupted if you had the same problem she was wrong this okay so this is not true this is not true a PNG image hey let me just try to find another image that I have on hand let me just give me a minute okay my dear wine suddenly the images are very hard to find give me just give me a second we just look for you can see whether it is a fake image there some of it are really sick I need to have one okay open image a new tab hopefully this one works save image as and I'm going to just review this okay this one seems to open alright so hopefully I'll keep my fingers crossed this time this time this one works yeah okay finally and I think this one might be a thing that might be a fake likewise let me just bring this in and let's do it yeah this is a fake image with the thick alpha because it doesn't really have an alpha Channel alright so by anyway I just want a drink do it this quickly first I'm gonna drag this in if you have an alpha Channel right this area here non-connected it to the image plane and it should have a transparency on top of it alright so unfortunately I don't have to don't find a proper image so if I were to play back you can see that with the tracking on right and see the image is now locked onto the ground now to make it a little bit more interesting I'm going to move this sponge pop okay now I'll probably be repeating this entire process again in tomorrow's lesson okay so what I want you guys to do right now is just to try to figure out how to get the tracklist don't worry about all this much 3d stuff because I'm gonna be going through this again the most important thing is for you guys try to understand the process of tracking and solving and then getting the final result ok so right now I'm in my 3d view and I can rotate this again you burn any me okay I can you burn any main by adding a keyframe for the rotation so as I scrub through the timeline I can rotate spongebob here to face me so once you're activate the keyframe right you can actually have him face the camera all the time just by animating the keyframe okay so you can see now if the plane is facing now the checker pattern is supposed to be transparent but because this is a thick PNG image this is not really transparent that's why we ended up with this and unfortunately I couldn't show you why the FBX it's not working I'm gonna try to fix this problem because in school it worked fine for me so I'm gonna find out what's wrong why I couldn't pull it in alright so once you got this done already the next thing is you have to plug this out from the camera renderer plug it out to the media output and if you go to your edit timeline you should be able to see all the effects right on the edit and all of this right is exportable now now by going over to D deliver okay so if you go to deliver can click on h.264 and start to create your my first track then save it to your location and add to render queue and then start random we're not saying the much of the animation going on okay that's updating it's gonna take about two and a half minutes to render so after this I'm going to stop the recording and I will be uploading the recording on to my youtube channel so that you guys can watch it but it's gonna take about maybe ten or fifteen minutes to upload and process then after that you guys can watch and then you guys can try it out on your board alright and I think you have more than not time to do that okay at least right do the tracking portion now for those of you who are done your tracking using nuke you know that nuke one then the nukes method might be more straightforward okay but the concept is more or less the same right the steps involved is amol SSE perhaps the alignment portion might not be as convoluted as this one but you just need to get a hang of it and yeah just just do it a few times and remember the don't just try out this click here because we have other clips right there is their shot at different locations I think all of these clips right there should be able to to film or you should be able to do the same tracking easily because I deliberately filled them in such a way that I know that it's easy to track and solve so when next time when you guys go out and if you are shooting your own footage right so these are the things you need to consider the dust that your footage contain enough details for the song title song is there enough parallax is there like sufficient movement between the features in the distance and of the features floating closer to the camera this is very important if you don't have any of that right then it's very difficult and remember today we are only playing with this car free cameras off these are free cameras all I have yet to talk about another tower cameras off called the nodal cameras off okay the no the cameras off so maybe tomorrow or the day after tomorrow we have basically the foodening yeah maybe tomorrow we will go through this entire process again and then once you guys done enough exercises then we will talk about doing that notice all because I think doing this once it's not enough you all need to do this a few times just to get a hang of it okay and then if this is even written in the manual self they are just some shots right they are just impossible to solve especially the camera moves too much okay so now we've exported and I'm just going to show you the final result and this is the completed clip okay so you can see everything is locked down okay so ideally I want this to be transparent okay but it's not because this is a not real PNG okay so basically this is it I think as I said to one student this is gonna give you guys bring down especially if you've never done this before so I'm going to stop the recording now and I'll be uploading the video shortly and I will send you the link immediately once the video is ready so I hope this information that presenter is sufficient for you guys to at least get the first track done all right you don't have to export out the video you don't have to do that what you need to do is you have to go through the lens to start okay do the camera tracker and then do the export and then make sure you do the alignment properly okay so with that I'm gonna stop the video recording right now

Channel: weelian soh

Views: 593

Keywords:

Id: lzQwkmfppB0

Length: 118min 53sec (7133 seconds)

Published: Wed Aug 19 2020