VIVE Mars CamTrack and Aximmetry DE setup guide

hi guys welcome back to the channel in this video I'm going to be showing you guys my workflow in setting up the HTC Mars cam track tracking system with eximetry de or dual engine asymmetry that's using Unreal Engine now I'm not going to be discussing how to set up the Mars system with unreal because you can connect directly right without using asymmetry using the live link protocol and that is not something I'm going to be talking about so if that's what you're after this might not be the right video for you I'm just putting that out there so to not waste your time but other than that welcome to my studio so this is our very own virtual Production Studio and some of you might think wow that's a really small Studio but it is actually big enough for the types of Productions that we do here which is mostly Esports we usually never have more than three people on the green screen so this is just about the right size for us so we have the control room over here inside with the you know the PCS with the vmix a symmetry and all that and this is our green area where we have our casters or our talent so the first thing that I want to do is first of all I have to make sure okay are we going to be having our next production here fine it's locked great once we have the location lab because we don't always just shoot here sometimes we shoot at a bigger Studio that we ran out but once it's locked so I know that I'm going to be building my Set uh based on the actual location because when you're working with a tracking system um I don't know if my English is correct or not but everything needs to be designed to scale basically what I'm trying to say is that the measurements from your actual set your green your studio should match your measurements in Unreal Engine and in asymmetry so your real set should match your unreal set in terms of measurement so that's the first thing that I'm going to do is actually measure my green screen and it's always handy to have one of these this is like one of those laser measurements and I'm going to be you know like measuring my green from side to side foreign the measurement from my green screen I'm gonna grab one of these uh Rovers from the Mars kit I don't need to have the tracker on it right now and now I have to Define where my zero or my Center Point is going to be and I want to choose somewhere in the middle here in the beginning of my green screen and the reason why I'm doing this like all the way on this side is because for me personally I'm not really going to have my camera moved too much with within the the green it's going to be moving somewhere around behind so I'm going to be building my unreal set from this point uh forward and so this is going to be my zero zero zero so X Y z00 with an unreal and also in asymmetry and it's going to be my center point for The Trackers as well later on I want to show you guys that you can recenter each of your trackers and this is the point where I am going to recenter the trackers and if you have more than one tracker you want to recenter all the three trackers at the exact same location so this shouldn't move and to make sure it doesn't move I'm just going to use these gaffer tapes I think you call them and just sort of like Mark where it's supposed to be thank you alright guys before we continue with the tutorial I would like to make a quick stop to the unreal editor for X Symmetry and this is just something that I like to do it will make things easier for the production and you guys can choose to do this or you can skip it it is totally up to you so this has to do with what we just did where we just set up the center point or the zero of our real set now um sort of want to set up my zero in the virtual set so I've made a new level here it's called virtual stage this is where I will be building my virtual set and the first thing I'm going to do before anything at all this is an empty level is ADD camera and add track cam so I'm going to set it to the unlit mode so I can see it more clearly and here is my camera so now because I've done this I know where my camera is pointing and this will be the same direction as where my real camera in the real world will be pointing as well and it is positioned at location zero zero zero so this zero Center Point is the same as the zero Center Point in my real set and when we load up the stage later on in asymmetry I won't have to move my set too much around because it will already be where it is supposed to be therefore just making the production just easier overall you know less time fiddling around so uh what I want to do now is because I've already measured my green screen right I'm actually gonna recreate my my green screen area right in front of the camera so I'm going to use a pack from the super grid here it's a free pack from the Unreal Engine Marketplace and I'm going and I'm going to be talking a lot about this super grid when I will be making my video uh how to make a virtual set from scratch but for now I'm just going to use an asset from there called the super grid box centered and I'm going to give it the size of my green screen so it's uh so it's 3.4 meters 3.4 meters and as you can see these big boxes represent 50 centimeters so it's like one two three three point four meters and I like this a gridded box mesh because it's sort of like similar to the one that uh asymmetry has when you're trying to set up the Studio's Green Space which we will see later so um I want it to be right in front of the camera and it's gonna be 3.4 by 4.6 six there you go and I'm just gonna supposed to be on the floor so I'm just going to make it zero point one and now I'm gonna go to the like top and side view with me from the right and just like bring it up two making sure that it is in the center yes and from the top view yes it is so I have it exactly where it is supposed to be all right so I have my green screen area I have my camera it's pointing directly to the green screen and now uh I can actually just save this and I can send this file to the person who will be making the stage because now that person will know that okay this is the area where the talents are going to be standing so the designer can build the stage around this area so there's like a starting point for them to build the set and once it's done and we load it up it's gonna be really uh easy to set up the cameras because it will already be where it is supposed to be in the virtual space all right so now let's continue with the setup and I want to talk about uh the gear a little bit here because uh when you purchase the Mars cam track you get um the Mars of course and you get three Rovers these and you get two trackers right two trackers and you get two base stations so um I really recommend that if you want to you know track three cameras you need to buy a third tracker since you already have three Rovers right and for the base stations I really recommend you buy an extra two base stations because the Mars system can make use of four base stations so you get two PlayStations 2.0 you should get another two position 2.0 and even though I have a small you know like area for my production I still like having four because just in case if you know like some of the base stations is like being covered by blocked you know the view to the trackers being blocked by a person or by a light or something the other two can still pick it up and your tracking data won't be messed up so as you can see I have my base stations uh set up it's just drilled to the wall on every corner of my studio and they're sort of like pointing down towards where my cameras will be moving and as you can see I have already set some a power outlet on the ceiling so that I don't have to have any cable you know like dangling down if you just like plug directly into the power source and I actually have a switch on the wall that controls those power sources so I can just like flip on a switch and all my base stations will turn on and it's a dependent switch separated from the lighting okay so since my base stations are set up now I'm gonna set my tracker on the Rover and connect it with the provided cables right USB to USBC and of course it's you know it's using a screw right here that you can use to unscrew and screw the tracker on the Rover and as you can see I have a little tape here and I know this isn't really pretty I might find something else to replace this but the reason why I have this is because if I just connect the tracker directly we found that it actually sort of like moves a little bit so it's not really stuck on like that hard or like you know like permanently and when it comes to tracking I really really want everything to be as you know we don't want any unnecessary movements so I you know I don't wanna I wanna clean track I don't want to have any Jitters or anything and I don't want when the camera makes a set of moves and then the you know it'll like move around or something like so you really want to have everything stuck on there good so uh I've made sure on that see there's like no movement at all so that's uh that's about it you know set up your tracker to your Rover and then we're gonna connect the Rover over here see with the provided um cables and I'm going to be plugging it into my port number one here in the Mars main unit so if you have three cameras you know you'll have three Rover Street trackers you can you know populate all three plugs so the Mars can handle three inputs maximum all right so once that's done right I'm going to connect the Mars unit to my main router that's going to be connected to my PC Okay so this cable is going into my router now I wanna I wanna talk a little bit about connecting the Mars to the PC because uh some people you know they they don't use a router or a switch and so on they just connect from the Mars directly to the PC me I use a router um because I still want my PC to be connected to the internet while I'm doing my production some people have those motherboards which you have uh two ethernet inputs so they use one for their internet and they use one for uh the Mars that's also possible but me I haven't had any issues by just connecting the marsh to the network where all our PCS with the same network as all our PCS within the control room so that's just the way I do it I haven't had any issues with that but if you want to connect it directly to the PC go ahead all right so next I want to talk about the sync generator that we're using we're using the Blackmagic design mini converter sync generator of course you can use any sync gen that you want you know I know Aja makes them and so on and um so basically we need to have everything synced right the camera the Mars and the PC so as you can see I have an input an output sorry a reference output for each this one is going to go uh inside the Mars and one is going to go to the uh camera and one is going to the PC all right so a sync generator is absolutely essential if you want to get a clean track if you don't use one you're going to have drifting because the frames aren't in sync and all that so they're not that expensive it's definitely a must have and I'm going to turn on the Mars system so as you can see as soon as I turn on the Mars I get a system update available and this is one of the reasons why I like having my Mars connected to our main Network that's connected to the internet so that I always know when there is a system update if I connect it directly to the computer and it's not connected to the internet you know I can't really tell when there are updates and so on so I'm going to update my system all right now that I'm done with the update and now it's asking to update my Rover so the the previous was the update for the Vive now we're going to update the Rover itself so update now and that's done so now we are at the Mars main menu and a few things that I need to check now is first of all is my tracker being detected and yes in fact I do have one tracker connected to the Mars system and I have the slots for the four base stations next to it and I right now I have three base stations being detected and one that is not so I'm gonna go ahead and fix this okay so I'm gonna do a little bit of troubleshooting here and I you know I like including these things in the video you never know if it's gonna happen to one of you out there so I have four base stations uh three of them is being picked up by the Mars and one is not so one of these is having an issue and the problem is we don't really know which one is the one that's not being detected so what I like to do so I'm not gonna go you know on a you know get a ladder and like turn it off one at a time and all that so I'm just going to grab my calibration board I want to place my uh a Tracker somewhere in the middle where you know all four base stations can detect it and I'm just going to start you know covering one of each you know like for example I'm gonna cover this one and you know I'm gonna have someone see on the Mars if it will you know if one of them will turn off so I have three if I cover this one and there's one less that means this one is active and then I'm just gonna go to another one for example like this one see if I cover it up you know will it reduce one that's active or not if I cover it and none of them turn off I still have three of them that means this is the one that is having the issue and then you know I can look into it maybe it's sturdy maybe it's uh there's a problem to it or what not so that's how I troubleshoot it alright so now as you can see we've fixed the issue and all four base stations are being detected and actually we had to replace the one that wasn't functioning because we don't know what the issue is yet but that is why I actually recommend you know not just having like four base stations we actually have six so we have two that serve as backup because you never know if one of them will have an issue then you have a replacement okay so the base station issue is done for base stations one tracker all is good and now I can see that the Gen lock is not properly set because for my production I'm actually going to use 50p so then I'm going to go to my sync generator and I'm going to set it to 50p but actually like this is not really relevant because this will depend on the sync generator that you will be using each will have different ways of uh setting the desired uh frame rate all right so I set my sync generator to 50p and as you can see now there is written on the screen gen lock at 50. now if you go to settings and there is here that something that says synchronization frequency FPS and mine is set to 50. you can click on the drop down and set it to your desired FPS now if you do this and you don't see some of the ones you see here it means you might need a software update because I know some of the older software versions only included like 30 and 60p or something like that but you didn't have all of these you didn't have the 50 for example and now you do so just get a software update and you should get it so set your synchronization frequency fps to the Gen lock that you're using all right next let's start talking about the networking all right so let's start with the Mars Network and you know I'm plugging this into a router and I can just use you know obtain IP address automatically dynamically and all that if you want to use a static one go ahead and I just want to let you guys know something I think this is worth mentioning that I usually use A1 uh sorry I usually use a192 IP address why because we've had some friends in the community that use like a 169 and they've had issues with the Mars you know the tracking data not being detected within ximetry so I don't know I thought this is just something worth telling you guys that if possible try to use a192 IP address but for me I just use it automatically and I don't want to you know worry too much about all this and apply and I'm good so next I'm going to talk about the camera tracking protocol and here as you can see there's live link and 3D now as I mentioned in the beginning of this video I'm not going to be talking about live link live links to connect directly to Unreal Engine if you want to do a production with it on your engine using a camera tracker we're going to be using 3D because this is the protocol used for the Mars to send the tracking data to ax symmetry so here there's an IP address and this is going to be the IP address of your computer so you can you know check it on your computer what is the IP address set it here and set your Port so once you set this you should be good to go and click apply and the last thing we need to do is re-center your tracker so this is what I was talking about earlier about setting your tracker at your zero or your center point so I'm going to click recenter and here it says recenter virtual space for tracker one because we have tracker one only connected it's going to show the option for tracker one and I can just tap it but before I do that I have to make sure my Rover and tracker is set at the zero position all right so we already marked our you know Center Point or zero in the beginning of the video so now I just need to put my Rover right here and now I can just tap this button and that's it so if I have three Rovers or three cameras to track I just have to repeat the same step with each of The Trackers all right so once you've re-centered all your trackers um you're actually done you don't really need to touch the Mars unit anymore we can move on to the PC but I just want to note that if for some reason you know there's a power outage or you turn off the Mars and you turn it back on again you should re-center all your trackers again at the zero or at the center point otherwise you might have issues that had happened to me before so if it goes off re-center again and um if everything is re-centered properly and you have a multi-cam system uh everything should when you switch between the cameras the positions and all that should be correct because the trackers will know where they're supposed to be in the 3D space in relation to where the uh point of zero is so that's that let's move on to the PC all right so I'm going to open up back symmetry now and I'm going to go to manage devices and I'm going to go down and look for the 3D protocol under camera tracking there it is so I'm going to click add so I'm just going to use my select my you know network adapter and you know I set the port to 2000 on the Mars and I'm going to set this camera ID to one because I only have one camera so you need to set the camera IDs if you have three cameras camera two camera three and so on so that's done and I have so I've added my tracker to uh oximetry now apply okay and now the only thing left to do before we can you know you know start you know testing our set and the tracking is to calibrate the lens with your camera and the tracker together now I've made a separate video on doing this because it's quite a process and just to make it easy for those of you who only want to you know sort of like uh go directly to the tracking process so I prepared a separate video for that so feel free to pause the video watch that video again and once you're done with the calibration process get back to this video alright so we're done with the uh camera lens tracker calibration process and which is why we have now moved our camera from the tripod to the for the chip and we love the the portrait because you know you can get so many different angles with it you have like buttery smooth movement uh so yeah and then you can like Mount all your power Terminals and all that on the uh portal jib itself because you're not really restricted to weight and things like that so we're done the only thing left to do is to set it up in asymmetry and test it out let's go all right so I'm heading back to Unreal for a bit and I just want to show you guys I added some pillars and some colors on the ground so just that it's not a plain stage so that we can feel the movement of the camera there's a parallax so okay so I'm done with this uh make sure my camera is a tracked camera and I'm just going to save all and cook content for asymmetry all right so we are inside oximetry and I'm gonna make sure my FPS is correct I'm using 50p because we are gen lock at 50p so this is correct uh so I'm gonna create new compound and I'm gonna go under a camera because I'm using the latest version of asymmetry and it is a little bit different here and the file locations and I'm going to choose a track cam unreal and with the three cam option I'm going to drag this in here and I'm going to drag my project the tutorial that I made and uh Mars tutorial project bring it in here and let's connect all the wires all right and now let's set up the camera camera one that's okay so let's go to inputs and turn off input two and three input one and for the camera I'm going to be using my input it is the Blackmagic SDA 4K and I can see the feed now my camera and for the tracker I'm going to be using the 3D one or tracking system okay so we can see the camera feed here and I'm just going to test that our tracking is working I'm going to go to Studio mode here and can someone help move the camera a little bit tilt down tilt up all right so as you can see the tracking data is being fed uh we can see still see the green so the king is not working and I'm going to go into my key and it's because it is still set to clean plate tree generator and because I'm using the latest version of uh asymmetry compatible with the Unreal Engine 5.2 it has a new feature called clean play 3D generator and I will be talking about this feature in a separate video for now I'm going to go back to Advanced gear B and now it's being keyed and I'm going to try and turn it off for a bit and just try and get the color of the green screen okay and then turn it on again so that looks not bad I'm gonna get rise the low cut a little bit okay now I'm going to go back to final and as you can see now we can see the stage that we made in unreals alright so now that I know my uh tracking is working I'm going to go back to the studio Tab and now I need to define the area of my green screen so that asymmetry know which areas to key and which areas to just cut out so if my cameraman would tilt down a little bit okay stop and you see the area that's green over there is going to be keyed out and the area that's not that's not having any green on it is just going to be a cut out and if we tilt down a little more you can see that square is the zero zero Mark and if you remember earlier on we set our trackers to be centered right on the same location so now I'm gonna adjust my green and I'm going to start by the floor the green floor over here and I'm going to pull it Forward okay hold on hold the camera until it gets to the green over there I'm gonna just I usually leave a little bit of space so not too much to the edge because I'm not going to be moving any Talent all the way there anyway so we're done with the bottom part let's look to the left side and I just want to point out to you guys that these squares right these represent 50 centimeters if I look at the floor like there's like one two three about four meters series of green and that is correct right so um this is actually already like pretty close to my actual green screen but on the left side as you can see I have a green as well so I want to key that out as well so like I think it's wait this one green front left oh no it's not that one Ctrl Z so green left okay that is the one so look left please cameraman all right over there and go to the other side and so the right side I'm going to add some green as well so let's check out the top part look to the left side [Music] okay so I think that should do it for now all right once you've already defined the area of your green screen now it's time to adjust your tracking delay and a few things that you need to check first is first of all you have to make sure that the calibration profile is already set to the profile that you used uh to calibrate your camera lens and tracker so uh again I had a separate video on how to do this and this is my profile right here I'm using the bgh1 with the zuico set at 14 millimeter and then next I want to make sure my camera is Gen locked so I'm going to hit the SDI here on screen and it's going to show a g if you are gen luck with the Mars and the camera so we're all good here and so now for the tracking delay let's go back here to the studio mode and so I have put a chair here and we have two markers that we can use so here marker position one I'm gonna pull this up and I'm going to set it to one of the feet of the chair like okay for example this one marker one and I have another marker which I'm going to set on the other feet of the chair all right so this is a good setup to check your delay and you can have your Mr cameraman to move left and right and as you can see it's uh dragging quite a lot so I'm going to select my input and if I go down here there's a part that says track delay and the way I do this is so you can either go minus you can like you know like go minus or you can go plus right and the way to look at this is like this as you can see from the image uh when he's pending left and right you can see that the video input is faster than the tracker because you can see that the feet of the chair is moving first and then the tracker tracker markers is following behind it so I need to add a number to the tracking delay so I'm not adding to the delay I'm telling asymmetry like okay I think there is a delay of maybe two and as you can see see now it's less dragging and maybe I can go up to three and now the feet of the chair is getting closer to the markers it's not really that much ahead of it so this number is the number that you think there is the this certain amount of delay between the tracking and the camera so I need to add just a little bit more maybe three point five and now it's really sticking to it right and maybe I can just add number and you need to like really fine tune this afterwards like maybe like five points three and or maybe like five point five five you know but let's just say like this is good enough for me right now you guys can really get that perfect number but anyway this is the way to do it and once you're happy with the delay um you know you can go back to your final mode and take out your chair and add a proper talent to it let's go guys okay so now we have our talent and we can test moving the camera around go ahead [Music] and that is all there is to it that is how I set up my Mars camera tracker with X symmetry dual engine I hope it's useful to you guys that are just you know getting started setting up with the system that just purchase it and you know I'm only human if you guys know better than me have more experience you guys looking watching my videos like yo this guy forgot to mention this this and that by all means write it down in the comments and I will pin it or add it to the description okay or maybe I will make another video in the future and add your tips into it other than that thank you for watching don't forget to subscribe to the channel if you find it useful and I'll see you guys in the next video bye

Channel: David Alexandre

Views: 4,889

Keywords: htc mars camtrack, htc vive, camera tracking, tutorial, setup, workflow, aximmetry, unreal engine, virtual production, rover

Id: 1fA2YD34WJI

Length: 31min 23sec (1883 seconds)

Published: Tue Jul 25 2023