Ultimate Beginners Guide to Virtual Production in Unreal Engine 5

hey guys the idea today is that we film a talent on a green screen because an LED wall will cost me two kidneys and I really need those then we move the real camera which needs to move the virtual camera so that when we put the two shots on top of each other it seems as though I stand in that virtual environment that is the fundamental idea behind virtual production once you understand this you can build further on that and make your virtual production more complex now we're not gonna do live chroma keying and unreal you could do that as a reference but it's not a necessary part and you're gonna need additional Hardware this tutorial is for an offline workflow so that we can do a much better keying with something like after effects Katie was so kind enough to sponsor this video they sent me a couple of TC ones and their SL1 slates and I was like a little child so happy when they arrived although I'm not sure if a child would be happy to get a Time code generator for Christmas anyways these time code generators are gonna be super useful alright another yapping let's start with step number one good get on with it design your Unreal Engine scene and I just slapped something quick together from quicksilon as an example there we go step number two I'd like to place a meta-human or a simple character in here that gives me an idea of the light we can go ahead and click the menu here on top to create a new cine camera actor at the place that I'm currently am there we go this is the initial setup for unreal step number three put up your green screen and make sure it's tight well lit and use the reference character in your unreal scene to match the light in your studio I tend to take a screenshot and compare that lighting with my setup alright we're ready for tripod shots but that is boring so for step number four we want to move the camera so we're also gonna need camera tracking and here's where the first piece of Hardware you're gonna need comes in now there are super expensive systems like the one from Vicon which is used in high-end Productions but that's not an option for us the cheapest solution is to use a VR system the one from Vive is the most recommended and best supported and even within Vive there are two solutions either you go for a VR headset buying such a package gives you two bait stations these register the tracking and two stations can cover an area of around three and a half by three and a half meters which is approximately 11 feet but you can go up to 10 by 10 meters or 33 by 33 foots if you install four base stations here in the studio we've got four of those which allows me to set up the camera anywhere I want and I can play VR games in the weekend which is truly awesome in such a large space but from not here to play games now there needs to be a device that can be tracked by these base stations and we get a lot of options here if you've got a VR headset you can use the controllers for that or even the headset itself now it's not so user friendly though that's why Vive also has these trackers which have a mounting option on the bottom much easier to connect it to your camera so once you've got that installed you just launch steamvr and then you can find The Trackers back in unreal but there's also a different solution from vipe and that's the Vive Mars which is specifically designed for virtual production if you get the Mars box it comes with two base stations we already know what they do and three trackers so that stays the same but it also comes with the Mars and this is essentially the brain The Trackers are also complemented with these adapters which they call the Rover this allows you to connect the tracker into the Rover and then connect that Rover onto the marsh so instead of transferring everything wirelessly you got cables which is more accurate and reliable I can connect three trackers to the Mars but we'll leave it at two for now one goes to the camera and I'll put another one on the floor the Mars gets connected with an ethernet cable to your network and it doesn't require you to launch or even install Steam VR you just open unreal and device trackers will automatically be there Step number five time code so we've got a real camera and a virtual camera it's like recording audio and video separately now we need to synchronize those two signals unfortunately we cannot just clap our hands as Unreal Engine is not a sound recorder so that's where deity comes in which is absolutely the best and most affordable solution out there you can get these time code generators which you connect to your camera and the cool thing is that this works with almost any camera as you can also plug in it into the microphone Jack it will then write its time code to the audio Channel but it also records audio itself because there's a small mic on the tc1 so you'll still have reference audio which is super useful now I've got a pack with three generators so one goes to the camera I can hook another one up to the Mars and it needed another one to a sound recording device now it's very interesting about these generators is that we can connect them to their apps situs audio you'll find all your devices in here as well as their slate which I'll get into in a moment the idea is that we set one box as the master this one is going to determine the time code all the rest needs to follow so we set them to jam so that means we can change all the settings further on the master let me just set the right frame rates here so let's go for 30 FPS there are also a bunch of channels if you're working with multiple sets or Crews but that's not needed for now finally we can select the output either L out or a out a out is for if you don't have a sync port on your camera as mentioned before but the deities should automatically recognize that and that's pretty much it it is very important to use one brand and multiple time code generators because there could always be an offset or by the end of the day you'll see that one device is like a frame or second behind now they also have a slate which goes into the app as well and it has a beautiful unibody aluminum design with wooden sticks it feels super solid and it is built for one hand control as well there's a backlight so even in dark scenes you're able to see what's on the Slate on the back there's a menu to change a bunch of settings but we can do everything from the app which I recommend you get over 25 hours of battery time and it has two batteries that means you can swap one out with having to disconnect the slates it's truly amazing products and you can learn all about it by clicking the first link in the description Down Below guys alright step number six let's connect that camera tracker to Unreal Engine and the first thing you want to do is head over to the menu on top and select plugins from here search for LifeLink and we want to enable the standard LifeLink plugin but also the LifeLink lens plugins we're gonna need that later on to get the lens information from the real camera into the virtual camera otherwise your tracking is not going to work properly finally look for camera calibration and enable that plugin as well then just restart unreal and once it's back up we can go to the window menu on top and under virtual production choose LifeLink click on Source message bus source and you'll find device Mars in here then on the bottom you'll find all the trackers that are currently connected to the Mars we've got one on the camera which is the Rover one and Rover 3 which is the tracker on the floor now some people have trouble connecting the tracker to LifeLink unfortunately there are many causes so I can't really give a straight answer you can try to disable your firewall or if you go into the project settings in unreal check UDP messaging on the site switch the IP address and unicast endpoint and see if that solves the issue alright step number seven we're gonna link the tracker to the camera so select your camera actor and in the details panel click on ADD and we're going to look for the live link controller then with that component select it we can assign a subject to it which is going to be the Rover one and that's pretty much it when we now move the real camera the virtual camera will move along now what I'd like to do next is Select my meta human and also add the LifeLink component to it here we are going to assign the Rover 3 to it which is the one on the floor now this is all optional though but if you place the floor tracker on the spot where you're going to stand it gives you some kind of a representation of the distance between the camera and the subject step number 8. there's a problem the camera is floating somewhere and we can't move it to a different spot because its position is controlled by the tracker to solve that issue we're going to go to the ads button in the viewport choose basic and then actor this is a nothing object it doesn't do anything but we can attach the camera and The Meta human to it by just dragging those into the action so now we can freely move the actor around see it as a group and place the camera where we want it to be now one thing to keep in mind though is that you want to plays the actor at the same height level as your floor in this case that would be zero however the tracker sits on the Rover adapter which is five centimeters high so you want to add 5 to the Z position of the actor step number nine the lens data and as you know the virtual camera has its own focal settings and unfortunately it's not as easy as just typing in the millimeters from the real lens plus the tracker sits above the camera this is called the nodal offset and that needs to be calculated somehow now if you have the Vive Mars it comes with a lens calibration tool I'm going to show you how that is done in swipe has their own tutorial and it's pretty easy but you'll end up with a lens data file you can just import that lens file into unreal then select the camera actor again and add a new component this time adds lens to it and from that component we can assign the lens file finally set the evaluation mode to use camera settings and don't forget to enable Distortion that will set the focal length and everything else the same as the real camera step number 10 we've got the time code running thanks to deity now we just need to bring that into unreal as well for that we're gonna have to right click in the content browser and choose to create a new blueprint expand all classes on the bottom and look for LifeLink timecode provider give that a name and open it up on the right hand side set the subject key to either one of the Rovers hit save and compile you can then just close that window now going into the project settings look for time code Provider from that drop down menu here you should be able to find the blueprint that you just create read it select that point and we've got time code running into unreal thank you so much deity Step 11 and I'm missing a finger here you know just assume my head is a finger as well yeah sure weirdo Rome was there guys everything is set up so we can start recording go to the windows menu cinematics and choose the take recorder although and 12 you should see your time code running in here everything that you're gonna move around and want to record needs to come in here so that's gonna be the cine camera actor and we're also gonna need the attached actor to retain the correct World position so we can start the recording sequence alright everybody ready sounds rolling camera rolling unrul engine recording and action and congratulations your first virtual production is a fact you can find your unreal recording back in the cinematics folder just open up the correct scene from which you can preview it and on top you'll find the render button to export it out as a movie then loading both into a video editing program like Premiere we can easily synchronize the unreal scene the camera shots and if you have audio as well the audio recording because as they all have the exact same time code running then just pull your green key and you're done hey here we are inside Unreal Engine the camera is spanning we got some movement although the space is a little bit empty here isn't it I should have put more stuff in here anyways it's working hey now if Unreal Engine is still new to you and you want to learn all the basics for using Unreal Engine for filmmaking like how to get DMX to work perform a live green keying and whatnot check out my course in the description down below I'm gonna give the first 100 people 50 off so you definitely don't want to miss out on that one thank you so much for watching and as always stay creative

Channel: Cinecom.net

Views: 115,612

Keywords: unreal engine, virtual production, unreal engine 5, how to, virtual set, ue5 tutorial, virtual camera

Id: OgxfvtT17CY

Length: 10min 58sec (658 seconds)

Published: Fri Mar 10 2023