Virtual Production at Home with an iPhone and TV - Unreal Engine

[Music] [Music] hello and welcome to my spaceship it's it's not a PNG here come take a look look we can uh can see what's that way can take a look at that that wall on the left there there's a ceiling there's there's floors it's it's totally it's totally legit it's so legit it even has zero g now I know what you're thinking Max you really need a haircut and yes you have a point but hear me out unreal virtual production right it's the Holy Grail of modern film making it's like the hottest thing right now we've all seen the Mandalorian and we were all willing to overlook how bad it was because of how good it looked and the cool thing is if you have a projector or a big enough TV you can do it and I'm going to tell you how now this is not a proper tutorial and it's not a complete setup I haven't gotten to all of the display outputs all that stuff but this is like how to get as far as I got so far let's do this now first things first you want to go to unrealengine.com click the blue download button scroll down click a bigger download button and once it's done it's going to open up the epic games installer you want to click on this Unreal Engine item in the sidebar and click the big yellow button in the top right it's going to say install and once it installs it's going to turn into the launch button it's going to want a lot of gigabytes so make sure you have the gigabytes now first time you launch unreal you're going to see a list of projects you see I have a bunch here already you want to click on the film video and Live Events and then blank name your project something cool like super awesome YouTube just not too long and then hit create your unreal window might look a little different so make sure you have your outliner and your details view enabled we're going to need those all right it's time to enable some so let's go to settings plugins and you want to make sure these plugins are enabled you want to go and enable live link you want to enable virtual camera and you want to enable Apple AR kit you also want to enable pixel streaming I forgot it in this recording because I already have it enabled but pixel streaming is another one you need once you've added those it's going to ask you to restart don't do it yet we'll do it later go to window virtual production and turn on live link it's going to give you this live link panel we're going to use this later now before we continue we need an environment I'm going to use some assets I got off of unreal store I have the spaceship thing that's like 20 bucks so I'm going to add that uh you can use any Marketplace package you like just when you have it click this add to project button and then select the project that we just created all right once that's done we're going to want to go to content drawer select our spaceship and in this package there is a overview map that has all of the assets neatly arranged for us to use it's going to take a moment to load and once it's done we can move around the scene and explore the assets we're going to go into one of these cabins in the front let's uh let's get get rid of Mars fire let's get into this cockpit and sort of look at what we're going to be filming I'm thinking this triangle window looks kind of neat let's get rid of that other planet back there and now what we need to do is kind of line the Earth up in a way that we like let's change the scale a little bit move it around just play with it until it looks kind of like what you want okay now we want to go to the plus Cube and select virtual production vcam actor this is the magic component you'll see the camera is going to change and we get some weird new UI but you want to take note of this vcam actor thing in your outliner we're going to need that later specifically these camera component and vcam component these two are the important ones all right now let's move this new camera back to our original spot and set up the angles so it's going to be roughly here now let's go to camera component and change some things we want to go to the lens settings menu here and match the lens with whatever lens you're shooting with in my case I have a 24 to 72.8 which is perfect and I have it set to 35 so I'm just going to set the minimum focal length to 35 and maximum for that matter just just fix it to 35 we're not going to be zooming it so now it gets a little tighter so take a couple of steps back and we more or less have our shot now let's set up the iPhone motion control we want to go option click your Wi-Fi icon and note your local IP address right all right now we go into edit project settings and we want to change two things here first you want to scroll down to rendering and then in the search field you want to type in pixel buffer or frame buffer and you change your frame buffer pixel format to 8 bit rgba once that's done it's going to ask you to restart let's not do that just yet instead we want to scroll down to UDP messaging and in your unicast endpoint you want to select the IP that you just saw or just type it in remember the call and zero at the end now you can restart your engine and you should be good to go now let's jump over to the iPhone and set everything up on the phone so first you want to download this live link vcam app it's free just look it up once you have it click open and it's going to give you the single input field in this field you want to put in the IP address that you just put in your unicast endpoint but without the colum zero now at this point if you're me you will have an error saying it could not connect to the server a couple of people had similar issues a couple people had this exact one they weren't able to solve it here's how to solve it you want to go into settings and in the connection type you want to change it from Pixel streaming to remote session this is a legacy protocol that seems to be more reliable uh even though pixel streaming is the new fancy one if that one works just use that if it doesn't switch to remote session all right now let's go back to the main screen hit connect and at this point you should be seeing your unreal viewport on your phone there's one last thing we need to do in the engine which is to go down to this live link panel we've opened up earlier click Source message bus source and select our live link vcam it's going to pop in the iPhone live link subject we want to go to the vcam actor vcam component so it's the innermost one scroll down to live link subject and select iPhone and holy moly guys we are controlling the camera with with our phone holy crap would you look at that this is honestly just super awesome on its own but once we put it on top of a camera it's going to be really sick all right one last thing we need to do is turn off the UI so we want to go down to Output providers select whichever one you're using so pixel or unreal remote which is the remote session one make sure it's active but then go down to umg Overlay and just turn it off now with the UI gone we can just plug into our TV and film stuff now the lighting setup is pretty simple it's just one key light and the lightsaber this one is just there to accentuate the blue light that wraps around me in this shot so it just goes on top of the TV now I'm using DJI rs2 with the motion control feature because I have no friends and I have to do it by myself so at this point I just need to program the motion track hit record and now it's time for some zero g acting oh boy look at him go look at me being all floaty believe it or not the thing that actually sells your virtual production is not your background it's your foreground it's stuff that belongs in that world that exists in this world in case of the Mandalorian it's stuff like the rock props so I needed to come up with something that belongs over there that I can put over here which is how the cup came to be I took a vector mcrn logo I found on Reddit I put it in this Cricket machine I have for another project and then after some brief arts and crafts class I got myself a martian Navy cup that I can suspend from a string from a sewing kit that I stole from my hotel once high-end visual effects over here unfortunately I couldn't film the camera with the action because I don't have anyone to move it to match the movements of the world like what I'm doing see here when I filmed it with myself in the shot I can move to match the movement of the world as the camera moves but the cup remains stationary which kind of looks out of place also this wasn't a particularly easy setup because I don't have SE stands I would like to State for the record that this is stupid if you're going to try something like this please for the love of God buy a SE stand so I green screened it three tracked the scene and put it there with some light lens blur now obviously this setup has plenty of limitations and not all of us unfortunately have ilm budgets but for what I'm trying to do with this this is honestly pretty good it about doubles my range of motion from TV's edges at about a meter and a half away and if you're using it for something like product shots this honestly works really well if you want to do more character stuff where you need more Headroom that sort of thing you might want to invest in one of these short throw project they can go up to about 150 in and that should get you covered all right that's it for now if you found this video useful make sure to hit that subscribe button make sure to like that hit button stay tuned there's a lot more cool stuff coming to this channel so you know subscribe or I will be forced to intervene

Channel: Max Rovensky

Views: 90,132

Keywords: unreal engine, virtual production, iPhone, live link, vcam, livelink, live link vcam, unreal virtual production, game engine, filmmaking, the mandalorian, mandalorian, vfx, icvfx, unreal engine virtual production, projector, green screen, game engine filmmaking, unreal filmmaking, unreal engine vfx, unreal environment, unreal engine tutorial, tutorial, after effects, htc vive, htc mars, vive mars, unreal engine filmmaking, unreal vfx

Id: 6PYYvBORXvI

Length: 11min 16sec (676 seconds)

Published: Sun Dec 03 2023