How To Use Unreal Engine 5.3 To Map Your Scene/Room Using The Meta Quest 3 for Mixed Reality Gaming

hey everyone so welcome back in this video we're going to be doing some more pass through stuff except this time we're going to be mapping our room so in the headset it has a scene detection uh setup and we're going to use that to be able to map our actual physical room to our environment inside of our headset so we can do stuff later on down the line um what we want to do first though is make sure in your headset that in your settings that we have physical space and we've created a new uh space setup so space setup I won't be showing how to do it through the app in this one that'll be over on patreon there'll be a video showing how we can actually blueprint this to tell the user to set it up but um for now we're just going to make sure that you've got it set up so space is all done and then be able to map the room pretty sure mine's done so we're going to just leave it as it is so we can Jump Right In and then the next thing that we're going to do is hop straight over to Unreal while that is connecting up I have gone ahead and set up pass through for my level um and my project so I don't have to restart stuff but we are going to create a new level and it's just going to be an empty level in here we're going to need a player start so player start and if you want to set up pass through by the way I do have a video link in the description and on the channel so you'll be able to access that and see how to do all this cool so we've got our player start game mode override which going to use a default character um details for the default game mode Auto receive input we want to set that to player zero and then we're going to add a sky there's a few things we got to set up in this so we'll go through it so for the sky I'm going to set Source type to specify Cube map and select just a default ambient Cube map and then we can drop in a cube so with that in the scene it will will allow us to actually see what's going on see if it will actually let me do this so we can see pass through and there should be a cube somewhere if I'm actually in pass through so yeah so we got that there's going to be a cube there it is it's a bit lower down so high cube with that we've got pass through up and running in our new level and I need to make sure my project settings that level is set as the default because we are going to have to build the APK to the headset to test scene I can't make it work through play editor so we will have to build it to the headset which sucks but that's kind of the case of where we're at so save all VR template maps and then call this pass through doing this just cuz it's a bit faster than having to delete everything in the other level and it makes sure that we're all in the same Place cool so pass through I spell that one right pass through so now if we restart the project it'll be in the correct place and it'll load up properly cool let's Center that Cube and just move our spawn point back not too sure whereabouts will spawn but it's what we want so for scene detection what you need to do is go to Project settings and then with oculus plug-in So Meta XR plugin enabled and the open XR XI tracker and XR hand tracker disabled we want to go to our project settings find The Meta XR we want to make sure that anchor support and scene support is enabled if you've set a pass through that's great you'll have all that set up as well check the video in the description but we need scene support enabled and what that'll do is when we build our APK It'll ask our headset for permission to use our scene data which we can then pull in and we're going to pull that in by using a actor and it is called an XR scene actor so we can just go up to here and then if you hover over get content you can actually search for scene actor and you see Oculus XR scene actor we just want to click that it'll be a null object empty there's nothing in it however it will give us access to this scene actor information and what we need to do is we need to set these up to work with our room so these have a whole bunch of default settings um if you have Oculus Source available they are available through that you can download them from there what I'm going to do is I'm going to zip up the folder with all the content in it and I'll put it in the description so you can download that and then just drag it into the project so if you're working from Source or you have Source set up what you can do is open up the Unreal Engine uh Oculus source and then 5.1 and you can find the folders in here and what we need is the materials meeses and textures so I've already got those downloaded and I'll put them in the description to make it easier for all of you so what I'm going to do is take these so you can see here I have Oculus Source installed so I can just access these files from here and rather than migrating them it's going to mess up the references and it will have to be sorted out but I can simply just drag them over so what I'm going to do is I'm going to go to content create a new folder I'm going to call this pass through content so pass through content hit that make sure we are still recording and then with that folder created I'm going to right click showing Explorer and I'm just going to take those files I have so materials meshes and textures and I'm going to drag those into here so when we do that it'll update our actual pass through content folder and in here we'll have materials meshes and our textures so the main thing is the textures folder is absolutely fine however the meshes and the materials folder are now broken because the references they look good but they're bust so if I open up bed material you can see that it says it's errored so what you want to do is you want to go through each material as tedious as it is and set up the textures to match the one that they are so couch is going to be now couch ouch text and then we're going to go through all of those to set those up so once I'm done I will come back from this and then I will show you how to set up the measures also don't forget to click apply cuz I think I forgot there yeah so make sure to hit apply and then go through each one and they don't have a texture like the ceiling then just ignore it but we want to go through each one and make sure they're done so I'll be right back cool so I've been through and I've applied all the materials and textures so or all the textures to the materials so you see them all up here we can go through and close those down but what we want to do is we need to add these to the meshes that they correspond with so while I'm trying to fight these buttons what we're going to do is we're going to go back to our pass through content we want to go to meshes and you can see here it looks like they're already applied but if we select one so bed mesh and open that up you'll see that it just defaults to World material grid so we want to search for bed bed material and you see that we can add that back in and we're going to do that through the whole thing so bed volume is also going to be bed unless the material says volume then just assume that it's going to go on that box but they're all pretty much the same ones so couch mesh and couch volume they're also going to just be the same material couch couch material save and then go through each one of those to match the corresponding material with the actual mesh and and then once that's done we'll come back to it excellent so now that we've been through and we've added in all the materials to our meshes what we want to do is we want to go through our Oculus XR SC actor and we want to place these meshes in the correct places so they pretty much line up with their names and what we want to do as well is make sure active component is set to Oculus XR scene anchor component and do that with all of them so I find opening these from the bottom up works best a little bit faster so we just want to set these up as we go and then we can search wall and then mesh and you can see how we just go through each one of these find them stealing for sealing mesh and then that's not the right one or seiling RH as well so we can kind of just go through this and fill these in um I can just go through each one and show you so you can see here that we have two different ones we got couch mesh and then couch volume couch mesh goes into the top ones which is part of our um spawn scene Anchor Properties and then our volume meshes go into scen volume spawn scene Anchor Properties so volume meshes go on the bottom and then the other ones go at the top so couch couch mesh and then you can see I would just do this all the way through but make sure they are in the right on so meshes on the top volume on the bottom and then I'll be right back so I just got two invisible wall mesh for this one I just do wall mesh cuz it's not wall art so just thought I point that out and then we got other mesh and then now we're on to volumes and I'll be right back so that's pretty much it once you've done the the setup you could actually build the APK and then test on the headset um what you'll find though is if we hop over to screen share wherever I had the link so you guys can actually see what's going on um I've lost it let's set up Oculus casting again so you can all see what's going on you'll find that when you play it asks for permission to do the scam and then it doesn't actually display but if you quit the app and then load back in it does so I'm going to set this up to show you what happens and what I've done is I've going ahead and already built the file out so I've got an install scene detection then you guys can see what's on my screen let's do wide screen install we're just going to wait for this to finish which it's just done so we're going to hop over and see that I'm in my head set got my room you go to the app the detection load this up will be asked to enable permission so allow this app to access your spatial data we want to say allow however you see with Cube and we've got the my desk is basically mapped out but we can't see any of those meeses so if we quit and then we open it back up now that we've allowed that spatial uh setting or permission we should actually see our entire room with boxes and mapped out so we're not done yet um hopefully you can all see this you see our Cube down there what we're going to do is I'm going to show you to do a wire mesh of the actual room and then a wire mesh of the actual room and then we're going to set up a new level to check for that spatial permission to then open the level so what we're going to do is we're going to quit out of this app let me do that so we keep it open on the screen delete that Cube and what we're going to do is we're actually going to set up that wireframe so we'll do that before doing the permission stuff and what we're going to do is we're going to be able to add and then we're going to search for Oculus and think it's seing Global mesh component yes so Oculus xrc Global mesh component is a child let say Collision on collision profile name could be whichever one we want so block all material we're going to create a new material and this is going to be a wireframe material so material um mcore pass through uncore wireframe open this up and then we're just going to add a default color so holding through on the keyboard and then select a fitting light blue and then if you want to search go to the search bar and search for wire we'll see wire frame so we can select this hit apply and now we can add that material to this pass through and we can make it visible so what that'll do is it'll show us our actual room mesh so you could use that how you want and kind of use it to whatever degree so we can see what's going on so file save all and then then what we can do is we can go to actually make a new map so V template maps and then level we could just duplicate the one that we got delete pass through duplicate um M check commissions SP wrong nobody cares so let's open that up and we're going to remove that xrc acor we don't need it in this level it's just going to start black and this is going to be our main level to begin with so the reason we're doing this is so we can check that permission and then if it's available we can open up the new level and if it's not then we can actually delay it and then we can kick the user out because I haven't give that permission so check permission is going to be our start map and then we're going to be pretty bad we're just going to use the level blueprint for this so onlevel blueprint begin play and I'm going to double check the Corde for this because I've already done it once and I forgot pass through cool so in the level play print we're going to do Android or check Android permission and then we're going to set permission right click actually we'll drag off and do literal name make literal string it's more accurate and we want this literal string to be this so com. Oculus permission. youen what this does is it basically says okay when the level begins check to see of Android permissions which we've given it in Project settings so down here in meta XR scene support if that is enabled if it's if it's enabled then we're we've given it permission to use scene and then we can do a branch and if we don't have that enabled because we might not we can do request Android permissions and then we can do that by make array cool and then chances are it's going to be done but you could have this fire something else to be like okay we've got to request the permission and then maybe restart it again so we could have I'm pretty sure there's a console command for restart but in this case we just want to say if check for Android permission is true we want to open level open level by name and this name is going to be our pass through so my spelling is bad enough I'm just going to copy and past paste it in pass through and then that means that if we have the permission then we can open that level which contains our static measures so if I build this up to the headset now package project we should ideally go from this level or the check permissions level to our pass through level and then we should be able to see our environment hopefully and once that's done we'll be right back and we'll take a look so I test the A and it was still doing the restart thing so what I found is if you use the event begin play you can add a 1 second delay that gives it enough time for the check for the initial popup to happen to check the permission before check Android works and then it kicks in the actual scene stuff so with that done we've got the pass through in check permissions uh pass through level sorry so alcus scene extended we have the child and and then the wireframe so if we build this now and then we add that APK in there what we can do is we can hop over to the headset and we should now see it work as intended so that's currently installing hopefully it will only take a second there we go now we can actually hop over to the headset you guys can see pass through cool so it's looking all good protction back a little bit allow access hit allow and then it boots up and then we can actually drop into the level and we can see our room layout and you see because we added that scene mesh with the wireframe material we can see that around our room room so that is the mesh that the UN like the headset is see so we can use that to our advantage if you want to make the stuff in the scene invisible all you have to do is go back to Unreal and then put the pkal whole material into it so quit and then we can hop back in set the Oculus XR scene Global mesh to Hidden so we can't see it Collision on and then for the actual meshes so all of these we can change their material so now we can actually take these meshes which is tedious but we could go through we could add the pkle material so P whole material and then we can do that for each one and then that means when it shows in the environment we won't be able to see it and things behind it won't appear as well so I'm going to do that and and then I'll be right back so now everything's going to appear invisible rather than jumping into the headset I'm going to bring in the pistol from the project and I'm just going to spawn a couple of these around the player's head the idea is that they'll spawn at least somewhere in the room CU if they don't they'll be outside the mesh and then we'll be able to pick it up and interact with it so let's drop a few of those and because we've got Collision on for the room through the XR scene actor it will actually land on the floor in our room and then what we'll do is we'll come back once we make sure that works and we'll change the projectile to be physics based so we can bounce it off the walls which be super fun so let center of our room a little bit find our application we start allow where we can see all the guns on the room so they're a little bit jittery but it's because I've got too many skelet meshes in the room oh there we go it's kind of bounced out but you can see how one's falling down the back of the sofa and because we're using that collusion mesh it's actually hidden so now I can actually shoot around the room and we can see basically it hides the bullets where they're not where they pass that occlusion mesh so let's say we want to have physics we can jump back in and we can set that up cool so we'll delete a couple of guns as well I know I keep going the video is getting really long but this is the fun stuff that I wanted to show off and everyone over at patreon makes it possible for me to do this because this video is taken a couple of hours maybe an hour and a half with editing just because I keep breaking things I want to make sure I've got it right for you guys cool so so let's see if that works just have them directly above our head so we got two guns so what we want to do is we want to go to projectile um we want to disable a simulating physics which is going to set that to true we're not going to destroy the actor on projectile stop we're going to disable that so they no longer Collide static mesh we want start to move able and simulate physics Collision preset block all projectile movement set this up and initial speed I think is something like 500 I have I might have to double check this and while we're in here what we're going to do is we're going to set our projectile so instead of the sphere Collision it's going to be our static mesh we're going to delete the sphere Collision yeah we don't really need that we just want to see it bounce around staic mesh should be a good size pistol hopefully that should scale one okay that's going to set it to be crazy big let's set that to 0. 1.1.1 I think projectile should be .1 I forgot that inside of our pistol spawn active projectile always spawn ignore collisions and then we want to move our muzzle location forward so it doesn't interact with the barrel I'm going to put it about there should be cool and then we're going to test this you can kind of just do what you want to be for at this point we're pretty much done Collision is going to work things are bouncing around we've seen the the guns on the floor I just want to see the guns shoot and then bounce off the walls so we're going to get that set up then we'll call it while this is doing it big thank you to everyone over on patreon without you guys I won't be able to put the time into this to be able to figure everything out build the tutorials make it all possible for you guys to follow along but um yeah thank you so much and then hopefully this won't take too long be right back that said the new Mega thank you tier gives you early access to videos now and then we're going to try to do some more stuff over on there for you guys so maybe some live streams we kind of still planning it and then I'm going to try and just do a lot more for you all which will be super fun and it'll be really nice to do I think so that's finished we can go to install our APK hopefully hopefully the bullets bounce around the room and they are the correct scale I've got a feeling it might not be something's going to break it always does these things are a nightmare cool so we're back in got the APK up and running let's go to scene detect all right guys doing a voice over now cuz I realized I walked away from the microphone so I'm setting up the permissions that it asks for in the first part and then it loads in and because we've set the meshes to be be transparent with a pork hole material we can't actually see them but you can see that shooting around the room and using the pistols the physics can actually now bounce off the surfaces and then they would roll around the room so that really is it for this we're just kind of playing around making sure the APK can build and that we've got these the the projectiles interacting with the scene in the world things so anything with that wireframe mesh and then the cubes we're going to interact with so this is how we set up collisions room scanning room detection weapons bullet bouncing and everything else we've covered so we've done quite a bit in this one sorry it took so long but hopefully you guys enjoyed it and learned something new in the next video what we'll do is we'll make the weapon spawn on top of the desk so we can read actors in the scene so the sare the TV and we can make things spawn on top of those that'll be over on patreon through the mega thank you TI early so if you want to check that out make sure to head over there so once it's up and loaded you'll be able to check it out as soon as possible cool um for everything else all the files will be up on patreon as well make sure to join the Discord if you have not already there's a load of people doing stuff like this which is pretty cool and you can see the behind the scenes and what's going on hopefully so cool that's it I'm going to call it there cuz the video is now half hour long so until next time stay safe and I'll see you then bye everyone that

Channel: GDXR

Views: 17,377

Keywords: Gamedev, Indie, Indie Gamedev, Indie game Devlog, Game Devlog, Game, Dev, XR, GameDevXR, Blueprints, Coding, Unreal, Engine, VirtualReality, VR, ExtendedReality, tutorial, how to, how, To, develop games, making a game, Dev Log, beginer, Unreal Engine 4, archviz, architecture tutorial, beginner tutorial, basics, blueprint tutorial, blender, beginner tutorial series, coding, devlog, Jonathan, Bardwell, code, unreal engine 5, passthrough, scene, mixed, reality, game, Quest, Quest 3, UEVR, bardwell, log, development

Id: rgOi_RjmMJs

Length: 28min 13sec (1693 seconds)

Published: Mon Dec 04 2023