Getting Started with VR/AR on Windows | The Blazor Power Hour

welcome to a special edition of the Blazer Power Hour usually we talk mainly about Blazer and Blazer only on this channel uh today's episode is going to talk about the tools that are going to get us to working on Blazer uh from an interesting perspective that you might not have thought of before we're going to do some Blazer development with augmented reality maybe even publish our app to an augmented reality platform form so I thought this would be a fun experiment to have on the show uh but today we're going to focus on what tools we have to do that because it's um kind of a a lot of context that revolves around all of this and if I don't give you that context before we start having shows about the device there's going to be a lot of questions um which I know you have a lot of questions for today and that's what this episode is about it's answering the questions that you might have about this guy right here the quest 3 how you can use it for development whether it's uh a good device for your daily desktop window uh what do you call monitor replacement um all that type of thing that's that's actually one of the questions I get most often about my VR headset is um can I use it as a monitor replacement for my daily coding activities and all of that stuff and um it's a mixed bag it's maybe not as uh clear and concise as you'd hope it would be it's not a solid yes or and no um a lot of it just depends on your habits that you have around your work activities um so the the long answer will get to the short answer there's a lot of awkward things that happen when you're trying to use this thing as a monitor for example I drink coffee in the morning or any hot beverage the the short of it is it's not a 100% replacement you can spend some good time in here doing some Dev though uh so let's talk a little bit about how we get there about the device and all of that um if you're if you have a quest three or any of the any VR platform chime in in chat just let me know what you have out there I'm kind of curious see if people are using these devices like I am um I'm particularly interested in them I think augmented realities one of the coolest Tech technologies that we have going and possibly uh a strong future of development itself um and you know devices themselves like I can't see uh the phone being the the go-to device forever I just don't so um quick disclaimer also I do not work for meta I work for Progress software uh although I am a fan of meta in the the device that they make um I don't work for them it just happens to be one of the most accessible devices out there um from a price perspective and per performance perspective so it's most bang for your buck device right now uh so this is the one that I have that you'll see me working with on the stream today and um in the near future um I've got the quest 3 Model I opted in for the 512 uh gigabyte because I play a lot of games on mine as well um and I didn't know for development purposes you know if I'm going to be deploying a bunch of sample apps to this thing if I need extra space but your mileage may vary um there's a lower end um memory version of it uh it's still same capabilities just doesn't have all of the storage uh that the other one has so for about I think it's another 100 bucks or so you get the extra storage one of the things I'll note about just just the hardware in general here um if you are considering buying something like this uh one thing meta is not great at is Comfort the original head strap which I'm trying to see if I have it sitting around my desk I don't uh the original headset is kind of blah not great it's really not great and you're going to want to invest in a third party or a first party um comfort strap this one is very uncomfortable especially given the the topic today which is using this for uh long-term Dev purposes uh the included head strap is a nogo the upgraded strap is more like the hollow lens uh 2 type of a strap where I'll try get this in front of the camera it's got the uh ratcheting knob in the back uh fits snug around the back of your head here um and then you just kind of Ratchet it in and it's got the uh velcro top on it there battery lasts about an hour and a half to 2 hours depending on what you're doing some people say even less uh the mixed reality mode eats up battery like crazy it does get a little hot on your face as well but the battery in general uh or the device in general consumes a lot of power okay this thing eats power um it does so so much that the included uh USB cable and adapter that come with it cannot power the headset and charge it at the same time for this headset I don't know what that time span would be ought to estimate maybe four or five hours of uh on time while plugged in it's probably still going to die okay so you would need to take breaks which I'd recommend anyway that's kind of the hardware specs if anybody has any questions pop them into chat I'll try to answer those um and then we'll we'll jump over to talking about some of the software involved the first thing I want to point to is the meta quest Quest developer Hub so if you have a quest and you want to do any kind of development for it make sure you go to the quest developer hub download the app that you find there so this is both for Windows and Mac I have no experience with the Mac version I would assume they're the same um but I can't vouch for that uh it does give you you know the outline of what's available here manage devices monitor performance record and cast which is something we're going to try to do today hopefully it works um I had issues with this on a prior um webinar where it wouldn't start up because of my camera um but now I know what's causing it so hopefully uh we can do that today drag and drop on uh to drop your builds you can drop apks kind of s side load things uh with this app and you know so on and so forth so actually have that tool I can open it up and reshare my screen let's do the entire screen this time and this is the uh meta Quest developer uh Hub application and uh you kind of get like a little news dashboard here but the more important thing is up at the top here you've got device manager see if I can zoom I don't have my zoom tool up I know it's probably hard to see on stream there we go uh we've got our device manager which is what we'll spend a lot of time in here on the show uh performance analyzer I haven't tried this out yet because I'm not really taxing this for for any reason uh file manager is really nice just uh gives you access to the file system on here um and they do have code samples and some other things one thing that's pretty good so far is the documentation uh from meta has been quite good and I can't get out of my zoom [Music] tool there we go um the documentation has been quite good um and they have varying um degrees of documentation so there's things that we'll talk about later more in detail but you can do immersive applications obviously like VR and AR um you can also do things like um Progressive web applications um and you can use things like web XR and they have that all splited uh like a dropdown that you can choose and jump between their documentation I'm going to try to fix this display real quick you'll notice the resolution pop in and out when I start using the device we'll talk about that as well but just want to give you a heads up the screen's going to kind of Bop around a bit I'm going to try to set this at 1920 by 1080 and see if it'll stay at least close to that while we're using uh the device uh so download that install it you get the nice dashboard here in dark mode fuel enable notice that we we do have dark mode here uh this will let us manage our device and this is where I was having an issue with my camera plugged in and let's see if we've got the quest 3 is active looks like it's finding it this time it kept finding my camera last time which was really awkward uh because I was trying to do stuff with my camera that only The Meta Quest can do so I was trying to access proximity sensors and things and it and my camera is like what are you doing and then the app started blowing up and crashing my system so um we're we're good today for some reason all right uh so this is uh something that's really helpful down here is our device actions uh something you'll see me use while we're doing development stuff uh we can uh do casting so you'll be able to see from my device uh let's actually see if this is working right now and this is where I was having an issue before yeah this is something to do with my camera I'll have to turn my camera off possibly to get that to work um apparently if you have other Android devices attached to your system they will interfere with this so just a little heads up there I haven't figured out how to get that to work together yet um so I'll just I'll switch to a different camera that does doesn't have an Android operating system um it's a little grainier you don't need to see my face anyway um so you can also record videos do screen captures uh you can um request the meta Quest browser to open a URL which is awesome uh so that's in there as well um these are super helpful here if you're doing Dev uh you can turn your proximity sensor off and Boundary detection off I do not recommend that if you're gaming in VR you will hit the wall no matter how good you think you are you will run into something don't ask me how I know um various other um test Utilities in here as well you will notice there's also some sideloaded apps that I have here one of them is a Blazer app another Maui app here so we do have the ability to publish some net stuff to this headset using net Maui we'll talk about that in another stream um again I haven't used the performance analyzer and it doesn't look like I'm properly connecting because of my camera again but uh we'll take a look at that as uh we do other shows um then we have our device manager here which is really great uh for recording videos from the device and uh sharing them on social media you may have seen this one I posted on uh LinkedIn um of me working in augmented reality we'll do a little bit of that later as well uh and that's that's about it for the developer Hub okay so this is where you're going to manage your device uh do screen recordings captures um turn developer options on and off uh it's it's a nice dashboard uh for debugging and that sort of thing okay all right uh and again questions in chat are welcome so please post them if you have them uh next on my list something that uh is pretty invaluable is the uh the remote desktop software for the uh PC or Mac desktop um what this is going to do is allow you to access your uh develop v m computer from it doesn't need to be a Dev box but we're all developers here I assume um this going to let you access APC or Mac uh in my case it's my development system um from the quest itself uh and this is where we get into the question of can I use this as a monitor um yes you can is it going to be something you're in more than an hour or two a day that's questionable does it have good enough re resolution that's a question I get a lot does it have good enough resolution for me to code in Visual Studio yes it does it's not a problem the resolution is not a problem on this thing that is uh something it does very good um whether you personally can sit with something heavy on your face for hours at a time um and drink through straw that is the question that you need to ask yourself so uh my my long answer is always no this is not a a computer monitor replacement um is it interesting to use as a uh Dev environment yes um is it easy to work with if you're doing VR development or ar development absolutely uh but if you're just sitting there coding and reading all day I don't know I really I have High Hopes but uh it's a little rough all right so you'll need to download the app to your desktop it's just a little VPN style thing um it looks like this and uh for some reason it says no device available let's try plugging it back in again it should actually wake up and it will say ready on it um this is a really kind of a weird UI for one um because and again it might be because of my camera but it's funny because you have to put the device on your face before the proximity sensor will light this up and tell you that it is ready uh so you don't really see that it's ready ever and as soon as you take it off it the proximity sensor kicks in and it says no devices available so it's kind of weird but uh you can as I have done place a small piece of tape over your proximity sensor and uh that will help alleviate things you can also turn that proximity sensor on and off I believe from the device manager here and you can see I've disconnected so we'll we'll have to reconnect that device here in a minute and I will show you uh from VR how that works this is a very quick and easy thing to set up uh and again allows you to connect to see your desktop monitor I think it only does one um one monitor only your primary um it doesn't let you interact with the with the monitor at all from the VR headset you can't use the controllers any of that stuff to interact with your PC you'll still need your mouse and keyboard with this app okay uh so you would connect to it and then go back to your keyboard and mouse and work um um if you wanted to stay in VR and kind of tool around with things um there is a thirdparty tool called virtual desktop that you can buy uh on the quest store it's about $20 it's well worth it even if you're using I use a a combination of some of these things uh but virtual desktop has a lot of extra features that you might enjoy if you're watching movies playing games things like that so it's worth the $20 but it will connect and you can actually navigate your PC with this using the quest controllers um you know open apps uh click on you know buttons and files and things like that um wouldn't really recommend typing in it or anything but uh you can at least Mouse around from inside the the VR headset without switching to a physical Mouse attached to a computer uh which is nice if you're not standing in front of the computer right take this uh to a couch or something thing you want to open some video files to watch movies things like that it's super helpful um or play uh some games uh PC VR games um this one also I believe only lets you have one um desktop window open um so as far as I can tell and I've used it quite a while I've only been able to have one window open at a time if you want multi- Windows if you're looking at doing um you know you want like this infinite number of virtual uh desktop monitors um to open up from your PC uh Aztec Consulting is one tab ahead of me here and there is a mer so m is another option this one lets you add as many virtual monitors I think as your PC can handle um I haven't I you know I've added a couple I think think up to like four just to try it out and I think you need to have a um uh subscription there is a subscription model to this one uh so virtual desktop is $20 immersed is a subscription model there's a a co-working space a private space um the multiple monitor set up okay spawn up to five virtual monitors it says without any additional Hardware um the the remote collaboration stuff um I I forgot to mention meta has that as well um with uh this guy here the let's see what it's what is it called oh Horizon workrooms so something similar to that screen sharing whiteboarding uh so you can have kind of that remote office kind of a feel uh with this app but Mo most of the time I see people wanting it for multiple monitors we probably won't be using that much um on the show but I just want to mention it's there in case you wanted to try that desktop monitor replacement type of a thing progress software company that I work for uh we actually have VR AR tools uh they are for Unity so if you're considering any U Unity development um which Unity uh they going through some changes right now but uh with you know that aside they are kind of the goto um engine 3D engine for any mixed reality uh if you're on the Apple side of the fence they want to call this spatial Computing now um any of the XR stuff so if you're going crossplatform and you're doing 3D development uh you know immersive applications then Unity is a good way to go and we have some nice tooling for Unity people know us for our UI components and we have some UI type things that you can use in these immersive applications one of them is the teler XR motions and this is like if you've ever played a VR game where you need to navigate your environment so you kind of have your um uh your stick and you press in on the joystick and it kind of shoots a beam with like a circle of where you want to teleport to and then when you you know let go or press a button you you know jump to that location um that is what this does so you get like that uh snap movement stuff um smooth Locomotion teleport and curves that type of thing and then we have like interaction uh things controller tool tool tips um so this is all based around like you have a controller or or you have your hand tracking and you want to have some kind of menu system or interactivity with your hands or your controllers um these are functionality to uh kind of build up those kind of UI systems or interactive systems um with hand tracking and controllers and then finally there's some UI stuff we're going to be working mainly on what VR nerds like me like to call Flat apps uh so the traditional um web app type of uh you know Blazer applications inside of VR we're not going to be jumping into 3D yet uh might be something we do later on um but what I want to do with the rest of the show today is just talk a little bit um about our options when we work with these tools I'm going to try switching to another camera oh there we go like I said it's a little washed out it's at an odd angle too but not that you need to see me that way well all right so this is connected now and then hopefully this is coming through on stream it's a little hard for me to see uh how detailed this dashboard is um but what I'm focused in on here is these device actions we're going to try to hit cast and see if I can share my screen from my VR headset so inside of my headset I'm just getting getting a quick check here to make sure that uh developer mode is enabled casting and streaming at the same time has become a little bit of an issue so trying to get my uh debugger um there's a prompt a challenge that comes up actually the best thing to do right now might be to reset the headset and it's like a typical Android device um you get you get a prompt inside of the device it has the device ID in it and then it says would you like to uh connect to this device for debugs in you have to like check a box and then hit okay so um it got locked up on that screen and now when I'm plugging it in it's not prompting me for it and what's odd is I get the dashboard here but uh for some reason it won't get pasted it won't let me cast there are other ways to cast I might be able to try uh let's see here see if it'll let me do it now um one thing is uh waiting for connection there we go it's a little Hollow mirrors but we did it we got the casting working and you should be able to see on the stream now what uh my view looks like from my headsets you can see kind of my office area here um you can see the the lovely I don't know if you can hear me well facing this direction we got lovely gray weather outside um the overall video quality of the headset probably uh looks better on stream here than it does in the actual device so uh one thing that uh you might notice from videos um like on YouTube or social media is the video quality in AR looks phenomenal um however inside the actual headset itself that video is stretched quite wide because it is about 180° projection um around you uh it looks really great on a small screen like many of you are looking at now um however in in the headset it's grainy um you can make out uh faces of friends and things but you cannot make out um their facial features and things like that so uh be aware that that uh you know what you see is not necessarily an accurate representation of um the device so uh we can cast from it now so that that's casting so we can just hit this button here and uh enable casting from the desktop and that will let me share my video with you now if I want to uh connect um you have your eyes on the xreal Air Pro 2 give me one second my microphone I bumped it when I was moving my head there and knocked it down there we go apologize that back up um I who makes X real what's the company behind that one smab is it is xre the company or is that I I need to be more familiar with other devices I just mainly use this one right now and haven't looked back for a while to see what's out there um so we've got our cast here and inside the headset let's see you yeah there are no menus open it does hand tracking so you can see my hand here and notice there is a small ball inside of it that's going to be my menu so it has similar gestures to the um the hollow lens so if I hold my hand up and I pinch my fingers together um oh xreal has the the glasses one okay um you can you can do various things here um it's just like hitting the menu button on the remote uh one thing that's nice that does voice commands um so it has like a Siri type of thing built in um so we have our menu there's our menu system so there's our real desktop monitors there's a virtual augmented reality monitor there or um I guess inside the OS itself um this is actually easier to use with controllers by the way it's not that hand tracking is bad it's just sitting in front of a computer at a desk you're going to run your hand hands into the desk trying to do this you get a lot more accuracy with um the the controller which is something that makes me wonder why Apple's choosing to not ship a controller with their VR headset um even if hand tracking is great when there are real world things in front of you you're going to bump into them trying to manipulate these screens so it's often nice to just have something that's on your wrist um that you can aim very easily with and uh not have to gesture and be accurate with uh so uh we can use this bar here to move the uh screen uh closer further away change the orientation um there's some small buttons here we can tap um and then it's more like a tablet mode so you can pick this guy up and move it around um kind of like a tablet uh then this is the projected mode uh where you get a nice big screen and this is a very large screen in person and you can change the side of it size of it as well and what usually ends up happening is this is going to be in front of you because you're using your keyboard and now um you could see your monitor through it it does have a decent amount of resolution that you can read the monitor through the the headset like this um but what I want to do is actually mirror my monitor so I'm going to go to my apps uh down here and I have that remote desktop display that I talked about earlier and you'll see I have a couple computers in my house that are hooked up to this and now I can click on that there I'm going to go ahead and put the headset on it's actually kind of heavy to hold and there we go so now I have a giant monitor um we're going to get this hollow of mirrors because that's the only way to make this work um and and cast this right now so uh this is um my monitor again I can't interact with it not with the uh joystick or the the the controllers that come with the quest but I can Mouse around um this this monitor this big virtual Monitor and it's pretty large I couldn't tell you how big that is I'd say around 50 in or something and it's uh kind of a square aspect ratio um let's see if we can actually see what the resolution being output is um let's try this here and uh sorry I'm trying to see my stream there we go so you can see my desktop uh I'm looking at the desktop through VR right now and what I really want to do is go to display settings and see what it's outputting here uh it's 1,600 by 1024 that's the resolution uh that I'm getting uh when I'm connecting the remote desktop so it's kind of an odd ratio and I believe that's per eye what what we' get but when we're streaming um a desktop like this is a 2d app here you're only going you're getting the one [Music] um the one eye out of it uh so we've got this nice big monitor you can see it's it's almost as big as the two monitors put together uh so we've got that and then uh we can also grab this monitor um and change the size of it as well so we could make it wider but you'll notice there's black bars on it it so it doesn't actually change that that one fixed resolution there and we can also take the monitor uh at the top here and we can scoot it over and now we've got space for something else so we can come into the app and open up a browser so now we've got a web browser um and and we've got the uh we've got the remote desktop side by side and I'm trying to figure out what's the best way for you guys to see this it's kind of awkward to do on a live stream um so I don't know what kind of hir tricker we've got going on but uh you can see I've got both a desktop and a browser window open open okay so um this is kind of how you can do some development you can open Visual Studio here uh work on things just fine while you view documentation in your browser um you can also um you can use this if you want to have you know like video You Know YouTube going whatever social media you've got you've got access to that as well and then again you can resize um the application here you want to go to like a mobile mode or whatever that is uh you could do that as well and you could also have the app you're working on side by side here and you can do up to three windows with this too um trying to remember if we can do yeah we can snap a tab off uh we can have three of these open at one time I think that is the max uh for this mode go ahead and take the casting away for a minute uh let's see uh no questions so far um and I have to watch the stream back later and see how this actually projects out it's a little hard to wrap your head around when you're doing it live um so that that is one way you can get uh some workflow going and the resolution you saw was pretty good 1,600 by 1024 it's not bad at all uh the text is easy to read um you could sit in there and code and see your docs in the um the quest browser and you can always use the dev tools here to um uh type into that browser um Ur long URLs and things like that paste into it um and then open them um as well so you could take a long URL like this you can cop copy and then you can come back here um and you can say paste and you just say open and this will actually tell the VR headset to open that page in the browser uh so you can kind of remote control with this as well some of the things that the headset's doing um let's see here so that's the general gist of uh the workflow um I also o have let's see if this will open up I could try to launch a side loaded app on the headset let's go ahead and cast that and see if these things are open in the the headset for us uh it's looking for a boundary for some reason not quite sure why it wants a boundary uh let's see we can confirm that boundary continue and pass through there we go and you'll notice I have AET Blazer application open Okay so we've got AET Blazer app here um and we've got the hello world Blazer app so this is the um typical hello world Blazer app that you get with a file new project this happens to be a Maui application so we can build net Maui apps deploy them to the quest and they generally just work so far I've I've only experimented um but we'll do some experiments live on the show soon see what kind of apis we can actually use with it maybe see if there's camera access uh stuff like that I don't know what it will do to be honest I don't know what camera access and file system access and all of that will work um like if we'll get errors or if we'll get enumerated cameras because there's a couple on here I don't know what that's going to look like uh so we'll we'll play around with it we'll play around with getting that working uh but it's really an interesting thing because we can let's see if we can get this going um I don't have Visual Studio up and running at the moment but what we'll do is we'll bring over um our Google BR or uh Chrome browser has to be Chrome by the way this does not work in edge for some reason um but we'll bring Edge over or Chrome over and then uh let's go in here and say devices or Chrome inspect and that app I think had set up with debugging and there's uh there is my web view for my blazer Maui application right so now if I hit inspect on that I have my Maui application that is running in my quest headset all right and I'm able to interact with it um see do I need to interact with it I can inspect it and debug it let see if I can interact with it in the headset it looks like it's frozen so uh that's not not a great demo oh there it goes there was some lag there all right so I'm able to interact with it in the browser and you can see it updating um on the debugging tools uh for some reason I can't click on it from here it doesn't seem to want to work like that but what this is nice tab inactive the sensor must have got kicked off this is not a foolproof experience uh as you can see I'm seeing nothing but the real world through this right now so uh mileage may vary with some of these things things but it is neat that you can uh get in there and debug um what's nice about this debugging experience is your web debugger uh so you can go in and Traverse the Dom elements uh it's still able to pull something up in here um so you can pull up your Dom elements ins things see console messages and stuff like that so you can use your debugging tools uh to connect and um work on applications in in the VR headset uh let's see if we can terminate that uh there it stopped casting and maybe I'll be able to get my device back to normal um in just a minute so uh that's kind of a good summary of all the tooling uh that we have we've got the quest developer Hub um again you saw me casting with that uh we could do recording captures uh talk to the browser with it you can run ADB Commands if you're familiar with the Android uh development commands send those to the device um I probably should have had my boundary off I think that's part of the issue with um uh the device acting the way it is um and then you can side load and run and terminate apps as well and then if you want to add new things you can just grab APK files directly in here um it will also connect through Visual Studio which I think we should do on the next show so we'll we'll get a um file new project up and running and uh launch that into the headset uh do a little bit more debugging and then maybe I can work out some of the Kinks of trying to cast um The Experience uh to a live stream because uh again it's kind of kind of awkward I don't I can't see what you're seeing uh while I'm in the device working so I don't know what your experience is like I have to go back and watch a stream and see um and then we'll Circle back on a couple things that that we talked about earlier as I was doing this the device has been plugged in okay remember I mentioned the battery uh the device uses a lot of juice uh the battery cannot keep up even if the device is plugged in we've been plugged in here for an hour uh the device is being uh it's I think it's been on most of that time so we'll say it's been on one hour while it's been plugged in and we've lost 133% battery so you would expect maybe that if you were plugged in um and you started at 100% an hour later you should still be at 100% not the case so keep that in mind even when you're plugged in you can run uh out of power it's not using the recommended power adapter by the way I am plugged into the PC um so I have uh a nice sharp casting stream without bogging down my um Network because that could affect the live stream here so there is that um you may get a little bit better experience if you're plugged in um to the actual Outlet uh Aztec said he's been on six hours that's a long time in VR um on the quest 3 Aztec or the quest two I think you said you had both uh that's that's quite a lot of time to have this on on your head I get a little sensitive um around the face to it after a while wow that's impressive yeah it gets it gets a little headachy after that for me um yeah I don't get the motion sickness by the way a lot of people run into that issue um that hasn't been a problem for me thank goodness um very few things cause motion sickness for me in in the headset um that is something that's going to vary quite a bit from person to person so uh for me if it is a very Bland cartoony uh experience with uh very like plain Graphics which there there's a lot of stuff out there like that like wck room um what is the one that Microsoft has that similar alt space VR uh rec room alt space VR those type of things um these cause motion sickness in me for some reason uh so if we uh v r uh so stuff with like this type of Graphics here it's like very vibrant like uh cartoony colors and basic shapes something about that makes me incredibly queasy and quickly um alt space VR does the same now to be fair I haven't tried them on the quest 3 yet but um that's actually way better than what it looks like uh I haven't tried them in a while maybe the graphics get a little bit better something about the movement in the though always threw me off so some of those things do uh games generally do not um things that have a good amount of realism realism to them do not uh but um it does get uncomfortable after a couple hours on your face does dig into the forehead and stuff a little bit so it's uh again if you're looking at it for just doing development full-time some things to consider before you buy one if you're not interested in gaming if you do not game on this thing or enjoy any of the other experiences on it um there there's some good like sporting packages and things that you can get into to watch like live sports television stream video if you don't plan on doing anything but going into Visual Studio on this I wouldn't recommend buying one if you're into any of the other stuff plus you want to get into Visual Studio or vs code or whatever um highly recommend it so like it's very very little gray area there uh it's you know not the cheapest device out there uh 500 virtual travel I have to check that out what what type of is that just a category genre virtual travel or is there an actual app virtual travel app Aztec um yeah so and there's little little gray area there like um it's $500 entry fee for this thing it's nowhere near the cost of Apple's crazy expensive device that doesn't seem to do a fraction of what this one does um but for that much money you could have a really nice monitor if that's all you're looking for uh if you're you're looking for a device that does many things and and you want to try those out then it's perfect for that uh highly recommend it if you're you're going to experiment with it um any other questions before we end the show today I'd like to take another show to do more uh structured demos I guess and experiments um in the device itself making some um different types of apps for it uh and again they going to be 2D apps to start with we might get into some VR stuff eventually uh it's kind of a different realm of thinking though um different Frameworks are involved probably Unity something like that um I've tried um I've tried some webxr stuff does not mix well with blazer uh lots of video some apps like blink and wander and Traveler I'll have to check those out um I've seen uh some cool games that are similar so they wouldn't be real places but they're more um like getting off topic here but it's still uh in the realm go to steam there is like a virtual kaying one that I thought was interesting store home I spell it wrong kayak uh thought it was there it is Mirage um this one has phenomenal graphics and I could see like doing this for like a virtual like vacation type of thing like just to take a 15 minute break or something from work and just chill out and like go into some of these environments might be cool um yeah but on on some future shows here we'll we'll do some Blazer development in Maui we'll do some Blazer development uh with pwas um we'll try to see what we can get working with web XR but the thing that I've seen with web XR that is a problem it doesn't mesh well with um other like 2D framework Tech so we can even try let's see it's 3js there's some really great Frameworks out there it's not the framework problem really could be I don't know uh if you look at let's see trying to remember where to find these there are some webxr samples I mean they're really cool samples but what happens is let's see if we can find uh might need my emulator up inspect uh web XR this is a web XR emulator that you can get by the way um trying to remember the name of it I'll have to look it up in my plugins plugins web XR API emulator um this is something that something that you can search for I think you get out of the Chrome Store and uh install it it's not perfect but um it will let you get into some of these I think you have to refresh the page once you open it and then you can open the demos any live demo that you see this says VR not supported uh I might have to try it on Chrome even though the plugin is working showing a UI there may actually need to do it in Chrome uh threejs a cones this is the one I was looking for AR not supported all right let's try Edge again cones a are not supported come on let's do inspect again uh the tooling from the web perspective is not quite there yet for VR and AR um it's getting there there are things that work and things that don't work um I think you might have to have a certain device um selected there it is so you have to have like a smartphone device I think and it see this is behind this is the quest go an Oculus Quest the browser is not detecting it um and this is like a very simple emulation uh Samsung Gear VR are not supported so the these are all older devices um and I don't know what the status of this web XR plugin is if it's under current development or not but if you do Samsung Galaxy S8 plus with ar you can get these demos to work in your browser um we'll look up where to find this in a minute uh I can't remember where I installed it from but basically it's it's kind of a a dumb emulation where you can move a phone around the environment and look through it and then you can kind of spin it around um in this demo you're placing these virtual cones so it's drawing a 3D model in this space and you can see the cone with the you know through the viewfinder of the phone and if I can remember how to move this side by side side to side there there we go it's kind of clunky I'd love to see like a a quest emulation a proper one but you can see the the cone there um if I could center this better there we go so we can move around the cone you can see it there um it's kind of should be in the space right here but you can only see it through the AR device right so the that's the demo you click on the screen and it places 3D objects in a uh in inside of your space the problem with this and Blazer when I start this experience um you have this start AR button this is part of 3js this button uh it's being rendered with a canvas um and basically there's there's just no room to interrupt with 3js and uh Blazer there's just not kind of an a interop barrier here that other than show this button using JavaScript click the button and now you're in a JavaScript world and you're completely outside of net and you don't you're not going to see net again until you end your AR experience so hopefully that made sense like uh I would be inside of JavaScript programming now to do anything here in this mode and there's no real intelligent way to interrupt with Blazer and then you know other than on this screen before I press this button I could have Blazer dashboards or whatever and then say start AR then I have a complete AR experience and then jump back um even customizing this button is a pain I I guess that's kind of 3 js's fault uh there are other Frameworks as well um the only thing I could see being kind of useful here is uh something like Amazon has where you put a virtual device in the room with you um or virtual product I mean and try to like place it and see uh so you might have like a shopping experience on this black screen and on that product page you might have a button that you click to see that product in your room um then you go to this EXP experience you place the product in your room you look at it and then you go back to uh the the shopping screen that's the only thing I can see you doing with this and those are going to be two completely different programming models uh you're going to be in Blazer C up until you hit start AR and once you hit start AR you're 100% in JavaScript land writing JavaScript code uh to do all the 3D stuff and again there's no real room uh to talk back and forth between those things so that that's kind of disappointing um the other I think it's Babylon JS is the other one is this is the other um yeah this is the other pretty prominent JavaScript based webxr platform and uh see is this the live Nike Site I guess Nike deploys this for something maybe you can see the shoes in real life or so or in VR but um didn't it didn't really say it just advertised that they use Babylon for something uh so this is a little bit more prominent College room planner some games stuff like that oh I know that the Xbox Design Lab where you make your own controller is using Babylon that's pretty cool so this is kind of some of the things I guess you could build with Babylon uh but again when you you have this Frame here everything that's going on inside that frame is going to be JavaScript programming um you might be able to interrupt with something web here uh but that's not really an AR experience that's that's just a 3D panel um when you get into AR what happens is the experience that's in this box where this controller is overlays the entire screen um even more than this so it's um an immersive mode that the browser goes into and all of the UI goes away and it's basically rendering a canvas and the you can render some 2D things over the top of it but it's actually not recommended for the immersive experience mode so there's like an overlay panel and there's some rough guidelines that are still in the works about it being transparent um it shouldn't be 100% opaque um and should only contain like certain interactions and things uh so it's basically like a little tiny menu that might display like frames per second some statistics and things um over the top of your immersive experience there's no real room for like a Blazer Style app to fit there so it's it's kind of a disappointment I was hoping there was something we could do a little more interop level um to to work with these type of things and again if you look at the samples on these um um see learn it's under the learn Pages possibly the documentation all these have good documentation by the way there's nothing negative about any of these it's just the app model and Blazer don't play well together um crossplatform API search playground there we go we do AR might have to do web XR to find the sample I'm looking for web X not web X XY web XR there we go um so they've got some similar samples uh like plane detection right that's something that would be a nice feature to add uh to an app where you're looking at like a product a product in your real space um but again see this goggle button this is rendered by babon and it's another one of those situations where like you can't even render this button with blazer like the button comes with the 3D framework you can customize the button um and place it in different places but essentially you're still telling Babylon JS to Ren render this button and how to style it so once you're in there you can you know exit the the thing and that's about it like that's that's where your AR experience is going to cut off with um with blazer so not not a lot of Hope here to actually do uh any web XR stuff with blazer directly um but what is nice is we can build 2D app app s and put them in u the quest and you might think that's kind of weird like why would we want to do that but there's actually a lot of that going on right now because uh developers are realizing that this is another device that people are using on a regular basis and you saw earlier that side by side uh type of thing let's see if we can cast again and uh maybe our our device will start working um we were having a little issue earlier okay so it went to 3D mode here and I can hear it opening a window let's see if we can get um an example of there's my menu uh so not only do we have our browser right we can we can navigate to normal 2D apps normal flat apps in here so for example we've got Instagram in stagram also has a dedicated app because meta obviously so we could go to Instagram um in the browser and we have this flat experience or we could actually open a 2d flat Instagram app notice there it says beta because they're still testing this out this is all new ground to break here but it's essentially the same app right it just dropped the Chrome right this kind of this is a pwa basic basically so we can deploy pwas into the quest store and I think this is where the actual Blazer opportunity is Right building our apps so they run in this environment um we can do it in VR it's not letting me go back to AR for some reason that's weird uh so you see me tapping my head that's me trying to get into augmented reality I'm stuck in Virtual for some some odd reason um it might be because I turned off boundary detection uh let's see stop casting real quick or now we have our AR toggle I forgot about that feature I don't know if it's a safety thing or what but proximity sensor will turn off your AR environment so there's my screen in real life there's my Instagram app again chromeless essentially uh HTML based application right pwa uh app right there um and then another example of this VR or uh flat app uh we've got what what's app um messenger uh move it's a great example this is the the quests um Fitness tracking app so that that's a good example of something that you might want to build um dashboards for something uh maybe you maybe you're using the quest device in VR for something um at your uh facilities Factory whatever it is you need a dashboard in 2D to do some kind of reporting uh there's also see if still under new releases um Xbox cloud gaming is technically a flat app but this is the one I was I was aiming for here PowerPoint Word and Excel so you can essentially open the standard word application inside of your quest so I meant to install this over the holidays it's a quick install uh probably because it's just a web app um I would need to sign in and I'm not going to be able to do that live I don't think oh it did it for me I don't know what it's I don't know where it pulled those credentials from but more power to you for doing it blank documents here some some single sign on somewhere made that work uh that was a little bit of magic um I've never opened anything office related on this device but yet somehow it knew the credentials to get in to office um and I saw work rated material in there so it it definitely definitely was working um I think you have to pair a Bluetooth keyboard for this but again got the full office suite here in VR um I'm assuming it's saving this opened it in a tab that's is this oh that's okay I was mistaken this for a browser tab um so we've got uh a little bit of a different UI than you might be used to I guess for tab interface but still this looks like office right full office um Suite here so yeah so we've got office our word we've got Excel we've got PowerPoint and um it might be even interesting to check out what PowerPoint is like if I wanted to do presentations um at a conference with my VR headset that might be a thing so again 2D apps have their place in augmented reality here you can see I've got Instagram open I've got uh PowerPoint open um and then I can easily go to my app library and hit the remote display button and pull up my desktop uh and access my desktop remotely as well again the only negative to this remote desktop thing is you need the physical keyboard and mouse um actually might might be wrong on that I may need to just attach it uh USB be to my um my headset and it might work as well but there you can see I've got my remote desktop session um I've got PowerPoint and I've got Instagram open and they're all 2D apps in this 3D space and we can build some Blazer apps I think that could take advantage of this there we go I should bring my resolution back to normal I don't know how that looks on stream but it's uh I get black bars on the sides of my monitor when I do that so what do you all think uh about building apps for that AR experience um if these devices are going to let us work in uh in the headsets in the future we need apps to work in the more native apps that we have um in in the stores the better uh for that um they're only going to get better this again this one's a little heavy um Aztec said how many hours were you in it I lost the chat history on that you said it was four or six hours was it it was quite a while you've spent um a lot longer sessions in there than I have that's for sure six for an event okay six hour hours in there uh this one's a little hot a little heavy um I would say you know a couple more generations of this it'll be uh more mainstream um I'm interested in seeing what people say about the Apple device that's coming out I had high hopes for it and then the more I see about it the less impressed I am um I'm not an Apple fan I will admit that but prior to actually seeing the device um I had really high hopes for it like I was like they have the technology they have the processors they could build something amazing then they came out with this um I it's very gimmicky for the the price that they're asking for um if you look at these demos it's all flat apps and I know I just said like there's plenty of room for flat apps right uh this is what we're going to try to build with blazer because there's not a lot of room for Blazer uh to do 3D anything and you if you want to do proper 3D you're either going to be in JavaScript with Babylon or 3js and um you're or you're going to use Unity right and it might sound like I'm being hypocritical cter addicting myself when I when I say that the Apple Vision Pro doesn't impress me because it's all flat apps and then I'm turning around telling you to build flat apps the catch here is this is $3500 and it's this is the only thing they've demoed is flat apps the quest is $500 and it does everything right there's room for flat apps sure but that's not the only thing the quest does this seems to be all it does so far and they're they're asking for seven times the entry fee uh I just don't get it did anyone figure out Maui and AR beyond flat app yeah that's um you're you did start right after I said it thle um I'll jump back and I'll talk about it one more time I still have the tabs open so with uh so for one with Maui you cannot do 3D um there is a limitation currently actually I missed this earlier when I was talking about this from Maui I I said Blazer but from Maui specifically you cannot do um any 3js or babylonjs um because um the view that is in um the quest headset so the way Maui works it uses the web view that is supplied by the device uh the web view that is supplied by the quest 3 headset does not allow access to web XR for some reason so when you visit a page that is using webxr um I have a plugin on this one so it's still it's showing up let me turn it off turn off my XR plugin okay so when you pull up a Maui application that has uh web XR embedded in it you will see this app or AR not supported uh that that is what you'll see so you cannot you just can't do anything the API is not there um you can get into your Dev tools and do console uh trying to remember the thing you can write in the console uh web or notes it's uh bo I'm trying to remember this off top of my head there is an API here window do web XXR something like that um you can do the basic tests uh to find this I can't remember the the command for it but um it will basically come back undefined or whatever um meaning that it the API is not even there you can't call the functions in the browser they don't exist so um yeah that is a limitation of the quest software So Meta would have to enable that um they have it in their browser so they they're using either a different browser for their web view or um they're limiting their web view for some reason um I I would say it's probably a different browser in their web View and I don't understand why uh their their web browser uh on the quest does have XR um API capability so that shuts down Maui really fast because there's no way around that um it's going to it's going to use that web view you can't force it to do anything else and you don't have API access therefore you just can't do 3D okay so Maui is a no um we can use Blazer though so the other Avenue for Blazer to launch in um the quest is why is that not going away let's try killing everything uh if we go to the quest developer Hub we go to Quest developer Hub resources uh this is what I was talking about earlier I didn't actually show it on stream there's a little toggle here and it may be this note right here limited PD pwa distribution Progressive web app Distribution on medaquest is for developers with apps listed in the meta Quest store I haven't quite got to the bottom of what exactly that means but at the very least you have to publish your app to The Meta Quest store once you do I don't know if you'll get the button in the browser to install it or it will just allow you to put a meta tag in the HTML that will direct you to the quest store to install it and then the app in the quest store is a progressive web application that has been wrapped uh similar to how Maui Works what was that didn't do what um so if you go into pwa design getting started I think uh there is yeah so create your an map app manifest that's all normal uh this all web standard stuff so far scope and extensions uh might be some specific stuff in that but it's still web standard right um web standard web standard uh it's tools and packaging here we go this is where things diverge from web standard all right so normally with a pwa you have all of the things that I think this is mostly just Google uh has deemed as um a progressive web app so you have your manifest file um and the um the background Services um trying to remember the name uh of those Services what are those called basically everything that's outlined in your app manifest uh your service workers right all of that defined that generally qualifies uh pwa to be installed on a platform not for the quest um again they've disabled this for some reason and you have to deploy to their store um I think think they they're just trying to control uh what gets installed on the device and the means of how those things get installed because they want you to go through their store so what you have to do is once you have an APK or once you have a pwa built you have to get their CLI and you have to run it against your um your folder uh and I think it looks at your app manifest um and it will wrap it in an APK file so it will create some kind of wrapper around um your website so that gets packaged up you get a APK and then you have to install the APK on your device to test it and then you could submit that um APK to the App Store uh that has to go through their submission submission process uh similar to like an Apple app into the web store or Apple Store uh so you have to you know create a developer uh account which is free um and then you have to register the app and submit it against their guidelines so you can't just build a pwa and expect it to download to the device through the browser and just run like you can on the Chrome browser browser or edge browser all right that's not going to work uh example of that if I go to tel.com and I go to web components on Blazer demos uh there is the Blazing coffee Warehouse demo this is a pwa if I run it here in the browser uh let's see here I'm not getting a prompt I should get a prompt sometimes Chrome's little well this is Edge but the chromium engine's a little tricky about this uh maybe it's up here there we go install pwa files uh this site can be installed as an application yada y y I can hit install and it will install it this functionality does not exist in the quest I can pull blazing coffee up in the quest do not get an inst button here or anywhere else just doesn't happen I would have to take this blazing coffee Warehouse app run it through the CLI spit out an APK install the APK onto or not install but um submit it through their um their App Store and meet all their criteria okay but the the original question was like can we do uh 3D uh with blazer hybrid no but we can do pwa but then we run into the issue that I outlined where we're still on the web we have to rely on 3js or battle on JS or something similar there's more these are just the popular ones um and when we go to the these examples here and look up um some examples you'll see that all of these have this uh this is a nice reppel by the way that you can run some of these U examples in Babylon has better document documentation than 3js does and as I say that it's not going to work for some reason it's not running uh let's try another one I did plane detection earlier and it worked loading webxr demo and their website's gonna not work for me right now so uh let's try the 3js one because it's basically the same problem so you get these buttons basically um that come up and when you have a VR or ar sample or or application you get this view you can do you do any kind of 2D HTML stuff in here that you want and then there's a button the button is supplied by the framework work whether it's 3js or babylonjs once you click that button this one says VR not supported let's change that let's put our emulator back on uh that and then we got to refresh the page so with our emulator on it's still VR not supported seriously why I've got the right device chosen neither one of them is going to work right now uh let's try it in a separate window yeah it should be this one well let's try the VR gear VR one it's the same idea anyway all right so this one's specifically looking for VR uh but it'd be the same with ar so you'll get this button that says either enter VR AR whatever um when you click this button the API that takes over here I was doing this stereo yeah stereoscopic thing it's bad bad example uh it's the only one I got so oh stereo effect off there we go so this would be your VR mode mode the browser kind of hands over everything to a canvas and you draw everything here so you're like in this 3D mode um Blazer doesn't have really anything to do here um and you you maybe could interrupt with some net apis or something but I I think you're you're swimming Upstream at that point you're causing a lot more complication you might as well put a web endpoint behind whatever you're trying to do and just use JavaScript and talk to the web endpoint and um you know store data or whatever it is do crud with uh with a web API backend built with.net if you really want to do that so this 3D stuff is all JavaScript all of it all of it all of it all of it or typescript Babylon is typescript as well but it's certainly not C if you want to do c um you could create a Unity uh application and it's just going to be Unity app if you want to use Maui or Blazer the 3D part of it doesn't seem like it's going to be an easy or the best route like there's other things that you can use to build 3D apps um there's just a lot better platforms and Blazer is not one of them Blazer is not not the go-to option if I'm thinking 3D yeah uh so Hollow lens quest uh Apple all can use uh Unity Unity also has a web um a web view of its own um so they have a cross browser web view I guess it's third party but it shells I guess into or it goes inside of a Unity app uh so there there's something like that and then you can deploy from Unity to the web so you can you can host a unity game inside of there it is create publish and deploy you can create a webg build and uh you can do that so there there's other crossplatform tooling to get you onto the web and do VR and AR specific apps so there's kind of a dividing line really there's there's not so much of a mixed mode between these apps there's not like a there's not an example that I've seen of a 2d app that transitions into 3D and then back for any good reason except for maybe the Amazon store and some other stores um I don't know what it'll pull up if uh maybe if I turn on my web AR thing here I might be able to see it let's do that and then we just click on some random Furniture living room tables and um let's see if their their thing pops up on here so they have um uh view it in your room there we go so that view in your room button oh it's it's going to push me out to a phone so it it doesn't want to do it here let's open image it looks like it has the URL there but product search it's just opening it on on their it's it's in their app it's not in it's not even the web so they're um oh am I my bad um so they have the view in your room but this actually just opens the mobile app so in their mobile app only so it probably uses u a platform specific API like um uh Google and um Apple both have so Google and apple have uh I'm trying to remember the name of their apis AR core is probably what they're using so um you know you jump out into what essentially is well you have to have it on your mobile device um so this is a a platform um this is a Android application that Android application um that Android application then handles all of the AR stuff so for Native mobile apps you would use something like this these are using openg G AR a core um and then Android or um iOS has their own because you know platforms can't agree on anything um I guess this can be crossplatform or some I I guess there is some crossover here so you might not have to code it twice I don't know I don't know how these things work so yeah uh web XR isn't quite there yet uh these on platform apis are more advanced but I don't feel like you have access through through um uh Maui with this I don't think so Maui uh core air core uh somebody says they'd love to have it it's closed issue so a core bindings are in zamin compon so looks like there's some way to get that that would be more the route to go that would be in a native uh Maui part of your application wouldn't be Blazer anymore wouldn't be a Blazer hybrid thing I mean it could be part of a Blazer hybrid app right but when You' navigate from your Blazer hybrid page it'd have to go to a net Maui um Native page or native Island at the very least uh for AR it' have to be an entire uh that entire page that entire view because immersive immersive apps um don't really allow mixing like there's no framing or anything like that it takes over the device's screen completely so there might be something that you could do from that route but again square peg in a round hole um you really have to have the right use case for something like that so that's where we're at I think we'll we'll be doing some 2D apps um next time we'll just see how you can launch them we'll see what apis are available we'll Tinker around a little bit experiment um probably keep that one at an hour this has been two hours today uh so again this was a special addition to try to dip our toe in the water of VR uh we looked at um quite a bit of tooling that is available uh we've got the uh remote desktop app let's put that back up we'll just look at the tools again real quick before we end the show uh so we got the meta remote desktop app that's one that lets you see your desktop um I'll have to get a Bluetooth keyboard and mouse and connect it in my headset and see if I actually remote control my computer with those um it doesn't let you like point and click with the the controller for some reason um there are other softwares that let you do that so um like the remote or virtual desktop lets you use the uh the quest controller um but it's really a must-have this remote desktop it just makes things really easy if you're sitting in front of your PC already to jump in and work um on an app s you know in Visual Studio or something with an app in the device open and then there's the meta Quest developer Hub which is the other necessary thing uh for this it's kind of like a dashboard um So Meta developer Hub just punch that into Google download that that'll give you this um you know you have your device management stuff here um sideload apps uh run CLI commands against it do performance analysis and uh get your files on and off of uh the device um there's also debugging stuff up here too uh there's a logger and you can kick your device uh stuff like that um and then probably too much more doxing there and again my my headset's dying as I talk so uh yeah this thing's a power hungry device uh no I didn't never do Java the where do you see the Java part oh that's that's a Blazer hybrid app on Maui that's a Maui app these are two Maui apps you can do Maui they're just not 3D they're they're 2D apps in um in ar oh it well I mean it is a job a environment right all right uh that's all I've got folks it's been another interesting Blazer Power Hour we're going to mess around with ar and VR soon and um hopefully you will join me next time when we build a Blazer hybrid Maui app and launch it in the headset and play around with it answer questions about what we can and can't do with it and and uh thank you again and join me next time for some more Blazer hybrid stuff

Channel: Progress Telerik

Views: 222

Keywords: Telerik, vr, ar, mr, virtual reality, meta quest 3, quest 3, quest 3 mixed reality, mixed reality, coding in vr, coding in meta quest 3, coding augmented reality, blazor, dotnet, blazor ui, blazor hybrid, flat apps

Id: P25jkY1gyKE

Length: 99min 30sec (5970 seconds)

Published: Fri Jan 12 2024