Getting user input with Unity XR.

hello there well Pattillo here and welcome to this series on VR development in unity this is going to be focused on Unity's new XR system as well as using the oculus quest I'm not going to be covering the basic project settings and hardware setup since VR with Andrew has a very good series on that I'll link to it in the description below this particular video is going to be I diving straight in to getting input from the controllers and doing so in a way that is very simple and extensible and doesn't couple together a bunch of other features that we don't necessarily want so starting out we're going to be creating a very absolutely bare-bones camera rig so I'm going to create in the hierarchy right click go to XR and get our room scale rig here and I'm going to open this up and these parts the XR rig camera rig offset leave those those are fine as they are so is the main camera I'm actually before anything else I'm just gonna make sure that this works so I'm just verified that my oculus quest the device is connected and that if I put the headset on I am in the I have the oculus link I enabled and that that has to be done in when you open up with the oculus quest going into settings and setting it there so anyway if all of the hardware is set up correctly then when I press play I should be able to look around with my camera and I'll see some lines coming out of where my hands are so first thing is I want to get rid of those all these line interactors and that kind of thing because we'll be creating our own systems for all this stuff so that we can control exactly how it works so in each of these controllers see if I can get both of them at once and I will remove the XR interactor line visual and also the line first remove that component and remove the XR ray interactor and so that's right there at the minimum and then next also want to create a ground on the scene so we can see where we're standing so we'll just go game object create 3d object plane and I'm a zero out this location next I want to be able to see where my hands are especially since I got rid of those lines so I'm gonna create an empty game object and call it model and then under underneath that I'm gonna create a 3d object cube and the reason I'm structuring it this way so that I can very easily swap out this model with anything else so right now my model is just gonna be a basic cube and I'm gonna set his trigger to his trigger so that it's not running into things and I'm gonna shrink it way down so it's kidding I'm not if my head isn't inside of it and I'm gonna set the right hand up the same way so create a model and then a cube inside there shrink it down and set it to trigger okay so now I pick up my controllers I can see my left hand and then my right hand over here now one thing I'm also gonna go ahead and add is to make it visible right away which hand is which now in unity I can tell because they're just labeled left handed right hand but the player might not be able to to see that it with the quest you can kind of the the controllers are not symmetrical so you can feel which hand is which button some other headsets it's it can be hard to tell and they especially the viable sometimes randomly like swap the hands when you start out so sometimes having a little bit of color on this can be helpful or something asymmetrical in the game can be helpful to let the players see which is which so when I create a couple of materials I'll just make a red material set that to red and then create a blue material and set the Obito right here zero that to set this to blue and then on the left hand just bring that in and same on the right hand just bring the red material on to the mesh renderer and so now when I visually easy see which hand is which this is a nice kind of default setup right here and then once I get more complex hands maybe something that can open and close I'll just replace these cubes out for those now one thing I'm also noticing right now is the cube disappears when it gets close to my face and that is because cut off distance set on the camera so I'm going to go ahead and fix that right now so on the main camera the camera components the Athas the clipping plane right here I'm going to shrink this way down to point zero one and also just so to save on performance a little I'm gonna shorten the far distance we can always change these and make this further out as needed so now close to my face I can yeah things don't just clip out and this is nice if you want to inspect things up close all right so that's enough for my basic setup next I want to be able to get input from each of these controllers now Unity's XR system has some built-in ways using direct interactable x' and other kind of systems built-in although I've having worked with those a little bit I find the couple a lot of things together that don't always go together and also sometimes it's better to just get input that you can then use however you want one of them is if you want a keyboard controlled character and a VR character and an AI character to all work the same way just have the input systems different and and have the rest of logic the same I find it much more helpful to use a unity event based system so I'm gonna be ignoring some of the built-in interactable stuff and using just getting input from scratch so to start off I'm gonna create a new C sharp script and I will call this just X our inputs attach these for now to the controllers directly later on when things get more complicated I'll be putting them on specific systems and I noticed for some reason there's two of these attached I'll just get around one of those okay and now open that up and I'll get rid of some of this boilerplate that unity so helpfully generates automatically I get rid of those and we're going to be using the X our controller so we'll need to get a reference to that so I'm gonna type in serialize field and X our controller controller and unity doesn't really know what this is right now so I need to add a using statement using unity engine X our interaction toolkit all right there and now I can drag in those references just drag their right hand controller into itself like that now I'll start by just checking for a trigger press as a test before we start adding versatility to this so private void update create a temporary variable bull is pressed and controller dot inputs dots our controller dot input device dot try get feature value and in here we'll be using common usages dot trigger button and then out he is pressed and then right here I'll just say debug log actually I'll just say if is pressed debug log trigger pressed okay I'm gonna test to verify this works and then I just kind of go over and explain what's what's happening in this particular lot in this line right here now one thing that can be helpful in testing is to bring up the window and analysis and XR interaction debugger and the first thing we should see is in our input devices that we need to be able to see all these left and right hand controllers if these are not showing up then they then that means a cue list is not detecting that controller the next thing is we should be able to see whether this trigger is pressed or not so right now here we're looking for the trigger button and when I press it it should switch between true and false so if that's not running and that's not any issue with the code that's on a connectivity issue so next if we look in this console window we'll see it's I have it just saying whether it's pressed or not right now so when I have the trigger pressed then true it's printing true when I'm not pressed it is printing false good so that's our little test right there to verify we're actually getting something so now to take a little look at this line right here and so we're getting access to the controller getting the input device from it cashing this doesn't really seem to work so just yeah use them both and then we'll be using this try get whether a particular button is pressed we'll be using the common usages and checking for the trigger button the trigger button is just a boolean true or false whether the trigger is pressed or not there is another one I think if I just look for trigger that'll get a floating point between zero and one I'm saying how pressed it is for now we'll just be using the the boolean 's and then it also takes an out parameter which basically you you pass it in and then it mutates the value of whatever gets passed in kind of like working with Ray casts rather than printing right here I want a different script to be able to know whether or not this trigger is being pressed or not so for that I'm going to create a serialized field unity event and for now I'll just call it on trigger pressed and for that I'll be needing to add the unity events so there so we're adding using unity engine events also unity engine dot XR and so now in here instead of just printing to the console right there I will just say if these pressed on trigger pressed dot invoke like that and now so now we have this unity event show up and now I'm going to create a object that I'll just use for debugging purposes so create empty game object debug create a new script C sharp print to console and I'll attach that to debug and then go inside of here public void print string message this is just a general purpose test script that we're writing and inside here debug log message okay now on these inputs for so that the left-hand I can just drop in this debug here look this print to console print and here I'll say left trigger pressed on the right-hand controller I'll just call that debug again and print a console print message right trigger pressed okay and now when I run the game and I look inside the console again nothing's happening that's because I'm not actually wearing my headset right now so just to be able to see the console while also running the game the oculus has little camera right above the eyes I just put my thumb over that so that it thinks I'm wearing it and then I can you know be looking up I have it off of my head while I run this so I'm pressing the left trigger and I get left trigger pressed message I bring up my right controller press the trigger and nothing is happening that's because this XR input is on ticked for some reason so the update wasn't happening and so now I press the trigger and yes now I'm able to see whether either trigger is pressed and in this way using these unity events that already separates out whether I'm using the left hand or the right hand in a script that doesn't know that it's working with XR so I could just if I were to set up keyboard input I could get methods and passing information that way all right so that's our basic little test setup right there next we'll be going on to get some other inputs so I could here just now copy this line and instead of looking for the trigger button I could say look for the grip button and then have a separate event for a grip pressed that's getting a little bit messy we get you know a bunch of different unity events most of them are going to be blank so I would like to have a way of just having an array that I can just you know set in a number of how many bindings I have and then for each binding be able to pick which button I'm using and I then have unity events associated with it and right there when I'm mentioning that you know we have a button that we want to specify in a unity event associated with it so there's there's two bits of data that are associated together that's telling me I'm gonna need a helper or a data class below here so I'm gonna go in serializable and you know he doesn't know what that is right now so I need to bring in I using system and say public class and I'll just call this an XR binding this will this is what I'm going to be using for associating a button with a set of events it'll be some other data that goes into it's this as well now if you're wondering why I'm using class rather than the struct it's because I'm gonna be having some methods in here later on in fact actually I'm gonna there's gonna be a method right off the bat so I'm just gonna instead of I'm not gonna need this unity event anymore and instead and I have a serialized field X are binding bindings and I make this into an array by putting those brackets in right there I'm gonna comment these out since I'll be reusing them down here and going here and I'll just say for each bar binding in bindings binding dot update and so that allows us to just delegate all the logic down to the individual bindings just so that they can look for a specific button so in here I'll say public void update a serialized field something that specifies what the what the binding is and then the unity event that will be Asif I come with it so I'll get the unity event right here let's call this on active and now for serializing this common usage is that trigger button there isn't really a direct way to get that that I know of so I'm going to create an enum that that includes trigger grip and all the other things and then convert between the enum and this value right here so let's see create a public enum I'll let X our button and here I'll just list out all the different button types so I have a trigger grip primary this would be the a button on the oculus quest secondary this would be the B button believe the on the vive the primary is the button that's above the the trackpad I think it's called the menu button I don't think there is a analog for the secondary and that could be wrong about that then oh and then also I put another set up for the primary touch since we're working with boolean values secondary touch just treat that as a as a separate option and then also just to see whether the player is clicking or touching the trackpad or joystick so primary I'm gonna call this primary 2d axis click and primary to the axis touch and now that I have these I can in my X our our binding add a serialized field and X our button button and that gives me on these bindings right here the kind of interface that I want to be working with so I can have one thing that's looking for a trigger or something something else that is looking for the grip and actually I think I'll just die yeah and then what happens when they're active and so for now I can just bring in the debug right here to each of these and have it print to console so this would be the right trigger pressed and in the grip print to console print right Rick press okay so that's that's the interface now to actually translate between this button and this enum I could put the method in here but there might be other occasions where I want to translate between a button and one of these common usages so I'm going to create a static class inside of here so that it's universally accessible so public static class x-bar statics and I'll just add to this if there are other XR related things in the future that I want to be static and now public static get feature that is looking for an X our button and now as for the return type this I'll need to be to return this or actually to return this whole whole thing I need to be able to return a input feature usage and we're looking for whether it's true or false so put in a bool right there and now inside of the static put in a switch statement for each of the possible buttons so switch button and oh right and I need to actually do that up here so in the case of the XR button dot trigger that's just grabbed right there I will return common usages dot trigger button just exactly so we're basically just getting this value right here it's actually I don't really need that anymore and then I'm gonna repeat this line for all the other different cases so I've just put some of those in so in the case of the XR button grip I will be looking for the grip button and what else we have primary primary button primary touch look for the primary touch in the common usages again for secondary a secondary button a secondary touch secondary touch and what else do we have to the access click primary TV access click and finally the primary two D axis touch ok so that's a conversion from all the enum values to actual common usages now to actually get it in our update so this is going to need a reference to that device that we had or that input we had earlier so I'm going to look for an input device I'll just call it device and to call this in the update I'll use a controller dot input device and then here we'll just put that logic we had before device dot try get try get feature value and here we'll use our static class X our aesthetics dot get feature button so that's the one the user specified and then out is pressed okay and we need to define this is pressed I'm just gonna cash a reference to it up here so to say rule is pressed okay there we go and then lastly this update on to say if is pressed on active dot look okay there we go so we're getting a reference to device whether it's the left or the right hand checking whether a whether the feature is pressed or not and we're getting that by taking the button that the user entered in passing it through the statics to get common usages and this and that all and that'll change this is pressed to or false and then if it's true then we'll broadcast that so now if we go back into unity the right hand we have this trigger and grip messages setup will do the same thing for the left hand just to make sure both hands are working so add two elements one for the trigger one for the grip you can use different buttons if you want of course and just send these into debug have both of them print to console so this one is going to be the left trigger pressed and then this one will be the right are sorry the left grip pressed so now we have four different options they can be either trigger grip left or right and we should see all of those be able to print to the console now there is error right here not all code paths return a value let's see what's going on with this and right here so it's oh so we need just some kind of default case in case the option that's passed in is not one of the things listed here so this should be a comprehensive list if we get a button that is not recognized in this anew that is a problem so I'm just gonna print out a error message at this point defaults debug log error and button button not found and just to return back something so the compiler is a happy I will return I'll just use the trigger button okay and that this line here should clear the error we were seeing right now if I run this and that's because I need to put my thumb over the the view so that the it's getting the input so I am able to get the left trigger pressed and the left grip I'm going to switch to my right hand and let's see that's seen anything yet oh right first last time I had this XR input kicked off and I set it in play mode so that didn't get saved let's try this again so my right hand got the grip got the trigger and left hand trigger grip and I can easily extend these two I get all the different buttons that I could like you know these all of these should be able to work as to whether they're being touched or not now two things that I want to add still before I finish out this video one is to have options as to go whether it's pressed down press up or continuous and another is I want to be able to get the position of my on the joystick not just whether it's being touched or pressed so I'll start by getting the different options whether it's a down up or continuous I think I find it very user friendly to just set up another enum to give a drop-down so say public you know and I'm going to call this a press type I'll be used to begin and and continuous and that these would correspond to press down press up and press but I think this naming is a little I've easier to understand I'll set up a serialized field and look for a press type and I'll just call it press type and now in my update after I have gotten set is pressed or not I can say switch press type and in the case press type dot continuous I'll start with that because it's the simplest then I can just take this line right here say if is pressed on active does invoke and and then they break next deal with these some of the other cases case press tight dot begin and I'll just go ahead and copy this line to deal with the end this case as well press type dot end these check for a change in state so if begin happens when we are pressed but we work pressed on the previous frame and end is what happens when we are we're not pressed on the previous same frame and we are pressed now and we have we have half of the data we need we have the is pressed now as so for these we just need to be able to know what whether we were pressed on the previous frame so I'll need a new boolean up here bool was pressed and then here are conditions for begin if is pressed and not was pressed on active invoke and then for n is the opposite if not is pressed and was pressed on active invoke break there's a lot of kind of extra language here so I'm gonna simplify this down a little bit I'll just create a little temporary variable right here full active equals false and just a little something to make this all easier to read and here I can just say active equals is pressed and I'll get rid of this active equals his Preston not was breasts and here active equals that just a little lesson for boasts right here and then down at the bottom this is well we'll just combine all of that and just say if active on active dot okay alright next and now the last thing is we need to set this was pressed so I just say was pressed equals is pressed so at the end of an update will set was pressed so that we can use it for the next frame and then when we get here since we haven't reached this yet was pressed will still be the last frame but is pressed is calculated right up here and so that way we'll be able to compare the current frame with the previous frame so next we'll make a use of some use of this and rather than try all the different possibilities I'll just go with my left hand right here I'll just use trigger for all possibilities add one let's say you add one more binding left hand trigger begin and and continuous and left trigger pressed for end left trigger released and last here for the continuous left trigger held I'll try this out let's make sure I put my thumb over the view and now I press down on the trigger and I see one left trigger pressed right at the beginning I get a continuous stream of left trigger held and when I release I get a left trigger released and I'm just gonna assume that works for all the other buttons and for both hands okay so one last thing before we're done with this video is getting the joystick and right now all we have is whether a button is pressed or not and we're not getting any data from the button like how how how pressed it is or or any kind of access value so for to get some access value I'm going to create a new script and I'm going to call this X our input to D because it's kind of an extension of this X our input so create c-sharp script X our input to D and bring this up and this is going to need a reference to the XR controller I need to bring in some libraries yeah there are XR interaction toolkit this one's gonna be a bit simpler because there's only one joystick so I don't need to delegate down to a helper class that you know I make into an array so it's I can actually just put my settings including the unity event right here however the unity event when I print things out this needs to be our dynamic unity events unity events by defaults are just static where they can call things and you you set what the value is right here that's not going to work for something that's gonna be changing depending on the value that it has so I need to be able to have a form of unity event that outputs a vector too so have a serializable which means I need to be using system public class and I'm gonna call this a vector to events and it will inherit from unity events but then take in as a generic a vector too and then open and close curly brace which means I'll need to be bringing in unity engine dot events to for it to know what unity event is now this is giving me an error right now because I already have a script in this particular project that has a whole bunch of unity events this is just a script that I carry over with me from basically ever every project I work on as vector to event but also vector three event float event in to vent string event basically everything that I can come up with including lists and arrays passed through through unity events and so I'd actually recommend rather than adding it specifically to X our input to D other things might want this so to just create your own scripts called unity events put it anywhere in the project doesn't have to be attached to anything and then just add to this as you need it so anyway I'll I'm just gonna get rid of that here and now that means I don't need those libraries so I serialize field vector to events and I'll call this on active and next in I'll have a public void get axis value one assumption that I am making here is that this script is only going to be running when X our input has detected a some kind of touch or press I could run this in update and just be continuously spitting out zeros whenever factor 2.0 is whenever the player just isn't touching the trackpad and maybe you want that but I think it'd be a little more efficient to have X our input drive X our input 2d so I'll have get axis value and I'm going to assume this is only being pressed when we there's some kind of touch happening so I'll just say in here controller dot input device dot try get feature value same as before common usages dot primary 2d axis so remember before we used a 2d access click and touch these just return boolean x' so we can see here this one actually returns a vector 2 so we get primary 2d axis and then out and then we need to need some variable to contain this value value that's being modified by the out parameter to save on memory I'm just gonna cash it here so vector to just call it the value I can change I can set it in this out parameter and then I'll just say on active dot invoke Oh one thing that also occurs to me if you wanted to put this in updates and you didn't want to print out a bunch of vector 2.0 x' you could just say if value not not equals vector 2.0 I don't really feel like I need that with the way that I'm setting it up but that's another valid option that is it for this script for forgetting two dimensional input so to hook this up I'm gonna go over into my right hand controller and I'll just I'm just gonna change this binding so rather than a grip I'm gonna be looking for 2d axis touch and go with continuous and now instead of calling this debug directly I'm gonna add a component so X our input 2d and have this bring in right hand controller X our input 2d get axis value so touching the joystick we'll call this script which can then call the debug and print to console however I want to be able to check so this this string just prints a static value I want to actually be able to print out the value of where my thumb is on the joystick so I wouldn't need to make a little bit of an addition to this debug script right here we're going to print a console and be able to receive a vector to so public void just make an overload of this print here vector 2 and I'll just call it value and then inside here just call it self print value dot Q string and if you want some precision by the way then you can just actually just put in a say an f2 or both an f4 so that we get really precise to this is to four decimal points and so now let's see if this works make sure I have everything set up this is activated okay yeah the last step to set this up is in this printer console I need to use this new dynamic vector to print method and also assigned this controller so now I run the game put my thumb over the sensor bring up the console to clear this out for now so it's not printing anything I touch the trackpad and now it's printing out zeros because I'm not really moving my thumb at all I move the trackpad around actually I'm gonna collapse this so I can really see all the values move this around and I press up to the front and I have close to zero and one go back to zero negative one there's the left it also detects if I'm just moving it slightly I get different values all the way through okay and then just lastly just to be for the sake of thoroughness I'm gonna do the same thing with my left hand controller and an X our input 2d and bring in this XR controller and set up another binding so I'm having all these different options for the trigger and a fourth binding to get the 2d axis touch and I can use 2d access click that would base work basically the same way except that will only print out when I have the joystick fully pressed down but I touch is more likely to be useful and also when you're using this 2d axis you're pretty much always going to be using continuous so now here instead of going to debug I will bring in X our input 2d get axis value and then the X our input 2d those goes to debug and prints to console and prints out to be value I only run this just to verify it works with those controllers so I left hand I'm able to see the values actually just make sure of it by scrolling down yep see the values that I'm pressing and only when I'm touching and same with the right hand and then I can also get these right trigger pressed s' and then with the left hand pressed held and released okay and now with that you have basically a pretty universal purpose XR input system you don't really need to muck about with all the settings of select and activate and and all that kind of stuff you can just be able to call public methods in your code and have them work however that you want and just with the input taken care of we've been just running everything through this print to console line just for simplicity but since I mean you could just take any public method and just have this whatever you like so that is all for this video next time we'll be actually using some of these events that we're passing around to actually do stuff in the scene but for now thanks for watching and if you want to have any difficulty following along or would like to just get the end results of these or you come up think with things that you want to contribute all at once again all of this is available on github it's public it's under will 9 3 7 1 that's me and character template and there will be a folder for XR which has all the scripts and materials that we were just working on and the interaction test scene this will be updated as I add new things and also while you are here there is lots of other things inside of this particular project I'm just kind of using it as a repository for a lot of personal libraries that I feel are fairly well established and worth sharing the main thing right now is under character we have a third a physics-based firstpersoncharacter third person character a AI system in here that can follow you around and lots of other things so absolutely you should absolutely check that out but in till next time I will see you in the next one

Channel: Zen Marmot Digital

Views: 784

Keywords:

Id: o1qA_dE2rfI

Length: 47min 33sec (2853 seconds)

Published: Sat May 16 2020