Augmented Reality Tutorial | Gaze Interaction in Unity

okay so I made this coronavirus visualizer where you look at a map of the US and wherever you look it brings up information about like the number of deaths and all that stuff and a bunch of people in the comments are asking how I did that gaze interaction so today we're gonna go over doing that so I think we're just gonna make some cubes and when you look at them some information is gonna pop up so uh yeah let's do that okay so the first thing we need to do is go to github.com slash third - aurora and let's get this AR foundation example downloaded so I hit clone or download download the zip and then let's open that up in unity okay so once we're here let's open up the main scene this is what we're gonna be working in and then again if you haven't been following along with the other videos you know this video just literally gives us the ability to place an object on the ground with AR foundations so this is what we're gonna build from today so let's go file bills set build settings and let's switch our platform to Android & iOS and let's just get this stuff set up to build out whenever we're ready okay so let's go to player settings and then the only thing I'm gonna really do here is just change our bundle identifier mat let's go gaze test - because I did this once in practice okay so cool that looks good now the first thing we're gonna do is expand this content parent and we're gonna make something - we're gonna we're gonna make something to wear when we look at the cube it's going to display some information so this cube I think it's pretty small right now so let's make this point five across the board and then right click on here and create an empty game object and this is going to be called I don't know section info and then right click on this and create a 3d object quad and this is gonna be our info parent why can't I spell info info parents okay cool let's see so let's raise this up a little bit here raise up the section info and then an info pant let's rotate this 180 and let's make that looks good we're probably going to want to do some materials so let's right-click in the assets folder and create another folder called materials double click that to open it let's create a material in here and let's go we'll call it trance and we're gonna put that on our info parent and then we're gonna make this I think we'll go black with like 0.7 opacity and then change rendering mode to transparent cool so we've got a nice little see-through thing here now on this info parent let's create a 3d object cube and this is going to be like a little handle I don't know so let's go point one point one point one that might be too much point oh five okay that looks good and then if we scale it on the Y yeah let's do that and let's just put it right there five oh is that okay okay whatever I think that's okay for now let's put our transparent material on that as well all right that looks alright and then we got to create some tests so on the info pantry click create 3d object text mesh Pro import TMP essentials okay cool so we got some text here the text is distorted because our parent is scaled weird so let's scale this by 2 on the Y that should make it normal and then click gizmos and what we're going to want to do is scale this box to where it fits only on that info let's zoom in here and we're gonna want this yellow border just to fit inside that box okay that looks good now on our TechSmith Pro let's Center everything we'll set it to auto size change the min and Max to like zero and a thousand something like that and now our text will sit nice in there but it looks like it's behind our box so let's move it out a tad and then let's just make it say I don't know um here yeah here is some info cool alright that looks ok for now let's turn gizmos off to get that stuff out of the way and now you'll see if we moved everything up on this y-axis so if we scale it oh that's not what we want ok so let's drag the info pan out of here section info we want this anchor to be right at the top of the cube here so let's go about 0.5 and now let's drag the info parent back in and now if we scale this on the Y yeah it should look like it's coming out of the box that is ultimately what we want but where did our text go and why is it not showing let's go down to the text and extra settings there should be something for a layer default let's just put that on one now we should be in pretty good shape and then let's make our cube I don't know I want to make him black maybe her let's make it let's make it blue sure like a blue material drag that on to our cube and let's go some kind of blue call area cool just to set it apart from the info all right I think that looks pretty good okay now we need a script that's going to be able to control this info what we're gonna want to do is if the info is open we're going to interpolate its scale 2 1 1 1 and then if it's close we're going to interpolate the scale to all zeros so inside our scripts let's right-click and create a new C JavaScript and let's just call this info behavior okay so let's put this is going to go on our cube because our cube is what's gonna take the gaze interactions because it has this box Collider on it so let's drag the script on here and then let's open this up in Visual Studio okay so the first thing that we're gonna want to have here is we're going to do serialized field and we're gonna have a transform and we're going to call this section info and that's what we're gonna scale and then we're gonna want to say we're going to go vector three desired scale and this is what we're going to interpolate to oh you know it's so then we should probably set this to vector 3.0 in the first place because we want it to default to the info being off so then inside the update function we're going to go section info let's see the local scale is what we're going to interpolate equals vector $3 and we're going to do section info local scale and we're going to move that to desired scale by time.deltatime times speed and so then at the top let's set that speed usually something like six or seven work so cons flow speed equals six F I don't think we need these using directives let's get rid of that I don't think we're going to need a start function because we're set in the desired scale here and then the only thing that we really need is two public functions to public voids and the first one is going to be called open info and here we're going to set desired scale with that desired scale to the vector 3.1 and then we can just copy this function and we can go close info and we're going to set it to vector 3.0 so I think that should be all we need here and we got that script on our cube so that's good oh so we need to drag in our section info which is right here cool now the other thing that we're going to need is we're gonna want this section info to rotate on the y-axis and face the camera so let's right-click here create a new C sharp script and let's call this face camera and then this is going to go on section so drag face camera on to section info open this up here and then here what we're gonna want is let's just delete everything for now we're gonna want a transform we want a reference to the cameras transform and then we're gonna need we're gonna need a vector3 and we'll call it side target angle and we'll just set that equal to vector 3.0 for now and then we need to get a reference to our camera so in the start function to delete that in the start function we're gonna go cam equals camera main dot transform that'll set that and then inside the update function we can do something like transformed look at cam so this is going to make that whole element this whole element face the camera but we don't want it to move on the X and z axis so what we're gonna do is target angle equals transform local Euler angles so now we have that current rotation saved in target angles so then what we can do is just zero out the two axes that we don't want to move so target angle dot X equals zero and then target angle dot Z equals zero and then we can just do transformed local Euler angles equals target angle again because now that should be everything that we want and then let's just make sure this is working so let's go execute in edit mode and then delete these unneeded using directives up here and let's test this out and see what happens okay so now if we grab our camera oh so that we did here camera domain transform we need to tag this a our camera as main camera so once we do that then we should be able to move our camera transform and this thing should move to look at it but that does not seem to be happening here so we might need to enable and disable this again now let's try just removing that script at this point put it back on and then yeah now it seems to be working so now when we move this you'll see that that element is always going to face the camera on the y-axis no matter where we are in the world okay so cool that looks pretty good let's do this back out and then the last thing we need to do is actually get this gaze interaction working so let's right-click in the scripts folder c-sharp script and let's just call this gaze and then this is going to go on our main camera so let's click on the main camera and let's drag this gaze script onto there and let's double click this to open okay so now in here the first thing that we're going to want is we're gonna want a reference to all of the info behaviors that we have because we're going to need to loop through them all and like turn them on and off so let's create a list of sorry type in list and then if you right-click quick-fix it'll give you the using directive so system dot collections generic so we're gonna want a list of info behavior and we'll call this infos equals new list info behavior and then we need to load that list so inside a start function we can do something like infos equals find objects of type find objects of type info behavior so that way we're only finding objects of type of once on start because I don't think that's a very efficient thing to do and then we can do two lists because I think that finds them yeah as an array we want to do two lists so quick fix using system double link that should get everything in the list so that looks pretty good there and then we're gonna want an update function and this is where we're going to actually do this raycast so inside our update function we want to go if physics the raycast I think it is the raycast and then we're going to set the origin his transformed out position so that's the position of our main camera because this script is on our main camera and the direction that we want to do a ray cast is the forward vector of the camera which is that blue axis in the editor and then we want that to output a ray cast hit and we'll just call it hit and then here we want to do let's make a game object go and this is going to be the object that we hit so I hit Collider game object and now here we're here's what we need to do a check if we've hit an object that is interactable so I think the way that we're gonna do I hear is if go game on other side if Godot compared tagged will just use again a tag and let's call it has info and let's just make sure this is working so print here and then this has info tag let's copy this and we need to go back to our cube and this is what has the collider on it so this is what we're going to be interacting with so we want to tag this what that has in front so it's add a tag pasted in has info cool and then cube we're gonna tag this as house info and then let's make sure this outputs something so let's hit play and let's move our camera and get the forward vector to point at this cube so let's rotate it on the Y and is that printing something yes so you can see in the console when the blue vector is pointing towards that cube it's printing here okay good and then one thing I noticed in plane mode is this in landscape mode this toggle is like giant so let's go to our canvas and let's just make this smaller let's go 25 by 25 no 35 5 35 whatever that looks fine and then let's just move this way up in the corner okay cool hopefully that still works it's a little bit small but hopefully that works ok so now what we need to do is we need to go back to our gaze script and we need to get these infos actually moving so what we can do is we need first of all we need a function called open info and we want this to take an info behavior okay actually we want to call this let's actually call this desired info and then we want to loop through all of our info behaviors that we have so for each info behavior info and infos and then we want to turn off all of them but only turn on the desired ones so we can do something like if info equals desired info we want to do info dot open info that'll set the scale to ones or else we want to info dot a close info okay cool so in here if our game object has that right tag we can go open info when we can pass in a game object dot and get component info behavior so that handle is opening everything but we want stuff to whenever we look away from a cube we want all of the info behaviors if they're open to closed so here we can do something like else maybe close all so now we need to make a function void close all and then here we should be able to just simply loop through all of our info behaviors like we did previously let's get that formatted and we can just do info dot close info okay that should be all that we need so let's go back in the editor and test this out oh you know we're gonna need more cubes for this so let's go back to the scene view and let's duplicate these and let's just put a couple I don't know let's just put a couple around like that okay that looks good and now we should be got to hit play and we'll go back to the scene view and we're just gonna move the camera so let's move it out let's turn it around here and now we should be able to look at that and that pops up goes away beautiful everything seems to be working let's build this out and see what happens yeah so if we look at the cube the info pops up everything seems to be working pretty good alright so that's it that's all I got for today let me know down in the comments what you guys want to see in the next video and with that we'll see you next time goodbye

Channel: Third Aurora

Views: 25,521

Keywords: augmented reality, artificial intelligence, mixed reality, technology

Id: OE66gtiF8QQ

Length: 17min 18sec (1038 seconds)

Published: Mon Apr 20 2020