LeapMotion Hands and Fingers Walkthrough (with Processing)

this is a video walkthrough for the leap motion and processing assignment called hands and fingers first thing we're going to do is open it up and I should note it is important that we're using processing too for this assignment at the time of this video there's a version 3 but we don't want to use that so go ahead open up your sketch hands and fingers be sure your leap motion is plugged in and your green light is on and let's take a look at what we've got here so before we start this is very important we need to import the leap motion for processing library this is going to allow our leap motion to communicate with the processing to do that open up sketch click on import library and add library under our search bar we're gonna type leap motion and the first one you see here leap motion for processing by mr. Darius go ahead and install that one and once that's done we are ready to start so you'll see here at the top of the file there's a few objectives we're gonna get done there's one more thing to do here I'm gonna go ahead and copy this hand-drawn we'll use that here in a second but let's talk about how this sketch is structured and how you're gonna use processing to get these objectives done so in these objectives we're gonna add some shapes attach some shapes to your fingers move them around change some colors what you'll see underneath that are some shape templates I've included for you this will make things a little easier for you when you're putting shapes in there and we'll talk about each of these properties what they're doing here a little bit underneath that you'll see some location hints well we're moving some things around the screen these will come in handy and we'll get to that let's talk a little bit about the structure of a sketch before we start changing some things some of this will be familiar some of that's gonna be a little bit new if you've taken programming classes already first thing you'll see here is your instance variables right just like when you're making a class we're going to declare them but not initialize them that's the purpose of a constructor is to initialize your instance variables and that's what we see here instead of a constructor the Java sense we've got a method called setup what this does is it sets up your sketch with the dimensions and renderer that we'll be using we set the background color and we initialize in the instance variables in this case we're using a leap motion object this is the object we're going to use to communicate with the leap motion underneath setup right we've got another method called draw and these are generally your two most important methods in in any sketch right so your setup is run only once when you start the sketch setup as run it sets up your canvas that's a anything else that you put in here it's not just limited to size and backgrounds and then after that the draw method is run consecutively and this is dependent on your frame rate so let's say for example you can manually set your frame rate and setup and if your frame rate would be say 30 frames per second your draw method will be called 30 times per second so once for each frame that's rendering to your canvas I don't call the default is so we don't have to mess with that just know that draws run several times in a second so in draw it's important to note that what's run here runs from to bottom in order and the order in which we put things can matter a lot that's something you'll see when we get into some of these objectives so let's take a look what we're doing a draw very first thing we do is we set the background color now the format for background you see just 255 here you'll be familiar with this format if you've gone into HTML or CSS 255 here is the same as saying 255 255 255 right these are your red green and blue values if we've got all of them at 220 255 this is representative of the same thing on the code you see here is white if you've got 0 and 0 at 0 it's all gonna be black and any combination in between so you may notice we're setting the background twice rightly set up once here and once in your set up now what's the difference remember we said that the set up runs once this is what's done initially and then every time we call the draw right I think the default may be 30 frames a second so 30 times a second we're refreshing the background you'll understand why we do that in a little bit some of our objectives will illustrate why we do that just know that the first thing we do every time I draw is called is set the background now past that I've done some frame work for you already this is how we're going to access hands and fingers and get all that drawn on to the the canvas what you'll see here is a for loop right it's an enhanced for loop so we're saying is for every hand object that we're going to call hand inside of our leap objects get hands method we're gonna do this now this method extends through to the bottom of your sketch which means if there are no hands present none of this runs if there are two hands presents this four loop will run twice right one for each hand if there's three ends or four hands you get the idea now what you see directly after this isn't incredibly important to what we're doing here today but it is good to know so what we're doing is we're setting up some variables based on information we're getting from the hands objects now remember hands are coming from our leap object as an array or probably an array instead of the get hands method we're setting up positions we're setting up movement this is all sort of a behind the scenes stuff but I put it all here so that you can see what's going on and maybe get an idea for how this is being run in the background we also assign fingers per hand so for the first hand that's present we get a list of fingers for the second hand that's present we get another list of fingers after this there's a drawing section we'll be putting some stuff in here and inside of the hands enhanced for-loop you'll see that we have a finger in apps for loop so for each hand that's present and for each finger that's present on each hand we'll be doing some things here's some framework in the background that you can use to get various attributes from each of your fingers will be using get position for example here in a little bit and you'll see here that we have a switch case now we'll break this down a little bit but know that each case represents a single finger right so when a finger is present get type will return 0 1 2 3 or 4 depending on the type of finger that's available and you see with some comments here each of these cases represents a unique finger they said we'll get into that in a little bit there's also a touch emulation which means the distance from the screen or more specifically the distance depth distance across your leap motion right so if your hands are closer to you or directly over the leap motion or further across the leap motion this is what your cases representing we'll get into that so let's go ahead and come back up here to the top and take a look at some of these objectives and see if we can't do some fun things with this leap motion first things first we want to take this hand draw copy it and what we're gonna do is place it right down here in the drawing section now tabs aren't super important here but I like to keep things tidy so what this is gonna do is remember the location where for each hand we're gonna draw it now these are all things built in the early motion library so we don't have to worry about what's going on behind the scenes just know that for each hand we're going to draw it let's go ahead and test this out hit run you should see your canvas right and this is what we're seeing in our setup right our dimensions are 1024 by 768 little white backgrounds now when each hand is drawn we should see them represented as wireframes you can play around with this a little bit there's some neat information there your fingers are labeled and there's some other information on there about each hand time visible there's even a representation of grab strength how close is your hand if your finger is being extended you'll see that finger is highlighted in black and it's also represented as an integer above each hand under outstretched fingers now once you have this working we're ready to knock out some of these objectives go ahead and close this and let's look at the very first objective first one is at a red rectangle with a dark red border to the center of the screen now to do this we want to do this regardless of whether or not a hand is present we just wanted the book to show up in the middle of the screen so what we're going to do is we're going to do this right into the background so that everytime draw is called we get a fresh background we're gonna put in the fresh rectangle right here now grab one of these presets I've made for you we're just gonna copy that and toss it in right here on your background don't need that but we've got some properties here that I'd like to go over with you well I get sort of straight down alright so rectangle mode this represents where the quote-unquote center of your rectangle is going to be located when we say Center here it means that the position we give this will be oriented to the center of your rectangle object and this is in contrast to say using one of the corners in some location right so if we give it say 1 and then 1 the center of your rectangle will be at position 1 1 this we can leave alone strokeweight if any of you are into digital media you may recognize this terminology this is going to tell processing how thick you want your lines to be in movies say lines I mean specifically your border lines right so in this case we're just going to give it oh I don't know it's start with five it's not too big this is going to be your stroke color and in the objective we needed a dark red border all right regular red rectangle dark red border so I'm gonna put in here wonder if we can show line numbers this might help do maybe not all right maybe next time so right so into your rectangle mode stroke lead we've got stroke color they want a dark red so I'm gonna go with let's say 150 zero and zero right so a full 255 on the red would give us a solid red any number lower than that will give us a darker red closer and closer to black now so this should give us a nice dark red color under the fill now this is going to be your rectangle fill color you'll notice one of these is an RGB one of these is in hex they are interchangeable we can do hex here RGB here doesn't matter I just wanted to give you guys an example of both so in this case we want to read we're gonna use ff0000 in hex again if you've taken digital media this may be familiar on a base eight base 16 yeah base 16 we've got red green blue we max out our red this is gonna be 255 which should give us a nice bright red now we want to put in the rectangle position and I've got labels for what each of these spaces do better x-position y-position and remember this is gonna be the center of our rectangle we've also got the width and height of our rectangle so let's refer back up to the top we're we've got our location hints center of the screen is what we're looking for right on objective one we want a rectangle in the center of the screen so let's use this guy right here height and width both divided by 2 copy that and we're gonna post these right back in here now what this is doing is height is a value kept in processing same as width and those are represented by your height and width of your canvas so if we want it in the center we can divide the height by 2 which should give us halfway up and we can divide the width by 2 which should give us halfway across this represents the center point of the screen now how wide and how high do we want this let's start with I don't know let's go with 100 or both or if this is a rectangle I guess we should do just a 50 hi I'm going to do a hundred across feel pretty good about that let's double check at a red rectangle with a dark red border to the center of the screen let's give it a go all right so not quite in the middle of the screen but our border is good and our color is good let's see if I can figure out so x-position y-position this is important and I think this is where we messed up our exposition has to go first along with our width our exposition should be width so you just have to swap these around and I just to be 100 across and 50 tall to try that again looking good okay so objective one is done if you've got this far we're looking good all right let's close that let's look at number two I want to make a red circle with a black border appear over your index finger no placing your shape in a different space is going to matter and remember that this is happening sequentially top to bottom so we draw a background we draw a rectangle it doesn't matter if your hands or fingers are in view of the leap motion this will always be drawn directly after a new background now in order to get a circle with a black border to appear over your index finger we're gonna move to a different part of this sketch and the place we're gonna move to is going to be inside our enhanced for-loop for him so we know that a hand is gonna be present and in fact we know that a finger is going to be present because we want this to appear over our index finger now look inside of your switch we're gonna look at our index finger so remember for every finger we're gonna go through this right typically I guess not everyone has five fingers but most of us have five fingers so your for loop is gonna go through five times once for each finger every time this goes through we're gonna see a different case right so the first time this runs through what we get here is an integer return from get type that starts with zero then it goes to one then it goes to two then it goes to three and four so every time we go through this it's gonna run through one of these caches so first time is a thumb second time we go through is index third times Bill finger and on through the rest of the hand now if we want this into our index finger that means any time that the get type returns a 1 we want to draw something so we'll put in our circle here let's look at what we need I want a red circle with a black border over our index finger so we're gonna take this circle preset red circle black border yeah let's put this in our index finger try and go to your comments and same deal the stroke weight is going to be the width of your outline in pickled pixels we're gonna stick with 5 I like the way that looks we do 5 yeah stroke remember we want this to be black so we are going to stick with 0 which is the same as 0-0 0 but we don't have to draw that or we don't have to write all that in let's just stick with 0 representing black our fill color needs to be red just like a rectangle so our fill color is going to be ff0000 for red green and blue be sure you keep that pound symbol in there now the position is a little bit different right we know we want with a knife and this is gonna be a circle so let's stick with let's go with yellow with it 5550 hi we don't need an oval but what about position right how do we know where the finger is well this is going to come back to this variable right here we've got a finger position that's gotten from each finger now for this particular finger how do we get that position well I've included some location ends up here for the finger position we just call finger get position dot X for our x position in the same for our Y position so I'm gonna copy these guys and post them right back in here so we're getting our x position and our Y position of our index finger fortunately this works out to be the end of our finger looks like we've got everything in here let's go ahead and give it a shot so our rectangle is still there we know this is gonna show up regardless of whether our hands there when you move your hand in you should see that any time any time that there is a finger and index finger it attaches a circle to that that can be moved around regardless of extension or hand right so anytime there's an index finger we see a circle awesome looking good all right on to number three make the rectangle appear only when your hand is on the stage now this is gonna illustrate a little better about why we place this up here and where we're gonna move it so just to review we know that this rectangle is gonna show up regardless of whether or not there's a hand on the screen right what we want to do is move this so what I'm going to do is cut out lips need to cut out the whole rectangle we're going to move it where do we want to move it we want to move it under our drawing section why do we move it here because for each hand that's drawn do everything here and in this case we're going to draw a hand when it's present and we're going to draw a rectangle when a hand is present let's test this looking good so far no hands present we don't want to see the rectangle we move the hand in and we see a rectangle now why does it disappear remember that every time a frame is drawn we get a new white background that overlaps any previous background drawing or any other objects being drawn so this is how we get a rectangle will show up or an object to show up only we're not hand is present not so bad huh alright we've done three next thing we want to do is change the background color when you are in the hovering zone all right this is going to touch on the zone emulation or touch emulation remember talking about zones so what we want to do is change the background color when you are in the hovering zone so let's head down to the hovering zone which is gonna be this guy right here I may see that there's a little bit of feedback here and I'll show you where that's coming from or where that's going as a matter of fact you can see just from a previous run you'll see hovering present here along with a number what we're seeing here is finger ID and then touch distance so this is how your leap motion represents touch distance what do we need in here we need a background we want to change the background color so let's add background we're gonna do this with red green and blue don't forget your semicolons and what color do we want to make this let's say a hovering let's go with yellow so how do we get yellow we need red and let's think about our digital media so we take away green I don't remember well whatever color we get we'll just go with it we've had a background of hovering let's see what we get hey we got purple that's totally not what I wanted but we change the background whenever in a particular zone so now my hand is a bit closer to the screen I'm give it back into our hovering zone and then back again to our closest zone so you'll see that only in the middle we see purple and we're going to change that kiss I'm not a big fan of purple green of course you can make whatever color you want if you like prayerful you can stick about now you'll also see or touch distance let's see what's next we've got change the background color when you are in the touching zone so same thing as we did before except we're gonna have a different color when we're in the touching zone of course you can pick any color you want I'm gonna go with blue this time don't forget your semicolon so when hovering we're gonna see green in touching we're gonna see blue let's give it a shot so I've got my closest zone it's kind of small so you may have to take a couple of tries to see it our hovering zone is green and our touching zone is blue and change these by moving your hand closer to or further away across your lead motion or closer to and farther away from your your screen now last one make the rectangle show regardless of the background color change now remember we talked about doing this draw method of running this drawing method top to bottom so why is the color overriding our hands how do we get that rectangle to show up regardless of where it shows up well in order to get this rectangle to show up we know we're going to need to move it so let's go ahead and cut this whole thing out and think a little bit about where we want to move it so top to bottom each time draw is done we draw off a new background white we draw a hand we draw a circle over fingers and then we redraw the background depending on where our touch emulation is so in this case we want it to show up regardless of where our hand is that means that the very last thing that we need to show up are the very last thing we need to draw on our canvas is going to be a rectangle which means we need to put a rectangle not only in hovering but also inside touching you want to be sure that this is taking place after you read you all the background you don't have to get your tabs straightened out I just like to make it look nice so let's review what we just did the very last part of the sketch has to do with touch simulation when we're hovering it redraws the background green on top of that green background we're gonna draw a rectangle same thing for touching if we're in the touching zone we're gonna change the background to blue and then draw a rectangle on top of that so layers wise I think Photoshop or or illustrator any of your drawing programs the very first thing you draw is going to be your bottom layer in this case our white background and then we just stack layers on top as they're being drawn so we draw a hand on top of that white background we draw circles on top of that hand and down here we're gonna draw a background on top of the circles on top of the hands and very last we're gonna draw a rectangle on top so we should see rectangles on top of our backgrounds let's give it a shot white background is good let's keep our hand in from the back until it gets into view any second there we go okay so in the closest zone there we go we see a hand we move it a little further and we see green with a red rectangle this is good we move it further in let me get blue with a red rectangle and that's that's so hopefully throughout this activity you've learned that the setup is run once draw runs many times per frame I'll guess it's once per frame but many times per second draw is processed top to bottom so as you go you're stacking the layers on top of layers and how to manipulate this sort of framework that I set up for you in the next assignment you'll be using something similar similar principles a little bit of critical thinking if you need to help that you can refer back to this video at any time

Channel: Jake P

Views: 8,711

Rating: 5 out of 5

Keywords:

Id: _SfrBicrRNM

Length: 31min 28sec (1888 seconds)

Published: Mon Sep 26 2016