Mouse Picking // Game Engine series

hey what's up guys my name is jonah welcome back to my game engine series so last time we talked about how we could clear specific frame buffer texture attachments check out that video if you haven't already and today we are hopefully going to be more or less concluding our whole mouse picking trilogy i think it's a trilogy might be four episodes actually but um basically uh we are going to be uh making it so that instead of just outputting the value 50 every time we have to render a pixel into that additional kind of entity id buffer we're going to actually hopefully be able to uh write in our actual entity id into that texture so let's take a look at how that's going to work and get on with things okay so um again last time just a little quick little recap we basically uh the state that we're at right now is that if we look at this like pink cube for example we see the value negative one if our mouse is over empty space because we're now clearing this specific texture attachment that's holding this data to negative one at the beginning of every frame and then if we are over any any entity and by the way even though this looks like um [Music] uh even though this looks like it's uh all one entity this is three different entities left right and top and now we have a fourth one you can see it's still 50 right so we are actually these are three entities and they do have different ids and you'll see that you'll see that in a minute hopefully today so how do we make that happen well luckily for us every single entity in hazel has an id right if we go to the entity class we can see that we have this ant entity uh entity handle the type is ant entity what is that let's take a look at it i always encourage you guys to actually dig into the source code and see what things actually are so it is an id type what is an id type it's an ant id type so much in direction i love it i don't um that's the type it's a uint32t so we are what are we storing inside our um buffer inside our frame buffer as a texture attachment we are sawing a gl red integer well that's the texture format so basically it's a signed integer this is an unsigned integer that is a signed integer we could have stored an unsigned integer i just thought i wanted negative one just to be really clear for you guys that negative one is not a valid entity because zero is a valid nt so we couldn't clear it with zero so we have negative one when we have two billion other ntids i think we'll be fine ultimately but the point is it's an integer right this whole thing is an integer so what we should be able to do is for every entity that we render i want to be able to take this ant entity and somehow get it into my shader so at the end of the day really i just output it here so that should be my kind of end entity value right i want it to be over here instead of the value 50. so what are our options for this okay well how do we get data to the shader in the first place from c plus plus well the simplest way is to use uniforms right but uniforms obviously have the limitation of you can only set a uniform in between draw calls you can't just set uniforms i don't know you can't like batch together uniforms right like um you can't set the same uniform for three different it doesn't make sense right it's not kind of like a pervertex thing or a per object thing it's a per draw call thing and so um that doesn't really work for us at the moment without 2d rendering because as you probably know or you should know if you've been following the series if we look at sin um our renderer and the way the renderer the way that rendering happens the way that we actually draw a quad i just completely it just lost where i was the way that we draw a quad is we call this draw quad function which does it draw the quad no it does not draw the quad it adds it to the vertex buffer and then at the end of the frame when we call end scene and it calls flush and i pressed on the wrong one uh i was going so well um and i press and uh we flush right then it does a single draw call with all of our quads in our scene so those four quads that you saw with the white square as well as the three kind of faces of the cube that was a single draw call so there's no way that we can actually add it as a uniform right what we can do instead and there's a few different things you can do based on like if you're instancing you have ways to do that if you're not interested in a few different ways we could do it but but the uh simplest way and the way that we're going to be doing it is just packing it into the vertex attributes so what is what happens for every quad developer powershell go away what happens um to every quad that we draw we obviously set the vertex buffer to particular values like the texture index like the tiling factor like the entity id as well now this has one extra little piece of limitation right if you want to do this um and i want to i want to mention a few things here right but if you want to do this the renderer now somewhat needs to be aware of entities is that acceptable well i see no reason for it not to be acceptable i mean some programmers like to hide things from other things as a matter of principle and that's fine i guess but ultimately speaking this is renderer 2d is hazel's renderer hazel works with entities why would the two not know each other right that that's not a huge deal to me right um tying things together and again it might not even have to it's not like i'm suddenly going to include ant into renderer 2d all it needs is an integer all i need to know is that hey this quad that you're rendering i want like an additional little payload on it we'll call it the id of the draw that's what i want you to do and that's what i want you to output but the other important thing here to note is that this is a editor thing this is not like a runtime thing right you you are not going to be outputting an extra like you know you're not going to have an extra frame buffer attachment just for entity ids in your shipped game that could potentially be running on like a phone right that's just not going to happen first of all that's a drain on performance but second of all unless your game specifically needs mouse pit like little tap to pick things which isn't that uncommon um and if it does need that maybe you would do it in a different way anyway and maybe that should be your job not the not hazel's job unless you do like you know you expose some kind of like um you know raycast or something into the 3d world that maybe you want to do there whatever but the point is what we're writing here specifically for the editor so this does not need to exist everywhere so what we end up doing is first of all this shader we're quickly realizing that this shader is becoming an editor shader this isn't a shader that we use for like for shipped games because having an additional attribute here that's going to be our entity id right this is what we're going to do in fact we've just written that code this is not something that needs to ship in a game right and so um take just remember that because a lot of people you know are very quick to point out hey man that's that's so inefficient that's so i can't believe you're doing this okay first of all show me how inefficient it is show me some actual profiling data and then maybe i'll listen to you but also hey man it's for the editor it's for the editor that you know this is hazel hazel's gonna be a game engine you don't expect unreal engine 4 or whatever to run on like you know a chromebook or something like it requires a decent computer to run properly at least right so in other words my kind of allowance for performance here is slightly different my budget will say in terms of frame time is slightly different for hazel's editor for hazelnut than it is for hazel's run time which we will refine when we actually get to crossing that bridge right so it's a bit of a different story um but also some things are just a lot faster than people seem to think and so people always like like saying oh my gosh that takes that's too slow it takes point five milliseconds and it's like well dude maybe that's something that i'm willing to uh part with especially if it for example only happens when we click or something like that right because well actually this isn't none of this is slow really the the only slow part we've really introduced is this kind of read back which again as i mentioned only needs to happen when we click technically unless you want to do certain other things anyway i don't want to get into some rant again um but this is what we're gonna do right so how do we set this up and and i guess the the primary reason i started talking about this is because at the moment right what we have is we have a quad vertex i'm gonna add a little section here that says editor only and then i'm going to add my stid the reason is this doesn't need to exist for the run time but i don't want to start making like a editor quad vertex and run time quad bit i don't want to start doing that yet right because when we actually get to the runtime that's when i want to start doing that for now we're very much focused on the editor and the other thing is guess what you can run the runtime with this and not use it right um unless you're actually getting to the point where the performance is really tight and you're like okay what can i do let's try and optimize this now then you can start being like okay well this doesn't need to exist let's remove it from the runtime but um you know just uh just just kind of a um just a just a general word to everyone to just relax a little bit because uh you know it's okay it's okay we'll remove slow things when necessary don't worry i'm very capable of optimizing code okay so let's go into our drawing functions the big thing here though is again i want to draw a quad i don't care about the entity id can you please just not record it that's valid um that's totally valid what i'm going to do is uh i'm going to uh actually add a different set of these functions there's a few ways you could do this we could again add another overload to this function with like int ntid set it to negative one by default so that well you can print negative one if you want it's like the entity's not there it's like it's the clear color right but um what i want to do is actually add a different kind of entity right um so and by different entities sorry i'm in a different function so one thing i started thinking of though is we probably want to provide a default for this in fact we probably should provide a default for everything we haven't because they get they all get set but this is something that maybe we don't want to worry about right i also don't really like the idea of duplicating all of the functions either um we could obviously create another because we have a lot of these right we've got one two three vectors vectors um textures not textures yes we just have that and then again adding another set of these overloads i really don't know if if we should do it this way um because what i was going to do off the top of my head was just draw quad entity or something like that and that would include like entity id um which is fine but the only downside is that well yeah because the thing is like again what this enables you to do is first of all for things like ui like you know you're not only drawing quads as part of like a sprite renderer component that's actually a good point though right because specifically remember the way that we're using this the way that we're actually drawing this maybe this is worth doing um so let's go to like scene maybe this is worth exploring because um whilst i did write this code on stream ages ago during like a live stream that i actually posted on youtube i'll have it linked up there as well i'll also get um that was uh like that was more rough i wasn't really refining the api i was just getting code to work so uh now i'm thinking more about the design actually live during this episode but you know we're passing in the sprites sprites are going to eventually have textures maybe it's worth having a function called drawsprite drawcord is fine right but remember that there's there's there's kind of different use cases for this there's different contexts for all of this right on one hand draw quad can be used for rendering like you know things that have nothing to do with entities things that have nothing to do with your scene like for example i want to render a button i want to render something part of the ui i want to render a little uh i don't know like a little visualizer for like a light or like a little bounding box visualizer that's not that's not that's just like a debug thing right but it's still like a 2d rendering in a way right because it's a 2d primitive so um these things don't necessarily have to do with a scene in hazel or like the entity component system so maybe we will actually use a function called draw sprite and the thing that makes this special right this draw sprite function which first of all let's go back to scene and see how it's handled so it actually does get taken in from a transform right um so we'll actually grab this draw sprite will grab a transform i don't want to use the position but the thing that this will actually do is instead of taking all of this stuff in i wanted to take in a sprite component controversial perhaps because now i'm using the entity component system i mean this isn't really the anti-component system it's like the component but i'm using it inside here right now by the way this is going to open the door to a lot of cool stuff because at the moment i mean what are we even doing man at the moment look at this right this can only draw a colored sprite we can't use textures we don't have tiling factors we don't have tint colors these are all things that we can add to this bright renderer component and they will all be within here it will take this in and then it can potentially call one of these functions with the right parameters that's kind of what i'm doing in a way but we'll probably end up writing our own draws anyway so inside here um let's move this over here i'll do this here so at the end we'll do our kind of sprite stuff so sprite render component now what else do we need we do need the entity id right so i'm just going to do int entity id that's it so now we have a function that we can run instead of what we did uh with render a draw quad we're going to do renderer draw sprites right because these are sprite renderer components uh transform get transform sprite and then the object which guess what is going to be just entity cast into an int that's it let's get rid of this function now and let's make that other function work i think that's i actually really like that idea to be honest um i just came up with this now uh but i like it so there you go sometimes i do stuff live um and i think this will work out well so let's go ahead and implement this function um we'll have to also include components here i don't know if it's work it might be worth like dividing components soon into various header files just because we don't necessarily need to include all the components here um wonders in scene what's even in the components file though it's just like we don't we don't include um oh we do include entity which probably does include end so we do include end technically but um it's a scriptable entity includes entity what's that even used for if there's a native script component we might move this stuff out of here because yeah this is most of this stuff is just data right but this scriptable entity is a bit more complex and it has an entity which includes entity which includes and so it does end up being a little messy but um we can deal with that later all right so draw sprite is implemented um how do we make that happen so i mean ultimately at the end of the day what are we trying to draw we're trying to draw a non-textured sprite with a transform which is just this function at the moment i'm going to copy this um and uh yeah i'm not too happy about the divergence of code [Music] but the problem is that the problem is that it actually has an ntid but they all do at the moment so maybe it's okay maybe it's okay okay let's take this away this should be negative one anyway i don't know why we're at zero um let's take this away and let's do this let's call draw quad we're going to call that with transform with src.color um so that's this one right it's got a tiling factor it's got a tint color i'm going to add an entity id of negative one okay two specifically but this takes in a texture okay ah because you can't have tin colors if it's not that okay it's okay we'll extend both of these right with default parameters so draw quad now we'll have an stid this i think will only exist until we remove this because we actually have a runtime thing but it's negative one anyway so you'll be totally fine to do with uh you'll you'll definitely be able to do that this stuff with um you know if you don't want to draw an entity it's just going to set negative one there which could be a bit of a waste but um i think it's okay these are kind of the things that i think about when i write code like this okay so that's done uh and then we also have our this one which again will do the same thing uh one more thing we have to do by the way is actually add it to here so shaded data type int um i don't think we have any we might have to do something special here because this layout may not be able to deal with integers because there's actually a special attribute pointer function you have to call for ins um so vertex buffer where does this happen though we have a layout where we settle out in vertex array yeah i think so okay can someone please help me by the way with where this code came from because i didn't write it and this is like this is for like instance rendering which we have not covered yet so i don't know why this is here to be honest um so maybe someone will track that down for me in the comments or on chat or whatever and let me know because i'm very confused anyway um so basically what we need to do for these right is something a little bit different we need to use a different function um and i think for bull as well be wrong whatever i'll just write the code and then you guys can fix it and pull requests you know me and that is basically we can enable it right but we use vertex attrib i pointer instead right um there's no normalization for this and i think that's the only difference right but this is used for integers not for floating point types um this is still fine i think there's just some weird offset stuff going on but this is uh yeah everything else happens the same so it's just important to use that i function for integers uh otherwise i don't think it'll work properly so back over here now so we've got everything basically figured out now right we've made sure that inside renderer 2d we're setting this as an int which it is uh let me just line this up a bit i always i never like it when stuff isn't well i don't know it's something i like align everything it's just some things because i'm definitely not one of those alignment freaks where everything's aligned but some things like layouts and stuff i like to have me and i think i did the same thing with this um during the stream i aligned it and like this as well you know that's why i have the plus zero here it just looks nice okay and it's easier to read so um we have the uh color and then i think just the entity id and that's all we need to call so that will draw quad um and then obviously what this at the moment this function looks fairly useless because it just does this and we could have just done that from scene renderer but this function will actually check to see if the sprite renderer component uses a texture instead maybe use a different one not a different shader but it'll pass that into the shader it'll deal with tiling factors it'll do a lot of stuff that's why it's here okay um now we need to make the shader do what it needs to do so let's go into here and so position color texture quad okay all this stuff um so in entity id let's do this let's do out flat and i don't think it matters if it's flat or not to be honest because there's nothing there they're going to be identical values just like with text index for each uh for each um vertex but anyway so yeah we'll do that and then v and cid is just simply what we'll output here and that actually should be it like if i just hit f5 i think we should just be good we had some kind of issue here so let's open pink cube all right not quite so it's coming in as negative one oh actually i know no actually do i know why um let's go check it out so okay well that's kind of promising in a way negative one at least it's not like garbage values probably just forgot something to be honest um so yeah draw quad takes in what now a transform a color and entity id which we are providing here into draw quad and then that's what it's setting for the quad vertices so why is that not good uh because that definitely should be okay um i may have just uh i mean is this the only point where we draw stuff on update editor what's this function arctic runtime i don't know i don't even know if we're using that so yeah we're just using the wrong function here i think that's the problem i don't know yeah i don't know why we have a yeah run time so that's because we added the editor camera and then we also have a camera in our scene and we kind of can toggle between the two technically all right so um there you go so if you look at this pixel data over here now guess what when we move our mouse over these three faces you can see that it changes right negative one and then it's three it's zero and it's two pretty cool stuff right so the beauty of this and i'll leave you with this before we so we won't do the actual clicking today but i will leave you with this right we'll be able to see what nc we're hovering over so when we actually have uh where do we have all this stuff usually so stats i mean we'll put it in stats for now but um maybe over here um so let's do this text harvard t uh and then we'll do a little s so let's do this um [Music] when we read this pixel back instead of pixel data and instead of printing it i want to do m hovered entity so we'll do entity and hover empty we'll set it to that right uh and then we'll um that's actually an entity okay sure so let's do let's do that and then we'll do m hover density equals pixel data um we'll do uh okay so we'll do this if pixel data is negative one poverty um equals nothing right um [Music] else we'll do pixel data and this scene so out of scene is that a how does that work handle and scene what's that a ref saying so why are you upset oh because i'm handle nice and see okay there we go right so um i mean ultimately i'll combine this so it's a little uh bit more so if it's negative one we'll do that otherwise we'll do that maybe that will make more sense all right there we go so hover it nc equals either just a blanket like an an invalid i guess empty entity or an entity that's actually real based on what the value is and then what we can do here is basically say that if um so we'll do std string i don't really need to do the sd string but we'll do se string name equals um none and then if hover density we'll set name to hover identity get component tag components and then dot tag name dot c string all right so now what should happen right is when we hover our mouse over an nc it should tell us what entity we're hovering over over here so hover density is none at the moment and then if i open up pink cube and i hover over a particular entity you can see up here hopefully it's very small maybe i'll uh go to i'm gooey layer and actually just make the font size larger which of course we can't do it as we can 38 yeah let's do 38. way bigger so if i look at pink cube you can see now that depending on what i hover over it actually says right left top if i create a new entity and give it a sprite render component and i'll have to move it over here then you can also see that as i hover over it it says empty entity because that's what it's called um and then all those still work obviously and everything is good okay so hopefully uh that all makes sense let's revet the font size um next time all we're gonna do is instead of printing this for example we can actually just go into like our event on event and add a little um mouse clicked event and if we click then well we've got the hovered entity so technically with this system we could just set the selected entity and scene hierarchy panel to be the hobbit entity and our work is done um there are going to be some bugs and some other things in the way for example you'll note that uh you know if i'm moving an entity i probably don't want to select other stuff just because my mouse button's pressed or you know sometimes when like for example i go to left and i want to drag right clicking in this case shouldn't select the entity because if i'm trying to drag this and i click then technically the this uh top face is in front and you can see up here it says top so i don't want that to be so there's a lot of caveats to kind of figure out and fix i guess with this but that's basically the gist of things and that is how mouse picking is going to work so i hope you guys enjoyed this video if you did please don't forget to hit the like button below thank you all for watching hope you've enjoyed this series we're getting into some some of the media stuff now don't forget that you can help support the series by going to patreon.com slash the channel you'll get access to like hazel dev and past live streams where i work out this stuff and uh there's lots of really exciting stuff going on in hazel dev as well check out those devlogs definitely if you're interested in where this series is eventually heading to hope you have an amazing day and i will see you next time goodbye [Music] you

Channel: The Cherno

Views: 17,014

Keywords: thecherno, thechernoproject, cherno, c++, programming, gamedev, game development, learn c++, c++ tutorial, game engine, how to make a game engine, game engine series, mouse picking

Id: GKERbQ7Sqyg

Length: 30min 14sec (1814 seconds)

Published: Thu Feb 11 2021