Handmade Hero Chat 015 - Interview with Team from Carnegie Mellon

all right all right we should be live now in theory and there yes the twitch buffer has like you know finally catch up with with however many seconds they need yeah we're good all right so yeah I guess we can start just kind of introducing ourselves so we'll go around the table I'm Jake I'm one of the programmers on the team I haven't told you I'm also a programmer I'm PD I'm Jared I'm the producer I'm sitting off screen taking notes okay the fourth the off-screen programmer yes okay and yeah just kind of give you a little bit of a background about our project so we're at school at Carnegie Mellon at the entertainment Technology Center and so part of that is after your first semester you do semester long projects where you just kind of focus on doing one thing and so this semester we're focusing on building a game engine seeing how that goes so we kind of been like doing our research in it of like okay what before the semester starting up like what goes into a game engine there's a lot of things out there like handmade hero and Jason Gregory's game engine architecture book it kind of go over all of it but we really didn't feel like we know how to do it until we kind of sat down and tried to do it ourselves that's very true that's very true and so we're going through that and then we're the goal of the semester is to kind of go through and when we were looking at how other people did it we found a lot of people who would wait till the end a lot of like this other student projects they wait till the end and then tell you all about all the cool features they did but no one really there wasn't many others like your hand made here a series that kind of went through like their decisions and their thoughts throughout the whole thing and so we kind of want to go through the whole semester of like documenting okay we made this decision and this is why it's probably not the right decision but we'll see where it goes and then two months down the line kind of say oh this is why it was the wrong decision and then talking to professionals like yourself about okay we're doing this project what what kind of advice do you guys have and where you started and kind of what other engine developers have to say about the topic that sounds really cool actually keeping track of why you make decisions is definitely something that I wish I did more often there are certain projects where I've been like I really want to make sure I get some of these things correct and on those I always take meticulous notes on them and other ones I just don't for whatever reason and you usually end up wishing that you did because you're like why did I think this was a good idea again and like it helps to be able to go back and know because sometimes it wasn't a good decision and but it's interesting to know why you thought so other times you forget why it was a good decision and then you go to change it and only after like a couple days of doing other stuff do you run into something like oh that's why I have a trite few you know future self was dumber than past self in that sense so yeah I do think that's a very good good practice to try yeah we're very curious kind of see how it works of like being further down the line like how much we learn over just doing because we only have about three months to build this game engine so it's very simple in yeah we've just seeing how much progressively I bet by the end of the semester we'll kind of turn around and like all that stuff we made at the beginning was just almost garbage compared to what we're doing yes yes that's that's probably true but I mean I guess the other thing I would say is there's definitely a tendency to undervalue code that you have written that is working a lot of times those programmers it's very tempting and and you're not wrong right when you look back at code and go wow I could do so much better but one of the things interesting about that that I find anyway is that a lot of times there are a bunch of things you did in there that you're forgetting are important and that work and like it's almost like I tend to look at code in a little bit of more of a Darwinian sense these days than I used to where it's like code that can do something has a certain value to it that code that's in your head that is better doesn't have yet right and depending on how complex the code that your the problem that you're tackling is sometimes you can drastically undervalue those things that went right simply by virtue the fact that the thing works you know that they did and so I would also I often try to encourage people especially you sort of programmers with less experience to sort of go don't be too hard on your existing code if it works because there's probably more to learn there than you think there is or another way to say it was you did better than you think you did I think there's a lot of talk about people you always say clean code or is it elegant or is this sort of thing and there's a tendency to discount it worked as a really important part so always be a little bit proud of something that worked and and take more time than you might otherwise to appreciate why it worked because there's often more there than people think there is and then they go to like do some high-minded redesign of it and it turns out worse than the original version because they didn't spend time appreciating just like what went right you know so yeah separate thing but yeah this whole whole stock we're just kind of trying to get pull as much information like that out of you so as much like sure so how we're thinking of doing this interview as you saw in like the questions is a kind of like a pre-interview where we kind of try to get to know you a little bit better with some kind of more basic questions and then kind of go into an interview more focused on the technical details of game engines feel free don't feel free like you need to hold back or anything we might not necessarily understand but that's kind of the point of this okay yeah and so I guess first kind of question simple question is a key tell us a little bit about your background your your schooling your hobbies and stuff like that but you think find it interesting that could be interesting for game and development type of work so I guess you know this this answer is probably not going to be very interesting so I'll try to keep it short essentially I started programming very very young compared to people probably from my generation simply because a lot of people wouldn't have had access to very much that they could do to program but my father happened to be a programmer and that was somewhat more rare in the 1970s I was born in 1976 so the number of people whose fathers or mothers didn't that in that day and age actually it was more common for females to be programmers as well actually I think percentage-wise there were just very few programmers at all right yeah and so the chances that you had a parent or you know in relative or something who is programmer or even a neighbor was low and the chances that you had programming materials like compiler or books that were about it or someone you could ask was also low so it was pretty rare to learn as early as I did I learned when I was about seven years old my my dad taught me basic on deck rainbow computer and I just loved programming from then like forward and even before then I used to love typing on a computer just the act of typing so I really never did anything else I think I maybe gravitated towards programming game engines because it's a really interesting technical thing to do and kids love games so at an early age you're attending that's like what you see most I you know I I wouldn't have been naturally drawn to like the the simulation of you know thermodynamics that someone was doing it Lawrence Livermore Laboratory at that time as interesting is a technical problem that may have been a child is not aware or thinking of these things today with the internet that may be very different people may be getting exposure to lots of other things that a computer can do and they might think more about that but you know I wasn't so what ended up happening is I just really I worked on that sort of stuff just on my own at home on various different computers that we had and I literally just never stopped I didn't go to a college or anything I just ended up going into the game industry sort of right out of high school and I never have really done anything other than game engine programming hobby wise I do think I've I've always been interested in sort of the artistic things that go along with game development so hobby wise I do tend to look into and practice other things like I've learned to do 3d modeling and texture and I've learned to do sketching like concept sketching in that sort of stuff and I and I learned to play piano I took lessons as piano as a child and that was very helpful to understand music stuff so there's been a bunch of other things that I've done that helped to understand and allow me to communicate with other people who work on other parts of the project in ways that are much better than I would have otherwise yeah but I never really took seriously any other profession or occupation oriented thing I've never really had that experience for all I know I would like some other occupation better I have no idea what I would say is that the only interesting takeaway I think from any of that it's just that I would say that learning to do other things even if you don't intend to do them is very useful from not only the fact that you can communicate more effectively with the other people who are going to be doing those things so an engine programmer who knows how to do 3d modeling is kind of very valuable in a lot of ways on a team because the modelers have no way of communicating with you other than in the language that they understand and about the tools that they understand and if you literally just don't know all of like what goes into making a 3d model if you if you don't know how to paint something in ZBrush or you don't know how to like resurface something in 3d studio or whatever um you don't have to be great at it but just knowing what goes into that can be very nice because it means that you can then understand exactly what they're going through on a daily basis and be more effective at helping them or understanding their complaints when they have them or assisting them in getting their tools into a state that is better for them and I saw I feel like those things are very useful for engine programmers to have if you don't like doing those things it's not a bad thing there's plenty of room for engine programmers who just want to focus on optimization there's a whole suite of people who just do that but if you do find yourself being more I don't know if polymath is the word for it but if you do find yourself wanting to have a broader experience those things are helpful and you will find those valuable as you as you sort of work on technology even though you're not technically doing any of those other things yeah question follow-up question to that is do you find artists or like designers coming to engine developers and asking those questions or is it more you as an engine development kind of need to go out and see what other people are doing and communicate with them that way it's both so one of the things that you'll find is that artists in particular I think are very boxed in in their understanding of what a computer could be doing oftentimes and this is because they have been sort of brutally treated by tools like Photoshop or any Autodesk product you might care to purchase and so you know they may have gone into art or computer art with sort of a more boundless imagination of what was possible but they have been beaten down I mean you can pretty much expect that any artist has been beaten down to the point where their ability to imagine what a tool could be doing for them is very limited because they've constantly faced just I don't want say rejection but they've been constantly faced with barriers to them ever realizing any thing in their head they may have had right so anything inside that boundary you can usually count on them to communicate to you so if it's just a case of like I think this thing should be working in Photoshop but it's not or we would like a way to be able to use this Photoshop feature but your engine isn't supporting it those are the kinds of things that an artist might bring up to you things that they have sort of been taught that the computer does not do they don't know programming a lot of times and can't be expected to realize that that's just because Adobe doesn't do that right like it just yeah that thing isn't exist so anytime that you can observe an artist working or spend time just looking at how they have to go about creating things or working with with the tools that they use to make things is very valuable because you will see things that a programmer obviously knows could be fixed but then an artist just may not realize can be fixed and again I think that's just because that you know they're an artist they think in art terms and they have never have the ability to change the tool in any meaningful way a lot of times yeah so they just they just think that what they have is what they have to live with and and they especially the older the artist is or the longer that they've worked in with computer stuff the more they will just accept the reality of the tools and you can no longer expect them to bring you any kind of pie in the sky idea even though that may have been something they would have thought of before they were sort of taught rigidly what is available to them another way let's say it would be artists are used to adapting to their medium when someone gets a charcoal Penn State have to use charcoal and they learn to use that medium so it's exactly the same computers after they adapt to the medium they can no longer tell you what they wish they would have because they've been trained out of it essentially most the time all right yeah that's a really interesting point of view on that guy actually going back a little further to when you were saying you started programming at seven and you're interested in games too when when you were around seven were you making your first games or were you just kind of experimenting making like hello world type programs yes so when I was seven both of those things are accurate statement so when I was seven in basic basic the the the basic language was very much what you might think of as on like an Apple 2 or something if you've ever seen one of those old form you know in a history museum or something so I wanted to make a game but not only was I not really capable of understanding a lot of the stuff that would have gone into anything that you would think of as a game today but basic wasn't super capable of doing anything at any speed in those days anyway I mean even if you just wrote a semi language you couldn't do things at very high speeds in those days for obvious reasons so all I made for games were things that would basically print out like some text like imagine a text adventure where instead of actually because I didn't know how to implement the back of a text adventure it's more just like a multiple choice right so it would print out like a line of text and like be like what do you want to do and it would be like one or two and I would just hard code the results right so you can think of it kind of of games that are sort of on the level of interactivity of you know a today's game by say David cage do you want to push the button to do the thing or not right like and that's all you get right so it was that without all of the fancy graphics was all I could do for obvious reasons a to my brain was just not capable of understanding at all no experience and then the fact that computers are so slow to write so so yeah they were games but yeah yeah I think everyone's kind of first game mm-hmm so then going off off of that when about would you say you started creating your first engine like about how old / like and water around what were you thinking of building so the first engine that I liked I would say that I attempted to build I guess I would say that I did build it is in eighth grade for a science fair project I built this thing that was essentially a I guess you could call it King's Quest without animation so - maybe a better example would be like a magnetic scrolls kind of a thing but from the old days where it had like a picture and the picture could change based on the state of the world so it wasn't like it was completely static it could show you different pictures depending on like what room you would go to or like if you open the door it would change the door from a closed version to an open version but there is no real like animation in any real real time sense like you didn't see characters walking around and there was really no spa animation so you know pretty pretty back back pretty far in sort of the tech center and what you do is you type in a line of text and it would execute it so opened or closed or North those sorts of things that's yeah you could do any you know whether or not you want to call it a game engine as opposed to just a game in an engine that we're sort of together I don't know but it was design started to be an engine so there were like separate pieces in a sense and that was the first one that I really did and you know obviously it wasn't good but it was it existed something definitely so kind of moving a lot forward in time now to handmade hero so we're as we're kind of building up for this project we were checking on a lot of your videos and you're at like 470 now I'm sorry something like so I'm going on as long as this how do you do you ever refer to any of your past streams or how do you keep track of things going on so like with such a long project by yourself so I think there's a couple things to appreciate about handmade hero which is that the development streams are very short they're an hour to two hours there's really there's only once or twice we've done like a marathon where I did four hours you know so 477 or whatever right now let's say let's just say 500 500 hours or even a thousand hours of development I is really not very long for a 42 year-old programmer that's you know a couple months of development time if I was working full time and so I think that the it's easy to think well this is so many episodes because if you were just coming at them to watch them you know when you were gonna like I'm gonna sit down and watch handmade hero the whole thing it's like oh my god right but it's really not very much programming like if you think about how much programming each of you has already done in your life it's probably pretty similar I mean you spend a lot of time programming if you're a programmer and the amount of time it would take you to get to five hundred or a thousand hours of programming isn't really that much you know and if you haven't already hit that now you'll probably hit it before you graduate from whatever program you're in now but you know even if you don't do that your first year of full-time work you'll definitely hit it right so I don't know how much program guys have done but you've either already done it or will soon do it and so it's easy for people to overestimate the difficulty of of keeping track as I get that question a lot and I think it's just because people don't realize it's bigger it sounds bigger than it is it's actually a very small engine splendor a hundred thousand lines of code every time I'm programming I'm explaining at the same time their entire episodes that are nothing but whiteboard essentially or a blackboard in this case discussions and Sue the amount that I actually have to keep in my head in handy here is very small but if you watch some serious I'm like I will say on a day I don't really remember how we did X like you can find episodes where I've said that and what hopefully you can see is what I then do is I just go into a very straightforward process that I always I always just assume that if I wrote code competently in the first place a cursory inspection of it later should reveal to me how I decided to do things it won't necessarily reveal why because I may not have encode anything I tend to not comment code until it's done done cuz I find that the comments end up being out of date and end up being counterproductive I write it's actually describing the thing that was before the latest one you know and it's actually worse than having a no comment and so I I think that part of developing a good programming style it has a lot less to do with things that people normally focus on which are like these rules you know did i overload operator equals to prevent a copy big they focus on all of these rules that they really have never even really tested or proven in production really save you any time and they ignored like the most important things which in my opinion are how easy is it for me to read my own code later and know exactly what it does right I mean and that's like one of the most important things you can do and so that's really I think what keeps hand made heroes so flexible and easy for me to not have to worry when I come back to it on a weekend to start doing a stream is I just know that's part of being a competent programmer is writing in a way that doesn't require me to keep it all in my head I can easily go look at the function names and I just know what they do and this is doubly easy if I'm the programmer because I know what assumptions I make and if I just always make those same assumptions then I know I don't have to investigate those things further you know I know I'm not gonna call new in the middle of a thing because I never do that right so a lot of things like that if you develop a programming style that you find is effective simply leaning on that style in your own code will allow you to keep it flexible keep it easy to remember and recall when you do forget so I think those are the two aspects one of them is about becoming a good programmer in a sense which is is is making sure that your style actually pays real benefits and not hypothetical sort of textbook unproven things but the other one is just I think people overstate the difficulty of the handmade hero thing it's it's not that hard when the code is that small you know try doing that on the Unreal Engine codebase right I mean I would not be coming back to it and going like oh and I'll just do this I'd be like what is going on here like let me step through it in the debugger for six hours before I really know you know so yeah and you said like interesting there where you're talking about your own personal kind of standards and conventions so when you're on the team of people who have maybe have different kind of things that they lean on do suggest that everyone kind of you kind of start recognizing people's styles or do you try to force people and conforming into one style which might get them uncomfortable with something yeah like I I know exactly what you're asking and it's an excellent question and I think that honestly I think there is no good answer to it I'll I'll say a few things about it but I guess what I would say is that is probably one of the biggest problems in programming today if you imagine that programming in a sense is a two-part process because I think it is one is and they're not separate processes but there's two you know sort of things that are going on anytime yeah one is coming up with a language and the other is speaking in that language so we talk about having a programming language but we really don't have the programming language that we use what we have when we talk about a programming language is a building block for the language that we will use it's more it should probably be called a programming alphabet if we're honest or a programming phonemes because usually what happens and in any game engine this definitely happens is that first you take the language that you have like C++ and you build your own language on top of it which is sort of a functional language of core things that everyone will talk about you know how is the memory managed and how do we implement sort of the render pipeline and what how do we pass things back and forth what is the job control story you know all these arts things yeah those things form a secondary language and this is a it's in some sense a very recursive process you could think of it as making lots of languages on top of each other and some perming languages and some program projects are so poorly thought through that there's like languages on top of languages on top of languages in some sense right and so if you think of each programmer is sort of having their own ideal language that is what they would like to see that substrate that's right on top of the regular language that they all had to conform to yeah they're all fluent in that presumably their own one so when you bring two programmers together you essentially have a problem where it's like hey you and I should write this book but you speak French and I speak Spanish right and it's like okay well how do we come together to write this book that's you know still in a Latin style typography but it's neither of our languages and we get right and so I think that the jury is still out I might even say that the appellees and the plaintiffs and defendants are still out on what on what the right way to do that is there's certainly procrustean approaches that are like here at you know company X we demand everyone have this many tabs and their thing and every class looks like this and there's a file for every block there's people who specify everything right and it's like everyone must have conformed there's other places that are just like do whatever you want and if that will figure it out the boundary right will do something and and everything in between and I think that we don't know the answers to what's right about that and I would be absolutely lying if I said I thought I had a good solution to it what I can say is that there is no question in my mind that there is a difference in speed efficacy and quality of code that comes from shifting from your native language into another one so shifting from your native programming style into somebody else's costs you and you have to balance that cost against the fact that when we work together we can get more done if we have more competent programmers on a project assuming each of them is capable of writing something useful for this project we can get more done you know up to a certain point obviously there's communication costs that might kill you at some point but four or five people on a project should be able to get more done than one person on a project my experience with working on the witness was definitely that I took a pretty massive productivity hit working in that codebase compared to my own and and there's nothing you can do about that i I do build some of my own language in there like I put some of my own tools in there as overtime so that I can be effective but it's on an on-demand basis and there's an entire article I wrote about a time when I had a bug that I had only because I made a wrong assumption about the math library right and that's a perfect is like the reason that me and John blow are both able to write certain part types of code quickly is because we both make assumptions about what our math library does and if you couldn't make those songs we I had to literally read the code for cross-product or something every time to figure out whether it was right-handed or left-handed I would be much lower at having way more bugs right blah blah blah and so it's really hard to overstate just how important that is and I think a lot of people don't necessarily realize it because if you've never programmed by yourself on your own engine at a time when you were a fairly mature programmer if the only time you ever did that was like in your basement or whatever right and you've never done that after you've had a lot of experience and are more self-confident and have more of an opinion about what should be going on you may not realize how fast you actually are when everything is done the way you expect because you've only ever done it at valve or something where everything works maybe a little bit differently than how you would want to write or something yeah and so it's it is a very important thing to be aware of I don't have an answer but it's important to understand all right yeah thank you so then I've kind of going off of that you mentioned a little bit about are you working on the witness and kind of adding your own layer into there has there been like a hat per se that you've done recently or seen recently that you thought was a either really unique or kind of interesting to to do or like way around the code base or something like that how can you be more specific what does that mean a hack so kind of going off of like not necessarily following the rules that are kind of laid out by like the other person programming it or so typically probably not like handmade hero but maybe like when you're working with someone else's code so I may not be understanding the question correctly but if I'll restate it to see if I am so what you're saying is that if you are programming in a code base that was not that you did not yourself design yeah is there anything that I can think of that's a particularly interesting thing that's done in that code base that goes against the grain of the code base but that was interesting is that yeah yes that's what I mean so I guess what I would say is that I would really I would probably never consider that to be a particularly grad of flying experience so know that to me is just always a sign that there is a problem with whatever the code base is because generally speaking if I have to do something that I feel is antithetical to how the code base is architected or works in order to get something to work properly then fundamentally speaking that's really just more of a statement about the poor quality of the underlying thing and not a statement about something good about what I did so if I you know to put it in other terms I might say so you know have I ever felt particularly psyched about something I accomplished when programming on Microsoft Windows and the answer is no I've just been aggravated the entire time that it was so annoying that I had to go to these Herculean lengths and maybe you know I could look back on it and and warp it into a sense of pride that I was good enough to make this thing that worked around a really bad system but I guess at this point anyway in my life I don't have a pride about that I just feel bad I just feel like this is it was a waste of my time and it's gonna be a waste of everyone else's time and it's just a shame that stuff like that's still around and never seems to get fixed I'm sorry to answer that question with a downer but yeah going off of the kind of your answer of that when you notice something that you're like you feel like you're doing something like that is there you do tip and like you're working with other people is it typically kind of go to address address the person or do you kind of try to dig down into what they've done to figure out why it's like this just kind of curious about your thought on that yeah okay so that I that I definitely I think I understand that that question um so I guess what I would say is I would not take my advice or perspective on anything that has to do with that so I don't like working with people on code I like working with people on other things I love working with artists and so on I don't like working with other people on code because I don't care to do exactly what you're saying I just want things to be done well and if they are done well I don't mind using whatever paradigm the person who did them well has if they are not done well that I'm just aggravated and I don't really want to go have a conversation with that person about it that's not to say that I I'm not trying to belittle that I think working on teams and learning team communication and how to work effectively on a team is something that's a great thing for people who want to work on teams but the reality of programming is that you know everyone has their own preference and one of the reasons I like programming is that there's a lot of things I can do and I can earn a living not having to communicate with people about what I'm doing I can just do it the way I want and get it right so I would say most of the time my preference for working on things is I work on my thing you work on your thing we communicate about those things but not about the code and I don't care how you did it and I don't want to butt in on how you were doing it and to the extent that I can I just want those things walled off so that your code doesn't affect mine and mine doesn't affect yours so we really just don't have to worry about it that said I appreciate how it is not always possible to put yourself in that position especially if you're working a larger company or on a very large project and that is just why I don't work on those things okay and so kind of continuing off that so having like a fans and like backers to hand-made arrow have you felt like that's and the people watching you as well do you feel like that ever influences the way you do things or kind of changes your thought process about any of the things that you're doing on the engine or is it more they're just kind of there along with the journey with you um it's the latter in terms emotionally I guess I would say meaning that I don't think of people who buy handmade here as backers it's just a pre-order for a game you get the source code if you want it it's not a Kickstarter I don't require a certain amount of money to do it I'm doing it whether no one ever bought it or not so I don't really think of them as backers if they were backers in the traditional sense of the term meaning I had promised some specific set of deliberate deliverables or something that they were funding to make happen I might consider things in that way but since it's not that I really don't think it's that way I made here it's think I wanted to do I wanted to have video of a programmer making an entire game from scratch so we could go back and look and see like what where do things get done how do they happen right and all that sort of thing and you could jump around it and all that source I just wanted to have that yeah that said it is unquestionably true that having people watch on the stream affects the series and the reason for that is twofold one is that there are a surprising number of experts who show up to the stream from time time and from time to time yeah and those experts know things I don't know so for example you know someone might be more of a platformer expert on a particular aspect of OpenGL or something than I am and they'll tell me on the stream so I'm there's something that I might have programmed worse and they'll pop up and say oh you know you can do this or that's not support on this card but if you do this it is right those are things that are just better now like the it's like having a second it's like having a free code review that I don't have to really spend any time on right it's just happening I don't even have to wait it's just happening over the internet with thousands of eyes waiting to point out something that I could have done better and that's kind of a cool free resource it definitely means that there's sections of the Hammad hero code that did things right that would have done them wrong or you know if right and wrongs may the wrong term but did things better than the way I would have written them or did write them and then corrected yeah similarly like there's been times when people have pointed out resources I was unaware of I the last time I had really done any x64 assembly language work well mattress emulation intrinsics work I didn't know about Ayaka the the the Intel code analyzer mm-hmm cuz didn't exist and so when I saw stream I was like I really wish that I didn't have to count all the instructions and that there is a way to just you know compiler should just tell me what they are and you know I think it was Fabien who was just like there is like go you can go download Ayaka and use it and I was like oh my god and like that tool is awesome like it's exactly the tool I wanted and who knows how long it would have taken me to find out that that tool existed if he hadn't randomly been watching the stream and told me you know I mean yeah so it's definitely paid dividends having a stream that I have nothing to do with the original stated purpose of the stream just having experts in various domains especially ones that I am NOT an expert in has been helpful even in my knowing Bayaka helps me in my own programming dirt day job right like so that's been great yeah been really good I'm kind of going on the other side of you of like the the people following along kind of in the same boat exhaustive like learning how to do things with regard to handmade hero do you think people can use handmade hero is like a sole source of learning kind of watching from the first to well following you kind of through it or do you think it's a really good use of like watching this and doing some other things and kind of doing your own thing all at the same time because like you said you have the zip files available at the end of the days and stuff like that that's a really tough question so I will start by saying that the goal of handmade hero was to do it the way that I'm doing it because I wanted to have that complete record and it was designed to be there because I thought there were things people would learn by watching an engine programmer just do what they do rather than them telling them the result they ended up with mm-hmm and I think it's been really good at that because I've had many people actually tell me that that was a big breakthrough for them they were like just seeing what you actually do totally makes me feel more comfortable about doing how to go about this right yeah and so that I think was the only thing I was trying to really do with handmade hero so that's the thing that I think is important to take away from the series and what I hope people get out of it everything else I have no idea because the goal wasn't to teach people 3d math I just do that as of course too serious as I wanted it to be I wanted people to see explanation of those things yeah I don't know if it's a good way to learn it um I don't know what other things would be valuable so I think I'm not really sure probably the answer is I mean one answer is if I was gonna make a course for people to learn 3d math for engines let's say it would look like handmade here oh yeah right so that's definitely true whether that means there's other resources available right now that are better than Hanneman here oh I don't know but I know that handmade hero isn't the best we can do I could say that for sure because if I made one I wouldn't make it like this and I'm the one making handmade here I'm already saying it's not the best for that right yeah um so I I guess that's really all I can say about that part of it the other part that I would say is just to focus on the part you said about using as a sole resource I don't think you ever want to use any one programmers ideas or code as your sole resource and the reason for that is because it's assuming everyone's brain works a little differently and that the most efficient way to think through a problem is not the same for everybody yeah then unless your brain literally works exactly the same way that mine does and there may be some people out there who it does right but it's only gonna be some people then the chances that all the things you might want to know about programming are encompassed in what I have to say is zero there's going to be very valuable approaches and things that I not only might not be aware of but even if I am aware of them might just not like and not think are good that you will think are good and that are effective for you so I think it's important to be exposed to multiple different programming styles so that you and and even more so than that I think it's important for our programmers brain to be able to distinguish between those styles in a concrete way because that's the way you can find the primary styles that work for you and I think like when I found myself I probably say when I was like 23 24 maybe I don't know how old I was at I'm about the same time as I found myself actually thinking how do I figure out what elements of programming style are good and bad for mice like I am NOT taking anyone's opinion you know I don't care what people write in these books I've realized that they don't really know what they're talking about all the time I want to come up with mech I want to come up with ways I can evaluate this stuff for real and I want to see how it works when I do it about that time was when I felt like I became a much better programmer like it was a you know you always feel like you're getting better but that was a really big divide and I think that it's important for programmers to try to cross that divide for themselves and it they're not gonna cross it by watching a series like mine and assuming that everything I do is right because even if everything I did was right for them they still haven't developed the critical faculties necessary to know why and that's a problem so not only is that a problem but I also suspect that for most programmers not everything I do in the way I do it will be the way that they should do it because they will find different techniques are better for them okay all right so we're gonna kind of now move on so we have a few questions that we so part of our project is also as we said to talk to professionals we're gonna edit and then transcribe these so that way other people can kind of learn from them because we found we think that blogs are a really good way to kind of learn process that's how we're doing of our learning and so as I was saying we're trying to help novice novice engine programmers and so when you first started programming like larger scale game engines what was like the most confusing part or maybe even not even large-scale but when you first start programming game engines well we've kind of the largest confusion that you might have had approaching those ah so this is a very intricate question and I'm not sure how I'm gonna answer it in a way that can be concisely transcribed and useful okay let me say a few things about the question and then maybe you could ask a follow-up question that will get at the heart of what you want to know and again I transcribe that all right perfect okay so there are two fundamental there are two fundamental problems that I think are involved in game engine development that set it apart from a lot of other types of development the first one is that it is it is not a stable problem in any way 99% of everything that programming does in the past or today looks a lot like standard in standard out right you can phrase it as here are set of inputs and here are the set of outputs that come out of it so most of the things that programmers are used to thinking about in the way that they're used to thinking about them is I'm working on deep learning here's a set of input images and a set of output tuned neural net parameters how do I make the best translation between these two things right I'm a natural language processing person here's all the corpus I want in and here are the like noun tags I want out or the sentence tags I want right and so I think one of the core one of the biggest challenges for an early game engine programmer is making the leap from input-output thinking to this amorphous systemic tons of things in crazy states come together in this weird way to produce an alternate reality that you can experience right and it's very confusing how that happens at first because even though you're not necessarily aware of it everything you've ever done prior to that looks a lot more like this input-output phrasing right yeah um so one of the really good things that you can do at first right is to try to figure out the the sort of what I want to call the the core loop the the golden loop the the the golden differentiator of a simulation which is like a flight simulator a game the things that have this right is they look like this real time loop of I have a set of entities States right I go through a simulation to change the states of these things that's a certain process I then go through a way of presenting those things and then I return to the beginning right yeah just getting yourself comfortable with that and that everything that you build has to build out of that is just the first big step it's not input-output anymore it doesn't look like a web program it doesn't look at the neural network it doesn't look like a parser doesn't look like a server does look at any of these things it looks like that hopefully a lot of people are able to make that leap without too much trouble but it's just worth noting that's the first big thing you've got a kit so you figure out how to do that make a very simple thing that has those properties and get it working understand that everything from now on will have that structure not the other one right yeah okay the other thing is the complexity explosion right so again because it's not input to output where you have this very clean sort of thing you have all of these systems that are all happening the same time they all interact they all overlay on top of each other the physics and the rendering and the blob is that the other thing game design entities scripting all this stuff comes together you really need to be able to do discoverable architecture in order to do these things correctly and most people are just not familiar with that process and what I mean by that is I called it compression oriented programming in the past what I mean is the technique of writing something in the simplest way possible I think this is Jonathan blows term that he uses often he says like write something the simplest way possible just whatever is obvious to you don't even think about the rest of the game engine I just need to do this one thing think about it more like the standard and standard out that we're all used to how do I just freeze it to just get the input and the output to do the thing I need to do just in isolation anything else no code design no cleanliness no nothing just no abstract thoughts just concrete you need to be able to do that clear everything from your head and just do that and then pull that out into the architecture and one of the biggest mistakes that I think even experienced game programmers will often make is they go the other direction they try to start with the architecture and drill down to the thing they want to do and I have never ever in my entire life seen a good result from that it almost always has to be refactored at the end because when you start from a conception of how something will plug together you don't see all the details you don't know all the things that you're really gonna have to do when you go to solve the problem so you always forget some whereas if you just implement the real thing first and then pull it up into the architecture fit it in you almost always end up with better results so I like to think about those problems and this gets away from question a little bit so I apologize but I like to think about dough so much there's thing called boundary value problems in mathematics right yeah and there's different ways of solving them there's like shooting methods I start here I see where I end up they're solving backwards where I try to see what I could different um building up a skillset of how you work on code that allows you to work from either side of the problem ends up being really valuable in this case because like I said you usually want to start with the solution loft it up into the architecture but in addition to part of that prop process when you get to a certain level of that lofting you want to think from the other side down as well you want to go what should a good API for this look like what are some things I maybe didn't think about when I was implementing it that I should maybe change that wouldn't break the algorithm but that would let it work better with this integration so it's kind of like this you you almost want a ping-pong game process where you start at the solution work backwards a little go to the architecture side work forwards a little come back and it's that the best programmers at doing this sort of thing are able to come at it and make it fit really nicely at the end so both sides can be happy yeah I'd like to give a shout out to Allen Webster for that by the way he's he works at rad game tools now but I've talked about those two things separately and I never really talked about them together for some reason and he actually asked me a question he's like I've heard you say two totally separate things I've heard you say start from the solution and work backwards to integrate and I've heard you say always write the usage code first well which one is it and I'm like you're totally right like it's like it's that you do the first thing I said until you have the thing correct and working and integrated then you work forwards and go how should I talk to this thing what are the changes I need to make and why never connected those my brain no but I give him full credit for that realization that you should always talk about them together just talking about them separately doesn't help anybody yeah I can see talking about him separately cause like a little confusion after hearing the second even in my own brain it's it's it's clear that I was doing something intuitively that I'd never internalized and probably was therefore forgetting to do sometimes when I should have done it so I'm I'm very thankful to him for having like identified that rather important gap all right so we've been reading through some of the handmade hero forums and we came across one where you were kind of like restating the the goal are talking about who this handmade hero was actually for and I think one of the lines you're saying is that it's not about making a game but it's about like I think you said inspiring engine programmers or something along those lines yes and you said there there's a shortage of people who are trying to be engine programs yes why do you think that there's a shortage of engine like people trying to get into the engine programming space versus versus like the other in spaces so you know I I don't really want to do too much data free analysis on this I guess is what I would say because it's attempting to provide answers well if you ever read the newspaper you will know that's very common for people to like to provide answers to things that they obviously just don't know what they're talking about right and have no data to support them so this is a question that sort of demands if it wants to be answered correctly demands a data answer which is are there really a shortage of engine programmers and if there are why right what's the reason for it I don't pretend to know the answer to either of those questions what I can tell you is that if today you told me I needed to staff up a 10-person Engine team which is a modest engine team I would have almost no idea where to get eight of those people right and the two that I maybe do know where to get I'd have to hire from some other team right and now that team has to hire two engine / that's right yeah uh and so I'm strictly talking about this from a personal experience in the game industry it is an anecdote not a statistic where I say that I just don't find that people know where to hire engine programmers and I mean I know so many people who I know plenty of people who have hired people who they really don't think are even that good at engine programming but who were definitely the best they could get I mean they could not have gotten better intrapreneurs but they wished they could have right and similarly I know plenty of people who would hire another engine programmer right now if they could find one but there's no one who meets their cutoff right and so some of that is just like well you know if especially on smaller teams you can't afford to hire lots of junior people maybe so maybe there's some raw talent out there it just needs to be developed you know and I can data wise I don't I don't want to make claims about this that I can't substantiate because I don't have the data but in terms of real experts on this it's just very hard to find and I can't offer you a reason for that but I just assume that the reason is that I even in today's day and age were a lot of people license engines the bottom line is even something as simple as using unity kind of requires you to understand 3d math at a pretty decent level if you're not gonna ship one of these games with all these like obvious bugs in it right and so it's just it's a hard profession it demands that you know a lot of things it's not for the faint of heart it requires a lot of expertise a lot of time spent learning even to just be a competent engine side programmer like I'm gonna modify Unreal Engine in a small way I'm going to modify a real in small way I'm gonna make a plugin that does something I need to do or even I'm just gonna competently program a motion controller in unreal or something yeah these are just skills that people just don't seem to have in abundance and so you know even today when it's like well you can just go get an engine so you don't need to build it from scratch we still haven't it's just we have not gotten to the put to the point where we've we have enough talent verses and at a strong word expertise really we don't have enough expertise versus how much we could be using to make games better yeah no sorry I wasn't trying to make you pull any any yes it's it's that's just a personal experience and you know it's I I suspect if you talked to other people in the game industry they would have similar opinions about how hard it is to find good engine programmers and I and you know I don't want to put words in their mouth so you should ask them but I just it's not the kind of thing that you can just go find someone to do and I think we're getting better at pipelines for finding artists it is much easier to find a highly qualified artist today than it was 10 years ago thanks to social media thanks to online tutorials education about what goes into making a game asset probably thanks to even school curricula that was not around in those days what I have not seen change is the number of of engine programmers the last three engine programmer hires I know about or was in any way involved with all came from handmade hero if it wasn't for handmade hero never would have found him and who knows if they even would have done it maybe they would have something else I don't even know so you know that was really again the reason for handmade here was exactly what I just said it's in that that intro sort of fluffy video I did there it's right it's like it says I believe this way of life is worth preserving it's talking specifically about a practice and a discipline and a mental model it's not I want to show you how I make a game never said that right I said I this way of life specifically this and I meant that like I don't mean what's the easiest way to make a game what's the most cost-effective way I don't care at all about that and I feel like that's something that's covered well enough elsewhere right yeah yeah all right so then talking a little bit so you're you're making handmade here and you've obviously made engines before handmade here it is there certain aspects when you I was listening to on your videos and you're saying you've made at the platform layer like 15 times and you kind of know how you like it but is there certain things like that that you kind of always just pull in from each project or do you ever try to like say oh I've done this ten times before I'm going to try it this new way to keep exploring or they're just tried and true systems that you kind of follow along with uh so I guess I can't really answer the question because I am only 42 years old and not wise enough to know yet look maybe when I'm 80 I'll be able to tell you if I had a piece of code that was worth reusing here's a couple of things I can say along those lines for the very first time on 1935 which is the main project I'm working on now for the very first time I decided that I never want to write a platform layer or standard library again I'm like I'm just gonna do this one final and I'm gonna move on I'm just gonna make that statement so I actually on this project and this is a very long project for me it's it's you know I'm I'm expecting honestly this project is probably a five year project which is a long time for one programmer it's not that much I mean five programmer you programmer years whatever want to call those yeah whatever the mythical man-month would want to call that in you know I know if you guys know the the Brooks book the mythical man-month it was a classic of old yeah it's about IBM 360 don't worry know if you were alive so for the first time on this project I decided to do that I felt like I was ready to do that and I tried very hard to do that whether it succeeds or not I'm not sure but what I will say is I took an approach that was slightly different from approaches I had taken before and I did to bring it back to the beginning of our session today I did exactly what you were saying so I have a file and in it every time there is a question about what I'm gonna do in the codebase I document exactly why I thought there was a question so why did I think that the answer to how this should work is not obvious I discussed the solutions that I've tried and I select one of them and say why I ended up selecting that one out of the ones that I tried so there is literally for the first time on any product that I've ever done a 100% complete documentation of why every last thing works exactly the way it does right yeah and so I am trying to do that on this project and we'll see how it works out prior to this project I've never really felt like that was something that I wanted to do because I've always felt like there was probably room to grow each time yeah and that was usually because I never sat down and said let me think all these things through because a lot of time to do a platform layer you just do it right and this time I approached it with the other the other angle so we'll see how that goes but that's definitely I think even for myself who who advocates a very limber style of programming where I don't think you should really calcify or do a lot of upfront design sorts of things even I if I want something to be the be-all end-all this is the one we're gonna use forever I do want to spend up more time recording and justifying decisions I don't do them from the top down like a UML diagram disaster situation or anything like that but I do I do feel that a higher level of rigor is necessary and what I try to do in this situation is sort of play the devil's advocate as if I had more than one program the project come at it from different angles can I justify it from a speed perspective can I to fight from ease-of-use perspective can I justify it from a future like compatibility with possible hardware directions that sort of thing okay and so we'll see ask me in ten years if it worked I so so kind of going off of the last question a little bit more of when let's when handmade hero is done or like your engines and the pastor done specifically actually specifically talking about handmade hero when it's done do you expect other people to kind of use handmade hero to make games or do anything with or is it purely for the the kind of like that the process of watching it I know we kind of going over this a little bit but um the the latter it is designed to be a standalone project that shows all of the parts that you would have to do to put together a game that you could release on Steam where you didn't use anyone elses code to make it that's the entire goal of it doesn't have any any ancillary goal it's not designed for people to use for some other purpose or anything like this that said two things one I would like when the product is finished to essentially do an audit if you will of the codebase go through it all and I would like to point out because I know there will be many because this is a very limited time project the total number of hours that we spent on handmade here is less than a year of programming time so lots of stuff in it is very rough right yeah I like to do an audit find all of the places where I think that roughness really shows and write up exercises that programmers can go and take thing I'm a hero code base and fix something I didn't have time to do right and I just say like this piece of code should be able to be better in these n ways pick one of those n ways and fix it that's your homework assignment right yes I think that would be really good for engine programmers to start with in an engine that's made a bunch of decisions whether they're right or wrong you've got to live with them because that's gonna be your job when you're first starting out probably here's the Unreal Engine it does 50 different things wrong you can't change those they're baked in the architecture it would be massive change to fix them or whatever so you got to live with them how do you make these improvements or how do you fix how do you make this one part better leaving with those constraints so the hammer here will be no different here's a bunch of things that does wrong here's one thing we would like to improve you have to take it as a whole and just make that one improvement so I would like the codebase to serve as that in the future yeah and I think it can serve is that pretty easily the second thing I'd say is in terms of expecting people to use it now I kind of do expect people used to make other games because I've had many people use it for other games already I've had multiple people write to me and ask is it okay if we use this for our game and I've always said the same thing just make sure it doesn't conflict with handmade hero and you're fine I've had people already release games who did them for like game jams and stuff who use them at your codebase so there's games already shipping that are on the hammer hirako base so I guess if that portends anything the answer is yes people will use it however that was never my intention gotcha so moving on to so we while we were doing a little more research about you we all read your piece on the walk monster thought it was a really interesting tool but towards the bottom we like where you talk about you're like still open questions while you're writing it of like oh you could add this feature this feature in this feature at what point does investing into it like a test system like this become a net loss versus or like spending too much time investing in a tool versus just using the tool for what it was originally made to do I suppose what I would say about that is that although as programmers gain more experience if they are a talented and hardworking programmer they will gain a certain amount of what I would call taste I think I might want to attribute that term to Jeff Roberts the person who founded rad game tools and wrote the video codec that is used in every game that's ever shipped in the history of time he I think used that term taste and I still use it which is to say the things about decision-making that we probably cannot now possibly ever but at least not now boil down into a rigorous science or you know anything of that nature I suspect yeah as a parent gains more experience and if they are good and if their conscientious they can gain more taste about how to make a decision like the one you've stated okay I think it was probably completely implausible for anyone to be able to give advice on how to make that decision you either have enough experience to read the tea leaves to know and you're conscious of programmer to make a good decision or do or do the tests necessary to make that decision or you're not and I don't think someone can really help you gain that early not yet anyway maybe we'll learn so in the future what I would say is that people almost categorically undervalue visualization so if you were going to err on the side if you can't decide whether it's good or bad to make a visualization you're on the fence make the visualization that's about the only advice I can give the chance is that it will be not useful compared to its cost are almost zero if you get to a point where you're pretty sure that you don't need any more visualization then you can stop but if you're on that fence if you're like oh this could help us it might not I'm not sure go go visualization that's about the only piece of concrete advice I think I could give yeah I kind of going off that too so just a little I'm just curious of on how the tool so after you kind of did all the visualization did you have to kind of fly through the map and try to figure out oh like there's this red circle here or because there's like a leftover collision box isn't even better did you kind of have something that would pop up and like populated text file or something like that I'll answer that question briefly because on I think it's Thursday I'm giving a talk at the Busan indie connect conference where I will show demos of all of the tools we developed for walk checking and processing in the witness okay they are drastically more advanced than the ones that I wrote about in the walk monsterpiece and walk monster in fact was never used on the witness we replaced it with things that are many times better than it so it was more really more of an exercise walk monster was of determining that we had a problem and that we had a serious problem uh-huh but the solution to that problem was actually much more involved in went in a different direction so I guess what I would say is all of the problems that walk monster was designed to prevent we ended up either preventing systemically in ways that they literally could not happen or developed better visualization tools for that are much better than walk monster and you'll see those on Thursday they've told me that it will be recorded but it doesn't matter I'll post a recording of it either way so presumably they'll either be the official recording or I'll post one and you can that'll probably two weeks or something okay and you can link to that in the write-up if you want to get perfect so one of our kind of last questions about handmade hero is say you lost everything and you had to start from square one again what would you is there anything that immediately comes to mind of like I do this section differently or approach things differently well I mean yeah obviously there are but that's because of the point of handmade hero which is to say that the whole reason for recording the process was I wanted to see I wanted people to see me go down a few avenues before picking the one I wanted or to see how a amorphous unstructured piece of code resolves into a structured one so for example fairly recently one thing we did over the past three weeks in fact was we made the renderer standalone in a DLL that now anyone can use to make a renderer that does all the things that handmade heroes does so the depth peeling and like stuff for sprites and the your coat all that stuff is kind of in a separate library now so if you wanted to just make a game that has an instant 3d hybrid sprite block renderer you just have one right yeah those are exactly what I wanted people to see in handmade hero so if I lost the whole thing I almost would say I would start a different project because if I was going into handmade here with the knowledge of how everything worked out which I now know it would totally ruin the entire point of the series watching me type in code I already know how to do it get isn't worthless that's what a blogs for right or a github repo here's how to do it we're done right so the whole point the series was to show that so I on handmade hero it was critical that I not have a solution in mind when I started for at least some of the major parts of the project preferably all all right and then question about making builds so making a build of the game can be kind of a like a witness a build either like making the engine of dll or kind of compiling the game with the engine kind of together kind of doing that process can be a little bit of a hurdle or kind of something unknown at least from our point of view and so when you're doing something like that is there any good advice that you have to start it like early and keep it going on or is it like kind of what advice you have on creating builds early and like keeping them going rather than breaking ideal sue I might understand the question but I might not so okay if I may ask some clarifying questions yeah so if you were to substitute into that question rather than game engine some other thing like what have you worked on in the past even a small project just give me anything that you've done for school work or so we work we worked on like a like unity games and do us everything well so I can pin missile program um I worked on something with the had somewhat of a bill that we make you simple with C maker is just like a really really crappy version of Photoshop so essentially taking the different pieces we made it also something of like a switch to make a utility for hospitals to use it with things so okay okay I I think I had a question now so is it correct to restate the question as follows we have not traditionally worked on code where we actually built an executable from scratch so we have not we have no experience really with that process how should it work is that yeah okay okay so it's not yeah because I was not sure if you meant for a game engine specifically and I was like well what would be different about a game engine but you're just talking about the process of working outside of someone else's build they've handed you yes it's okay gotcha so I think what I would say about that is that it's a lot simpler than you think one of the problems so so i'll veer into an anecdote here at the risk of eating up more of your time at know if you guys getting bored sitting in your chairs here so if you've ever used an email program which by this point we all have you may have occasionally been kind of amazed in a sense at how how they managed to have so many bugs are so many problems in something that all it really has to do is just send a piece of text from one place to another right it's like it's like one of the simplest things you could do over a network and it's been done for decades and it still kind of somehow mysteriously impossible for them to get it right so back in the old days at Bell Labs which was obviously one of the like crucibles of software engineering of place that created see and UNIX for example yeah they had a term for this and it was called mailer science and it was a joke that they threw around because nobody could ever understand how the people who always ended up working on the simplest problems made the biggest mess so it's like why is this meal program so broken it's like mailer science man it's just mailer science so I would offer the term build science as today's equivalent of that we currently have some kind of weird programming culture where people you you would there's a term called build engineer all I can say about that is what is going on right so the way to approach a build is to realize that computers today are so fast that almost any game engine you care to make especially on a team as small as yours can be built with a batch file in like ten seconds anything you do more complicated than that is a waste of your time people throw in integrated continuous build servers and see make and ninja and distributed builds and Python all these it's like no just forget all that get rid of it it's not necessary all you need to do is just make a thing that says CL here are the files this is the exe I want build that's it I'm hand made hero we show how to do this the first day yeah it's trivial you don't need anything more complicated than that and I would encourage you to start there okay eventually somewhere down the line there are some arguments to be made and programmers who I respect would make arguments for doing some of the slightly more complicated things like a continuous integration server when you have multiple platforms so once you get to like oh we're shipping the switch the PlayStation 4 the Mac the Linux the PC build the Xbox one bill the ps4 build of this right uh-huh it can be hard for an individual programmer to have everything installed on their machines to keep those builds up to date and to know if they've broken some totally other build that they don't even have the SDK for at that point sometimes sometimes it becomes valuable to take that step if you do it sensibly and make sure that it doesn't interfere with the quick fast simple build that you use day to day so I would put that up but otherwise especially if you're only on a single platform a single line build is the build you want it shouldn't be more complicated than that single lines with a single batch file couple lines is plenty and I would also say that there's really no excuse for build times to get above 10 seconds there's a utility that I put out called C time that times builds and keeps a record over time you look at that graph if it's crawling upwards figure out how your builds are slow and fix them okay so that's the advice I would have and if you have any practical problems with that like I said handmade heroes first day just shows how to do that or just look at the code I can send you our batch file after this and you can look at it it's super simple I advocate using what's called a unity build not related to the tool but what that is is essentially just saying look make a C file that just includes all the CPP files you want build that one CPP file that's your build ok that's that's super useful information now it also speeds up the build quite a bit because the fewer CPP files that the the fewer translation units the compiler translates the faster it typically goes because every time you split into multiple translation units those translation units usually include some of the same H files which increases the processing load so by merging them all together you process every H file only once which can drastically speed up some builds like by orders of magnitude sometimes if those H files are complex ok birthday so we are actually done with all of our questions that we have for you the last thing we have would be so as we said we'd like to do like a company with like the kind of edit about a five minute story that you have on game engine development but you think it's interesting so it can be anything of your choosing so if you have any story that you'd be willing to tell and I'm not totally okay as well yeah I mean that's sort of the problem with that is there's lots of I suppose stories but I'm not sure which one are really all that interesting or or useful so oh go ahead yeah or just gonna say it doesn't necessarily need to be useful at this point this is one thing that we want to show people for the sake of programmers and maybe even other people who get into the space and see maybe the work we've done but they don't really care about that but they might be interested in seeing just something short that could introduce them and then hearing something interesting about space like oh wow I didn't realize dealing with that or like that just sounded really like a really funny problem that it's more or less of that so you don't need to necessarily have it be super useful with anybody as one unit five minute video you'd have to give me a second no take your time yeah absolutely yeah I really just don't know I mean one of the things that's true about game development is that the things that are interesting to you are not interesting to other people and so you know I mean I guess what I would say is that all of the things that I that come to mind when I think of them I can't imagine them being particularly interesting anecdotes to anybody most of the time you know I think when I think back about what were sort of the most interesting game development times they were almost always in sort of circumstances where I had been looking at something the wrong way and then you know after doing quite a bit of work on it or something realized there was sort of this other way to do it and that insight is kind of really really fun for for the person who's been working at it but I just don't know how to turn most of those into into an anecdote sometimes in lectures you know I've sort of told like I've had a lecture there's a at the papers we love conference where I talked about figuring out that you could blend quaternions using linear interpolation instead of spherical interpolation and how that discovery came about and like some of those things we're really exciting at the time and they just there's not much you can really anecdote anecdote eyes about them although you know I could I could sort of stretch one out I think if I had to pick something to say that would go into a compendium of the kind that you're talking about I think probably what I would say is that when I was little because I've always been a programmer like like I said since I was very very young I always wanted to program and in those days there was really no internet and no communication with anybody so I didn't know any game developers and I didn't know anything about game developers right I just had computers that I could program at home and I tried to learn to program them and make games on them as best I could and that was really all I had to go on and my father who was a programmer was not a game programmer he worked on traditional like tools for Digital Equipment Corporation which is a now-defunct hardware company right and so most of the things that I might have wanted to know he didn't necessarily know like he didn't know hardly any 3d math or anything like that that you would use for game development because it's just not that's not common knowledge among programmers right so one of the things that was really exciting for me as a young programmer was when I was still in high school I had met over this previous summer some programmers from Microsoft and they were going to lookingglass which is a company that again doesn't exist today but they made a bunch of games that a lot of people consider seminal like Ultima Underworld and System Shock and theif the dark project and flight unlimited just a lot of very important games in the pantheon of games when people are talking about history they were going there to port flight unlimited to Windows because this was at the very early time when Windows didn't really run real-time games for the most part because it didn't have a way of actually writing graphics to a back buffer like you would in das and so they were going to go do that and they asked me if I wanted to come and my mom let me take a few days off from school so instead of going to school I went into Cambridge and I like got to go work on the windows port of flight unlimited and it's the first time I'd ever really set foot in a game company and I think about that time often these days especially because as you get older you definitely become more sort of cynical about everything about development about companies about everything you know everything you do and it's easy to lose sight of what you love about things or what was important to you but I think back about those those days now and like it was like the most magical experience I imagine it was probably similar to if as a you know a lot of children grow up maybe dreaming of playing in the NBA or something and it's like you know some of those children probably get to randomly get to go out and shoot a free-throw you know with their favorite team you know or something like this and it was exactly that experience for me I you know just even seeing what it looked like to walk into a game development studio and see what the people there were doing and like meet the people behind it and I remember literally like everything about that experience like I remember we went to a grocery store next door called bread and circus and we bought like some bread and cheese and apples and stuff and brought them back and like ate them around a lunch table things that are inconsequential I'm sure to everyone else who work there still like stick in my memory as like I don't know if you can call it a formative experience because that's a sort of different thing I suppose but it's definitely like it just shows me how much of an innate draw towards this I had and I expect that other people who you know if if you want to know whether this is the kind of thing you might want to be doing with your life if that's kind of the way you feel about it if you have those kinds of feelings then you're probably right um but if you don't maybe not right I mean it's a hard job it's not like playing games it has nothing to do with that and so I feel like you have to almost approach it from that same thing and maybe the NBA examples pretty good if you like watching people play basketball that's not the same as liking playing basketball and it's good to know which of the two you want to do because if you just like watching basketball then there's no harm in having an ancillary job you could work on the communications team for your favorite sports team just like you might work in the production department of your favorite game company but engine programmer is the on the court job that's really demanding and demands the kind of dedication I think in a lot of ways that an athlete has to have you're expected to know a lot more you're expected to work a lot harder you're expected to know such a to have trained your brain in such a demanding way at some level that I think you kind of have to have a certain love for it to really want to do it and and to get back to maybe one of the questions you asked me earlier that might be why I think it's so important to try and expose more people to it because I think there's a very limited number of people who really have the desire to do it and finding those people is important to me and I think that it's worth doing and there's so many other jobs out there for a programmer to have that are not as hard as this and that aren't as demanding is this that I think that there's plenty of other jobs for people to go to and I think they will go to them if their heart's not really in it so finding out if your heart is in it is good and and I would encourage people do that as early as possible I don't know if that's the kind of anecdote you wanted but but there you have it yeah that was pretty spot-on it okay so we're actually done with any of our all of our questions I don't know if you have any questions for us regarding things but if if not we really thank you for all the time we've taken and we really appreciate all the questions you were able to answer for us and oh we got my pleasure those were excellent questions and I wish you guys the best of luck on your programming project making gay mention is very difficult the only piece of advice I would have is don't be too ambitious with what I've accomplished you know I always tell people make asteroids first and it's something to be proud of if you can even do that from scratch because most people can't most programmers can't so best of luck everybody thank you so much have a great day you know we'll definitely be trying to watch the view same comfort okay yeah if you want more information on waka monster it's got everything because it's not walk monster the talks actually called killing the walk monster because we got rid of it and and I will explain how and why so good [Music] uh so I'm still live here I didn't look to see what was going on chat I don't know how many people whoa we've got a lot of people watching actually I suppose I don't know if anyone has any questions SF gardener says that was fun cool what was that that was just there there is a team I believe that Carnegie Mellon they said so there they're in University who is working on a game engine development project and they were sort of saying that they wanted to an email to me they were saying they wanted to get some people's perspectives on just engine things as part of their project and that seemed like a really smart thing to do and so I just you know said sure anything I can do to help so these this interview was just the questions that they had I don't know how helpful my answers are gonna be but you know I I just tried to provide as much information as I could that me you know maybe is useful to the project they're trying to put together because they're kind of putting together some learning materials to it's well I guess for their own use and for other people's I suppose what is the killing the walk monster thing is talking about so it's it's there's this thing called the Busan Indy connect [Music] [Applause] there we go it's a festival in in South Korea I believe this is the shop from last year it's a combination event one of the parts of the event is that there are indie games that are going to be shown kind of like in the mega booth or you know like a look in a festival setting I think there's some judging to see what gets into what doesn't but I could be wrong about that I'm not sure how that part of process goes I wasn't involved in that but then there's also a small conference where a couple different speakers are speaking and Chris Hecker is I believe going to speak on spy party stuff I believe this is a spy party design lecture they for some reason haven't put the talk descriptions up i believe chris has sent them when I just don't it's just not up do I think it's a spy party design lecture that chris is gonna do I don't know what Tyrone is talking about but I'm assuming he's talking about publishing work they do it Nicholas I don't know I don't know what this blockchain one is about mine is about the wok system that I implemented for the witness and so it it's talking about the taking from the point that I wrote the article original article on wok monster let me see oh man this is okay let's try more specific guess there's another thing called one monster it's it starts with this article where I talk about what a thing I did on the witness and shows some of the techniques that we were using early on in the project or not necessarily early in the project but early on in the testing of the wok system and it starts there and goes through to discuss the final system that we developed for for doing all of the walk movement and everything and then I believe so I think these are sponsored sessions from various different companies like Amazon I think Nick Sutton er is another invited talk though and again I don't know I don't know what it's on because the things aren't up here but I believe this was also an invited talk it's not a sponsored talk I think like this one this one and these are sponsored maybe but I don't really know like I said once they put up the descriptions will be a little clearer I don't know why they're not up yet but you know they're not up yet anyway so that's happening and it's it's going to be recorded apparently so it will be available afterwards if you have a chance to come to Busan definitely come come say hi I will be there giving that lecture how do you find the source of excessive memory consumption when all you have to work with is a core dump as a further restriction it's not feasible to reproduce on your local workstation because the program is an online game server and the conditions only arise under certain circumstances with a lot of active players well I mean don't you have a memory tracking system in this thing I mean I wouldn't ship a game server without a memory tracking system I'm not sure what to say other than that all of my systems are we even handmade hero has the ability to track the memory if you want to and you know we could go further down that road we probably will at ship time to add individual allocation tagging which we don't do right now easily could so I mean that's the way you that's the way you track it down is you just look at for example let's suppose that the two ways that your thing interactions memory is malloc and free you know I mean I don't know what ways your programming interacts with it but you isolate those two points you overload them with macros to pass the file and line number then on every memory allocation you do a hash lookup of the line number and you increment or decrement that line numbers total memory allocation and then that way when something crashes you can just have it dump that and you can see exactly which line of your program is accounting for all of the memory right so salute all ski do you think a networked multi-threaded 3d engine is too ambitious for a three months dead period yes absolutely I would never try to do a multi-threaded 3d engine as my first engine and I certainly wouldn't do one if I was only going to spend three months multi-threading is is extraordinarily hard and and I don't think it was something you should do on your first time out now that said some people's idea of multi-threading is not really multi-threading so if you're going to do something more like a very weak multi-threaded thing then maybe that's fine but assuming you're actually trying to get multi thread performance meaning you're trying to use multi-threading to actually be faster than single threaded that is something that is not easy to do a lot of people multi thread things and getting slower so I certainly wouldn't recommend doing that like I would recommend if you do want to do multi-threading on your first product it'd be more like handmade hero where it's single threaded for the most part and it just lot launches worker threads that wait for workloads that are small and localized because that's something that it's not harder to think through that that'd be my recommendation on that I really enjoy the hard jobs and games but don't think I have the innate drive that you do do you have any advice or experience with other domains that may be just as challenging I don't know any other domains that are challenging in breadth so game development has a breadth challenge to it that I don't know any other domains that that really come close because games kind of intersects everything it's like photorealistic rendering and high performance optimization and servers and clients and real-time input processing and game design and all that like everything comes together for games everything that humans know how to do is in a game right so I don't know anything in breath but in terms of just finding hard problems oh I mean there's tons of things go work on molecular modeling for a protein synthesis I mean go work on figuring out how to how to better synthesize x-ray imaging or how to do 3d visualization of cat-scans you know like there's tons of things in the medical field you could be doing or biology you could go into security research right like how do how do we secure the power grid how do we prevent cyber warfare things how do we harden routers like all of these sorts of things you go into operating system development right hard real-time operating systems tons of stuff to work on there there's so many fields that have hard problems in them so you can easily find other hard problems in programming and and game development is is unique I think more not because it has our problems there Harper's Eric it's the breath it's the breath of the problems that's interesting game development that's what really sets it apart you know you know not the depth I think I think every programming discipline has the depth it's the breath that's different about game development there's anything else on here fubar count been programming for about six years hobbyist and currently university always struggled to push a game project to completion do you have any advice on how to see any product to its end no I'm bad at that I just like working on technology so I'm not really good at pushing game projects to completion pushing tech projects of completion I might have better advice for game products I don't I would ask John blow I would really seriously try to like get on his dream and ask John blow because he is someone who has both had trouble pushing game projects to completion and not had trouble pushing game projects to completion and I think he would be the perfect person to talk about that difference and the things you can do to try to get over that difference as fast as possible and and I don't have the necessary experience to offer advice you know kind of go haphazard through these questions here I don't guarantee that I'm going to answer all of these are you into isometrics how would you approach creating game in isometrics that using 3d well isometrics are a lot easier because there's everything has a universal sort order so you can usually just draw up front to back and be done with isometrics because there's no perspective happening there so I think that you probably okay but maybe not I guess now that I say that I it's not really true like I don't know if you read Andrew Russell's post about this I always spell his name wrong I'm sorry Andrew yeah we spell it more like Russell I think the first name but um this is kind of a good post about how it's not actually that easy isometrics this is kind of an interesting way to talk about how excuse another one right I might refer you to that because I guess I guess now think about snot it's not really that straightforward this year at a big database conference there were two papers about avoiding the penalties of cache misses by prefetching and then using KO routine to do work while the cache is being fetched do you see any applications for this in games it sounds a little weird I mean it's not the weirdest idea so first of all a cache miss is not that expensive right so a cache miss is going to be like what like 300 cycles 400 cycles usually in game development we're pretty good when we start getting down to that part of the project of figuring out how to interleave things we want to do so that we always have something we can do math on while we're fetching you know so like do I think I would want an entire system for doing a KO routine in a 300 cycle window especially given that hyper threading does that automatically so if I just have two threads that are doing stuff for my game on the same core when I hit the 300 millisecond bubble the other thread will take over I mean that's not literally what happens but the other thread is issue it can issue instructions into the AL use into the execution ports right it doesn't sound that useful but you know I haven't read the papers it sounds like something we've already kind of know how to work around or are usually good enough at doing that I can't say it excites me like I'm not itching to go read that paper it doesn't sound like one that's gonna be high in my list I guess I'd say but you know until I actually read it I'm not gonna I won't dismiss it because you know maybe it's more maybe there's more to it there you know so and he's good at six weeks I know I will be posting a schedule when I get back from do song is uh I'll be in South Korea for a week so until I get back I'm not gonna post schedule and I won't be streaming from there do you think John's language will make game program easier in a significant way as opposed to just the million list of annoyances yeah I mean hardly say I haven't looked at it in depth but the meta programming features that are in it are far so much better than the ones in C++ that you could really dramatically improve your programming by using them it is not as simple as eliminating an annoyance now how good those meta program features actually are in practice at the end of the day I don't know until I actually a chance to play with them boy I I mean the difference between C++ and jii in terms of the ability to like really leverage code is going to be I mean it's not even gonna be close you know C++ offers no meaningful meta programming at all it's it's almost completely useless for meta programming so you know that's gonna be a pretty pretty huge chris fehn a bit about AZ do um I mean I can explain a little bit about it but I'm not sure what I'm gonna say that's not gonna be the same as what you know if you just read just go read it like an nvidia tutorial or something but the idea a CD o stands for approaching zero driver overhead for those of you who don't know and I'm sorry if I've got one of those terms wrong I apologize the acronyms fly all over the place in this industry but AZ do is is basically acknowledging the fact that a modern 3d graphics card like an Nvidia 1080 like the one in this machine is a processor in its own right I mean it could run a whole game over there right like it's it's not made for that so there's a bunch of things it does inefficiently if you were to try and run an entire game over there but it's a full processor right and so the extent to which it needs the main processor to hand hold its execution is pretty minimal so what a video attempts to do is say look if we just prepare buffers in memory so when we're in the game we just prepare buffers that are formatted appropriate for the cards consumption we should be able to have the driver do almost nothing it should be able to just kind of you know knock on the cards door and say hey I've got a new set of data and code for you to execute do it and that's it so rather than the driver sitting there like manhandling when the GPU is gonna do this and do that and checking for states and changing the states and recompile shaders ad do is just about saying look let's just prepare actually correct execution buffers for the GPU that have all the stuff in them it needs that it can just read and then what we're going to do is just tell the GPU read it go and so the driver becomes much less involved and frees up a bunch of CPU time for you to use for your game that's all it is really you know Devils in the details but that's the concept I think that's all the questions all right I'm gonna close it down here you

Channel: Molly Rocket

Views: 7,624

Rating: 4.9593906 out of 5

Keywords: Handmade Hero

Id: 9-h6TPkQ6ko

Length: 114min 12sec (6852 seconds)

Published: Sat Sep 08 2018