Hammock Driven Development - Rich Hickey

as I said yesterday I swapped the stalks around so this is the more philosophical talk as well to the titles I couldn't decide some of them were semi cranky in those employments and then war can be a great a great feat or line because so this is about step one this is this is just an experience report it's not advocacy there's a little methodology here or science or anything else so I'd like you to try think about when was the last time that you thought about something for an entire hour like nobody bothered you and you had an idea and you sat for an hour thought about how about for a whole day does everybody remember less than a second thought about something for a whole day however over a course of a month we had something you're working on and obviously not spending all the time every day when you start staying into a month or a year these are tremendously valuable moments if you get to have them at all I consider myself extremely lucky to have had the ability to think about probably three different things for a year or more one of them is closure and there's nothing I prize more than that kind of time the other thing I'd ask is what was less than that you felt confident trying to do something you had never done before and what do you think it takes to become confident in turn something you've never done before right obviously a software developers a lot of times we're doing the umpteenth application that takes something out of a database and puts it on the web but the lucky you are luckier you are the more likely you are to encounter problems you've never done before and how do you end up you know how do you start doing that and not feel incredibly at risk so sorry about sit by talking about some software development you know things we all know to be true right we hate bugs and our programs of trying to write quality programs and we know if we let the programs reach the field it's incredibly expensive to you know fix inadequacies and the artists in the program right so we say okay we'll have a big testing process and quality assurance and even that we know is not so great because the sort of has this removal you know this distance from the development effort which is not good so now we know we know what to do in this area right we fix bugs while we're coding by testing and development and this is the best way to avoid bugs in our applications right no answer kids did you notice that I learned how to make each bullet this menu item no definitely absolutely positively not the least expensive place to fix bugs is when you're designing your software which everybody does yes right I will contend of all the things I'm saying here watch it which are very very extremely fuzzy that without a doubt most of the big problems we have with software our problems of misconception we don't have a good idea of what we're doing before we do it and then go go go go and we do everything you know we have practices and all kinds of stuff and we feel really good about ourselves after that point but if you mess it up you know as Mark said in step one it is not going to turn out they're not problems of implementation there are problems of implementation obviously and testing and other things help with those but problems of misconception are not generally addressed by testing or type systems or the things we use to correct defects and implementation right there aren't really type systems I can tell us if we've got a good idea or what we're doing addresses that idea so I'm going to talk a little bit about analysis and design I know that so nineties and ugly and was rightfully you know criticized and really dropped because you know people considered it to be about process and drawing pictures and you know knowing everything about everything and making comprehensive plans and the waterfall model and there was amazing amounts of stuff that was terrible about this but that doesn't mean that the the step before go do it is not an important step and I think we don't spend enough time and energy or or make enough time or get allocated time you know it may not be a matter of our choice if we say we'd like to spend some time thinking about it but we have to ship something next week but we are definitely suffering in quality because we we don't spend the time here and so I'd like to do is sort of just whatever you think analysis in the design is like you just forget for the moment let's try to make a really simple definition analysis and design is about two things identifying some problem that we're trying to solve and assessing our proposed solution in terms of whether or not it solves that problem that's really what it's about about anything else right we should be solving problems but we should not be building features there's nothing about the feature what is feature features just an attribute of something is the shiny you know chrome knob on something it's not the purpose of the car there's no guarantee you if you put together a feature list even if it comes from the customer that is going to solve their own problem or that solves any problem or that the features when you put them together don't introduce a whole ton of other problems right so programming and writing software is not about completing this features in particular features provided by users in spite of their best efforts to satisfy themselves or often really not good ideas and you've got to dig underneath it and figure out what problem they have and what's the best solution to it and then reconcile it with whatever they asked for we also have a tendency because we're we're all smart and we love being smart and sort of figuring out how to make things go that you know figuring out how to make something go is good no matter what it took to do it right so if we can find a way to get around a problem we're like whoo that's great and it's not great right or voiding problems which we're all capable of doing very capable of doing isn't the same as solving them so we should really try to support on solving problems and the thing I'm going to talk about today is really that there there's a bunch of technique and skill to solving problems and the first one is just to make an effort to understand the problem you're working on to recognize identify it put it somewhere and talk about it so problem solving is definitely a skill I think you know you shouldn't take away from this talk that you know there's a certain kind of person who's like good at problem solving and they get to do this part of the job and then we can practice these other things you can practice this part Polya wrote this amazing book called how to solve it in 1945 or something which is about how to practice how to practice and what are the techniques sobbing math problems in this case and it's a terrific book full of great insight and if you've never read it go on to Amazon right after my talk and order yourself I'm happy one of the things that's not so great about the book is that it is in the math space right and in that space this is really nice thing that happens when you're done and you think you have an answer if you have all the techniques of mathematical proof to determine if you actually have where as a software developers you don't have that right there's no way to prove that you have solution to somebody's ecommerce site problem right there's no mathematical techniques and there's not going to be any anytime soon that will let us do that but it is a skill and it is something you can practice this something you can learn about and and it's worth doing right because as human beings we get good at what we practice it doesn't matter what it is it's amazing examples of people practicing things that they seem to have no potential hope to become good at and they get good at it because they practiced it if you practice problem solving really practice a problem solving you will get good at it if you practice methodology X you will get good at that and I'd like you to ask yourself where do you think there's more leverage I don't care what X is pick any X you want would you rather be good at it or the general skill of solving problems so what do we need to do if we're going to work on solving problems what what is what is the activity like the first thing is to actually say I am solving this problem this problem is this bla bla bla bla bla and therefore a bla I have seen so much software made where no one ever said that no one ever wrote that down and then we have the whole system and no one said what problem is supposed to solve if we're not solving problems I have no idea why we're in this room we absolutely should be working on solving problems which means we should be we should be enumerated our and then from the mental standpoint which I'll talk about a little bit later it is actually important to say them out loud right as the person who's trying to solve a problem say say have a conversation with somebody in your group and say we need to solve this the problem the problem is you know rants or talk and you have a little conversation or write it down but just like you use the you know the practice of repeating somebody's name or introducing them as a mnemonic to help you remember their name it's the same thing this is the seed of solving the problem is stating it so the next the next part which is definitely trigger and Polly's book is great and it's got a lot of practical things many of these are overlap what he said is to understand the problem right so so we said we have this problem I think we need a no single database right there's something missing we have this problem we need a no single day to ransom we haven't actually said you know why what are the characteristics of this problem to lead us to this solution space and and this is where all the interesting work is I think in software development right and so the first step is what do you know about what you're trying to do there's definitely gonna be a bunch of facts there will be customer requirements there will be other things they'll be contact steal the system has run on this kind of box has to run for this long it can consume more more than this many watts or has to support than 10 million users whatever it is there are those kinds of things and constraints all this stuff or facts you know about what you're supposed to do right there will be things that right away you know you don't know right I wonder where we're going to get you know the inference of data as an input to this thing what we're going to do when our main data source word is isn't available do we have a secondary thing there'll be things like that of course there will be things that you don't know or you don't know well that's fair but if there's things you don't know you should think about them now the other thing to do is to say everybody says doing X I have this great idea for X if you know that you're the only person in the world ever had this problem self that's very very unlikely so go find some other solutions to similar problems you know are there any others that you know about and what can you find out about them because looking at other solutions the same problem is the number one way to get up to speed really quickly and start working ahead of you know the best known solutions in this space and then because what you'll have to do that will just be an incremental step above what the last guy did but if you're ignoring what the last guy did you're starting from scratch so you definitely want to look around in the space all right now I'm not advocating a methodology or anything but if you're going to bother to do all this work you should write it down somehow some way I don't care about the other thing you have to do is you have to be discerning yep that you have to be critical and we're at sort of in this world because this Willis community stuff and it's like I just hear awesome it's like awesome happy I just heard like 50 times a day not everything is awesome all right and so it's hard to talk about other people's stuff not being awesome so just I mean mainly focus on your own stuff in particular as you're finding solutions I should try to you know enumerate a solution to a problem look for defects in your own solution and of course you have a whole talk about this because there will be there will be technical errors there will be errors in logic they'll also be errors of taste and judgment and abstraction and all those kinds of things it all feeds into this and entire talk in this this area but whatever issues you can find in your own solutions try to solve those two right away upfront if you've added so the other thing you see is we're gonna do this oh he's using Ossie Davis oh that's great it has these Co 10 attributes is awesome it's really easy to get excited about the good parts of what you do but you should be looking for trade-offs the chances of there being no trade-offs in any solution are slim the other thing is just this again this what what don't you know that if if there's stuff you know you don't know there are questions you should be asking in order to find out what you don't know you don't know everything so there should be question marks on the whatever it is you want to use that you're going to write all this stuff down there should be question marks on that page if there are no question marks you're missing the step the other thing is to think about none of us are born knowing how to write software none of us are born knowing about sequel or the characteristics of the web or the protocols or anything else and if you're trying to solve the problem especially in a space where you haven't done it before you're going to have a very limited ability to come up a solution if you don't have a lot of input you're going to need to get a lot of different inputs so that you can let your brain go around between them and say oh yeah this idea and that idea are connected to each other and therefore I can do this other thing if you only take a really narrow slice of I see exactly what I'm doing right now right the second to deliver next week you're not going to have enough inputs to make decisions so you want to read about the kind of space that you're in widely all right very specifically with us other people try to do exactly the same thing and then broadly there's other characteristic problems and maybe even if you want go try to find research papers that are kind of in the same space it's amazing the cool things you can find by searching something like ACM 4 papers about the kind of it's like we can get a certain kind of hash code that does whatever you go into google type hash code that does whatever enter and if there's some scholarly and ACM references grab those papers even if you only understand like a tiny fraction of the paper it's likely to contribute to your ability to think about your problem the other thing is even if you're not going to tell the other guy when you're looking at other solutions be extremely critical I can't tell you how often you're going to find the next best idea by completely crucifying the last guy's idea at least in your own head all right take it apart right because when you take it apart you're going to find a couple of things maybe they didn't write down when they were doing it so everybody says designs about trade offs everybody knows this right but usually want to talk about trade-offs in their software let's talk about the parts of their software that suck I had to make these trade-offs that is not what a trade-off it is right you have to look at at least two solutions to your problem at least two and you have to figure out what's good and bad about those things right before you can say I made a trade-off so I really recommend that you do that and when you do it you might want to write that down somewhere okay so let's talk a little bit more about practice a big part of trying to do this work is maintaining your focus we had a really nice talk yesterday about flow and that that is a kind of a focus related concept and when you're trying to do design work you also need I think some of the most extreme focus you're going to ever need and so there's some cool aspects to the havoc one of the cool aspects to a hammock that you can go on a hammock and you can close your eyes and no one knows that you're not sleeping but they won't bother you because they think you might be sleeping so it's it's very cool computers are bad bad sources of distraction they're so bad especially for people like us something else besides what I'm trying to think about you desperately need to get away from the computer if you're trying to focus this it's almost impossible to focus sitting at a computer the other thing about focus is that you are going to be making trade-offs when you try to focus really intensely your gun to drop balls you're going to miss calling people back and responding to emails and doing your slides for conferences and the airport all the way there things like that that's just the game the one thing though is that you should communicate to people that you care about about this process and the fact that when you're doing it you're going to seem pretty far away and that's not a comment about the person that you're you know your care about it's just the nature of doing this kind of work so it is important to sort of do it a lot of people will not get time to do this all day every day or over the course of an entire week or if you're going to get some focus time you define what that is you know everybody knows about timeout time for little kids and what program isn't this focused I'm like look kids I need to go sit on the hammock and have nobody bother them so for me personally I think that that the process involves two parts of your mind and this is stuff that you're seeing those books written about this and whatever I haven't read them but but they seem to correspond to my personal experience which is that you sort of have this waking line and background line and you're waking mind is really good at that criticizing part it's extremely analytical and it's very very good at tactics right right now we need to make a decision you know the lion is chasing after us jump left we are really good at that that's what our waking mind is about keeping us alive and making short-term decisions and looking at the immediate leap present information and doing something about it however if you think you're going to sit down and look at a problem for the first time and stare at your computer and do whatever and have a conversation for 10 minutes and make a really great decision I don't think so I know I can't do that definitely not the problem with this kind of thing is it tends to push uphill who I see this movie that always okay here I have a choice left and right okay go right that's more up left to right it's right they left that's more up more up it's this part of your thinking is really good at finding the local maximum but it's not very good and getting off the track it's on and finding the fact that there's another hill over there that really takes you higher but this is a very very printable activity that you have to engage in I think if you want to use your entire brain and become very good at problem-solving and that is to think about using your waking time to assign tasks to your background line to actually think hard about something and create work for your background line that really is the point of the hammock and all this listing and all this we're going to talk about you're going to do when you're awake is actually to to give the other half of you stuff to do the other good thing about your waking mind is when you when you do think you have a great idea that you come up with in your background line your waking line is good at picking the other part saying you know you thought you woke up with this brilliant idea but now I'm seeing this this characteristic of it seems not so bright so let's talk about the background line I'm not going to directly equate it with the sleeping mind but the sleeping line is the number one instance of a background mind you can find access to your wrapper online during the day while you're awake but it's tricky it's good at making connections right the kind of thing like if I leave my if I make a Hut out of mud and it rains hard it will disintegrate is not necessarily the kind of thing that you can tactically figure out you or your background mind is going to know sort of aspects of all those different components that make the connections and synthesize them even when you think you're really hot at making decisions on the fly you're almost always just regurgitating something your background might has already figured out so the background light is good at synthesizing things it's about it's good about strategy right and so when mark talks about new abstractions and things like that abstractions are our software strategy right because the idea there is you're making some super global decision that's going to need to be correcting a whole bunch of contexts in which you can't make tactical decisions yet right what does it mean to make an abstraction you're going to drop drive libraries from you know what does it mean to put something in a programming language well I had no idea what you guys are going to do about you know with it right it's a it's a more strategic kind of thing you don't build a programming layers and say how will this programming language deal with HTTP requests what you want to do is give mark something that he can use and when he's got a tactical decision make about HTTP requests and that's a strategic kind of thinking and your background line is good at strategic thinking if you want to do abstraction you have to find time to do this thing because that's the part of your brain it comes from right it does abstraction it draws analogies right I think this is where you solve most non-trivial problems okay you can make good decisions in the moment otherwise but if you really try to solve something hard you've got to engage the other half of your of your head so I'm not just saying the scientific America say that when we're sleeping we process the information during the day because that's pretty obvious but that sleep reinforces memory which is good I mean it is important to remember what you're working on but more importantly it is a great sorter out of things so we had this whole I say I just advocated taking a lot of input my taking a lot of it but doing all this analysis of the requirements in the space right doing you know all the reading looking at you know competitive solutions and tearing them apart that's this is ton of stuff when you're going to decide what about that is important what is it when you're asleep and that's what happens evolution has solved this problem for us and that's the solution it came up with we can't ignore it we have to use it but the most critical thing is this one finding hidden relations and solving problems we were working on so imagine somebody says I have this problem this that and you look at it for ten minutes and say okay I'm going to go after the movies and do something else then you go to sleep you're going to solve that problem your sleep no or you're off and you didn't think about it did you no you didn't think about it you didn't think about it hard enough while you were awake for it to become important to your mind when you were asleep and this goes back to that feeding your background mind thing you really do have to work hard just think not typing it in just thinking about a problem during the day so that it becomes an agenda item for your background line that's how it works right it's when people were out there and they're like oh my god how am I going to find food and this is happening there but I know I saw help over there they seem to be by the water sometimes or whatever that's when you wake up as a caveman and say let's go hunt for the animals by the water it's not a logical deduction it seems like that when your foreground - or lysing it but there's no logic for that necessarily it's really a process of this very parallel kind of thinking so this is very important so we have a problem in general because we right we're just being cast right so for those more and more complex as time goes by and and we know there's a seven plus or minus two sort of working memory limit I and as smart as any of us are we all suffer from the same limit but the problems that we're you know called upon to solve are much bigger than that normal so what do we do if we can't fit the whole thing in our head at the same time how can we work on a problem with more than nine components right what I'm going to recommend is that you write all the bits down especially now you've written a lot about the problem right you know what the problem is you know a lot of facts about it you know constraints about where it runs you know you don't know you've asked yourself those questions you wrote them down I wish I do blog you looked at competitive things and said that works great over here but that part of that competitive thing sucks I hate that I wish that wasn't there you gave this huge agenda to your background line and when you're trying to load it up you need to survey and that's the point of writing it all down before if you've written all the stuff down including some sketch of how you want to problem you can go and just sort of jump around and look at that and sort of like you know how many balls can you juggle well you can only gentle something I can juggle at all but if we look at the seven plus or minus two thing to say we can juggle seven to nine balls but if you can imagine having an assistant who every now and then can take one of those out and put a different color in then you could juggle balls of twenty different colors at the same time as long as there were only nine in the air and any one point in time and that's what you're doing you're going to sort of look around at all these pieces and shift arbitrary shapes of seven into your head at different points in time maybe you'll draw pictures don't use that you'll melt it's not a methodology so go over and over but then you must again step away from the computer there's another really important important part of doing this which is to go and sit somewhere and have no input and close your eyes and not go to sleep close your eyes because we have this other thing right everybody knows what it is it's really hard to describe but does everybody have a concept of their own their minds I write what you see when you close your eyes and you start thinking about something it's this weird I mean is that actually technically visual though some people are really vibrantly visual I know for me it's I don't know I can't describe it but it's not realistic but you need to do that that that part is important for your brain because at that point you're switching out of sort of an input reception motor if you're just looking at your lists you're sort of in the mode of I'm getting input but when you're sitting and contemplating something and hashing it over in your head you don't have any other input which means you're exercising the recall I had looked at those 20 points let's say it was just 20 I looked at those 20 points over and over and over again and I jumped around with it put between them now I close my eyes and I'm trying to recall them and think about them a little bit more in my head and you're going to find if you've done the last step going over and over you will actually be able to sit on a hammock and pull all the different parts of a fairly large problem in an aggregate admittedly maybe one at a time and think of a thing about them that way that exercise is really really important I don't know why it just is because it forces this recall thing that definitely makes those things agenda items for your background line so we'll call that mind's eye time now you're done cake is in the oven you just have to wait it's so good and and and one of the things I would say is at least wait overnight no matter how ill you you and your buddies talked about it and you like you just feel like such a hotshot today I have got this thing you know sleep on it at least one night at least if it's an important decision now how many people woke up this morning with the answer to a hard problem is a science science at work no it's really kind of an unfortunate thing if you're not thinking about this do you think what happened I worked hard all day right who I've done working time to relax unfortunately if you if you believe in what I'm saying today you're actually doing something kind of important when you're sleeping so occasionally you really have to give your brain a chance to do that other part of the job if you always deny it I don't think you're going to have the best results unfortunately sometimes overnight is not not enough some big problems especially finding really good abstractions or finding answers to things that satisfy a bunch of simultaneous constraints take a long time it just does and I'm I know everybody has to ship and everything else in that case a lot of what I'm saying doesn't apply and like I said before I consider it a huge opportunity when I get an extended amount of time to think about a problem because I know it I'll come up with a better answer but one of the ways you can deal with this and not get stymied by well let me just think about that for three months because most managers are not incredibly receptive to that sentence is to just work on more than one thing right not inside one day okay try to work on one thing each day but if over the course of time you have like three projects right it's quite possible to load one up and work on it for three days and find you're not finding answers to any of your question mark items or able to enumerate new possibilities so you kind of stuck a little bit just switch to another project and do that for a few days you have to advertise the loading up time it can take between an hour and an entire day to load up something so once you've done that you try to get at least the rest of the day or three days or more on it but don't get don't get hung up about the stuck thing just switch don't stay stuck switch or get morning but talk about it more you keep stimulating the pathway don't don't say stuck on but yeah then eventually cake comes out of the other you wake up and you have a great idea you think you know the answer to your problem or you have a good idea for a solution unfortunately sometimes you have an answer to that not the problem you were working on you're working on three projects and you're loaded up the project C and you woke up with the answer to project a that has to be okay right you just just switch and take advantage of it at least capture it right if you wake up with an answer to some other thing that you can't work on that day capture the results of this background process they're really useful finally you do have to take your great ideas and figure out if they're actually great by either analyzing the more which is certainly important but sometimes you have to write them and type them into your computer actually we all have to do this so you do eventually have the code it's fun Stu has this great sense he's seen some of my design sheets and document of despair or something it's like it seemed to be all like hey we can't do this this doesn't work like that question marks blah blah this other thing it was just like a whole it's all negative but it's all challenges to the problem-solving process it's not like it's not despairing it's positive it's saying I know what my challenges are and therefore I can work out but you spit this thing out now you have something so you tried it you try to try to avoid a lot of typing I know I do because if I think I've got an answer and the answer is small that's one of the most telling attributes that is probably good and what I would hope from doing this whole process is that you gain confidence in it after you've seen it work for you so that you say you know what I have never done this before but I really have thought about it and and this solution I came up with overnight feels awesome and whoo let's go it is important to look at what you did and to run it and see and find out new things about the solution and say I you know I had the supposition it's not correct I thought it would have this characteristic it doesn't et cetera et cetera I am not advocating a waterfall model you're going to try stuff and go back that's fine but don't lean on this right test-driven dentistry I don't think I could come up with a better thing right we can't really do that the last thing is you are going to be wrong I'm frequently wrong that's part of the game you're going to think of better ideas I think that's one of the most exciting things I think no matter of what I've ever thought of the fact that I know I'm going to think of something better as much as it will suck a little bit because I sometimes the best means that I'm still going it's still working so you will think of better ideas also the facts change you can change because of two reasons right one you missed some of them early on so they're due to you because you skip them what else do we have changing requirements right this just you know we all know this the facts will change when the facts change do not dig in do it over again and see if your answer is still valid in the context of the new requirements and new facts and if it isn't change your mind and don't apologize sometimes you'll just make mistakes errors in logic or you know miss you know you just you just get it wrong that's fine if I could advocate anything do not be afraid especially do not be afraid of being wrong so in summary this is a ramp there's no sir I have two minutes for questions yes

Channel: ClojureTV

Views: 233,518

Rating: 4.9705133 out of 5

Keywords: clojure, programming

Id: f84n5oFoZBc

Length: 39min 49sec (2389 seconds)

Published: Sun Dec 16 2012

Reddit Comments

[u/progfu]

While this is old, it's very inspiring. I've watched the talk at least twice.