GeeCON Prague 2017: Kevlin Henney - Full Stack Developer

[Music] [Applause] ah there it is I'll talk about these these this is a little bit okay because that's obviously gonna be somehow vaguely related to the subject of the talk which is full-stack developer what do we mean by this and I've been very very careful with my punctuation it's I'm not actually talking about what most people to think of as full-stack developer if you've read the abstract you will know this if you haven't you're in for a surprise what I want to talk about is genuinely knowledge and techniques that can be Illustrated and I've it's just one day I worked out that actually messing about with just stacks the kind of computer sciences favorite abstraction it turns out you can illustrate quite a lot so just in time for this talk friend of mine Chris old would put this wonderful quote up yeah trouble with full stack developers you can't push them if you're not sure why that's funny it's okay I can explain afterwards and also it will be available on video so that this is you know you can find out why this is funny and then use it on your friends so what are we going to talk about um I'm going to talk about romance as to start off with love and romance this is a picture of Stonehenge and you might be thinking what is Stonehenge got to do with love and romance well I took I took this picture on Valentine's Day this year because my wife and I are obviously very romantic we got up on Valentine's Day the kids are off school it's a school holidays it's like right what are we going to do we're gonna go and visit Stonehenge that is how romantic we are okay so forget bottles of wine and candlelit dinners you go you go with your partner and you show them rocks you know arranged 4000 years ago it's there you go that's real remote so yeah it's very pleased with this picture why am i showing you this actually there's a reason I show you this because mostly when people talk about full stack development or sitting going like oh do we care about monolithic architectures monolith us the Greek for one stone big heavy stone one big thing we have a monolithic architecture and in common with monoliths these are surprisingly hard to move okay that was one of things my older son sort of said he'd see this he said I've seen pictures of this I didn't realize how big they were they're big they're hard it's like a description of somebody looking a legacy system for the first time okay how did they do this why did they do this we don't understand what's going on and it's so difficult to change so but then people so say no we have a solution we've got a new architectural paradigm yeah that's right micro-services yeah so you know this is this is this is how we're going to build systems okay we're going to go forward we're gonna build systems like this like you know what maybe maybe maybe there's other questions because in both of these there is a kind of a focus we're very much focusing on the very much the back-end stack and everybody sitting there going like here's the technologies that we want to use so I want to draw your attention briefly we're gonna go through we've done romance now now let's do some popular culture Michael Jackson now you may have heard of Michael Jackson there's a chance you may have heard of Michael Jackson the requirements engineering guy yeah you knew who I was talking about didn't you and he's done an awful lot of stuff and had a very strong influence on software development over over many decades in the 1970s he was responsible for JSP Jackson structured programming 1980s jsd Jackson's system design both of these were really very influential and some really deep ideas in there jsd in particular relates in some ways to an abstraction known as CSP communicating sequential processes which I will cover later anyway the thing about this I mean you look at this book and you're thinking it's not the most exciting book title you've ever read software requirements and specifications is just they are come on really but the subtitle tells you there's something else going on here a lexicon so it's organized alphabetic of practice principles and prejudices it's about 70 or so little essays between half a page and four pages long just organized alphabetically where he talks about software development with a particular focus on requirements and he makes very important observation this is too often we push the problem into the background because we are in a hurry to proceed to a solution if you read most software development texts thoughtfully you will see that almost everything is about the solution almost nothing is about the problem so we see this it doesn't matter if we're talking books doesn't matter if we're talking blogs we see there is a very strong focus for a very solution centric which is okay but every now and then we need to figure out what exactly is the problem now I have a solution to this I'm gonna go on bike Evelyn's book No I have a solution to this I'm going to ask the customer now in 97 things every programmer should know there's a lovely piece by Nate Jackson your customers do not mean what they say and he highlights it's very true said I've never met a customer yet that wasn't all too happy to tell me what they wanted usually in great detail the problem is that customers don't always tell you the whole truth in fact I think he was being I mean I was the editor and perhaps I should have changed this but I think he was being relatively modest the customers that customers don't always know they never tell you the whole truth this is actually almost humanly impossible to they generally don't lie it's it's not a question of honesty but it's more a question of understanding where somebody else is coming from they use their terms in their context that's of course they would why why would they do anything else yeah you're the person who knows about software and why should they use your terminology they leave out significant details because they just know it's very difficult for somebody to simply describe something that they know and are familiar with in an educational style as surprisingly hard to be able to educate somebody rather than merely just tell them what you're thinking and even that and they make assumptions of course they do this is compounded by the fact that many customers don't actually know what they want in the first place and again this is nothing - this is nothing special that you get when you call yourself a customer this is actually something you get for free when you're a human being this is this is a kind of part of the human experience you generally don't know what you want and if you say that you know what you want you will be surprised so so we go about and we say right what we got system requirements that's how we're gonna do we're gonna gather requirements we're gonna go out there as if the requirements were just sitting there in fields waiting to be sewn like corn you know basically we're just going to reap all of those requirements are just lying there waiting to be harvested so people just write stuff out now it turns out the writing is the surprisingly hard thing to do okay to try and put your thoughts down in a linear fashion in a way that communicates everything that you are thinking and is significant and communicates to another human being this is a non-trivial task and then somebody comes along says nobody's gonna read this you need to kind of I don't know break it down a bit it's like yeah that's cool I discovered the bullet point functionality yeah this is how we do our specifications come on okay so we need to have a point of view and perhaps we need a little more structure than just a shopping list okay good right enter use case so we got a trigger this is the situation under which the use case occurs we've got a precondition this is what must be true when do we actually start this in other words this is for the use case to be successful here is what must be true beforehand then we have a sequence of things that happens and then we have a post condition this is what must be true after a successful outcome now the problem is this is not a universally adopted use case form there were many different use case forms that's actually kind of a bit of the challenge people are not entirely sure what is a use case and indeed one of the bigger challenges there's a couple of things I think that are worth tweaking rather than saying here is the initiating action let us go first of all let us describe the situation when I'm thinking about things I would rather describe here's the situation here's a thing that happened in it right let's switch those around and then we might also try and rearrange it rather than doing it linearly in time we all use newspaper ordering or it's also formally known as the inverted pyramid newspaper ordering is if you're going to write documentation give people the headlines first and then give them the next most important thing and then the next most important thing this is not how we write stories as in stories for children's stories for adults films or anything like that we generally don't do this in this style but this is how a newspaper articles are written because it allows you to just read the beginning and you get a big picture view of everything but you don't have all the detail so here precondition trigger post condition that tells you this is what must be true our beforehand for us to be successful trigger this is the inciting incident the thing that happens that makes this use case interesting and then post condition this is what is true afterwards and then if people really want then they can read the sequence now I know a lot of people who are locked up in organizations for what felt like years I mean it historically you know historically people used to go to jail and spend their time there and then apparently in Britain we sent them to Australia and then in the 20th century we we gave them use case sequences to write I think that's possibly the cruelest thing to do but it's a great piece of advice from the author Elmer Leonard try to leave out the part that readers tend to skip so we can just get rid of all of that sequence but it turns out it's the most boring bit to write and the most boring bit to read and it has the added bonus of normally being wrong so what we're going to do is we're going to save ourselves a lot of time so this is the first thing we notice this is a skill if I want you to think in terms of understanding the bigger picture what are the skills that you need then really just leave out the part that readers tend to skip just prioritize now there's another interesting thing here oh wow that's magic it's a lot of people er we don't do use cases and we don't use preconditions and triggers and postconditions oh no no we're using BDD stories oh yes it's a reinvention the user story was inspired originally by used Kent Beck is very explicit about this but also given when then is a reimagining of precondition trigger post condition it's a very nice kind of three acts play here that we find very useful now so we think we now understand how to talk about requirements okay let's just see how that works out so first of all we need to if we're going to talk about full state development we need to extend the stack normally my experience is when people say I'm a full stack developer what I've discovered is that means they know some JavaScript and some angular or some node and they know something about some back-end technology and they know a couple of things about a database now I used to be a systems programmer it's just like that's the stack goes a long way down really a very long way down if you ask a full stack developer hey so uh device driver writing you ever written a device driver no well that's not much of a stack then is it okay so have you written the open source software that you are using no ok the full stack is a long way down it also goes a long way up it touches out into the world a full stack developer that doesn't really have a battery or an armory of techniques to go and talk to the customer or really understand the customer situation it is a game only a partial stack developer so let's just take this simple technique and apply it to that's right a stack okay it's the stack of books this is the stack of books on psychotherapy it's like analysis it turns out the psychotherapists are obsessed with objects as you can see from the title in fact it would not be too far from the truth to just say that psychotherapists are obsessed but that's a different story yeah my mother was a psychotherapist it's never done me any harm yeah don't do this to your children okay okay so option so it's all about object relations the share of the eyes it's all there okay so let's talk about this stuff we've got a stack stack of books right so we've got a stack how do we understand a stack it's got some things you can do with it you can create it push pop and you can find its debt and you can ask it's top element this sounds like a good start but let us try and take our way into this using the given when them for it turns out this is a very useful conversational format times but people do still struggle with it and we will see that it's not always the right thing so first of all come something given an empty stack it's a good start when an item is pushed okay then it should not be empty okay so I'm not entirely happy about this because there's a couple of things here I mean one of them is this should word but some people find it very comfortable when they're doing BDD to use should you know that's fine but if we look properly at what words mean should is a reflection of you know what maybe it may be maybe it isn't empty maybe it is empty who knows who cares it should is a very conditional word you know yeah so I don't know this talk should finish just before six o'clock but hey you know what maybe it doesn't yeah if you're nice it'll finish early if you're not nice I'll finish later again this is how it works so should is a very kind of like well I'm really not sure and it almost doesn't matter which way it is it's kind of like a very hesitating you know it feels like when somebody says should they're not really you know telling you something then it's okay with you I think that perhaps it should probably not be empty don't you think you know it's just like it's like talking to a teenager or millennial okay it's just like there's there's so much indecision you're really not sure what it is that they're saying but they're not very sure about it so okay go back there's some really great old advice from Strunk and white on English writing I love the self referential nature of this advice omit needless words not only is it short but it tells you what to do okay it's just just don't bother with this stuff and it turns out that Strunk and white also had a very strong opinion about tests and said make definite assertions they didn't talk a lot about mock objects but they definitely talked about assertions make definite assertions avoid tame colorless hesitating non-committal language when a sentence is made stronger it usually becomes shorter that's brevity is a byproduct of vigor don't don't don't put in those noise words people already got enough words to get through okay so with that in mind let's go back to this it is not empty that's it that's the end of the story okay how many different kinds of empty are there since that was quite a lot okay there's not you know it's all rather there's one kind of emptiness but there's lots of non emptiness there's non empty like the room in this cinema there's non empty like the universe there's non empty like the beer that I'm going to have later and then that will disappointingly become empty but there's a cycle there we can you know explore that one there's a point here is not empty is not very specific we should be specific it has a depth of one okay if you want to think about this in terms of 32-bit signed integers then there are two billion values that are wrong well she's strictly speaking four billion values but I'm going for the not empty one so two billion values that are not correct having a depth of two would be wrong having a depth of three would also be wrong having a depth of four you you see where I'm going the point is we can be specific it's not about sometimes using programmatic language I sometimes find that people when when software developers end up writing requirement like things they end up using all kinds of programmatic language they end up saying sort of things like well the number of items is incremented by one it's just like no people do not say that that is not a thing that people do see I've got two fingers here I'm now going to increment the number of fingers that I put no I just held up a third finger that was it there was no incrementing going on at all that is not how humans speak you can actually be surprisingly precise without being programmatic oh and it's not just that it has a depth of one it has a depth of one and the top item is the item that was pushed it turns out that doing this logic type thing is quite hard you really have to figure people often forget that it has a depth does it just have one of anything yeah it's one of the thing you put in there and then we turn that into a method name yeah that helped yeah okay so we're going to need a slightly better way of reasoning about this one but logic is kind of going to help us okay it turns out the logic is quite a useful tool let's start off with a very simple observation from Jason Gorman and kind of heads-up it software craftsmanship conference in the UK and it makes a very important observation given when then there's a kind of universality to it this three act play it's not just precondition trigger post condition it's what we call a whole triple name for Tony Hoare it is basically the observation that given a statement or sequence of statements and action there is some precondition that if it is true when we execute s precondition P if if it is true and we ask to us we will get the outcome Q and it's upon this that all formal methods is based it is just is just this is logic it says that to be surprisingly useful when you're programming but it also allows us to reason about and discuss our code more clearly okay so let us try a more formal approach to dealing with our stack let's talk about ADT's abstract data types one of the key people in the abstract data type universe was Barbara Liskov in the 1970 was she's still alive she was the recipient of the Turing award in 2009 I think for her work very much on programming language design and paradigms and distributed object well distributed systems and she made his point an abstract data type defines a class of abstract objects which is completely characterized by the operations available on those objects this might start sounding a little bit familiar a programmer is concerned only with the behavior which that object exhibits but not with any of the details about how how that behavior is achieved by means of an implementation so a really strict separation and in fact we can render this in very formal terms and and the only reason I'm putting this slide up here is it allows me to use an upside-down a Andover back to front II what we've got here is really we're trying to say for all types T there is a stack that is this this product it has a new that gives us a stack a push takes a stack and a type T thing so stack plus a thing push that gives us a new stack in fact we're gonna come back to this later there's an interesting implication of this pop takes a stack and gives you back a stack depth takes the stack and results in an integer and top takes a stack of T and Reed gives you a T other words we've got the signatures here okay well this notation seems a little distant from what we're used to so perhaps we revert it to something a little more like this and start sort of saying well you got push of T and pop pop here and indeed for the rest of the talk pop is a pure pop in other words when you pop you don't get a value back we're gonna do that for a number of reasons one of which is simplicity but one of which is it carefully works into something I'm going to show you later so it's a really simple stack and it's got a very simple signature set of signatures okay so that's great and maybe it satisfies some mathematical thing within you to have done it like this and you know some programmatic thing within you to have done that problem is I mean I can get an interface out of there in Java very easily but that's really not that's not really a stack is it I mean I've got some names but I haven't really given you much in the way of behavior when we talk about interfaces and this is quite important because we we've ended out with the situation where pretty much from the early 90s onwards and particularly from the mid 90s after the introduction of java the word interface has when used on its own has come to very much mean the language construct and it is blinder it's a set of signatures and it's kind of blinded us to the fact that there is something deeper it's not merely a set of signatures an interface is a contract to deliver a certain amount of service this come from it comes from a paper published in 1983 by Butler Lampson hints for computer system design which I do strongly recommend reading because it's still surprisingly fresh there's a lot of really good ideas in there and this was written a very long time ago a lot of other things from the 1980s have dated music for example you know although for some reason my kids seem to quite like 80s music when I'm driving in the car maybe that maybe that's because it's it's it's I stopped talking stuff like hey kids did you know what an abstract data type was no doubt play the eighties music okay clients are the interface depend on the contract which is usually documented in the interface specification I think that second bits quite sweet you know it's usually documented you can tell he worked in a research environment it's just like yeah out here in the industrial wasteland of programming where people get paid to do stuff for customers documentation is not a thing that generally happens so usually it's unusually documented yeah we normally find that things are normally documented if they are open source yeah they're more likely to be documented of their open source than closed source okay so we've got an idea contract now the funny thing is that the word contract was used a few years later by Bertrand Meyer and there's a very interesting this is a very interesting book object-oriented software construction published at the end of the 1980s you can in fact you can tell from the book cover it's got a certain 1980s quality to it and he kind of popularized the term and we still tend to use these days this term designed by contractor mean what he was talking about but I think the contract is much much deeper and richer than what he described which is in essence just preconditions and postconditions I think we can go a lot deeper than that but also we can also find a number of limitations it turns out that the view that he was projecting in this book is actually not does not cover all the situations as comfortably and easily as we would have hoped it would do so let's go back to our friend the stack and let's start annotating it without implementation because it's an abstract data type with its focus on what is it that we want from each each and every method okay so I'm gonna we're gonna focus on push first of all given that before so that's the is the depth so I'm gonna have a variable that is the depth before we call push the postcondition is the depth is whatever it was before plus one and the top is equal to the item that we push on okay that seems reasonable and then we're going to go for the same kind of story pop it's got a precondition before must be greater than zero it must be greater than zero so therefore no popping an empty thing post condition the depth is whatever it was before minus one now actually if we pause from open there's something here doesn't feel quite right there's something missing and it becomes a little more obvious when we go to depth depth given that the result is the depth post condition is that the result is greater than or equal to zero well that's a bit vague you know what can you tell me about depth it returns you a non-negative number yeah but which non-negative number I don't know there's a few to choose from which one do you like as a specification this is really it's not wrong but it's really weak you know so Kevin what's the weather like outside yeah it's it's not snowing yeah I mean yeah okay so true but not useful it lacks specificity we can be we can do better than this so actually that's a bit of a if we go back pop doesn't tell you what the top item now is push doesn't tell you anything about the rest of the contents and when we ask for top we get this this is the worst one of all top precondition I don't know it can't be empty what about the post condition I have no idea what am I going to get who knows are you feeling lucky it's like ah in fact the only one that's really precise is if we actually create a new stack the post condition is the death of zero and you can't call top and your copy on that's great but notice and this is a limitation so I'm always a little bit bemused when people's all yes great techniques like designed by contract no it's a great technique because somebody told you is a great technique they've never actually put it into practice okay when you put it into practice you discover really quickly that it has limitations the thinking model is very powerful but in practice it's filled with problems because it doesn't actually get you talking properly about the nature of the thing you end up with silent bits of the conversation so as a formalism it has limitations and people and to recognize these a little more clearly okay so what can we do about this let's take a different formalism hey same series of books I did sounds going to talk about CSP communicating sequential processes this is a formalism by Tony Hoare for reasoning about concurrent systems and it turns out there's a kind of a simple logic to this how do humans feel about multi-threaded systems well they find them difficult okay how do they feel about single threaded systems oh they find them quite easy okay tell you what here's the clever bit what we're going to do is we're going to pretend that everything is sequential yeah and then everybody talks to everybody else but we never actually see any threads it's a really clever sort of deceit it's a sort of a nice trick because it allows you to reason through these things how I don't care about the concurrency bit just yet we'll come back to the comparison a moment what I care is there are some ideas here which I'm going to sort of pull out and reinterpret slightly there is an idea called an alphabet in one sense it might be better called a vocabulary but an alphabet these are the things that we can talk about what is it that we can talk about with a stack well we could have a conversation about pushpop depth on top in other words this is the hole this is the set possible operations or actions or events that's his alphabet okay well we kind of knew that that's cool and then we have a trace ah the trace is interesting because what the trace is is a list of all the possible meaningful interactions you could ever have with a stack okay so we start off so it's a set we start off with angle brackets that means it's an empty sequence in other words we create a stack and we do nothing with it in the second line we create a stack and we push something on it and then we never see it again we never do anything with it again the third item we create a stack we call depth and then never do anything with it again third one we push something onto it pop something onto it then never do anything with it again in fact you can keep on going as you might have guessed this bits quite long it's sort of infinitely long so you know if you have a junior programmer who's just joined your team you get them to write this out okay it'll keep them busy for ages or if you have one of those members of the team where whenever they actually develop code it seems to have a negative effect tell them we've got a really important task for you formal specification we need detail yeah my previous talk is about detail yep this is what it's about we need all of this so what we've done here is what we're doing is rather than talk about preconditions and postconditions we are talking about effectively saying what are the legitimate sequences so one of the sequences you cannot call is create and then pop create then top those sequences do not exist in the legal set of traces this sounds all quite exhausting it turns out that we do have a simple way of doing this it's called a state machine disappointingly this will generate you all of that work that you've got what we've done is we describe the life cycle here again there's this problem in software development that we get attached to one formalism and we say that's the formalism and it I'm gonna use it everywhere okay and you may have heard of the single hammer syndrome I have this hammer it's shiny it's new and then what happens next everything looks like a thumb so there is this idea that we need to do we need to explore is ok so ok yeah I like I like the pre and post condition model I like the design by contract model I like the giving win them all I've liked a lot of these things but what you're doing is you're saying oh hang on my set of skills tells me that instead of sitting here for an infinite amount of time writing out the possible traces perhaps I could generate a grammar so to speak what I've got here is a description that it's very easy to visualize and people have sort of turned away from visualizations which I think it's a shame because while some of them are terrible some of them are great we create a new object it's empty we push onto that becomes non empty ok if you're in an empty state you can't call top and pop because those aren't available to you but you can't call depth in the non empty state you can call depth top push and pop and it pop is greater the depth is greater than 1 then we remain non-empty if the depth is equal to 1 then we become empty this is a really simple lifecycle model this is great it turns out that there's a very simple way of capturing what we're seeing there and it's not the other approaches it's called unit tests it turns out that actually if we think about it from a unit test point of view this is far more constructive let me just reformat this and then just drop out all of the noise we've got a stack speck we've got a new stack it's empty and empty stack throws when queried for its top item an empty stack throws when popped that's what we're doing with the unspecified behavior an empty stack acquires death by retaining a pushed item as its top an empty a non-empty stack becomes deeper by retaining a pushed item as its top and a non-empty stack on popping reveals tops in reverse order pushing now notice this has a number of significant advantages over many of the other approaches all that I can use I can cross check it against my whiteboard state model but unlike the design by contract model the pre and post condition model it is I've got the concept of history I've got example data I use example data it turns out that humans are very much driven by example we understand examples or if you just present abstract propositions then your eyes eventually kind of glide off you need to tell somebody look I've got a stack I'm gonna push this and then I'm gonna push another thing and then guess what I've got in the stack well let's just pop it and see what I've got and let me assert on what I've got so there's a really important point here it's very very concrete and these are all propositions they are logical statements we've named these as logical statements and structured it using all the techniques we don't have nesting we avoid duplication in names by factoring that duplication out okay so that's all okay so we go I think we might have an understanding of a lot of the specification but what about the implementation let's try that so we've got a stack of tea I'm going to do this in terms of a ray list I got des I got top I got pop I've got push I'm gonna throw illegal state exception for things that are illegal States this seems entirely reasonable now let's try something different let's try something very different that relates actually to the original abstract data type observation that I was that I made about stacks this is a mutable stack let's try something slightly different let's try using a persistent data structure now persistent data structure as wikipedia tells us is a data structure that always preserves the previous version of itself when it is modified such data structures are effectively immutable the idea here is we are creating illusion that nothing ever changes we're able to share representation and so therefore the old version persists in fact we can actually sometimes I remember presenting this to somebody you know I said yeah the operations do not visibly update the structure in place but instead always a yield a new updated structure and the view is all chained together if you use version control this is what you're getting okay don't confuse it with persistence as in databases okay so what's that going to look like well let's let's try to understand imagine that you have the simplest aggregate data type an array we're going to treat that as a stack a points to the front that's great B refers to that the thing we're going to try and make sure of is that nothing we're going to deal with immutable stuff okay so therefore it doesn't matter if you share the same one so therefore B and a are both referring to the same stack and there's nothing that I can do well let's just try let's just try popping C is the a is the pop of a now here's an interesting one because now C is looking what we've actually done is we don't we have not modified a the person looking at a is is seeing exactly the same thing we've not created a clone of the stack what we've done big we've basically said look the pop of a is well you know move a long one because you can't change the body then it's effectively the same as having a separate stack but this is the nice illusion you can't really tell the difference unless you check reference quality it's a little harder when we're gonna do a push so we're going to change the representation to something linked that's fine there's a there's B of a there's the pop of a now the push is the clever bit is that now it goes the other way and you can actually have this forking out in memory this is great now this is actually a very old idea it's like 1950s old this is the basis of Lisp okay the the the list paradigm within Lisp is based on this concept and they're very much a mutable shared representation or estimate this is great so what's that going to look like well let's start off with an interface what's that going to look like if I implement that in Java so a persistent data structure let's try representing it using a different technique what I'm going to do is I'm going to represent the state model very directly I'm gonna say I'm gonna have a class that represents empty stacks now you might say that's not very useful carefully you know I have a class just for empty stuff yeah ah well what's its death it's always zero can you take its top notes always illegal State exception what about pop always illegal State exception why why we push when you return a new non-empty stack oh that's exciting we're not actually changing anything now I'll just factor out a trait method the effort for brevity because that's the same in both cases non empty here we actually have to hold something but again if we look what's the depth it's one plus whatever the rest of the stack looks like whatever the death of that is what's the top is the top item that we hold what's the pop well it's the tail this is quite interesting because if you look there are no if statements what we've done is we've actually represented very much like the Gang of Four State or object for objects for States pattern and we've represented two different types for two different states emptiness and non emptiness but everything is immutable it also satisfies a very amusing idea Jennie Vera come across this the anti if campaign the anti F campaign is like we need to get rid of ifs because they are dangerous there are no ifs in that code that the goal of the anti F campaign is to raise awareness of the effective use of software design principles and practices by first of all removing bad dangerous ifs okay you have something to do on Monday all right it's like fine knows if there are no if statements in the code I've shown you is all done through polymorphism the magic and wonder of polymorphism okay let's go back to that so here's the original mutable stack stack a string is a new stack and then I'm going to push geek on I'm going to push prog okay what's the change in usage I'm going to start off with an empty stack and then I'm going to push I'm going to chain it so what we do is that now instead of having modifier methods we always rich the resulting data structure but it's not expensive we're not returning a new internal ArrayList every single time or something we are returning a view of something something much larger however there are some limitations that unfortunately we hit in the current versions of Java the assumptions are whatever type you're dealing with it was to either be immutable so string works very well or you've got a guarantee in some other way that it remains immutable or unchanged just like the keys in a map okay you're not supposed to change the keys in a map once they're in the map and being used as keys otherwise you get surprises you know as you break the encapsulation so the illusion of this data structure and remember all software is illusions all code is illusion is that everything is share everything can be shared because it never changes but once you get immutability it actually has a fundamental shift so in languages that support that it's definitely that's definitely my favorite article title from last year immutability changes everything why why well it allows us to reason about a code it allows us to share representations but also it allows us to handle concurrency much more easily so I this is taken from a TV program it's the BBC and about 1984 I remember correctly the program was called threads and it explored what would happen in the event that Sheffield in Yorkshire in fact if there was a global nuclear war but focused on Sheffield specifically and it was utterly grim this is the height of the Cold War it was utterly depressing I remember going into school the next day and how silence Wow the year before they've been an American film called the day after that showed what would happen to America in the event of a nuclear war tell you what they have it they would skate look great threads told you about death but they were having a soap opera in America it was it was yeah it was this was much more convincing anyway this is elephant now apparently we've gone back to have a slightly warmer cold war than we used to so I feel I'm going to bring nuclear depression on those of you that are too young to have really experienced the joy of being on one side or the other of the curtain during the Cold War so welcome to the 21st century I hope that works out for you so threads and this is how I also want you to think about what happens to threads in your code when you introduce the threads it's a different kind of atomic devastation okay it's it's kind of like there are problems now the problem is that people say oh well that's really bad you know we have a solution we have locks we have locks there's it's like no no if you want a reason amount and stuff clearly here is a very simple quadrant diagram if you want to know how quadrant diagrams work there's normally one thing that's really really bad or one thing that's really really good let me just show you the really really bad one red is the color of danger okay it's nature's way of saying don't do this okay that's that bit that's the synchronization quadrant you can have data the data can be shared between threads or not shared between threads it can be mutable it can be immutable it turns out that three-quarters of the quadrant diagram it's really easy to program in because either you don't share things so you can change it if you want but nobody else gets to see it it's true encapsulation in other words you cannot tell if somebody else is changing state this is why we have this rather interesting outcome for example in UNIX based systems there's a very Strider of composing things on the command line you have the command line which allows you to pipe and filter things together which is actually a functional paradigm a dataflow paradigm yet the internals of the process can actually quite imperative and involves state changes but you can't see the inside so there's this whole point of like you isolate you genuinely encapsulate the degree to which state can be modified you can't see it alternatively you say you can't change it therefore we can share it freely 3/4 of the diagram is really easy to program in but guess where we end up coding yeah oh it's like oh you know what that's all too easy we're going to go for the top right-hand quadrant and you ask people so why is it that you are doing you know why why if you go these locks and people say Oh safety yeah but why did you do threading speed okay things to remember all computers wait at the same speed a lock the purpose of a lock is to prevent concurrency yeah it's just like yeah it's a really fast car that you've got there yeah and you know what I can put my foot on the accelerator and the brake at the same time it's like no no no no no no the point here is if you are care if you care about scaling composability in all the rest of it you want to get get away from this so there's a point here which i think is really interesting and and there's a relationship between paradigms I'm just going to pick on object orientation and functional programming rather than other imperative and declarative paradigms Oh makes code understandable by encapsulating moving parts functional programming makes code understandable I'm minimizing moving parts it's that minimization in both cases we're getting something the ability to reason about and potentially scale the code but they have slightly different philosophies but they're not so very different let's go back and revisit our friend the stack let's go and have a different stack this time gonna have a proper conversation what we're going to do is we're going to treat a stack as a first-class process and we our conversation is going to be based on the idea of we can push and we can pop but also it talks back to us instead of yeah we actually going to go for a message based paradigm instead of having a function return' we're gonna have a message based paradigm and so when we talk to the stack it can send us back a message it can say hey look guess what you tried to pop something and here is the popped item I deliver you the result or you try to pop something but I'm afraid I was empty I have nothing for you okay so we've now got a two-way conversation this is great so yeah there's there's there's the work that you want to give somebody else actually we can do this small iced it here so we have this very simple idea of I'm going to show you what messages are sent out so pop is an inbound event and empty is an outbound signal if I or an outbound message okay so you've got a nice kind of simple approach right and what language am I going to use well I'm there is a an ER lang for the JVM but it's only user lam because it does a really good job of this but it also gets us talking stacks of multiple types and it gets us thinking about things from a different angle first of all we can have an empty thing because we broke it into empty and non-empty right there is a primitive concept you can receive and you can you can receive an action in fact it turns out that Erlang has very few control structures you can receive something or you can have an if decision and that's pretty much it receive while you can receive a push request so I've got here pushes a symbol that's our push requests top is a variable that's the value we're going to push so what do we do well we call non-empty with top okay what else can we receive well pop and we receive a return address it's very polite you get to you know you get to say I would like to pop something please send the result back to me or you can send it to somebody else if you want so that's the return address fine and because in the empty state we have nothing to return we just return the symbol empty we say yep we're empty and then we loop back this is tail recursion we return there now some some people get very frightened of tail recursion because it's just going to disappear off to infinity fortunately most sensible systems have tail recursion a tail call optimization but this is the one true way of doing iteration in Erlang and if you're wondering this is what is called and this is very true and pure and original form of the actor model that's what we've got we've got a function that can receive interactions you cannot see the function from the outside it can receive events by our an identifier but that is it right we need to look at non-empty right we take a value we receive and we push top well we become more non-empty so now it turns out what we're actually doing is we're using the call stack to hold our state so function has state in its call stack when we return from being non-empty we continue being non-empty if we return if somebody wants to pop we return the value that we currently store as value and then we return from that function dropping down the stack so actually we're we're accumulating our state in the call stack and receiving requests and pushing and popping accordingly so it turns out they're saying very fundamental about stacks here and it's a really nice example to illustrate a very pure actor model actor model has become popular but they're always somewhat less pure in their in implementation so the the scholar model is much less pure than than this this is surprisingly straightforward to see you know there is there's this wonderful observation xkcd of course it's wonderful observations it's always wonderful observations why do you like functional programming so much what does it actually get you tail recursion is its own reward again if you don't get why that's funny there is time to look at this later but this I love functional programming combines the flexibility and power of abstract mathematics with the intuitive clarity of abstract mathematics it's like a comment I once read of C C combines all the power of assembler with all the readability of assembler so so we've got an idea there now we can see ourselves interacting with it we're going to create a process stack we're going to pop we're gonna self is ourselves so we pop something it tells us if we get a return empty there's nothing there we push 42 no response that's fine we pop self we get back a message a tuple that says hey guess what popped 42 okay we can push 20 push 17 and then we pop we get 17 back in reverse order snack order and pop so we can actually interact with this running function in a separate thread and notice nobody's seeing threads I'm using the term thread for familiarity but what we've got is really a communicating sequential processes now what it's fascinating about this is people say this is pure functional programming yes it is and it's pure object-oriented programming as well this is one of those interesting cases where paradigms end up unifying through similarity at a different level paradigms are not normalized they are not there's no kind of like you are now leaving the object sector and entering functional sector they overlap they're messy they're sloppy not all functional program programming languages have the same concepts not all object-oriented programming languages have the same kind that Alan Kay originator of small talk object-oriented programming to me means only messaging local retention and protection and hiding of state process and extreme Lake binding of all things that is exactly what the actor model is there's an interesting thing here so I'm going to bring back to a quote that I've used a number of times taking from William cook's paper on understanding data abstraction revisited he talks about abstract data types he talks about object-oriented programming and he makes a really interesting observation he says lambda calculus was the first object-oriented language this is a really interesting thing because we normally think of lambdas as being a very functional thing but it's this idea that they can retain some kind of state now he puts 1941 I did a bit of digging around turns out it's 1932 so you know when people got really excited it's just like yeah Java finally it's got lambdas it's just like yeah programming like you're in the first half of the 20th century yeah the ultimate retro that goes way way way earlier than any of the other papers I mentioned so the home straight I want to look at oh great I get to use lambda symbols as well so this is how we can define a stack and there in a way that pleases some people mathematically but instead of doing this I'm going to use JavaScript I'm sorry okay so so you know I could have used a programming language but you know time is short so what I'm going to do here is I gave you a different views their kind of function as object or module type approach this it's it's interesting hiding within JavaScript is actually a very pure object model it's just not the one that most people use so what I got is new stack I'm going to say right new stack when I call new stack it's a function but it yields what does it appear closure I have an array called items and then I return you I don't return you a thing in the classic sense with state what I do is I return you a set of operations I return you a dictionary or an object that contains operations there's an operation called depth and top and pop and push and these bind to these capture the closure of the local variable items there is no way anybody can get their hands on that private state it is truly private and so this is sort of simple implementation or we could use the pushpop approach both of those have exactly the same behavior and that's quite important ok so this is a very pure object model very very late bound but I've shown things like this before but I think what was more interesting to me was recently asked I think what hang on what about inheritance how do we do inheritance in this model well small change what I'm going to do is I new introduced the idea is that a function doesn't just associate what it does is it takes some state and then it associates any extra state and any extra methods that you need and returns those and it merges those that's what the object dot assign is doing so here we have you know something we're going to call it stackable and this actually follows if you've ever come across the mixing idea this follows very closely the idea of mixing so you can kind of get close to that with Java Java r8 traits but not quite but you know what we've got here is what you do is you take something and you say write my new stack I'm going to take that function that I had before and I'm going to say I start with an empty object open closed curly and it's stackable and that now gives it stack ability and so now I can call new stack exactly as I did before and this is great but it gets more interesting because I've inherited from not a lot now let's actually inherit from something neat more do something more meaningful let's make now stack clearable so what I'm now going to do is I'm going to add the ability for it to be clear so you can say clear and it will empty it while there is depth then call pop so now you can call clear on it so now what this looks like is Neustadt is clearable of stackable of empty object so you take an empty object and you apply you mix in stack behavior then you mix into that clear behavior so this looks like composition and in fact we can create a composition so it says that inheritance can be understood as a composition so is it so it's an interesting kind of crossover between functional and object-oriented thinking now job scope doesn't come with its own compose you can do that as a reduction it's a fold I'll leave that one for homework and you might be tempted hey wait a minute I can do overriding as well yes you can it's great you just sort of take something and we're going to rebind the push operation here I've got the idea of having a non duplicating top if you push something that's already on the top then it ignores it okay so if the top is the base top does not equal the new top then we'll we'll push it will use the the base version otherwise we'll just ignore it and people often use inheritance like this okay so I'm going to suggest one thing don't do that why Barbara Liskov again she's getting featured quite a bit today anybody come across Liskov substitution principle LSP yeah the L in solid principles it is widely misunderstood but this is the paper 1987 on data abstraction and hierarchy where she talks about it and she says a number of things and obviously she has to phrase it formally type hierarchy is composed of subtypes and super types the intuitive idea of a subtype is one whose objects provide all the behavior of objects of another type the super type plus something extra like stackable clearable what is wanted here is something like the following substitution property if for each object oh one of type s there is an object out this is the bit where gets boring of type T such that for all programs P defined in terms of T you can tell somebody wrote this out on a white board using upside-down s and then translated it into English it would be much better mathematically P defined in terms of T the behavior of P is unchanged when oh one is substituted for Oh two then s is a subtype of T now this is a few pages into what I think is a classic paper and most people their eyes slide off this paragraph it's very often quoted but it has some interesting implications let's actually look at what these implications are so this is a test you can you know it's an ad hoc testing framework that I sort of threw together to just check that my stack did actually do the right thing so I've got a simple one of my tests a non-empty stack becomes deeper by retaining a pushed item as its top and then I've got this stack called new stack I push geeking I push Prague I push Prague again the death should be three the top item should be Prague okay if I have new stack being the composition of clearable and stackable so it's stackable and it's clearable and it inherits in that sense then you know the test passes that's green that's great what happens when I also throw a new non duplicate top it fails it turns out that many people kind of misunderstood the implications of the Liskov substitution principle in fact and it's not an abstract idea it's a very concrete idea I can write a test to show you how Liskov is violated in this case the problem is that we said that for the base the super that whenever you act whenever you push something the height goes up by one and the top item is whatever we left there the height always goes up by one the problem is that here is a test that demonstrates that when you've inherited and overridden it so sometimes it goes up by one but sometimes it doesn't actually it fails so this is an interesting thing is that less cough is actually a thing that you can test you can write a unit test for and things can pass and things can fail it's not a it's not a fluffy abstract idea at all it's very very concrete so this stack with non duplicate tops does not satisfy the stack contract as I said the contract is much larger than just simply the idea of pre and post conditions it's it's something else about what we can expect and that's simply one way of phrasing it so this is the relevant bit for all programs P defined in terms of T the behavior of P is unchanged when oh one is substituted for o2 that's the relevant bit so in closing given I chose books given that I've chosen a number of papers given that people often refer to these papers I want to kind of close down on the fact that we've looked at a number of things that are of interesting knowledge but in some ways they're very fundamental and they are timeless all of the stuff is old but we often are in such a hurry to move to new solutions that we forget that there are ideas that could really save us a lot of work or that they're ideas we've heard of but have greater depths I think for example that the list Gogh thing is interesting because that paper is freely available as most things are on the Internet Angela Carter quite a book is simply the container of an idea like a bottle what is inside the book is what matters sometimes you have to really go inside and actually have a look what is this person actually trying to say it turns out that many of the principles that we overlook are hidden but known are hidden inside some really interesting blog posts interesting papers but also many of the ones that we misunderstand the truth is in there so we need to think of when we think of full-stack I've used stack as a gratuitous example to allow us to explore some very fundamental but are often overlooked techniques I've tried to show the connection between them I've also tried to use this as a way of saying you need a bigger picture as well when you come to this stuff it's not enough to know one thing well and one architecture and one tooling well there is a bigger picture as as Grace Hopper observed to me programming is more than an important practical art it is also a gigantic undertaking in the foundations of knowledge and I hope that if nothing else makes you want to go out for a beer or cut some code thank you very much [Applause] [Music]

Channel: GeeCON Conference

Views: 3,143

Rating: 4.8139534 out of 5

Keywords:

Id: JiQct3QixMo

Length: 59min 14sec (3554 seconds)

Published: Wed Jan 24 2018