Functional Programming in 40 Minutes • Russ Olsen • GOTO 2018

[Music] thank you so I am indeed Russells and I want to talk to you about functional programming this morning just briefly about me I am kind of a programming language geek I have been learning programming languages and occasionally writing books about them for most of my career and lately I've been doing functional programming so a couple of things about this talk before I really get started it's you might be thanking functional programming in 40 minutes how is he going to cover functional programming in 40 minutes and I'm thinking what am I gonna do with the last 10 minutes because I I think the whole theme of this talk is that functional programming has been mystified and people get very excited about new technologies or technologies that are new themselves and they start talking about them in big expansive terms about how it's gonna change everything and in fact functional programming really is a fundamentally simple idea and that's what I'm going to try and get across this morning I do want to warn you that I'm a closure programmer so my flavour of functional programming is with closure so necessarily my point of view and what I'm gonna tell you is going to have sort of a closure spin to it I'm going to try to point out in places where this is the way closure does it and other programming languages do it differently but keep in mind that I'm kind of a closure programmer so I really want to cover three things this morning and really one main thing and the main thing is what is functional programming where did it come from like what's the ideas behind functional programming how did we come to them and that's most of what I'm going to talk about this morning and then from there I'm going to move on to what's it like to be a functional programmer what's it like to actually do it and in particular what's it like to do it as a beginner I may be from the object-oriented tradition and I start programming in a functional style what's that light that transition like and then finally I'm going to try and answer the question which is maybe a little subjective which is does it work can you can you make it work and again I'm sort of a practical software engineer I write applications for my day job and so this is going to be a sort of functional programming for you no ordinary programmers who want to write applications and libraries so let's start with what it is and there is so when I was first getting involved in functional programming you know you google functional programming how do I learn functional programming and there was one thing that I saw over and over and over and it is functional programming forget everything you know about programming or even worse functional programming forget everything you think you know about programming yeah I think software developers programmers here it turns out we know quite a lot about programming what do we know about programming we know about things like programs are full of names and you assign values to those names programs are full of conditional logic they're full of iteration they're full of procedures and subroutines and things like that there's all sorts of data types that you find in every programming language strings and boolean x' and numbers one of the things we know about programming is indentation should we forget about indentation with functional I don't think so we have namespaces because you need to segregate those names so that they don't collide we know that there's a raised and hashes or maps another if you're doing object-oriented programming then you have classes and inheritance and methods and then finally we have programs right and this is I don't know sort of everything we know about programming if we're doing object-oriented programming are we going to forget all of this are we gonna forget about in pation really comments no of course not what we really want to do with functional programming I think of it as more of a refactoring okay and it's more like I'm gonna take some of the things that I know about programming and hold on to them but I'm gonna refactor my ideas of how how to do programming so I think of it imagine you're working on some ordinary application I don't know an accounting application and I apologize the people who actually work on accounting applications I think of those as sort of the dull applications but some application and maybe your application it has a design there's a series of modules or services and they all go together reasonably well and it's not perfect but it works and it's okay you have a pretty good design but time goes by and requirements change and you start fixing bugs and maybe you add a new module rearrange things a little bit maybe you add another one maybe you get rid of a couple because you don't need them anymore and then maybe you start adding dependencies that weren't there before right this is what happens in ordinary systems as they go through their life and after a while you start to develop what we technically call a mess right it gets to the point where it works but it works but it's hard to explain it works but the next bug that you have to fix is really a lot harder to fix them and then it should be and you spend a lot of time you know you're in this situation when somebody says how does this thing work and you say well it works this way except when this happens except when you have this except on Tuesdays when it's raining you know we've all been there right the thing is just it works and maybe the users or the customers they like it but it's it's just getting hard and messy and so what do you do in those situations I've got my counting system and it's just kind of become a mess well if you're lucky your boss or maybe you whoever is making up their mind you sit down and you say I'm starting with a clean sheet of paper I'm starting with the completely clean sheet of paper like it's the funny thing about clean sheet of paper we're gonna redesign the system there's clean sheet of paper what's the first thing you do when you redesign something from a clean sheet of paper funny thing is you pull in all the pieces that already work right nobody you like rewrite something from the ground up every single line frequently when you're rewriting a system refactoring redesigning it what you do is you pull in all the pieces you say oh this piece works fine it's just sort of out of sync with everything this other piece works fine maybe I'm gonna change them a little bit but there's pieces that work okay there's a pieces you throw away certainly but there's pieces that you keep so then you you have all these pieces that work and what you say is the reason my design is not working is because the pieces I don't have like an overriding idea of how they all go together I need a paradigm to pull this system together so a redesign is mostly or a refactor is mostly let me reorganize the pieces that I have maybe add some new ones in eighth with a new paradigm so maybe in our accounting system we have a message bus so we pull all of our pieces together but we have this new organizing idea it's a message bus all right well what about programming is programming and by programming I mean sort of the traditional kind of mainstream object-oriented programming is that getting messy does that need refactoring I think maybe it does and so you remember I said oh it works this way except on Tuesdays Thursdays and Saturdays right that's one of the signs well this is from the Java tutorial during type of racers process first bound if the type parameter is bound or object as the type parameter is unbounded right it has that feeling of sort of ad hoc Ness that you know it's the Tuesdays Thursdays and Saturdays thing and I don't mean to pick on Java right I think it's this has kind of been a sign that maybe the object-oriented thing has gotten to the point where it's not really working for us anymore one thing I am sure of is that if they ever build the monument to programmers the software development may be a giant statue of a keyboard or something there'll be a plaque at the bottom one thing I am absolutely sure of is those words will not be on the plaque right now but it's not just Java I quite like Ruby and here is a description of Hell protected methods work in Ruby but the receiver is always self it's the finding class or an object that it it it's sort of complicated but for the ultimate in this sort of thing you just look the C++ which is just kind of full of convoluted rules like this okay so maybe our object-oriented approach to life to programming has gotten a little messy maybe we need to do something else maybe we need to start with a clean sheet of paper and if I have any sympathy for the forget everything you know about programming it's starting with this clean sheet of paper except except the first thing we do with clean sheet of paper is we pull in all the stuff that works right the idea of a program or a service works we can keep that namespaces work or raise work now maybe we're gonna change these things a little bit as we go but all of the stuff works we are not going to throw away the idea of a string right in functional programming you don't have to forget everything all right so like our accounting system what we're looking for is a new organizing principle we need a new paradigm for pulling all the stuff that works into a whole into a functional whole so where can we find a a new paradigm well it turns out there's another group of people there's another bunch of people who were doing work very similar to the work that we're doing the mathematicians now it's no it wouldn't be any surprise if I told you right that there's a huge overlap between computer science and math I think computer science probably is a sub branch of mathematics but I think what we is working software developers don't really appreciate is that mathematicians you know the people who do the really serious stuff they have the same kind of problems that we have in terms of so if you ever like design the system and you say oh I'm gonna have this abstraction and and that's gonna be a part of my system and I'm gonna have this other one and I'm working to a point where I want to have my system do something and then you've got five or seven abstractions and you're trying to keep them all in your head you're making notes and writing all over the whiteboard trying to hold all of those abstractions together till you get to the point where where you've done something where it's functional well mathematicians do very much the same thing at least the abstract ones this is the cover of a book called principia mathematica it was published in the early 1900s by two British guys and what they were trying to do is derive all of mathematics from very simple principles right and let me tell you this is serious stuff let me also say I have never read this book so don't I don't pretend but it is just page after page after page of the most ridiculous abstractions you can imagine until you get to this page and on this page in the middle of this page they prove something really interesting one plus one equals two now you think your life is hard pity the mathematicians because the significant part there's two significant things on this page one plus one equals two and the page number look at the page number it's three 179 it took them 379 pages the proof that one plus one equals two mathematicians have the same kind of problems they've got these abstractions that they are trying to drive towards some goals some something in this case one plus one equals two all right is there something we can borrow from the mathematicians that will help us do they have some organizing principle that will that helps them do their work that maybe we can steal and the answer is yes they have functions and you say well we have functions subroutines procedures methods it's not the same thing a mathematical function is a relationship between two sets okay there's like the input set and the output set and it's a it's a mapping from one to the other okay there's no time involved it's not like you mathematicians in principle compute a function really it just is right this is very abstract stuff so this might be the function for like double the number one goes the two four goes to eight that kind of thing now these things are different the mathematical idea of a function is different than our idea of a method of procedure a subroutine or a function those things that we have because our things are code there are a mechanism and in principle our subroutines procedures functions can delete a file they can update a database they can compute some result from the from the input they can change the input they can compute nothing return nothing right we can do anything we want we it's just some code and it's not quite the same thing as a mathematical function mathematical function it's always the same for any given input it always gives you the same output it's not like five can give you a 42 on Tuesdays Thursdays and Saturdays and other time no it's that would be a different function and in particular they don't change or time mathematical functions they you know it's summer rain spring it doesn't matter they're always all the same mathematical functions are also a thing right they're values that you can manipulate and if this were ten or fifteen years ago I might go into a long talk about first-class functions I think first-class functions are kind of seeped into all of the major languages right a function is a thing you can make or really a procedure a subroutine a you know our computer e things are also things that you can make their values that you can pass around first-class functions in a programming language all right so what if we wanted to borrow this idea of a function from the mathematicians well really we can't like really borrow it because there's is completely abstract but we can have a bunch of rules on our functions that that make our functions act something like the mathematical functions people call functions that behave according to these rules pure functions it's not a term I really like because it's sort of like good bad you know and these are just this is an engineering decision so what if we we just made these completely arbitrary rules and apply them to our programming things and the rule the main rule is that you only look at your input you only look at the parameters and you only produce output you only produce the result okay so what does that mean that means no changing anything no side effects no going in and changing the fourth element of your array if if there's a array parameter it means no deleting the file updating the database none of that you just look at the parameters and transform them into some output all right we're doing this in the hope that it will make our programs easier to write easier to understand and that is the only reason we're doing it's not a good bad thing it's just we're doing it in the hope that it will make our programs to understand well would it actually make our programs easier to understand well maybe and here's why let's say we start with the three element array X and we call some function on X and that function is a function that a Bay's these rules so we produce some output Y from that function now the question is what's X at the end of this it's the same thing we those are the rules okay so now what if instead of just calling one function we call some incredibly complicated combination of functions on X right just just an absurdly complicated sequence of functions to compute Y what's X it's the same thing all right you can see how these rules might actually help you make sense of your programs right i I had this thing I call a bunch of functions I still have the same thing I don't have to look through 25 pages of code trying to figure out if some of they stuck something in the middle of X or something alright so here's what we're gonna do we started with our clean sheet of paper and we pulled in all of the pieces of our current programming language that we thought might work for us that we thought are working right now and now we're gonna pull in these special rules for functions okay and we're gonna see where that gets us alright so we think about that for a while and then we have a problem and the problem is that in most programming languages data structures like arrays and hashes or maps and lists and things like that they're mutable so if you had a little three element array anytime you felt like it you could look at that middle element and jam some other value in it you can mutate your array alright that's a problem because if we go back to our example and we have that incredibly complicated functional expression there in the middle and we changed it a little bit so that we just jam a new value in X now we are right back to head look through all of this code to try and figure out what's what sex and that we've just lost any gains that we thought we might get so I'd like to propose a radical solution to fix this particular problem let's make all of our data structures immutable once you make that array you can't change it once you make that hash your map you can't change it we're just going to make them completely immutable you can see how that fixes the problem right I am now unable to change any of the data in my data structures but it might feel like well how do you get anything done if you can't like mutate anything the answer is really quite simple we provide all of our data structures with an operation which says give me a copy of yourself except that's just like you except a little different so give me a copy of yourself mr. ABC array that's exactly like you but the middle element is Q if it's a hash or a map give me a copy of yourself that has all the keys associated with the same values except for this one which is different or this new one or delete one okay so we do that and now we no longer have to worry about anybody changing or data out from underneath us but now we have another problem and that other problem is copies right and and you could sort of frame it this way what if I have a million element array and I want to change three elements in the middle do I change the first one and make a copy of a million elements and then change the second one and make another copy and after a while I've got three or four million elements of arrays that I've kind of created and things like that turns out no you don't have to do that so there was the problem right am I gonna copy I've got a 27 LM in array here I'm trying to mess with the element in the middle do I have to copy all 27 elements it turns out we can do better than that turns out some very clever people have come up with something called persistent data structures and here persistent doesn't mean save to the database or saved in a file it's just an unfortunate word they picked for something completely different but what persistent data structures are is their versions of all the familiar data structures things like arrays and hashes and lists and trees and all of that there's basically one for all of the familiar data structures and they're there they have two key characteristics one they're immutable and two they support this VIP this copy on modification operation but they do it very efficiently without a lot of copying which seems ironic the way it works is think of that that 27 element array right and what if you implemented that 27 element array all the letters of the English alphabet I guess and the exclamation point what if you implemented it as a sort of tree and each tree node has has sort of three slots so each node has is like a block of three and that block of three if it's an interior node either points at something or if it's a leaf node it actually has the data okay and this is how I this is how I represent my array sort of behind the scenes well if I want to get at the twenty twenty second element I just have to traverse a couple of nodes and I'm there at my twenty second element right but what happens if I want to modify this thing which really means make a modified copy of this thing let's think of how much copying we actually have to do well if we want to modify the middle element there well obviously we'd have to copy that little three node block there we'd have to copy the one that point set it we'd have to copy the one that points it and then we don't have to copy anything else because we can reuse everything else from the old from the original so you get this the the new one and the old one share most of the data particularly as the data gets bigger and bigger and bigger which is exactly what you want now you might be thinking but what if that data in the in the old one changed this out from underneath me it's all immutable so you see how this all sort of hangs together all right so that's that I think solves our problem of immutability and principle now I'm showing this as three elements but in real in that's just to get it on the slide in real life people tend to use more like a branching factor of 32 so all of the nodes are 32 wide and if you do the math it turns out that a three element tree like this with 32 wide nodes you can you can support more than a million elements so it's really pretty efficient if you if you put your mind to it all right so here's my hokey icon it's a lock for immutable data structures so we've now got everything that we borrowed from our original language we've got these mathematical like pure functions and we've got immutable data structures all right and we've now built this lovely functional world we've built this lovely functional world where there are no side effects and the programs are easy to think about because you never have to like dig in to see who the heck is changing my array and now we have another problem and that other problem has to do with those side effects we got rid of you see if people still use two business cards I would print business cards and I'd have my name on it and I and the only thing the business card would say is side effects if you think about what a side effect is like writing a file or deleting a file or updating a database or hitting some service side effects are what we get paid to do all of the things that our customers care about our side effects our customer does not care about our code right we care about the code we want the code to be easy to understand fundamentally the the person behind that accounting system the banker or whoever does not care at all about the accounting system they care that the database is updated that the reports are generated that the files are created that the services are hit right we are we are all in the side effects business right and in fact our customers don't think of them the side effects they think of them as the desired effect and what we've just done is we've built this beautiful lovely functional environment that can't do anything right which seems to be a little bit of a problem so what we need is a bridge from our beautiful lovely functional code into the real messy world of side-effects right we want our programs to be able to do stuff so how do we do that well here here is the point where I have to sort of take off my functional hat and put on my closure hat because all programming all functional programming languages sort of approach this problem differently but in general they're kind of the same but here here's some of the closure solutions to the bridge to be outside messy world one problem in the outside messy world is how do i implement mutable state within my program okay think of like the classic hit counter in a web application congratulations you're the 57th visitor to this web application if you think through the functional world that I just described you can't do that there's no mutable state there's nothing that can change so what we need is some way to represent mutable state what closure does this through something called atoms and an atom is the closest thing the closure has to a variable it's a container for some mutable state that's what an atom is well that sounds like a variable what's the difference the difference is that what is the way you update an atom because an atom is a bridge between the lovely functional we're all that we've built in the messy outside world that people pay us to to mutate and affect and what the way you update an atom is you throw a function at it okay so if that's a hit counter you would throw a function at the atom that knows how to increment a number and what the atom does is you throw a function at it the atom takes that function gets the current value of the atom sense that into the function gets the result that comes out of the function so it evaluates the function and the value that comes out of the function becomes the new value of the atom all right it's a bridge between the nice clean functional world on the one side and the messy stateful world of the real life on the other side now to get a real feeling for like the power of this kind of approach let's think about what happens if you have to you're trying to update this atom on two different threads and that and the updates come in at the same time right this is the classic oh jeez I don't want to lose an update kind of problem that anyone who's ever messed with threads is familiar with well what what atoms do is you know both of the functions will come in they'll both see the same value of the atom 59 and you know this is where the trouble is lurking because we don't want to lose an update they will both calculate 60 as the new value of the atom but one of those functions will get there get in a little faster and change the value to 60 and what will happen at that point is the atom will notice that when the second function comes in and tries to update the atom it will notice that the value changed out from underneath it and what does it do it just runs the function again this is sort of collision detection and you know rerunning so it's it's similar to what a database does with transactions now you might be thinking oh is it okay to run that function a second time what about side effects guess what there are no side effects allowed the only side effect is updating the atom and that's what we're controlling so atoms are a lovely bridge between the clean functional world that we're building and the messy outside world and are the things that closure uses to maintain program state okay there's another stateful problem which is how do I update the database delete the file write the file hit the webservice the sort of I need to do something to the outside world and there's a bunch of different solutions to this including a bunch inside of closure but most of them all rely on some kind of cue and you put requests of one kind or another on the queue and then there is some nasty stateful side-effect decode connected to the queue which is pulling things off the queue and doing things with it so you put a request hey update the database and somewhere on the other side of the queue is some messy code that is pulling that request off and doing something side-effect e right the reason you have the queue there is then you can sequence there's a clear sequence of what of these operations so that things that sort of happen all at the same time close your agents which is one form of this thing have the same kind of bridge to the functional world that atoms do in that what you throw at the at the agent is not data but you throw a function at it and the function gets executed in sequence and it's the thing that does the nasty stateful write the file delete the file whatever all right so here is our functional world right we have all the things we pulled in from our old programming style arrays and namespaces and things like that we have these pure mathematical style functions we have immutable data structures and we have the bridge to be outside nasty stateful world alright and that's as far as I'm concerned that is what functional programming is that is functional programming in the not you don't have to forget everything you know about programming so what's it like to program in a functional style and in particular what's it like the program and a functional style if you're new at it yeah no there is no magic here okay I think it's a better way to write programs but there's no magic none okay you still have off-by-one errors like this is a new slide I put the slide in day before yesterday because the day before that guess what I spent half a day finding it and off by one problem you still have those kinds of things you can still have redundant code we can be working on something together and you go down the hall and sit down and write some stuff and I sit down and write some stuff and then at the end of the day we realize oh we just solved the same problem functional programming and not gonna help you with that maybe it'll help you talk about it and figure it out quicker but those kinds of things are still gonna happen you can still write bad code right I have as many conversations you know that conversation you have right oh you're going along you're you're you're in the system and you find something and you're like it's this as bad as I think it is and you go to your colleague and they're like yep yep that is as bad as you think it you still have that the database is still down right in the database always down one bit of magic that you do have is with threads one bit of magic that functional programming gives you is with threads functions take their input operate on it and give you output right it it reduces the amount of state but particularly overwhelmingly the immutable data structures mean that one thread cannot change the data out from underneath another thread one thread can get the wrong version of the array or the hash or the whatever it is but it can't it can't change things out from underneath another thread and in particular you cannot get the situation where one thread is in mid update and the data structure is complete garbage and another thread looks at it that cannot happen with immutable data structures because the thing is either there or it's not and the new version either exists or it doesn't when the new version of your array or your hash or whatever exists it is at least consistent so there is some thread magic it doesn't solve all your thread problems think about atoms but it eliminates a whole class of them and after all isn't that what funk our programming paradigms are supposed to do but so what's it like to program using effectively these are rules right these are arbitrary rules that we are saying we're going to adopt in our programs because we think that those rules are going to help us build better programs well initially it feels like programming wearing handcuffs right you know you have if you've never programmed with the beautiful data structures you spend your first week like oh I'll just change the third element in that no I can't I need to return something and it's a it really feels like you are burdened by all of these rules and it's hard to see why you have to program with these rules in your gut but you do it for a while and after a while you realize that oh okay my code is easier to understand I can reason about the code because I don't I spend a lot less time looking through all the code to see what's going on after a while those handcuffs feel like a bicycle right a bicycle right you got to hold your arms like this there's all kinds of rules in riding a bicycle right it's only most of us can only ride a bicycle and a pretty flat even surface but it reduces the friction between you and where you want to go and that really is my feeling about functional programming all right and we have left all right we're doing pretty well so does it work I think is the final question I want to try and answer and for that I'd like to introduce you to pedestal pedestal is a library or actually set of libraries it's produced by the company I work for a cognate ech and pedestal and it's open source and it's widely used in the closure world so forget about the fact that it's produced by my company that's not really important it happens to be a production library that I'm familiar with and pedestal is that library in the closure world that oh I've got a request that came in I need the parts of the parameters I need to route it to the right place I need to authenticate it closure provides or I'm sorry pedestal provides the plumbing to do all of that it's that library right there's basically one or more in in every programming environment okay but think of it pedestal is just a giant pile of production closure code written in a functional style so let's look inside pedestal what what what is in pedestal well there's eighteen thousand seven hundred and six lines of code and pedestal as of last week I gotta tell ya closure is a pretty dense language so eighteen almost nineteen thousand lines of code that is a lot of closure that is more closer than than is in most applications okay what if we look a little deeper what's in those about nineteen thousand lines of code there's 28 protocols a closure protocol is a bundle of method signature or function signatures and it's the way that you define interfaces between different parts of closure application or library makes sense that a library would define a bunch of interfaces so there's 28 protocols in pedestal there's eight of these bridges to the outside world right agents and atoms and things like that there's nine record types a record type enclosure is as close as closure gets the object-oriented programming it's like here's the thing and it has the fields in these operations just nine of them there's one multi-method a multi method is a completely general form of polymorphism all right and then there's 944 functions think about that there's nine all of those small numbers there's twenty eight of these and eight of these and one of these and then there's 944 functions and if you don't remember anything else from this talk remember this pie chart this giant pile of production functional programming is almost entirely functions and it has a little sliver of interfaces and then a trivial amount of everything else and if you want to know what functional programming is like for like I get up every morning and I write applications kind of programmer it is this you spend all of your time or most of your time just writing functions if that sounds a little boring yes it is boring it was boring in that good way of I can do the same thing over and over and over without having to negotiate with my language without having to negotiate with libraries I just write these testable straightforward functions for the most part and then maybe a few protocols so there we have it that's kind of the working programmers guide to functional programming does it work I think so I hope you'll try and find out for yourself thanks for listening you

Channel: GOTO Conferences

Views: 710,493

Keywords: GOTO, GOTOcon, GOTO Conference, GOTO (Software Conference), Videos for Developers, Computer Science, GOTOber, GOTO Berlin, Russ Olsen, Cognitect, Functional Programming, Clojure, JVM, Scala, Elixir, F#

Id: 0if71HOyVjY

Length: 41min 35sec (2495 seconds)

Published: Fri Nov 09 2018