dotGo 2016 - Dave Cheney - Do not fear first class functions

yeah yes thank you and welcome to dot go so two years ago I stood on a stage like this and I told you about my opinion for how configuration options should be managed and go and the cornerstone of this presentation started as a blog post by Rob pike it's called self referential functions and the design of options in the last two years it's been truly wonderful to watch this idea mature from Rob's blog post all the way to the GFC PC project who've really continued to evolve this design pattern into what I think is its best form so far but when I was talking to some Gophers in London at a conference a few months ago several of them expressed a concern that while they understood the idea of a function which returns a function this is the technique that powers functional options they were worried that other go programmers and I suspect they mean less experience go programmers wouldn't be able to understand this style and this made me a little bit sad because I consider go support the first-class functions to really be a gift and it's something that we should all be able to take advantage of so I'm going to I'm here today to show you that you don't need to fear first-class functions so to begin with we'll really quickly recap the functional option pattern we start with some options expressed as a function which takes a pointer to a structure to configure we pass these functions to a constructor and inside the body of that constructor we iterate over each function in order passing in a reference to the config and finally we call new terrain with the options that we want and away we go so hopefully everyone is familiar with this pattern now where I believe the confusion comes from is when you need to have an option function which takes an argument for example we have with cities which lets us add a number of cities to our terrain model because we're cities takes an argument we can't simply pass with cities to new terrain like the signatures just don't match instead we evaluate with cities passing the number of cities to create and use the result of this function as the value to pass to new terrain so what's going on here let's break let's just break it down fundamentally evaluating a function returns a value we have functions that take two numbers and return the number we have functions that take a slice and return a pointer some structure and now we have a function which returns a function and the type of the value that is returned from with cities in this example is a function which takes a pointer to config and it's this ability to treat functions as regular values that leads to the name a first-class function so another way to think about what's going on here is if we try to rewrite the functional Option pattern using an interface so rather than using a function type we just declare an interface will call it option and give it a single method apply which takes a pointer to config whenever we need to call new terrain we and we pass in one or more values that implement our options interface and inside new terrain just as before we iterate over each of those values and call the apply method on each of them alright so this doesn't look too different to what we saw in the previous example rather than ranging over a slice of functions we range over a slice of interface values and call a method on them so let's take a look at the other side what does it look like when you need to declare an option because we're passing around interface implementations we need to declare a type to hold that apply method we also need to declare a constructor to return our splines option implementation and hopefully you've really picked up this is more code now to write with cities using our option interface we need to do a little bit more work in the previous functional version the value of n the number of cities we want to create was captured lexically for us in the Declaration of that anonymous function because we're using an interface we need to declare a type to hold the count of cities and we need it then a constructor to make sure it's properly assigned and in putting it all together we call new terrain with the result of evaluating with reticulated splines and with cities now it go for con last year Thomasson art spoke about this qu ality of a first-class function and an interface with just one method and you can see the play out in this example the interface with one method and the function are to some extent equivalent but as you can see dealing with functions as first-class citizens is much less code so let's leave interfaces for a moment and talk about some other properties of first-class functions when we invoke a function or method we do so by passing around data the job of that function is often to interpret that data and take some action function values allow you either pass behavior to be executed rather than data to be interpreted and in effect passing a function value allows you to declare code that will execute later perhaps in a different context and to illustrate this let's look at an example here's a simple calculator it has a set of operations that understands and it has one method called two and do takes an operation and an operand also for convenience do returns the value of the accumulator after the operation is applied now our calculator only knows how to add subtract and multiply if we wanted to implement division we'd have to allocate and operate an operation constant then we'd have to open up the do method and we'd have to add the code that actually implements division this sounds kind of reasonable like it's only a few lines of code but what about when you want to implement square root or exponentiation or any other function that our calculator handles every time we did this du is going to get bigger because and it's going to become harder to follow because each time we add an operation we also have to encode in to do the knowledge of how to interpret that operation so let's rewrite our calculator as before we have a calculator which manages its own accumulator the calculator has a du method but this time the operation is a function and it takes a value as the operand to work on now this signature of the operation function might be a little bit intimidating so I've broken it out into its own declaration so let's talk quickly about what that declaration is the OP type as you see there is a function which takes two floats and returns a third whenever do is called it calls the operation function that we pass in providing its own accumulator and an operand that we provide okay so how we're going to use this new calculator as you probably figured out we use it by writing operations as functions so this is the code to add what about if we want to do the other operations subtract multiply they're not very hard to do either and as before we construct a calculator and then call it passing operations to apply add a our brand alright so now we can describe operations as functions we can probably go ahead and extend our calculator now to handle things like square root but it turns out that there's a problem square root only takes one argument not two but the signature of our calculators do method requires an operation function that has two up two arguments so what can we do about this aim maybe we can just cheat and just say interests ignore the operation that's kind of big gross I think think we can do a bit better so let's redefine add from a function that's called with two values and returns a third to a function which returns a function that takes one value and returns one value so our du method now invokes the operation function passed into it and it provides its own accumulator and then stores the result of evaluating that function back in the accumulator now in main we call calc do not with the add function itself but with the result of evaluating out of ten and the type of evaluating out of ten is a function which takes a value and returns a value and this matches the signature that do requires so subtraction and multiplication are the easy to implement as before what what about square root it turns out that that's now easy as well this implement square implementation of square root avoids the awkward syntax the previous calculators operation function as a calculator now operates on functions which take and return just one value and I hope that you've noticed that the signature of square root and the signature of main dot square root are the same so now we can so we can make this code smaller by reusing any function from math package that just takes a single argument so what's happened here we started with a model of hard-coded interpreted logic we move to a functional model where we pass in the behavior that we want and then by evolving this a step further we've generalized our calculator to work for operations regardless of the number of arguments they take so let's change tracks a little bit and talk about what most of us are here at a go conference for concurrency and specifically actors and to give to your credit the examples here are inspired by a talk by Brian Botham from Guerlain UK a couple of months ago you really should check that one out so suppose we're building a chat server we plan to be the next HipChat or the next slack but we probably should start small for the moment here's the first cut of the heart of any chat system we have a way to register new connections we have a way to remove old connections and we have a way to send to send messages to all of the registered connections now because this is a server and all of these methods could be called concurrently we need to make sure that we use a mutex or something to protect this cons map to make sure there are no data races and it's not corrupted so is is this what we call idiomatic go maybe this is our first proverb don't mediate access to shared shared memory with locks and mutexes instead share that memory by communicating so let's apply this advice to our chat server rather than using mutex to serialize access to this Kahn's map we can give that job to a go routine and communicate with that go team via some channels so ad sends the connection to ad on the add channel remove sends the address of a connection to remove on the remove channel and send message sends the message to be transmitted to each connection via the send message channel and so now let's look at loop rather than using a mutex to serialize access to the Kahn's map loop wait until it receives an operation in the form of a value sent over one of these channels and it will apply the relevant case and we don't need the mutex anymore because the shared State this Kahn's map is now local to the loop function it can't be mutated by anybody else there cannot be a race because it only exists within the scope of that function but there's still a lot of hard-coded logic in here loop only knows how to do three things it o knows how to add remove and send messages and as with the previous calculator example if we wanted to extend our marks it would involve adding a channel adding a helper to send data over that Channel and then opening up the Select logic inside loop and adding the knowledge of how to process that data so let's look at a let's look at our calculator so just like a calculator example we can rewrite our MUX to use first-class functions to pass around the behavior we want to be executed not the data to interpret now each method sends an operation to be exit executed in the context of the loop function using a single operation channel in this case the signature of the operation function is a map of addresses the cons probably in a real program you're going to have a more complicated type to represent clients but for the purposes of this example I think this is sufficient remove is similar we send a function that deletes the connections address from the map supplied and send message is a function which iterates over all the connections in the supplied map and calls right string to send each of them a copy of the message so now you can see that we've moved the logic from the body of loop into anonymous functions which are created by our helpers and so now the job of loop is simply to create this connections map and then wait in a loop for operations to come in and then execute them one at a time but there are a few little problems to fix the most pressing is the lack of error handling inside send message if there's an error of writing to one of the connections that's just going to be discarded so let's let's take a look at how to fix that now to handle the error generated inside and and on with inside the anonymous function that we pass into loop we need to create a channel to communicate the result of the operation this also creates a point of synchronization the last line of sent message blocks until the function we passed into loop has been completed and note that we didn't have to change the body of loop at all to incorporate this we didn't have to change anything about the existing code and this means that we can now go ahead and add new functionality like the ability to send a private message to one channel to one client we do this by passing in a function to loop that tries to find the find the client that matches this address and then we wait for the result to be transmitted back we check to see if it's nil which is an indication that that client is no longer connected otherwise we now have the connection and we can write our message to the client so in summary first-class functions bring us tremendous expressive power that lets you pass around behavior not just dead data to be interpreted now first-class functions aren't new or novel many older languages have offered them even C in fact it was only somewhere along the lines of removing pointers from object-oriented languages that somehow we lost this ability and really if you're a JavaScript program you've probably sat there in the audience wondering what I'm talking about for the last 15 minutes this is like second nature to you first-class functions like other features go offers should be used with restraint just like it's possible to make a an over complicated program by the overuse of channels it's just as easy to make an incomprehensible program with the overuse of first-class functions but that doesn't mean that you shouldn't use them just use them in moderation and first first-class functions are something I believe every go programmer should have in their toolbox first-class functions aren't unique to go and it's some and somethingö programmers should not be afraid of using and if you can learn to use interfaces you can learn to use a first-class function they aren't hard they're just a little bit unfamiliar and this unfamiliarity is something that I believe that can be overcome with time and experience so next time you define an API it has just one method I want you to ask yourself should this really just be a function thank you so much you

Channel: dotconferences

Views: 24,476

Rating: 4.9357944 out of 5

Keywords:

Id: 5buaPyJ0XeQ

Length: 18min 54sec (1134 seconds)

Published: Fri Oct 28 2016

Reddit Comments

[u/jimmeyotoole]

The first part scared me a little, but his calculator example was great and actually really applied to what I'm working on at the moment in go (an NES emulator). I was trying different ways to do like function tables and look ups in an attempted to try and avoid giant changing if or case functions. Thanks for sharing and of course thanks to Dave!

[u/google_you]

if functions were first class values, how do I serialize them, send them over network, deserialize and call them?

yes, i want to send iterators over wire, distribute, continue.

[u/mallocc]

Not sure why a dislike of first class functions translated to fearing them.

I certainly understand what he's advocating here but aiming for expressive over simplicity of readability at the expense of a few case clauses feels contrived. I didn't find any of the examples using this approach more readable than their more imperative counterparts.

[u/Paradiesstaub]

Looks like the visitor pattern.

[u/neoasterisk]

Although I enjoyed this presentation a lot, I felt a little off by the way Dave sneaked in the usage of his errors package at 16:36. Nothing wrong with advertising your own work but when you are trying to explain such a concept, it's not so helpful to sneak in an external package that could be doing god knows what. Obviously it does not affect the main point but it did make me scratch my head.

P.S. I mean no disrespect by this. I deeply appreciate all the work Dave has done for the Go community.

[u/Remi1115]

I am wondering why the PrivateMsg example at 16:45 isn't a data race. It looks like the function requests a copy of net.Conn, which it gets. Then it sends data to the net.Conn copy, but this is done off the loops goroutine. Reading the documentation of net.TCPConn, I cannot find information regarding if Write is thread-safe, so I assume it is not. I understand a copy of net.Conn is used, but this must have some form of connection with the original net.Conn, else Writes to it won't end up at the correct network. Can anyone explain?

[u/knotdjb]

Seeing as nobody is going to make a comment, I'll go first. I still fear first class functions.

[u/elcct]

I did come up with something similar in my chat:

https://github.com/elcct/taillachat/blob/master/chat/map.go