CppCon 2015: Stephan T. Lavavej “functional: What's New, And Proper Usage"

ok let's get started so hi I'm Stefan T la Wade and for the past eight almost nine years I've been working on VC's STL implementation and I'm here to talk to you today about functional which is a header that existed all the way back in CS plus 98 but has expanded dramatically possibly more so than any other header in the standard library through CS plus 1114 and the upcoming CS plus 17 some people who are pessimists refer to it as cos plus 1 z I'm an optimist I think it actually will be 17 so a couple things to get started please hold your questions until the end if you have a question about a slide I have slide numbers so write them down you'll put them on your phone and then at the end you can ask about that slide and I can go back and we can talk about it also everything I'm going to talk about today is standard so it'll work regardless of what implementation you use as long as it conforms to the standard there's only a couple of implementation specific things that I'll be mentioning and I'll point those out as they come along and in es2015 i over hauled functional support machinery so everything talked about today is available right now in vs 2015 RT M with only a couple exceptions and I'll mention those at the ends you know exactly what's available and what isn't also every feature I'm going to talk about I've cabbed with the standard revision so for some reason you only have access to CS plus 11 or CS plus 14 rather than the full everything that's coming in CS + 17 you'll know what will be available so although this talk is about the standard libraries functional header I want to start by talking about lambdas this is sort of prerequisite for understanding everything I'll talk about later on so they were added in CS + 11 + CS + 14 they became more powerful within a captures and generic lambdas so this is what a lambda looks like shouldn't be too surprising unless you have not used generic lambdas before here I've got a vector strings and I've initialized it with some stuff and I want I want to sort them but not lexicographically and still don't want to sort them by their lengths and I'd like to sort them stay bleed so preserve the original order when the links to the same so I could write this even in CSS 98 with a handwritten function object but with a lambda I can do it all in a single line wrapping for the slide and here it's very simple I simply say here's a lambda and I take Ln R and I return whether L size is less than R size and this distorts them by their sizing I can go print them out all of my code examples here are real code that will actually compile if you include the necessary header is pretty using directive and in me the generic part of this is that I say Const auto ref there's no reason for me to repeat string and this would the lambda would continue working if for some reason I needed to change the type to u16 string you throw to string or W string any of those there's no reason for me to repeat the type there so that's a lambda but I've noticed that many programmers when they they start using lambdas they get some very strange ideas into their head they think that lambdas are some sort of magical thing and the way that I think about it lambdas do something very simple a lambda expression which is what you type in your source code does exactly two things it defines an unnamed class and then it constructs an object of that type possibly initializing some data members which are there captures so the lambda syntax is very convenient as I saw in the previous slide I could cram the functionality of oh just compared these strings by their size into essentially a single line but it doesn't grant you physically new powers you could always have done this by writing a function object by hand the problem is that it would just be more verbose and you would have to put the definition outside of the local context of wherever you're invoking the algorithm so the lambdas are convenient but they're not magical in any way also I've noticed a lot of confusion lambdas they're a core language feature the part of the compiler and they're available without including any standard library machine machinery they're not stood functions they can be stored instead functions and we'll go over that at the end of the talk but they are not stood function so if anybody says oh the lambdas the stood function you need to correct them no they're not stood functions they're completely different lambdas do have a couple tricks if you have a stateless lambda it can be converted to ordinary function pointer but that's not super magical you could write conversion operator by hand if you know this in tax and you have a type def for the function pointer so remember a lambda it just defines a class constructs an object of that class it's a class type it's not a built-in function it's not a function pointer there are some people even people who refer to lambdas as anonymous functions do not do that or I will make this space seriously they're not anonymous functions they're classes ok so that's the core language feature that's the talk I'm going to talk about the standard library so instead of doing it in chronological order talking about 11 and 14 and 17 I want to present it in logical order and the story begins with invoke and now invoke is a function in CS plus 17 but you'd still need to understand even if you're only using CS plus 11 and cos plus 14 and I need to introduce a couple of bits of terminology terminology and books and articles is all over the app people talk about function objects functors the standard though has very specific terminology so the standard a function object is an object type that is usable like a function and there's three things that fall under this banner one is function pointers now even going all the way back to C 89 thanks to the wisdom of Denis M Ritchie you can invoke a built-in you know a plain function pointer just with parentheses DMR basically looks at that and says okay and C generally I map everything directly to what I'm going to compile what the machines gonna execute but if you apply parentheses to a function pointer what else could you possibly mean but B reference this thing and call it so I can just do that for you this is one of the very few times that C just does something helpfully for you and in CS plus we now have the ability to overload operators so if you have a function cooperator on a class it's called the like a function c s+ 98 used this extensively and lambdas count as function objects of class type also if you have a class that doesn't have a function cooperator but is convertible to a function pointer the core language is sneaky and says oh i can call that too that counts as a function object as well now the standard doesn't consider references to be object types but if you have a reference to a function you can form that you can also call that like a function and there's one place reference wrapper for this matter so I think of them as function objects but they're not technically objects then the standard introduces the terminology of call objects so Calma object is something that's called will in a generalized sense so not just function object that's call but with parentheses but also a pointer to member function or a pointer to member data now you may never seen these before in your life but they exist all the way back and see us plus 98 and they allow you to select a member function or a data member of an object without knowing that the object is ahead of time but the syntax for doing so is very strange and also has the long precedence in some sense because you have to wrap the PMF in parentheses before you invoke it it's awful now the the reason the standard library introduces this terminology is because the core language is different syntax for these pointers to members and function objects now in theory there's no reason why the core language couldn't do a DMR did and say oh if you try to call a PMF with parentheses and give it an object as the first argument what else could you possibly mean but say God star but the core language is not helpful they make you use different syntax for this and that's really awful so the standard library says okay I'm going to compensate for the deficiencies of the core language and I can just invent my own awesome world and this awesome world is called invoke now this was an imaginary function that didn't exist wasn't available for users all the way back in tier 1 in 2005 and cs+ 11 and 14 but there was a proposal that was recently accepted in CS + 17 to make it an actual function that's call by users so what does it vogue do you call invoke and you give it a callable object as the first argument and then a bunch of arguments for the call object if the callable object is a function object then it's just called with parentheses very simple but if it's a pointer to member function or pointer to member data invoke uses template metaprogramming and it detects what it needs to do and it's very intelligent not only can it distinguish between PMS and PM DS but if you give it a reference to an object versus a raw pointer to an object versus a smart pointer to an object invoke will just do the right thing it has five different expressions that it can compile it also is aware of base and derived pointers to member data if you think about this it's really nasty but invoked as does it all and it's extendable in the sense that it knows nothing actually about out smart pointers so you can call it with a unique putter a shared put or if you have your own user-defined smart pointer as long as its D referenceable invoke will work with it so if you have a C computer or some QT smart point or something invoke will totally work with that basically if it's not an object it assumes it's a raw pointer so here's an example of usage where you might actually want to use it in your code now this is not a super helpful function but at least does something I'm here I'm writing a range based algorithm and what I want to do is I want to iterate over every element and I want to transform it according to some callable object and then just print it out I could also insert it into some sequence or something by figure I just printed out so here I write transferred friend to pick sconce range ref it takes a call Bowl traditionally an STL we took all of objects by value I could do it differently and then for every element I just call invoke and I give it the call about object and then the element and by doing this I can now call my transform print function with either a function object like a lambda here IVA lambda that takes a pair and squares its first element or I could pass anything else that fits the definition of a call but object like a partner to member function or pointer to member data technically in the standard library you can't take the address of a standard library member function but you can't take the address of payers data members so I can print out the second element of each pair just by passing a PMD and invoke just does the right thing whereas if I tried to call C with parentheses then the lambda would work but the PMD would fail to compile so the reason why I invoke is necessary is going all the way back to cos plus 11 many things in the standard library use invoke and here's an exhaustive list of everything in the STL that uses invoke it's a bunch of stuff I'll be talking about all the stuff on the Left column today the things on the right and the multi-threading headers also use invoke so once you know oh I can use you on pmf's and PMT's like this you can give them to things like call once or stood thread and because they follow the invoke protocol it'll just work okay so some recommendations I promised I'd be talking about proper usage in addition just what the stuff does now if you don't write generic code then invoke is not super useful because if you know the type of your callable object you also know the syntax needed to invoke it but if you really really hate pmf syntax like you just cannot type paren object's star PM f / n / in args and yeah I have some sympathy for that then you can use a vote you know it will just do the right thing but invoke really shines in generic code where you can just take and potentially store arbitrary call BA objects and arguments that you need to give to it you give it to invoke and you let it decide what to do this is very simple to do and we use it now in the standard libraries implementation don't try to write helper code to detect PMS PM DS and dispatch on them it is an absolute nightmare PMF types are the worst types by far in the core language because they can be sieved qualify they can ruff qualified they can have ellipses you've got base and derived it is nasty so just give them to invoke let your standard library implementer deal with the headaches so that's invoke and even if you aren't using sis + 17 you still need to know how it works because everything else in functional almost everything else uses it okay so result of is actually not technically and functional but we need to talk about it because it's strongly associated with VOC and things that speak invoke and it gained a couple powers in CS plus 14 so result of is the type trait that corresponds to invoke it actually didn't live in the functional header err back into your one but it got moved to type traits which is probably where it does belong in CS + 11 n has remained there since then so the first thing that everybody tries with the result of is they try to ask okay what's the result of this function object you cannot ask this question it is a compiler error and the reason why is because the result of invoking Alba object depends in general on the arguments that you give it for example a function object could have a templated flush call operator especially like a generic lambda it could have an overloaded function call operator so you need to ask what is the result of invoking this call by object with so-and-so arguments and once you provide the arguments then result of is defined as the deckle type of an invoke expression and the standard says it's invoke called with deco Val callable deco val is this imaginary function that lives in the STL it's actually declared and provided to users but it's never defined it doesn't physically exist it exists only for the purposes of deco type to say what if I had an object of this type I've actually had users complain I can't call stood echo van yeah the crisp eyes it's not implemented I love users they keep me employed and entertained and in sales +14 result to have gained a power that if you say call bowl with arms but it's not actually callable the double cone type will spin a away and you can essentially ask is this thing call belit all in CS plus 11 it was a hard requirement that the thing be callable and then you could get its type so here's an example I think really hard to show an example that was somewhat natural and not simply superseded by Otto and deco type Otto so here I've got another sort of transfer function here coding it to work only with vector of T and I'm going to take another arbitrary callable object what I want to do is I want to transform each TLM tht element of my vector through this callable object and I'd like to push it back into another vector then I would like to sort the transformed elements and return that so I need to be able to ask what is the type of calling the kaabah object on my t's so I can make a vector of that type sort it and then return it and just to avoid repeating code I'm going to use cs plus 14 auto return but I could span out the vector to KT blah blah blah so the way that I call it is I have a vector string I have a lambda that's just going to take a string and return it sighs I'm here to sort of avoids just plus 14 overload I've made it just take a hard-coded constant graph but I could say Auto that Constanta ref and then I get arrayed through the transform vector and print it out so the crucial bit I've highlighted here what is the element type of ret I need to ask what's the result of invoking callable ref on consti ref but that Calma object might return say a Const int ref I can't have a vector of references and I can't have a vector of constants either so I need to use stood decay T which is a type trait that will strip off references and top-level CD qualifier so I can get a vector of the non-reference II the unqualified type of whatever returns so this works and it's perfectly generic but you'll notice I had to say call Bowl ref and consti ref there result-type is actually surprisingly tricky because it answers you what's the type of this invoke expression but it uses radically different syntax from the actual call if you look back at the example here I said invoke lowercase C comma T but the syntax for result of was totally different and you have to verify that they correspond and that's actually surprisingly tricky because the Seebeck qualifiers and the value categories that you give to invoke expression value categories as l value Nisour are valueless they matter functions could be overloaded to do different things based on the constants or the value category and in CS plus 11 you can have ref qualified member functions so functors function objects can be sensitive to whether their l values are values and if you get it wrong it may appear to work except if somebody calls you with a sufficiently weird argument it would fail to compile and also if you are not just using invoke directly but something like bind then bind manipulates its arguments before calling invoke stat acing does decay result of will not do any of that so you would need to do such transformations by hand I've seen people try to use a result of when they're actually giving something to bind and that in general is incorrect I can give you a placeholder and now your result of is incorrect result of it dates all the way back to TR 1 which was a library only addition to CS plus 98 or 3 it was extremely useful then before we had deco type and all this L value R value stuff but it really has not kept up with the times so what I recommend is avoid using result of if you can use deco type directly there are some cases where you do you don't never need to use result of because you could always write the deco type behan sometimes results of could be convenient but just be careful if you use it and if you have existing usage in your code base go back home go look it up in your source control and I bet you you're gonna find bugs look carefully at the value categories that you're giving for example in this code if I simply said result of call Parenti that would be incorrect because it would be asking what is the result of invoking an r-value callable on an r-value t and that could be different from the actual invocation which is invoking L values so you may find bugs in your code I've seen too many to count I recommend just using deco type directly or auto and if you're a generic programmer you already need to know that stuff so it's not an extra burden of understanding the general principle here is if you need to compute information don't do it through different mechanisms and if you need to repeatedly compute it and sometimes that's necessary try to just exactly repeat text because at least you can verify that by looking at the source code if you need to do some sort of mental transformation to verify this one thing corresponds to this other thing you're very likely to get wrong and even if you get a right it's going to be more work than necessary okay so that's result of now let's talk about mem Finn which was added in CSS 11 replacing some older stuff that I'll talk about later in CS plus 98 so here is usage I didn't even bother to actually vote count medals it's pretty simple I have a structure class it's got a member function that returns a bool and then I want to use an sto algorithm and have it call that member function so I'd like to call count if on every element of a vector and ask hey are you metallic now if I just tried to pass a pointer to member function or PMF we would fail to compile and the compiler error would say something like I can't call this pmf with parentheses because the core language hates you okay the answer is you can just cast it to infant and Men fun is a function in the standard library it takes a PMF and wraps it in a function object because stl algorithms expect function objects not arbitrary callable objects and then this allows it to be invoked with parentheses so this is actually very convenient but it actually breaks down if you look at it real closely so the good thing about Memphis is that it's very terse it's usually not possible to write less code than just Memphis address class double call block but it has some downsides a hidden downside is that it's actually going to make your code slower the time because it needs to store that pointer de member function as a data member and optimizers have difficulty seeing that a date a member of our class was initialized to some constant and then used later at least I talked to VC's back and dads and they said wow that's a really pretty fundamental limitation in our optimizer we might fix that but don't expect it you know anytime soon I don't know what the other compilers do but certainly if you write such code it won't be portably optimized and in any event you're gonna make the optimizer work harder than it has to also in general this will not compile if the code is weird enough if the member function becomes overloaded then simply saying address of the name of the member function is now ambiguous and you need to static cast to disambiguate which one you want same thing if the member function is templated I talked in an earlier talk you don't want to use explicitly arguments there you need to use that static cast to be really general and it's just awful to type that thing and if you have default arguments the signature of the thing isn't actually corresponding to what's going to be invoked so it breaks down as soon as the code is extended and I've also observed people doing things like calling mem Fionn only to give it to bind but behind already speaks invoke so you don't need to ramp pmf's when you give things to bind or stood function so something is in this CS plus 11 functional stuff you don't need mem fin at all you only need it or something equivalent when you're talking to something like SPL algorithms from 98 that expect function objects with parentheses so my recommendation is to avoid Memphis now it's not super horrible you can use it with a pretty clear conscience but the thing is if you're giving a member function to an s2 algorithm you probably care about the performance because it's going to call that for every element of your sequence and it's going to be maybe a million elements or knows that performance may matter and the fact that it just breaks down when code evolves that's pretty inconvenient so now that we have CSS 11 lambdas and 14 generic lambdas and so forth what I recommend is writing a lambda to call that member function for you yeah it'll be a little bit more typing but it's going to optimize away because the body of the lambda is just an ordinary function call operator and the optimizer can hopefully see through that in inline that without having to figure out that some data member is always some particular pmf and it continues to compile because it's just an ordinary function call okay so another feature those add Macias plus 14 are is the transparent operator func groups they're not actually named so in the standard that's just how I think of them here's a quick example imagine I had a vector in filled with stuff a vector string and if I want to sort them in reverse order stood sort has always defaulted to stood less that sorts things in ascending order where every element is less than the next element if it's different if I want a reversal I need to pass greater so in suza plus 98 and 11 I would have had to say greater int and greater string it's just plus 14 I can simply omit the type and just say greater empty diamonds I need to say the diamonds because it's a template and I can't admit it I really wish I could but you can just omit the type and you can just say sort by great earnest there's no reason to repeat the type because the compiler already knows hey this is an iterator into a vector in its and then I can just print them out so I've talked about this at CP Con 2014 and growing native 2013 so you can look at those talks they and slides they go into extensive detail about the transport operator functors and they enable a cool trick in the containers call in the map and set family called heterogeneous associative lookup so I won't spend more time on that my recommendation is to use the transparent diamond operator functors by default there's no reason to repeat that type the only case you would ever want to use the old style grader T and so forth is if you want implicit conversions to that particular T before calling the operator and that's extremely rare it's usually undesirable because if you don't perform those conversions you're going to make yourself immune which is good to things like truncation bugs my usual example is what if I'm sorting a vector of you in 30 toons and I pass greater u and 32 and then later the vector is changed to bu and 64 if you don't change your comparator then every time you invoke it you're going to get a conversion which truncates from 64-bit to 32-bit it will happily compile in a limited truncation warning maybe but that's bad shouldn't compile in the first place or it should do the right thing and the greater diamond does the right thing it also avoids efficiency problems like temporaries and copies now you might be asking okay what's the difference between mem phone which I'm saying is not so cool and the transparent operator functors which I'm saying are cool the difference is that operators in some sense are all known in advance there's a finite list you know less than greater than plus multiplies and the library can provide perfect definitions that are templated to accept arbitrary stuff they return arbitrary stuff they don't store any data members they're not vulnerable to all the problems of Memphis which has to work with a pointer to member function or data that's not known in advance that's why these operator functors are good to use and mem phone is not so good to use so now let's talk about bind bind was added into your Juan and soos Plus 11 and it's remained it has not really enhanced in power since then there have been some proposals to extend by and I'll go on about what I think about that here's an example of using bind and bind is actually pretty terse when you look at it but you got to learn what it does so here I've got a bunch of intz and I want to count how many are less than 50 so I can say bind less in here I can just say less diamond but I could also say less end and I need to say is underscore 1 less than 50 and underscore 1 is a placeholder and what this mind expression does is it makes a function object that binds less and leaves the first parameter unbound and that will actually be provided by each element to the sequence but it binds the second one to always be 50 so this function object is going to ask is 1 less than 50 is 4 less than 50 is 9 less than 50 and so forth if I reverse the order of the parameters I'm going to ask is 50 less than the first element is 50 less than the second element and so forth and I'll get different answers because there's different numbers that are less than 50 or essentially greater than 50 here so that's what bind us so I tried to fit and I succeeded all of what bind does onto a single slide but it's a pretty dense slide because binding does a ton of stuff and if you've not implemented bind it can be sort of overwhelming especially look if you look at the standard eaves so for completeness I did want to mention what bind does you give bind a callable object and then you immediately give it arguments that will be bound to it like here I'm gonna bind underscore one and 50 but I could also fully bind you I could say buying 49 comma 50 now we get a function object that takes no arguments and always calls less with 49 50 not super useful there but elsewhere it could be useful to bind all arguments at a time later you call B with any additional arguments those are called the unbound arguments so binder needs to store this stuff somewhere and the way it's specified to work in CS plus 11 and Beyond is that a copy removes them depending on their value category and stores them as data members then when it calls invoke either conceptually in CS plus 1114 or physically in Sisco 17 they need to be passed as L values and this is surprising so I've put it in bold there passed as L values because once you have a function object like B you can call it multiple times and if they were passed as our values then they would be moved from after the first call and then you'd be all why is my screen getting emptied out why is my pointer you know a smart pointer getting rolled out that would be bad so they're passed as L values also with these constants that implies that the function co-operators overloaded so some bound arguments are special like I've mentioned underscore one means fill me in with an unbound argument those unlike the bound arguments which are copied or moved and then passed as L values the unbound arguments are perfectly forwarded at the point of each call if you pass a reference wrapper it got some rap we'll talk about that later and if you have a nested binding which is I would say by far the most insanity inducing thing in all of the functional header those are called with perfectly forged and bound arguments and bind also has this weird little thing that's kind of unusual you can pass more unbound arguments that are then are necessary any ones that aren't used whether they're at the beginning middle or the end they're just dropped on the floor and completely ignored and there are some arguments for why that's good it's kind of weird though that the STL is just willing to ignore unbound arguments but that's how bind works so bian has many many issues I've seen lots of people use bind because it seems to be terse and it was good back in like 2005 but it has surprising gotchas so first it's got all the performance issues that lymph Indust ORS the call by object isn't it as a data member so if you have a pointer to member function order to member data or a function pointer which bind works with it needs to store that as a data member it suits can be resistant to optimization it's going to be fragile if the thing is templated or overloaded you'll need a disambiguate which is awful so same issues but then also if you misuse bind either by giving it the wrong stuff or by making invalid invoke call it will emit an awful compiler error now many people they're like oh I use the SQL and I get off a compiler errors and they're basically wimps I'm gonna say they're not really that bad you know yeah Pyrus can be confusing but you learn how to deal with them you know get over it yeah yeah concepts will make things better but they're not that awful but with bind bind errors are awful I'm a standard library maintainer and I just cannot bear to look at a bind error even if I wrote bind it's like Oh awful you'll just throw up your hands and give up and that's because it has so much library machinery to mess with the placeholders and the reference wrapper and the unbound arguments and okay so also when you call bind the syntax that you give to it is not ordinary cs+ you need to learn this sort of bind mini language you need to learn it how to write it how to read it and that's bizarre especially with nested bind you're basically delaying invitations whereas if you write a lambda function its body once you get past the introducer you know the square brackets it's just ordinary sickest but you just call ordinary member functions and you know how that stuff works well so that whole bit about bind passing things as L values is deeply surprising I've seen multiple people confused by the fact that they're passed as L values this means that for example if you bind a unique put ur stuff won't compile because it's being passed as an L value and unique footer is not copyable I've seen I guess I can mention some names just to reinforce here I've seen both herb Sutter and Scott Myers confused by bind and these are very very smart people but bind is even more evil than they're able to handle I was confused by bind while maintaining bind I thought oh I can just pass them as values and that will make it efficient and no stuff get stolen from that broke windows and I had to revert the change that is a very bad I have a test case for that it's called grand theft bind because buying Bastille entrants bad arguments very very surprising the only reason I knew about this issue is because I ran into it myself there bind is not obvious on when it calls things immediately versus when things are called later this has impact if you're calling something like a time function that's sensitive when it's called or if it's looking at some sort of global State that's changing whereas with a lambda function it's very clear what happens when the lambda is constructed that's all the stuff in the init captures and when the lambdas call because everything in the function body doesn't happen until you actually call the thing and then actually I've never seen anybody run into this but it certainly nags at me if you right mind expression and it's sufficiently bizarre you can actually move from arguments twice if you repeat placeholders or if you have enough nested bindings I find that extremely dangerous in theory because the STL generally is not willing to move through things twice unless you go out of your way to do something so dangerous the bind will happily do it because it was invented before our value references and then it was our very offenses were added on later hmm so very dangerous so what I recommend is and this is stronger than them fun avoid using bind it's just not worth it use the lambdas it may be tempting to use behind you because viable sometimes be shorter than a lambda but the verbosity lambda is worth the price it's the terseness of bind is not worth the price of it usually in the STL we like library solutions because they can provide a more abstract solution in the core language but bind is essentially a counter example we tried to make building up function objects in the standard library all the way back in boost and tr1 and it just doesn't work well and that's why lambdas were proposed and added to the core language and they vic they've been extended since then so don't use the STL in this case use the core language okay so I mentioned reference wrapper this actually lives in the functional header it doesn't always need to walk to the functions though but it has one useful power which is why it lives in functional so here it's such a small class I've actually put the entire class definition up on the slide reference wrapper Tootie is in some sense it behaves like a reference to tea except it's assignable so you can reseed it the other way you can think about it is it's an auto dereferencing pointer so it's got an operator T breath so it can convert itself to a reference or you can explicitly ask for get an reference wrapper also has this extra little bit it has a function call operator that takes arbitrary stuff and then invokes it invokes the T on whatever is given to it so a reference wrapper to a function object is callable like that function object you can also have a reference wrapper to a PMF of PMD but that's very obscure so here I've got an example of usage man I've got vector int and I would like to fill it in with randomly generated numbers from 1 to 20 so I can define D 20 which is a function object that every time you call it it will generate a random number and it contains a random number engine a uniform random number generator and a distribution so the function object is stateful and every time you call it it's going to modify its state and then return a random number from 1 through 20 so if I use the CS plus 98 algorithm generate and I call it with D 20 it'll seem to work but then if I call again I'm going to get the exact same sequence of numbers generated and that's because to generate takes its function object by value and you get a copy so when I copy D 20 the new object the the copy is invoked and modified and then it's discarded and then if I make a fresh copy I'm using the original state that's why I get the same sequence of numbers if I use stood ref which is a helper function that makes reference wrappers then I will directly invoke the original object D 20 that I've got here and I'll get a different sequence of numbers every time I call it because I'm you only using that original object State so it's more efficient and I actually does what I want which is usually good so because CS plus 98 oh three algorithms almost all of them the only exception is student shuffle that I can think of take function objects by value and are allowed to copy them and for each is also special but most of them can copy objects if you don't want to copy you can and should you stood ref to ask the STL Oh that's the sing by reference just put it in a reference wrapper and then invoke it later some people and I get bug reports they try to use explicit template arguments to make an SQL algorithm take a function object by reference and that'll work on some implementations but not others and it's not portable ESCA algorithms really are allowed to copy function objects so you should use reference wrapper in this case the standard even has a note saying use it it's also useful elsewhere even if you're not working function objects like stood thread but you should be aware of why stood thread wants to copy things before telling it no don't copy things also reference wrapper is unwrapped by a few functions in the STL I make pair and make tuple notably are aware of reference wrappers so if I say make a pair of make tuple and if I give it reference wrappers it will return a pair tuple with built-in references and I've got an example here so if you've ever wanted a pair of truthful references you can now get that easily with reference wrap okay so I'd mentioned earlier that medicine had superseded something this was stuff that was available in sis plus ninety 803 and in CS plus seventeen they have been removed outright this was at a very recent meeting they were deprecated all the way back in CS plus eleven so compilers could have been warning hater deprecated and in the standards u-17 they have been removed outright so the things that were removed and functional were the classes unary function binary function these were just helpers that would provide type deaths result type argument type first the second argument type there is a function put her phone that would wrap a function pointer and provide the type desc then the more things mem fun with you mem fun ref don't ask buying first buying a second and these have been superseded completely by the TR one STS plus eleven mem phone and bind so because these things have been officially removed from the standard and at least my implementation they're going to be removed at some point you should never ever use this stuff your existing usage you should start transitioning away from it they have been deprecated and removed for a reason they're not general they're hard to use they've been superseded things like that in fact I've seen lots of people using them when they weren't actually necessary they were needed these type deaths like result type argument type they were needed for adaptors like buying first and not one things like that but if you're just giving function objects to sto algorithms or STL containers like stood map they have never needed any of this unary function binary function stuff so if you had function objects deriving from unary function or if you were calling things with put her fun you probably didn't need to do so and you could eliminate it and make your compile times faster and your code simpler and all that now I wanted to break the world but I couldn't quite do it in one release so in vs 2015 we still provide all this stuff by default but you can define a macro project-wide and this will simply pre-process away all the stuff that has been removed so if you would like to join the New World Order of not having all this removed stuff you just define has Auto put or etc to zero this will also turn off the definitions of auto put ER and a random shuffle which were removed at the same time Otto Porter is extremely dangerous don't use it and but this talk is not about how to put it ok so finally let's talk about stirred function save the best for last this was a danseuse 11 and if has gained extra powers in CS + 14 + CS + 17 so here's an example using stood function stirred function can do lots of things here in the example is recording a sequence of actions in a vector and then playing them back later so here I have a vector of functions and these are going to be cobbled with two intz and then they're going to return another int so here I can store things like a plus function object a multiplies function object a pointer to function here I've said address some squares I could have just admitted the ampersand in word worked as well I can also store things like lambdas here I've got stateful lambdas that capture the integer I and then go use it later and then once I've built up this vector functions I can iterate through it and call it with arguments in this case I'm always going to call them with 4 and 5 but I could call it I could iterate over repeatedly or do whatever I want I can also do things like store astute function has a data member and then use it later ok so stem function you should think of it as a wrapper it doesn't do anything by itself what it does is stores a callable object of some arbitrary type now the type of the call object it could be a lambda or it could be a function object that you've written by hand or from the STL it could be a PMF PMD the type of it doesn't matter only the signature matters the stir function is templated on the call signature I take arms and I return ret and it erases the actual types the call object that's why I can have a vector of such things here in the example stood plus and a pointer to function and a lambda they're all different types I cannot have a vector of different types a B and C they all need to be the same type and that's what stood function does it erases the type of the Kalb object and like a Cheshire cat it preserves only the smile only the call signature so this is useful when you can't have code that's templated on the type of a callable object lots of different contexts it could be code that needs to be separately compiled because templates going headers in general if you need to separately compile something stood function can be put in that signature if something needs to be a virtual function for technical reasons you can I have a virtual template so if you have a virtual function you can make it take a study function of a specific signature and then it can be called with any callable object of whatever type and like I showed you container owns all need to be the same type stood function can help you have containers twisted functions so very surprising gotcha I would say that stood function requires copy constructible function objects and this is kind of unusual in the STL usually the STL is lazy in the sense that it doesn't need things upfront if I have something like a list and the way I think about it this is powered by a core language rule find something like a list of T T does not need to be less than comparable it can be that's fine but in general it does not need to be less than comparable only if you call the member function list T sort then it actually does need to be less than comparable if you call by default and the core language rule that powers this is that the definitions of member functions of a class template are not instantiated until they're actually need and the bodies don't exist in some sense until you actually call it and this is usually cool this means you only pay for what you need but state function is special because this type of raishin because when you construct a stood functions from some colville object f it needs to generate everything could ever need from that object F because it's going to erase its type it requires all the operations that it could possibly ever need regardless of if they're used so if you construct a stood function from some call by object F F is absolutely required at compile time to be copy constructible this is true even though F will be moved into the student function so even if you give it an R value and even if you never copy stirr'd functions anymore in your program F is still required to be copy constructible you'll get a compile error saying so maybe horrible maybe nice depending where you get it just cannot store moveable only function objects this is a design limitation caused in part by the fact that stood function dates back to boost MTR one before our very references and in some sense it can never be fixed with good functions interface as it stands alternatives are being investigated maybe we can cram something else in the pipe or have a different moveable function so we will probably get some sort of type erased wrapper that can store movable only function in the future but stood function as it stands at cos plus 17 right now cannot do that so just be aware steady function has got this trick called the small functor optimization so think about what it does to the function of a given signature has to be some specific size let's say it's you know 50 bytes or whatever but the Calma object that i give to it could be arbitrarily enlarged I could store gigantic Sturt array and my column object or whatever resilient pointers so how are we going to cram that into the stood function well the answer is eventually we're going to use dynamic memory allocation at some point we got to say operator new allocates a memory and then have a pointer at that memory but allocating memory is in general somewhat expensive so student has a trick and this trick is permitted by the standard actually we hired in one case if you're call my object is small enough it can be stored within the stood function object itself without a dynamic memory allocation it can simply be placement nude constructed within a properly aligned buffer and we store a pointer to it and we don't allocate memory in any circumstances that's good the standard guarantees sort indirectly by saying shall not throw exceptions that this happens if a stood function store is a built-in function pointer or if it stores a reference wrapper if you have a call the object of any other type even a pointer in a number function you're not guaranteed to get the small functor optimization it's up to the implementers judgment as to whether they activate it and as we'll see on the next slide implementations do vary in what they accept there's also one additional bit that is not outright specified by the standard but implied by its requirements and that's if you want to activate the small functor optimization your call by object must be no throw move constructible and the reason why this is implied is that swapping stood functions is required to be no except think about what this implies if you have locally stored function objects I need to move the first stood functions call by object into some buffer somewhere move the other one and then move the first one in and if those can throw then the swap would be able to throw exceptions that would be bad so your callable object must be no throw move constructible to activate the small functor optimization here I've got a slide I can observe through a test case what activates the small functor optimization so V is 2015 on x86 dude function is 40 bytes big and it can store a call by object that's no throw move constructible up to and including 32 bytes anything bigger will cause dynamic memory allocation and on x64 the step function is bigger at 64 bytes and it can store up to 48 now we chose these numbers in vs 2015 using the heuristic I should be able to store a call by object like a lambda that's bound one stood string that seems reasonable so I'm going to make the small func proposition at least as big as Astrid strength and I've put stood string size of Thera for comparison I've also put lip seus pluses and lips tootsies busses values there I observe that lips use plus also can store stood strings in its small functor optimization but lib stood cos plus cannot and just due to the constants that they've chosen that heuristic was suggested to me by James McNellis our CRT maintainer I asked hey you know how big should the small functor optimization be and he said a astute strength I was like that sounds good I'll do that in 2013 it was very small yeah where we chose it through a highly rational you know data science we get lots of no no we just make up an idea and go okay so stem function I mentioned it learned a couple tricks in CS plus 14 you know or one trick ensues plus 14 it's 11 this code was ambiguous if I overload me out taking a stood function taking one end and returning an end versus taking two ends if I call it with lambdas with different signatures in CS plus 11 this was ambiguous and were failed compile C is 14 can disambiguate and that's because did functions constructor is now constrained to exist only if the call object is callable with that signature there's a bid in the standard that says that but basically it means that this code compiles which is kind of what you would expect but in Lebanon did not and see just the 17 just voted in it also learned another trick if I have a student function that returns voyage specifically I can now store a function object here I've got a lambda that returns an int I can now store function object that returns something and that will be discarded essentially if I can always call a function to ignore its return value stirred function is now capable of this great the reason why this didn't work consists was 14 is it said stood function took the return type of the stored call object and implicitly converted it to the specified return type of the stood function turns out according to the core language you cannot implicitly convert something to void that's just how it works but like I mentioned the standard library can make up its own world so they voted in a rule say hey we can convert this thing to void why not package tasks also learn this trick and bind arms I won't going to now stem function is complicated enough that there was actually a couple issues with it that are issues in the standard that have not yet been resolved and I want to mention them for completeness first as the committee has discovered to its horror Stud function is not super awesome when it comes to multi-threading in a specific sense stood function this function call operator is Const because if you call us today to don't stay it can just call the function objects function cooperator the problem is the way it's specified sted functions constant function cooperator call a non constructs Co operator on the store called object even though it usually behaves like the value type and the reason this compiles is because it really stores a pointer to the call of object even if it lives within itself turns out this violates in some philosophical sense the STL's multi-threading guarantees when it comes to const usually the stl says if the user obeys these principles I will obey them too and constant means thread-safe stood function subverts that that's kind of bad the committee is looking into ways to fix that another thing that we have discovered is that because stood function wraps call by object it essentially exposes the deficiency in the core language if I have a called object that returns something by value like now returns a stood string I can wrap that instant function whose signature returns constrain graph that constrain ref that the stood function returns will be bound to the stood string that now returns by value but that evaporates as soon as the stood functions function operator returns so you immediately get a reference to a dangling temporary at least compilers will often warn but this code which is required to compile by the standard should not compile I think I can fix this in the standard I need a ripe a library issue but it's not trivial in this case is trivial because the type is the same but in general reference binding is really wacky I need to figure out exactly the core language standard needs to hook into so my recommendations for students it is super awesome the type of ratio trip that it does and that small functor optimization and all the other stuff it does it's really really great and super nasty to write by hand so definitely used to function but only use it when you need it I've seen people go crazy with good function when they don't if you're writing something like an algorithm that accepts some arbitrary thing it should probably be a template and with invoke now you can write such a template that accepts arbitrary callable objects your you'll be templated on the call by object type you won't be storing it anywhere you'll just directly invoke it things will be great or if at local scope your storing something like a lambda or maybe a call to bind you should use Otto to make the type of the function object the same as the type of what it's initializing you don't need stood function from that you only need it when you need that type of race your trip or something like I need to store stood function and then maybe later assign it to be something else that's a different type stood function is useful there it has nonzero costs and people often don't appreciate this usually the stl is like zero overhead essentially we're negative overhead Stern function though because of the fact that it needs to do type erasure it prevents inlining because it's going through the moral equivalent of a virtual function call it also has space costs we're going to have that small functor optimization buffer regardless of whether your function object is smaller than it or if it's too big it needs to be dynamically allocated you pay that space cost regardless of whether you use all of it and also the type of racial trick needs a couple pointers and you're paying that also template code blows template code bloat I'm going to say it's a myth I have almost never seen it matter in practice templates instantiate just what you have written by hand the one time I've seen template code bloat be an issue in practice is when somebody went crazy with stood function and used it all over the place one of the reasons why this is an issues could stood function emits code like stuff needed to copy and move the and swap the underlying function object regardless of whether you actually call it so that's a cost it makes your binder is bigger so if you do your stood function take into account all that I've said you should still use it efficiently it's a somewhat heavyweight objects you don't want to copy it unnecessarily you also don't want to move it unnecessarily and you don't want to make temporaries for no reason only construct things when you really need them it's kind of like a stood string but a little more expensive so more info I promised at the beginning all this stuff has been implement with couple exceptions first because of es2015 does not support expressions feigning in the compiler we have not implemented the result of and functions Fein a and I have refused to use library only tricks to do it in the absence of compiler support at some point in the future we will get that in library but currently we're following the 11 rules that say that result of his double chrome type does not Sweeney away and stood function does not have the disambiguation trick also while I rewrote functional to purge all the bugs in there the multi-threading headers use the stuff slightly incorrectly async uses bind it really shouldn't package tasks uses to function and hence requires copy constructibility it really shouldn't and also i tried to be sneaky and mem fun to handle the non-standard calling conventions I thought surely nobody will notice that my template arguments don't exactly match what's in the standard inevitably somebody notice to file the bug I fixed that already for update one and that will ship so I'll follow the standard strictly and I've included a link there to the c sp70 merkel paper so okay looks like we've got some minutes left let's take some questions I do we've microphone setup no microphones okay so I will just go left to right question there okay yes I would say there are there may be minor differences between a boost in the standard library but in general the booster interface especially if you're compiling it in CS plus 11 mode with a new compiler should behave very much like this I believe in a couple cases boost is added extensions that the standard library does not have I need to go look it up but the vast majority of stuff will port over now boost is probably not learned the very recent tricks like the conversion to actually I'd have had the conversion avoid bit I would have to go look that up but like the result of speen a I'm sure Eric me blurr has checked that into booze so almost everything should work with boost other questions yeah there okay yep slide 18 right okay there to detect uh something's called yeah yeah the question is with the library fundamentals void the detection idiom void T sort of thing I should forty was a danseuse 217 is a result of more useful that you can detect whether something is cobble and I would say maybe you could still write the Deco type by hand I mean that ultimately it is just a syntactic convenience for this deco type and VOC deck Laval thing but if you know the protocol that it uses then feel free to use result of just be aware that the value category like the L value reference or r-value reference that you put in there and the CD qualifiers manner but if you're at that level of a generic code then result of is not evil it's just requires skill to use properly like a very dangerous chainsaw and you said you had another question okay let's take other question I'll come back to you any other questions yeah over there right changing the behavior yeah right yes it's just it's does the question is what about movable only function objects the answer is it's a design limitation the standard library has no type erased wrapper for a move only function objects you can write your own you can write wrappers that sort of high jacket in the students like you could write something that is copyable but if you try to copy constructor it throws stood function will happily throw that happily store that and don't throw if you don't copy it but it's subverting the type system a little like I said something we'll probably propose from stare in the future but it has no support for that right now just like it has no support for say parsing XML or something it may show up in a library fundamentals chess first I don't think it would make seventeen at this point there's still a couple years out but there's no concrete proposal yet as does the committee has been talking about what would it look like to have you move baloney function wrapper other questions before I come back yes ask away yes we're still using dink more standard library I just take in PG up harbors functional header which I had been maintaining over the years and I ripped out almost everything except for the skeleton stood function and rewrote it all the process about four months and then shipped the changes back and at the old implementation it was extremely good back in the chair one ERA I could not have written it before be a depth type before forwarding and before very attic templates but as things evolved in CS plus eleven and fourteen the code was extremely difficult to maintain and extend as new features were voted in so that's why I rewrote it from scratch using an action a student vote that had perfect forwarding deco type and all that and purged all the bugs right yes please submit a bug yes absolutely you can also file bugs through Kinect or send a smile but talking to me is fine too yeah another question there yep the question is how to write things that take lambdas if you're writing the algorithm you could follow the stos convention of taking function objects by value the reason why did that 98 is so you could accept things that had either Const rely on cosmic sha function co-operators but you could also write it to take callable ref ref this would be like stood shuffle and that takes the thing in place and then invokes it and if you want to be real cool you would say forward callable to respect the original value category I'm going for it I'd say that's probably the right thing to do if you don't need to copy the thing game in it yeah if you call it then you would obviously not say stood forward but then if you took callable ref ref you'd be saying I accept an arbitrary thing and then I'm going to call it multiple times that would be acceptable I would say that maybe the new convention going forward it's usually not a good idea to just randomly violate standard library conventions but in this case it was due to CS + O 3 limitation it looks like our time is up for a 10 so I'll be around all week please come to me and ask any other questions you have thanks for coming

Channel: CppCon

Views: 32,877

Keywords: Stephan T. Lavavej, CppCon 2015, Computer Science (Field), Bash Films, Conference Video Recording, Event Video Recording, Video Conferencing, Video Services

Id: zt7ThwVfap0

Length: 60min 33sec (3633 seconds)

Published: Sun Oct 11 2015

Reddit Comments

[u/Rhomboid]

Great talk, as usual for Stephan. I always learn something from these.

[u/AntiProtonBoy]

Good stuff as usual, STL.

Regarding avoiding std::bind , I'm using it for packing callable types and their arguments into a std::packaged_task, as shown below. This helper function quite useful for constructing and asynchronously dispatching a bunch of tasks on a queue.

Is there a better way of doing this, without std::bind?

  template<typename CallableType, typename... ArgumentTypes>
  auto MakePackagedTask(CallableType&& Callable, ArgumentTypes&&... Arguments)
     {
     using CallableDecay = std::decay_t<CallableType>;
     using ReturnType    = std::result_of_t<CallableDecay(ArgumentTypes...)>;

     return std::packaged_task<ReturnType(void)>
        (
        std::bind
           (
           std::forward<CallableType>(Callable),
           std::forward<ArgumentTypes>(Arguments)...
           )
        );
     }

[u/redditsoaddicting]

Another great talk! Kudos to /u/STL! After watching, I have a couple of specific questions. I'm a language lawyery type of person, so I like the small details that casually get mentioned and thrown away in the talk. Thank you for mentioning them so I can ask.

1. 10:00
   Why can't you take the address of a standard library member function?

2. 11:47
   What exactly are the special cases for PMFs that involve base/derived?

[u/bames53]

At 51:52 STL mentions template code bloat. For anyone that's never seen an example, here's one.

Patches fixing template code bloat in lld:

• rL234808
• rL234931

Although in this case I think the bloat didn't impact the final binary size, the author describes the motivation as two fold: large object files with a lot of duplicate template instantiations was slowing down linking, and on Windows the excessive instantiations were breaking some limits on the regular (non-extended) COFF file format.

This isn't an argument against using templates. Templates are of course awesome. I just thought that anyone who hasn't seen template code bloat before (most people) might find a real world example educational.

[u/suspiciously_calm]

So, is that "uniform call syntax" proposal out the window in favor of std::invoke?

[u/[deleted]]

[deleted]

What is this?