C++Now 2019: David Sankel “C++: Engineers Wanted, Programmers not so Much”

alright shall we begin let's do it my name is David's ankle I work at Bloomberg I'm the the TL for micro service architecture that's enough about me you know what the talk is about I'm just curious how many of you feel like you know the difference between a programmer and an engineer raise your hand okay cool how many of you would consider yourselves predominantly a programmer or not an engineer okay like a few of you cool so this is a philosophical talk and I don't I've never really done a philosophical talk before I hope that I can do it justice there's there's an important insight that comes later on but we'll give it a try but just to give you an idea of like questions that we want to be able to answer so why do so many successful projects have such bad code my question comes up frequently why is agile by the book so rare what makes for a good API why is C++ still so popular for 30 years why hasn't functional programming taken off why does so many programmers avoid the standard template library why do so many programmers do non programming tasks all this kind of stuff so in order to be able to answer these questions and to kind of like build up this philosophy we're going to talk about things that we don't normally talk about in in this kind of conference so let's talk about alcohol what so let's talk about these two guys in alcohol so this guy is Antonio Rodriguez he's standing next to Joaquin Solano and these guys are very very much tied to alcohol and it's not because they drink it although who doesn't like to drink alcohol well many people but but anyway these are chemical engineers and this right here is a su carrera Montero so this is a a plant that's on the southern coast of Spain and what it does is it makes alcohol so they generally produce a high grade ethanol so ethanol like 99 percent ethanol so you use this kind of ethanol for I don't know all kinds of stuff you can add it into wine if your wine is not very good quality and you need to get add some more alcohol to it you can use it for fuel there's all kinds of things that you can do with it and one of the byproducts of this creation of alcohol process is this thing called Argan or diente it has an English word I don't know what it is but this aguardiente is what's used in creation of rum so that's what you saw on a prior slide there so why are we talking about alcohol why are we talking about chemical engineering so the idea is to see if we can gain some insights from these other engineering fields which are way more established have lived for a lot longer than software engineering so think of this plant as their codebase now one of the core things that a chemical engineer needs to be able to do is to design a column and what you see here in this picture is a column and the way that a column works when you're doing chemical engineering usually insert your base material in the bottom of the column there are a bunch of processes that happen as it goes up and what you can take out at the very top is your high grade ethanol and these can be several stories high you can see the stairs going up on this one they're pretty cool so designing of a column is like our equivalent of writing a computer program from scratch greenfield development and the question is what is this what Antonio spends his time doing designing columns what do you think nope sometimes right occasionally he needs to design a column and if you're doing software engineering occasionally you need to write a program write a significant program Greenfield development but that's actually not what he spends most of his time doing most of his time he's responsible for this plant they have to do renovations they constantly need to add things they need to deal with issues that come up they need to prioritize all this kind of stuff that's what they do now if you ask Antonio like what do you do his response is going to be something about how many thousands of liters of alcohol this plant produces right that is the core focus what do you do creates alcohol on a daily basis now if we go when we ask Jane programmer what do you do what is she gonna say well maybe something like this I write C++ and Python I've experienced with qt and I think boost is amazing right very much concentrated on the programming aspects now if we go to Elena software engineer and we say what do you do she might respond something like this I'm responsible for a message router that handles on average a billion requests per second under continuous load and is the backbone of Bloomberg's financial engine so do you see the the difference in the focus there all right now let's go back to this plan if you have a new chemical engineer and they see this plant they're gonna think wow this is this is amazing this is like fantastic I get to be responsible for that plant which is producing this many thousands of liters of alcohol per day like what an exciting prospect now if you take the equivalent of a programmer and and they get faced with this in the equivalent of a codebase what do you think the response is going to be too much metal it's gonna be like ah ah man this code sucks so I think that it's really interesting that we have this phenomenon in and software engineering here's another picture anybody recognize this yeah so this is this is a Falcon 9 launch right SpaceX they they launch things about it now one thing is really noticeable in this picture do you notice something kind of strange about it it's not painted right this this piece in the middle that reused first stage engine it's ugly right it's used and and when space s SpaceX did this launch Elon Musk had to respond to this to all these comments about how this ugly first stage looks right and he gave this response I thought it was absolutely fantastic obviously aesthetics are a minor factor in rocket design right I think this applies to software development as well we tend to concentrate on aesthetics you look at a piece of code and if it's not indented in your favorite style you think it looks ugly right this is the focus of a programmer but the focus of an engineer is a very different focus and I would argue the aesthetics are a minor factor and software engineering or at least they should be so you take something like this in an antonio's plant here and this is I'm not really sure what it is but there's a bunch of dials here probably measuring pressure of these various tubes that are transporting material between places now one thing that you might notice is that the paint on one of these tubes here is a little bit chipped off right just over time after it got painted it doesn't look too good so if a programmer approaches a code base and sees something ugly like maybe Constance should really go here or maybe this for loop would be better placed as a range base for loop programmer will typically like jump right into that and try to like fix that right away but a software engineer is gonna see these things in the context of their business value how important is this to the business because if they spend time painting that little thing that's time that could be spent doing something else it could be potentially adding business value to the products so so that's what that's about if we look at Antonio's ro role I kind of put it into these four different things and I've spent a lot of time with Antonio operations making sure the system is up and running and stays up and running is the number one most important thing if a piece of software is gaining business value I'll get you in a second if a piece of software is gaining business value the last thing that you want to do is shut that sucker off and not have business value anymore right that's number one number two renovations you have an existing piece of software and/or an existing system and you have to renovate it right maybe that you want to have more capacity for the alcohol that you're generating or maybe there are better systems that can be put in place and you do this to continually improve the system over time and there's also integrations and we'll talk more about that in a minute and then finally there's design right this would be the equivalent of programming or making that new column so all of these things are the responsibility of the chemical engineer and I would argue also a software engineering there's a question back there so the comment is that maybe there's a serious problem with that pipe because I think that chipped off maybe you should look into that for the context of this talk let's just say that that ship off is purely aesthetic to know which pipe is which that's why they're different colors all right maybe unless they're not rest upon anyway so this is an example of a kind of renovation that they did so here what you see are a bunch of screens that measure these sensors throughout the plant now this plant is very old like certainly before I was born this plant was created and they didn't start with this but what happened is by doing this kind of automation with their plant they were able to go from having maybe three or four people constantly needing to run around and check the operations of things and check various gauges and make sure that everything is all kosher they just need to have one person who sits here keeps an eye of the entire system and one other person which goes and runs around and if there's some kind of issue they'll go and like deal with some kind of it issue right so they're able to cut down on expenses automation of the kind of things that we do in our systems is a very important thing frequently neglected you know how quickly does it how quickly can you go from a function declaration to all the call sites right if you if you're using like some old stale editor without any kind of plug-in capability and you're grepping through code you're not using sufficient automation as a software engineer or if every time you make a release you have some person manually run all the tests and make sure that everything is good to go that's a bunch of time that's being wasted or that could be recaptured by inserting more automation into your process doing automation is an important part of software engineering one of one of the core competencies so when it comes to integrations Antonio had some interesting ones one is that they had this there was this other business I think that it was in Portugal what it might have been on another part of Spain that just went out of business and I needed to sell all their stuff and they had an existing plant for creating ethanol and so they were able to buy that plant and they had to figure out some way to transport that plant all the way to a zoo Carrero Montero and then integrate it with their existing system like that is just an immense project because plants aren't designed to be taken down and then just put up somewhere else so they had to figure out where they want to make the cuts where they're going to be able to stitch it all together all the tubes that needed to be designed in order to be able to integrate it with your existing system like it was an incredibly large project and these are the kinds of projects that we that we really kind of balk at software engineers it frequently happens that somebody will will acquire another business and they'll say ok now let's integrate all of our software together and then you know five years later it's just an immense disaster because we don't know how to do it we don't have the skill set and I my claim is that if Antonio can transplant a chemical engineering plant from one site to another site something which was not designed to be moved at all and you have to do with physical stuff then we should be able to do the same with software engineering we shouldn't be balking at that should we again part of our core competencies another project that he had is that there was a sugarcane factory and this another thing that he's kind of specializes in in the Dominican Republic that was just a funk and just hadn't worked for four years and they took over that sugar plant that sugarcane plant and I got it going again having to replace whatever kind of motors having to upgrade this and that and they're able to restore this thing so this is the idea of you take this old crusty nasty codebase and you take it on as a project and you use it to be able to generate business value right again for a program where these are the kinds of things we shy away from but a software engineer this should be like this should be the kind of problems that we solve I think it's interesting so some priorities that I was able to gather is based on my interactions with Antonio the first one there is long-term demonstrated success versus new tech the preference is the long-term demonstrated success it doesn't matter if you know some newfangled library happens to be using the latest promises or whatever kind of cool technology you want to name it if it hasn't been deployed on a large scale we should have a distrust for it right and makes more sense to go with something which has demonstrated success because that reduces the risk all right this goes a counter a lot to the things that we think of in programming a lot of the time brand reputation matters I know that there was a project with Antonio that they had to decide whether or not they wanted to purchase this particular product from China which was extremely discounted versus their normal European supplier which was more expensive and it was just like this big dilemma of like well do we go with the cheaper thing or do we go with like what we know can we can trust these numbers from these people we can't trust the numbers from the Chinese Chinese company so brand reputation absolutely matters ironically the preferences for use things versus new things because not only are used things frequently cheaper but they have demonstrated performance we know that they work if you get something new which has never been used before there's an additional risk which is involved with it so the preferences for use things and the last thing which I found really really interesting is in chemical engineering at least there is an other engineering fields there is a ton of sharing going on between companies I've been with Antonio on several trips where we just go and visit a plant we talked with other chemical engineers and they say hey you know we did this system and this worked really well for us and we tried this you know this thing from China and it was Junkin or you know whatever it happens to be like the information sharing is incredible there in software engineering it seems like we have this like adversity we think that if somebody looks at our code like oh they're gonna get the secret sauce right but realistically speaking are they gonna get the secret sauce by looking at you know some lines of our code in a you know 5 million line code base or 10 million line code base probably not right so we could do a lot more with sharing with other companies now I don't expect this to happen overnight maybe this is something a prospect for like 10 years or 20 years or 30 years I don't know but basically the way that we do information sharing right now is we have these folks that work for a place for a couple years then they go work for another place for a couple years and and they do all this right and then you can like figure out like oh this is how this other company works internally and such but that has its own drawbacks that approach you know first with that with this shuffling thing which we do in software engineering you very rarely have to feel the impact of your mistakes right you're usually feeling the impact of somebody else's mistakes if you work for a large system for a long period of time like Antonio has been working on that plant for I don't know probably longer than I've been alive you really get a wisdom and a knowledge that comes with working with the same system and learned things over time it's just a different approach so one kind of like success story that I've had and I've seen a lot is claim format so how many of you are not using laying format okay so for those of you who are not using claim format you should use clean format you should do it so so clean format so I learned about clean format at C++ now many years ago and and I immediately apply this where I was working and we started using it and what I did is it saved a ton of time now when it comes to like industry consensus in an engineering field you want to go with whatever the industry is doing right so something that would be sane would be to you know take claim format and say hey you know what's the most popular formatting this being used right now what's the one that's most well supported okay well switch to that format and that's it right end of story something that would be a little bit crazier would be taking claim format and then like forking it and trying to make it match your coding standard and and then support that thing in perpetuity right the idea is sticking with industry consensus from an engineering perspective makes a lot of sense wise and and in terms of like the kind of technology you're gonna get so when we talk about innovation I think that there's a very different set of things that you think of when you think about innovation when you're talking about an engineer and you're talking about a programmer - an engineer innovation is concentrating on that business value right if you do some kind of innovative thing which means that you're able to capture this many more clients you're able to shut down this many machines to capture some kind of expense these are innovations or your user base is going to be that much more enthusiastic about your software whatever it happens to be this is what innovation is - an engineer innovation for a programmer it tends to kind of like hyper focus on the minutia right doing some like cool library which maybe that you can use in like four cases in your in your system now from a software engineering standpoint that kind of engineer that kind of innovation is is really a misuse of resources it's kind of a waste of time the innovation that they're really looking for from an engineering standpoint are like these business value propositions I don't know is this kind of like ring true for you have you seen this yeah all right so let's talk about something else that we don't normally talk about let's talk about garbage look at that garbage beautiful stuff what should we do with that garbage recycle it out or that would take a really long time and you want to sift through all that stuff take a whole no but that's terrible then you know you use all that space burn it yes so let's talk about Covanta and Union New Jersey they burn trash and they burn 1500 tons of solid waste per day that's a lot of trash it generates up to 42 megawatts of electricity which they then sell and that electricity powers 30,000 homes businesses in the New Jersey area so pretty good stuff how I got a friend and his name is Bill span and he is the operations manager and chief engineer at that cooperative plan one thing that you'll notice that he does differently is that he wears a nice button-down shirt and a jacket when he gives a talk I don't know if we'll ever get there with software engineering maybe but anyway the way this plant works is that on the bottom left there you've got trash coming in you got the dump trucks dumps into this big vat and then you got this electronic arm which picks up the trash and puts it on this conveyor belt and drops it down here and it burns it the burnt trash boils water right so you get like this high energy steam the steam goes up over here and it's actually three of these boilers so you're only seeing one of them here the steam goes up here and it attaches to this turbine and this is a steam based turbine engine so you put in steam and just start spinning like a like a motor there the turbine engine is stuck into this generator with that which then generates electricity and then electricity goes to the electricity company and they give it to all the people who need electricity that's how their system works that's their software now I had a conversation with Bill's awesome conversation he told me all about what they do and the way that he thinks one of the things that he came he told me about is they have these service level objectives have you ever heard of a service level objective before raise your hand if you have okay almost everybody fantastic so each the boilers has a ninety five percent uptime target and there's three of among of those I mentioned the turbine has ninety nine percent of time because the turbine goes down they're not they're not giving electricity to the electricity company and they can get penalties for not giving as much electricity as they agreed to and I asked him how did they come up with these numbers and he said it's based on historical data on what it was able to produce and is a bit of a reach you know is it's not like comfortable level it's like a little bit uncomfortable so they need to continuously improve their system to improve their times and improve profitability for the business so we talked about wanting SL o--'s for software development a lot very rarely do I see people actually have them and measure them like one particular example well let me just ask you this do you have sporadic test failures with your continuous integration system raise your hand if you do okay almost everybody alright so in the ideal world you don't have sporadic test failures they either passes or it fails in the real world you're going to have sporadic test failures it's just the nature of the game especially if you're doing something with multi-threaded code someone's gonna write a test somewhere which is gonna have some dependency on the system or the timing or something like that and you're gonna get sporadic test failures so how many of you have a service level objective for your sporadic test failures okay just a couple of us so what this means is that we're allowed to have sporadic test failures because we know that from an engineering standpoint going and spending all of our time fixing them does not make good business sense you want to keep a certain level of what percentage of the time when you run your build you know on master or whatever does it have a test failure right so we have an SLO of 1/9 right 90% we're going for a 99% SLO for this we may not want to go all the way there because it could be very expensive but 90s where we're at 90 so another thing is you know think about whether or not you measure your service level indicator for the one of the person over there your SLO do you measure your SLO for your spread chest all yours ok so they monitor it weekly good so we measure our service level indicator we have what's called a canary job it just runs every hour or so just a plane build on master let's see what happens right and it's not just sporadic test failures it's also sporadic machine goes down or something is turning you know all these kinds of things feed into this to this issue we measure it we know where we're at and we can track it over time and see like oh something's falling down we need to do something about it and and our software engineers that are working on this I've done a very good job doing this kind of thing but anyway service level objectives measuring them this is standard fail fair for engineering and other disciplines for software engineering we're kind of rediscovering this stuff just recently so another thing that the bill does or use telling me about is they do a turbine over speed so this test so what they do is they take that that lever that right before it goes in the turbine open it up a little bit and see if they can spin that turbine faster than they normally would right and see what happens you know does it vibrate more than they expect as it actually shut down and they have some kind of problem and they verify whether not a greater load can be handled and and this is what he said he said a sketch this is a scheduled risk to prevent unscheduled risks if you open it up well everybody's there we know that we're running this test and there's a problem they can fix the problem if it happens to you know get overloaded on some weekends now they've got to pay people overtime and everybody's in a crunch to try to fix the problem right so we can do the same kind of thing with our own projects you know just just think about your project and what it means to have an overspeed tests if you're running something which is performance critical then you could run a performance test if you're running something which has other kinds of like points of failure you can try to do things one of the things that is commonly talked about is to have like an outage test like a planned outage if you happen to have like a cloud-based service or something like that and see what happens let's see how you react let's see how the system does all right another project he had was the inconel projects and they have a specialized coating on the tubes is something that was invented recently that can prevent failures a couple things about it is that this is a very very expensive stuff right so they're doing an incremental deployment of this coating they're starting at the critical sites sites which they know have a lot of stress on them they're gonna they do this coating on and they're starting there and they're applying it slowly over several years of time and eventually they're gonna have all their tubing coated with this this inconel stuff now this can apply to our projects as well take for example a static analyzer or something like that like Coverity or some other static analyzer if you have a sizable codebase it makes very little sense to run it on your entire code base and then fix all the issues right it would be way too expensive but what you can do is you can start by running it on your critical places and then slowly over time make sure you schedule it into your tier software engineering schedule to gradually apply it everywhere that may be eventually you do have full coverage with this kind of stuff but the idea is to is to spread the cost over time so the comment is that for greenfield projects one of the one of the things that they do is they gate new software from being put into the master branch or whatever the equivalent is until it passes these kinds of tests is that a ballot approach I think it definitely is right you know if you go into Home Depot or one of these other big-box stores and you and you see if you go to the electricity section I don't know if you've ever done any wiring in your houses or anything like that they have this section called old work and they have a section called new work and the old work is - it's made to build into your existing system right new work is something else but whatever you do you got to make sure that it's going to be up to code whatever kind of changes that you make so I think that that that applies as well so another thing they do is that they have periodic me meanness so two times per year each boiler is taken down and once every four to five years the turbine is taken down obviously the turbine has that really high SLO so they can't take it down as frequently and what they do is they take everything apart revisit everything put it all back together again and see if it still works now if you if you if you do this in your own software systems if you have an existing code base and you revisit things not because you have to do something there but just because as part of your periodic maintenance let's just take you know over a time period and look at all the critical pieces of our codebase and revisit and do another code review or do another kind of questioning of how does this work and does does it still make sense for us to do things this way I think that would be pretty cool I think that would have a lot of impact I've never heard of anybody actually doing that in the software engineering but outside of software engineering seems pretty big pretty common so the comment is that certain parts of Bell Labs used to do something like this where they replace 20% of their code every year cool so renovations sometimes cause outages so there was an example the bill was telling me about where they did some turbine improvements and they were really improvements they they it was able to make the turbine work much more efficiently which means that it spun faster but the rest of the system wasn't able to handle the increased vibrations from the things spinning faster they didn't before see this it just happened that way and so what they did because it resulted in a lengthy outage as they stopped the periodic testing for it for a little while right because testing every time you do some kind of testing it does increase your risk so the key idea here is that they're spreading risk over time this is why they this is why they did this right you don't do all of your boilers at the same time right you do one boiler then you do another boiler and the one after that so the other two can kind of compensate for it if you take everything down and you put it back together and they all happen to be messed up you're in a deep trouble so there's this idea of spreading risk in software development right so continuous deployment is one way to do that every time you make a software change you as soon as possible you can redeploy it whether that means you know putting it on some kind of cloud system or whether it means releasing the software and some kind of betas form that users can give you feedback on if you if you wait all the way until you have your major release before you have people start using all these new features you're gonna find out that basically all of your risk it happens like is at that one moment right it's better to spread it over time makes more business sense and then incremental changes right if you have an existing system and you're you're modifying some kind of system some piece of it and you have the option to replace the entire piece or make an incremental improvement to the piece the incremental improvement is gonna have much better risk implications up for it right and that kind of goes counter intuitive you you might say but it's really old and crafty and it could be so much better but increment but from a business perspective from an engineering perspective it can often make a lot more sense to just do things very incrementally and there's a skill to it a skill that I don't think has developed very much yet and software engineering and making these incremental changes too long to large systems so migration failures I could give an entire talk on migration values but I have compiled a little checklist I want to share with you about if you want a migration failure do one or more of these things so the first one is the 2.0 migration anybody want to take a gander to what I'm talking about when I'm talking about the 2.0 migration the second system syndrome yes let's redevelop it from scratch right sometimes it's worse maybe 10% of the time but basically the idea is is I understand why people are using my system I understand what their requirements are completely so well then I can write a new system from scratch and deploy it and know that they will everything's gonna be okay and when you think about it is kind of preposterous you know you quickly aren't the only one that's ever developed that system if it's been over several years there are probably people who've touched that code that are no longer with the company and you can go in and make some kind of assumption about why they made something act in this way and change it and screw it up now any time somebody does the 2.0 migration they always think but but it's different for me right yeah yeah yeah ninety percent of the time it fails but I'm one of the ten percent well the likelihood of you failing is ninety percent right so you have to look at it in the aggregate and and think about risk from that perspective maybe it would succeed maybe but chances are against you if you do 2.0 migration the second one there is the opt-in migration anyone want to take a guess as to what that one's about no what's taking a guess so this is the idea well well this is the idea of an opt-in migration is you come up with a new version and you tell your people please opt in to this new version when you have some time Python 3 is a fantastic example of this Python 3 as an aside I think that's probably the biggest software engineering disaster we've seen so far so the opt-in migration if you want to support two systems this is a great way to do it because you're gonna be supporting the old and the new forever and if you don't learn your lesson you're not just gonna have two systems you're gonna have another opt-in migration now you got three systems now you got four now you got five I've seen upward of this many systems going on at the same time because of this fallacy that the people do this because they want migration failures okay the third item in the migration failure checklist is tying one migration to another migration so this is I'm gonna make this change but oh they're also working on this new thing over there I'll use the new thing and then when they're done I should be about done and then you know it'll all work together right what happens if you tie your migration together as that it increases the likelihood of your migration to fail because all migrations have a failure rate right and if you tie a whole chain of them together and wait for all these things to happen in order for you to you do your you do your migration there's like a chaining effect you know this migration failure implies migration failures everywhere else so go ahead and use the existing system like even though they're developing some new fangled thing so that you don't tie the migrations together come up with C++ 11 I'm not sure what's meant by that oh yeah so yeah so I'll use C++ 11 and eventually we're going to upgrade our compilers and and it'll all fit together yep the last one is deploying everything all at once just deploy all at once and then BOOM right so the idea is incremental deployment works much better all right we're gonna shift now into the philosophical portion of this and so I'm gonna start with a quote from Nietzsche many are stubborn in the pursuit of the path they have chosen few in pursuit of the goal all right that's kind of true so ultimately when people do things in your software code base even if they're a programmer they have good intentions right there there is that goal in mind that this is somehow going to improve business value but they don't realize that the path they have chosen is not actually helping the goal but they just let go on that path and they and they fall in love with the path as opposed to the ultimate goal so software fitness theory this is the big idea and I'm not exactly happy with this formulation we can hit rate on it but the first point is that software is a means to an end and when I talk about an end I'm not talking about an end in the temporal sense as in like it ends here you know the ends can be a continuous flow the hen can be making some kind of improvement in society and you know that is what what I'm talking about I'm talking about end and it's not fixed as in the sense of you have this end and that's what it is or like all the time like the end changes over time like what you're going for and the second point and software fitness theory is that fitness is observed by measured proximity to the end how do you know if your software is fit well you know if it's fit if it's meeting those end goals right so this is we're gonna unpack this quite a bit has a lot of implications but let's start by let's do another quote from Nietzsche here convictions are more dangerous foes of truth than lies if you're convinced beyond any reasonable doubt about something and somebody comes at you with a counter-argument what do you think about that counter-argument well you're already convinced right there's there's nothing they can bring to you because you're convicted you know that this thing is true now if you have a false something that you know which happens to be false and someone says oh that's not right like here's the truth when you get confronted with the truth you can now understand the truth but convictions are really dangerous so this is where we're talking about ideologies ideologies make for really easy dopamine hits what do I mean by that let's say that you happen to be part of an ideology which says that software is good if every function has documentation for it now you open up some kind of code and you see there's not some documentation there and then you go and you add documentation where it's all missing and as soon as you're done adding all that documentation what happens boom dopamine hit feels good right yeah it's a new list alright you get the same kind of thing you write a to-do list and you check everything off and if you check everything off and that means success for you and so when you finish checking everything off you get a dopamine hit this is why people love ideologies right because it feeds back into our human joy systems now the fanboy is the most dangerous thing when you when you attach an ideology so the fanboy will measure software quality by proximity to the ideology so let's say you know we continue with that ideology that software is quality if all the functions are documented the fanboy is going to look at a piece of software and see hey this isn't all documented therefore this code is crap even if that code is has immense business value it could have a sustainable like advantage over all the competition because of the way that it's implemented or whatever's in there the fanboy isn't going to see any of that because they're stuck to their ideology so when we're talking about software fitness and software fitness Theory proximity to a particular ideology is unlikely to be a good measure of software fitness all right that's one of the key takeaways you're gonna hear lots of ideologies you probably have some off the top of your head that you have yourself alright ideologies of what what good software is but what software fitness theory is saying is like be very wary of this kind of stuff and then the last thing is the ideologies by their very nature are oversimplifications right the real world is a very complex thing and if you boil it down to some kind of like truths you're going to have an oversimplification and they do lead to absurdities I'll give you an example of that so one ideology that people are aware of is don't repeat yourself how many people have heard this ideology in one form or another everybody right it's very common so let's see what what happens here so here we have a main function and it's written like this int main int argc char star star RV how many of you have written this kind of code everybody write what a bunch of repetition so let's let's do what this ideology says let's make a macro now now what's interesting is a lot of folks in here are probably saying man this is just as this would be a terrible idea because of another ideology which is macros are bad right but if we talk about software fitness and you take a take this and you implement this into your system and you start changing all of your source code anytime you have the int main you use this new macro you're like don't repeat yourself this is making the code better when people that are on your team see this code aside from thinking the Year schmuck they're their first are not gonna recognize it like what the heck is this main Lib main thing like they don't know what this is they've never seen it before so their time to be able to get up to speed in terms of what's going on here it increases other people who might come and look at your code with the no macros ideology and say your code is crap so when we're talking about software fitness there are a whole lot of stakeholders here and this will actually decrease the software fitness in your code even though it gets closer proximity to the ideal of the don't repeat yourself all right here's another ideology that's out there so Adam Wiggins is twelve factors have any of you heard of this before okay a handful of you so what Adam did is he made a list of twelve criteria that if you do all these criteria then your service code is good and if you don't do it your service code is ahead and if you have a fanboy that goes into this thing and says hey let's say we want to do I don't know treat backing services as attached resources this might not make business sense if you have an existing service like someone could actually spend a lot of time to get number four here and it's just completely wasted effort because it didn't improve the business value of the software and didn't improve the software fitness so back on soccer fitness let me just go back to this slide is this code fit for something okay comment is fit for a testing framework anything more local okay people love this code but when we're talking about software fitness I would argue that this code is fit for this slide right it illustrates a point any kind of software that you write it has a certain context what you're doing it even if it's software written on a slide okay I talked about the twelve factors so I was once in a standardization meeting where the artist said this thing he said if you don't rely on a feedback then it isn't decent engineering he kind of said it said it in a flustered tone I think he was upset but I think that it has a really good point how do we get feedback with code what's our primary means any ideas it works I heard code review that's what that's what I was thinking of code review so code review is kind of a simulation of what happens when someone has to revisit your code three months three years down the line right and what you're doing is you're basically compressing that experience into a fixed time period so feedback is incredibly important we already do this and we see whether our code that we're writing actually fits in our team by getting these peer reviews and the more reviews that you get the more likely if you incorporate that feedback is going to be fit for that purpose but Fitness is not just a social construction right the code you're right is not just about the people that you they happen to work with there are more factors in play here and we'll get to that in a second but I want to talk about another ideology this one is fun correctness code which is correct is good code which is not correct is bad code with bugs is bad all right so does anybody see a problem with this what I have here on the screen ok I see one overflow exactly so this code if it does what it says that it's going to do is going to have some kind of overflow problem now we can say that it has inherently a bug but does that tell us something about the software fitness if you never happen to call that with integers that would overflow it might be fine for your software might be completely fit even though it has a bug now before you go tell everybody the David's ankle set bugs are ok like keep in mind that software fitness is a different thing in the correctness is an ideology so all right or another thing is maybe you do give these numbers that do overflow and it returns some kind of in realistic speaking I mean in reality it's not some crazy blow up your machine behavior you get some weird number 4 as a result maybe that doesn't matter for your system maybe that happens so infrequently that it doesn't have any impact on business value whatsoever if that's the case prioritizing fixing that bug may not make it very much sense when you have other things that actually can improve your software fitness because you don't see the limitation okay so the comment is that it could throw an exception over for sure but for the context of this let's just assume that it does the wrong thing all right yes comment the way I think correctness ideology has a component in fear maybe I mean there there is fear that compiler optimization completely break things but probably not right it's just at risk okay this is the kind of paraphrase what you said it could be useful to have somebody fix this bug even though it's not really improving the software fitness because you don't want them touching some other part of your code or whatever as an example so maybe yeah sure why not all right I knew this would be controversial okay when we talk about sorting and correctness ideology this sort could be correct but it could have another problem what problem might it have doesn't deal with null pointers different problem stability sure no I think this is valid but the actually the thing I was having in mind was performance right performance is a completely different thing than correctness it's not covered by correctness at all so you might say well here's how to improve this Big O and log n time all right so now we've added that into our contract is this fit as far as performance goes maybe it completely depends you don't know maybe this needs to be run on a very special platform or maybe that it has particularly list sizes which is being called with and so you may want to do one of the you know non Big O n log n sort algorithms like that constant factor can make a big difference and it's highly dependent on the hardware that you're actually using so just smacking you know Big O n log n on there does not help you with your performance software fitness not enough if you want to talk about software performance fitness you have to measure it okay but there are other Fitness's that aspects of fitness for code like this that are important another one could be security fitness maybe you have a system that you want to make sure that this vector int list that gets passed into here is only in a particular segment of memory and you want to guarantee that program wide so you may have some doctored up compiler some kind of extensions so the language to be able to give you that kind of guarantee because that's what Fitness means in your security context so when it comes to security how can we measure security fitness I don't know what do you think what would be a good measure of security fitness fuzzing okay another red team so to really know if software is secure is if it doesn't get hacked I didn't get hacked it was secure for that period of time but this idea of a red team I think it's a fantastic one so if you have a team of security experts that are constantly trying to hack your application and are unable to do so then that gives you some kind of indication of whether or not you have security it's very similar to the code review you know like what's it going to be like for someone modifying this code you know three years down the line compress that into a smaller time period and have a ton of people act as if they were doing that right now and then you can do the similar kind of thing there's a comment of what's the cost of having security broken and that also goes into what does Fitness mean for your application absolutely so to kind of extend this into a principle is that a good way to achieve Fitness is to put your candidate into a hostile environment and this tie is very much in to Darwin's theory of evolution that's why I'm using the word Fitness here if you put your stuff into a hostile environment that will give you some kind of indication of fitness that and use that feedback as be Erna was talking about if you just put into a hostile environment don't do anything about it obviously that doesn't help you right but if you use that as feedback into your into your software development then that can actually be a pretty good way so some hostile environments code review we talked about that extensively manufactured break-in attempts UI feedback taking your UI putting it in front of a bunch of users and say use this as you would you know do your normal workflow that's a hostile environment and incorporating that feedback getting API feedback a lot of people design API is based on ideologies and then they throw their API out there and it's not successful if you if you take your ABAP eye and put it in front of your actual users and ask them to do something with it and incorporate that feedback that hostile environment will actually help you a lot more than anything else and then stress testing is another really good one all right so this idea of software fitness I like it for many reasons one of the key ones is that I think it unifies our software engineering disciplines it gives us a vocabulary of talking about it and talking about why we do them there are things that on this list that are considered good practice already but this tells us why we consider a good practice and it gives us ideas for other things to come you figure out what software fitness means for your software that you're developing and then you figure out how to throw it into a hostile environment and incorporate that feedback right it could be something that you have you know we have when you do a like a pull request or something like that or or a sprint item this this idea of acceptance criteria something else that could possibly go in there is what is the software fitness what a software fitness mean for this change and how are we verifying that we're going to succeed so to paraphrase that something that's missing on this list that could definitely go on there is requirements analysis and I much agree that figuring out what Fitness means for your thing is like a very important aspect of software engineering engineering in general so a common is is that no requirements lists has ever survived first employment and that's also the idea of requirements change over time right there the end is not a fixed point in time or a fixed thing the end is constantly changing so the comment is is that doing a code review can bring in a lot of extra ideologies and as we see ideologies can waste a lot of time or not necessarily good business value and I very much agree with that and I've seen this happen quite a bit and I think that we need to start thinking of our code reviews as the simulation of the future maintainer not as this is your chance to insert all of your ideologies can you read the code does it make sense are you going to be able to continue developing this code if you answer those questions in the affirmative then it's probably good to go even if it's not matching your particular ideology another comment someone comment is that part of the product is the specification of the API so the comment is that you could I'm not gonna get the whole thing but but the example is that if you put in your requirements for example some metric that so much to be able to pick up this and be able to get working with it within an hour or something like that you can maybe quantify some of these Fitness factors all right common is that you can like disallow comments about style in a code review that's one way to kind of deal with this stuff so the comment is is that ideologies are a good crutch until you are able to transcend them and I I would argue that that transcendence is the turning of a programmer into a software engineer all right come on over here so with regard to the first comment which is how do you measure software fitness you just like an engineer can't say like a chemical engineer can't say what are the important things for any chemical engineering project I'm not going to be able to say that for a generalized software project right I think that a software engineer needs to figure out what these factors are like we talked about some things readability might be important some of these other factors but it really highly depends on what you're doing and that in figuring out with the fitness criteria are and how to measure them is your job as an engineer so the comment is the Zack Lex's ideologies you can tear him away from his dead fingers all right I'm just gonna take one more questions I do have a few more slides too I'm not sure I'm gonna be able to capture that but the comment is that our field is different than the other fields because of codes and regulations which I think are you're saying apply to those other fields more than us so I'm not gonna repeat that so all right we have this idea this this unification of software disciplines under a common framework we can talk about it I think that's cool I think it answers a lot of open questions why does so many successful projects have such bad code they're fit all right that's why you're measuring bad code by ideological standards um if the code is successful it is by definition fit I'm not gonna answer all these questions I'll leave that as an exercise if you can think about how software fitness plays into this I like what makes a good API and so on I look at as exercise for you to think about understanding machine learning so this is kind of a sad story but on March 26th the skywater Walter his Tesla took him out of his Lane pointer meta fix concrete barrier and then accelerated boom the car broke I think this guy is still alive he survived it oh we didn't survive it no this more sad but anyway machine learning is a different way of creating software where we're not involved this whole idea of correctness is not not a thing that you can apply to it but software fitness does write software fitness criteria applies to machine learning rated software and we can use that another aspect is how it relates to artificial intelligence like if you want to pass the Turing test what would be a good way to develop piece of software which which does whether it's machine learning based or somebody's actually programming it to try to simulate a human being the best way to be able to develop artificial intelligence is by using feedback and putting it into a hostile environment and being able to to get that and then incorporate that into your software you're never going to be able to create artificial intelligence by just sitting you know somewhere in Aspen Meadows just thinking about it and then having a and coding it up right the only way you're gonna be able to get it in my opinion is by checking this Fitness put into a hostile environment and continue iterating on it now one other aspects is C++ a success why why is C++ successful I mean it's 30 years old so in April 8th 2019 I saw I don't know if any of you have seen this article but ZDNet reported that C++ rose to third place in Taiyo B's programming language popularity index for 2019 displacing Python so what what's going on here why is C++ this old aging language still at the top of the stack and I think that it's because from the very GetGo C++ was designed by engineers why does C++ have C compatibility from the beginning it fits so much better that way all this existing C code could be taking advantage of why does it stay popular all the changes that we're making to the language by and large don't break programs by and large right we don't have an ideology that we can't break your code you know and the same standardization committee will break your code but we do have this idea of we don't want to break everybody's code we don't want to have people have a crazy migration path to upgrade to the newest version of C++ so it's certainly much harder than developing a new language extending the existing language but it has a much better software fitness implication and I think that as long as we keep this mentality C++ will continue to stay at the top of these lists so there was the comment that there are ideologies that are creeping up in the committee some of them are like don't break anything anywhere another one is like let's not do weird strange things and I totally agree and this happens like my the ideology that I hate the most in C++ is that the C++ standard library should only be low-level things why it's just an ideology right if we can add something which doesn't happen to be a low-level thing that improves C++ software fitness then we should do that right so I very much agree that that there is the danger of ideologies but thus far it seems like we've stuck with software fitness enough to be able to stay at the top now if other languages come out and then they're more like into this idea then they may be able to win but so anything that I've seen thus far you know whether it's rust or Swift or anything other kind of things they completely missed this point so the comment is that because sequence has so many implementations out there having multiple implementations can give us better feedback and so that we can improve our software fitness that way that's one aspect of it it's a good point so just hold your question we're almost at the end so fitness this idea of software fitness there is a generalized fitness theory that goes along with this and I'm not going to talk about it I'm just gonna kind of hint at it if you want to talk with me about it afterwards we can talk about it but it does have implications in music and in ethics and a few other different areas so the comment is that having a specification improves C++ --is applicability and heavy multiplications forces a specification and that helps in certain Fitness in certain fields like what were some of the examples you cited there like safety safety critical applications well not repeat it we could use analog those practices we were kind of I think explicitly but also implicitly asking the question these guys in these established industries are doing these smart things why aren't we doing it and I think part of the answer is accountable in your talk and see the econ points there's team talk about software capital and technical debt right currently there is no way under the generalized accounting principles to financially account for so technical debt does not appear on the books of the companies and frankly I think we all know if it actually did lots of companies with pet and lots of valuable disappeared and if the accounting rules actually take into account the idea of technical debt like they do depreciation on capital assets that you buy for a plant for example the changes that we so the comment is that the technical debt should be valued in an accounting sense and that could potentially improve things so the calm is in certain industries there is this accounting for technical debt in terms of like number of outstanding bugs and things like that and there are requirements to like burn those things down so the comment is that functional programming has taken off in terms of language design and is this an ideology thing or is it so the question is how do you go about determining what is merely aesthetics and what is actually has a real impact and I think that the answer to that is you have to be able to tie it to the fitness of your software and in like a very direct concrete direct way like I do think that consequence for example is a is an ideology and you can look at a piece of code and you can add constant a few places and it does not improve the fitness of the code so I think I think it really has to I don't have all the answers but I think that what we have is layered context for thinking about this now at least the first thing and that incredibly because we bought that piece of software into our custody exactly because there was an added requirement that wasn't a requirement before which was we need to be able to change this quickly so up so the comment is so the comment is that we took over this project and the first thing we did was add Const and then we had a testing and is so much better for adding cost and I'm just gonna say I don't believe you I don't think that I don't think that adding Const improves the fitness that software now I could be wrong I'm willing to be wrong but I really feel like this is like the ideology to understand the code alright so the comment is is that there's a priority like if there's new code which potentially could be introducing technical debt by not having constant redness like that would be something to prioritize fixing so don't get me wrong like the code I write I make sure that is cast correct but but I'm not sure if how important that is that's what I'm trying to understand so one common is that for for cos correctness ideology you could make a program which you know automatically and as well so the comment was that in their situation they have these three values which they measure against it was customers company and developers in the future also and the future is indefinite so the value of the future is is far greater than the value of whether or not it works today so the cleaning things up have potentially so the comment is that that I'm only focusing on like value in the present and not like in value in the future and I Knight and I definitely don't want to give that impression so glad that you you mentioned that so Fitness changes over time and it applies differently over time you know you you have a software projects that let's say it's for a startup company what Fitness means is that you know your software is able to demonstrate enough to be able to get more venture capital money to be able to continue your business so in that kind of a context that kind of fitness you're you're not going to care about a lot of stuff you're gonna take a lot of shortcuts on that that would make your software more fit but if you're also thinking the long term you know maybe you decide about how many of these shortcuts you want to take are you were going to pay down the technical debt after you get the funding that kind of thing so the comment is that regulations come down and like destroy your business so Fitness doesn't help there I'm not really sure anything can help there I I think Fitness is probably the best we got but it but it you know unforeseen events like you know nuclear bomb blows up her company boy actually as it was right there because it anyway you know so the comment is that adhering to certain ideologies like more rigorously might actually end up having a positive impact on your your ultimate fitness and I think that's true i I think that where we're at as an industry like the corrective is on focusing less on ideologies but you know there they do have a place like in terms of what exactly they're a place to sure to ultimately be I'm not exactly sure yeah there was there was from okay so the first comment is that ideologies should be taken and a little bit more nuanced form but I would I would argue that even if you nuance an ideology you will still it can still lead to absurdities right I mean it it can improve it this or doesn't happen as as much but just keep in mind that can always lead to absurdities and the other comment is that C++ doesn't have garbage collection because someone had an ideology that garbage collection is bad I have no knowledge about this like I've never heard before if garbage collection could increase C++ is Fitness maybe be more popular I don't know I I can't really comment on the X I I don't know the context because we would really like those they seem expensive and save time in the long run so so so one one so repeat your question like how do how do these engineers and these other fields convince management to do all this kind of stuff like we're NAB such an uphill battle my experiences I've worked with some of the managers of these engineers is that they have an awful lot of confidence in the engineer as an engineer and so they trust that you know their judgment in these things and they trust that they know what they're talking about with best practices how do we achieve that in our fields and not exactly sure so we're one minute over so I'm going to take this last question and that'll be he has many seen and so the question is do I do I think that will ever be regulated as like some other fields like for example that you need to have some qualification order to be able to practice I think that we already are in certain segments we do have certain requirements and stuff it software is very far-reaching so I think it highly depends on what you're doing if you're developing you know a triple-a game I would be surprised if there was some kind of regulation for that so do you want to take the question after the last question okay quick quick so the comment the comment is is that to practice software engineering in Canada for instance you do need to pass some kind of a test alright so that's the end I hope you enjoyed the talk thank you [Applause]

Channel: CppNow

Views: 12,712

Rating: 4.6842103 out of 5

Keywords: David Sankel 2019, C++Now 2019, Computer Science (Field), + C (Programming Language), C++: Engineers, Programming, Cpp, C++, CppNow, CppNow Conference, C++ Conference, Cpp Conference, software development, Bloomberg, C++ software projects, engineering discipline, David Sankel, programming talk, Programmers, C++: Engineers Wanted, Programmers not so Much, David Sankel Cpp, David Sankel Talk, David Sankel C++, David Sankle C++ Engineers, David Sankle CppNow, Cpp Talk, C++ Talk

Id: eRHLuuFMtx4

Length: 92min 26sec (5546 seconds)

Published: Thu Oct 17 2019