CppCon 2015: Neil MacIntosh “Static Analysis and C++: More Than Lint"

so hello everybody my name is Neil Mackintosh I work at Microsoft where I'm lucky enough to lead the static analysis team who work on C++ and unsurprisingly to me that's what I'm going to talk to you about today static analysis and C++ more than lint so I thought I'd mentioned lint in the title of the talk because according to the Internet which is where I get almost all of my information from it's a completely reliable source lint was written roughly 36 years ago and so I thought it was time to see it still relevant what have we grown out of that initial starting point for C and let's take a look at the experience we've had at Microsoft trying to apply lint like tools at scale what we've learned from that and hopefully the talk will give you a little bit of a story a bit of a journey that we went through and where we went ended up today and how we got to thinking the way we're thinking at the moment about things like the GSL and some of the checking tools that were talked about in B Ana's keynote or herbs others plenary session yesterday so let's make a start oh nice so the talks going to be example driven for a couple of reasons one is I lack imagination and I find it easier to borrow other people's code and put it up on the screen for you the other reason is I know I find it easier to walk through examples and and think about them and the third reason is I'm hoping to entertain you a little bit because in that sort of America's Funniest Home Videos since there's nothing as satisfying as looking at other people's mistakes especially if they're on a big screen so with that said all my code examples that I'll show you today I have adapted and sanitized to try and get protect the innocent who ordered them and that may or may not be me in some cases but all of the code examples are drawn from real live commercial code bases that I have looked at with my tooling over the years so although they look like toy examples I can promise you all of them are real bugs from production so you've all had a little while to look at this example in particular it's an easy one it's a warm-up because hopefully most of you think about writing correct code rather bugs so it takes a little while to flip you to the mindset of where's the bug in this so this is C like code but this is very very common it was an API which had to have C ABI compatibility and so you know the caller is expected to allocate a thing themselves pass it to a function that initializes a thing and then they can do other things to the thing without the function calls the usual sort of C style pattern for doing API is very common and of course when we get our thing in you know we we check to make sure we got one that wasn't know and then we make sure we mem set it out to zero and then we set some of the fields to certain values leave other ones at zero and so on except as I'm sure somebody in the room has noticed by now exactly we didn't maybe we set it to zero and luckily at Microsoft we run some tools some static analysis tools that help tell us this they point out that we're overflowing the buffer why are we overflowing a baffle well because we're mem setting the pointer the pointer variable thing that was passed in whereas what we meant to do was memset the memory pointer to by thing and so we get this error message that tells us you know more or less prettily but that's exactly what we're doing the buffer is 4 bytes long the size of our pointer on this particular platform we were building for and you're trying to access 12 bytes okay so struct earthing is slightly misrepresented up there ish should say 8 bytes and I've I've truncated it too much to fit it on the screen but you get the idea we're writing more memory than we have access to right yeah writes - right - so it's a simple enough bug it's easy to make simple fix one character the difference really easy bug to write really hard bug to track down depending on what the stack looks like what operating system you are on or what compiler you were using whether or not you've got stack checking turned on in the compiler and all these other bits and pieces because when you mint it passed it but at the time yeah okay it'll just write some bytes write some bytes somewhere to 0 and maybe that'll be a bad thing in maybe at one hard to find easy to find for static analysis tools so the point of that wasn't just a warm you are it was to give you the value proposition that lint brings to software development and I say value proposition because I work for a great big corporation and everything's a value proposition where we are but it does have a value proposition and that is to save you time and money right if you find some defects early before you check them in before you compile or before you commit your code to source control then they're easier to fix their cheapest to fix then ten minutes after you wrote them not down the track when you built them and when you're running tests or even worse when you've shipped the software and the customers are falling over them and it saves you money because it's really costly to update your software as I'm sure a lot of people in this room know and ship it out to customers I work at Microsoft I'm sure everybody's familiar with those updates that come around once a month I sit there and install on a Windows machine you either use or if somebody related to your users and usually asks you why is my machine doing this can you help me fix it no no I can't so let's value proposition has been pretty successful over the years and we see it's been incorporated into all sorts of tools right so compilers all have running it you know wall and a lot of the checks that run up wall is compiler warnings started off in the original lint tool right there's lots of free and commercial offerings for doing static source code analysis so you can use slash analyze and Visual Studio you can use clang tire your clang analyze Coverity you know there's a bunch of these tools out there so clearly the limit value proposition has resonated with people and if you're not running these tools some of them all of them in your source code you should be because they bring a lot of value to the table if you think they don't bring that value please let me take this opportunity to encourage all of you to complain really loudly to the people who write the tools as somebody who does that there is nothing worse than the mayor of epithelia at all says oh no it's too noisy it's not helpful and just turns it off again at least take the time to tell us why it wasn't useful to you so it gets better if it's an open source tool even better take the time to try and contribute and make it better at a patent for something that it missed another example I've talked enough so this is a slightly different example before we were doing something that was clearly wrong we were over running a buffer over running the size of an allocation and that was a bad thing this bird is different okay there's nothing illegal about this bug there's nothing where it's overstepping bounds are doing anything wrong it's a super simple bug and I'm sure you've all seen what it is by now it took me a while to see it when I first looked at it and it took the people who wrote it a very long time to find it in the source base and they only went looking for it because they were debugging and they were wondering why their service was so expensive in its idle loop answer was it was never reaching that idle loop because of course somebody copied and pasted it or maybe somebody just chose the first option given to them by intellisense or maybe they just made a genuine mistake and they should have set current to stopped before they went on and made that if test now there was a little bit of code in between setting the current state to stopped and then testing did we really stop in the original but it wasn't that much code so that you had it took you a real while to look at this and work out because it looks valid it looks valid to me I thought oh yeah I'm missing something of course they said it back to started and then they do something no no no no no they meant to set it to stopped and so they never went and waited for a restart but they didn't have to because they never stopped great some object bugs how do we find this bug all right so this is this is a little intro to thinking about now I've warmed up seen a couple of bugs how do we find bugs like this when we do analysis so remember I said the bug results from the fact that in current gets assigned a value and then when we go to the if test where if test is always going to be false because in current could only have the value it was just assigned this is not multi-threaded code that's how our analysis tools think about things it's a concept called abstract execution or symbolic execution or interpretation we construct from the source code a rough graph of control flow here and we just walk through it and we can store some values for EM current based on what we see going on in the function so we might say it's unknown at the beginning of the function we get two and current equals state started we know it has a very definite value at that point so we we can record it then we can walk on and we can look at that if test and we can say you know is the if test against unknown values or is it against known values and it turns out that on every path through this function it was against the same known value which was not the value being tested so in other words that if statement was always false provably false therefore wait for start is dead code and that looks suspicious and so that's what the tools tell the user it's suspicious sort of code you're never going to enter it it's always false so it's a different type of bug a logic error as opposed to a a a sort of a correctness error so Lintz valuable we've warmed up let's talk about how you scale lint how do you make it work on really big sore spaces and get the value out of it his some lessons we've learned from our experience at Microsoft and number one is the most important you need to be fast you need to be in that edit build debug cycle that you know some people at least at my company called the inner loop of the developer right if you're not mean in a loop you may as well build home in terms of getting their attention and bringing value to them there's a place for larger scale analysis there's a there's a place for you know test results run six hours after you've checked in or one or two days after you've checked in all those things bring value but if you really want to get people's attention and get bugs fixed early you have to stop them early too so your tools need to be quick they need to be obviously bringing value to the to the people using them they need to give them actionable warnings so you can't have too many false positives you can have some people sort of surprisingly will tolerate them particularly if they tend to point you at code that's suspicious anyway one of the things that first surprised me when I started working this area is even the false positives will often point you to real bugs just not the bug the false-positive claimed they're often just a sign of a bad smelling code so if you can get your tools generating those sorts of false positives it's a good thing where it's a true positive people have to actually understand what the problem is and how they can fix it then do it quickly you do those things you get buy-in seems obvious but buy-in is important when you're trying to appeal to thousands of engineers who are trying to get through their daily job just like everybody in this room as quickly as possible get home have a life have a weekend and they see your - you know they see that they see a tool that tells them there's a bug in their code is it's an interruption to that path to the sweep time so you've got to be helpful to them the other thing you want to do if you're running at scale is don't just look for safety and reliability bugs like the first one we looked at the second bug we looked at is equally important it's a logic error but there's also a valuable place for using static analysis to do other things that are useful to you so one of them is got a bunch of api's that you know a dangerous are old maybe you've got some crypto API is that your company once shipped that you now think are maybe not quite as secure as they should have been you want to move somebody to the new set of crypto api's it's easy to identify with static analysis which api's should be deprecated and how to move them maybe it's just a parameter that should be deprecated in the API again another really good opportunity to use static analysis and give some value back to the year the company performance has become an increasingly interesting target for static analysis we can point tools can know certain types of routines certain types of callbacks might be performance critical maybe they're only ran intermittently in sleep mode of a device or maybe you know that they they are on a critical path of some part of a kernel similar if you have that sort of knowledge in a tool then you can stop people making really expensive say cross process calls or writing little busy loops or doing too much work at all they've their certain API is you should never call where in your when you're in a particular callback or mode because they will cause the machine to wake up or wake up the disk and so on so again you can provide lots of useful hints with static analysis that help people get their job job done faster and with less stress so let's look at the first of these fast turnaround how do you achieve this with stack analysis tools it's pretty hard it turns out because the tools at least have to have the same sort of work done as the compiler they need to do parsing they need simple tables they want types they want to let sort of lit so we're piggybacking at least on compilation so at the very least people after what may have the same way as they do to compile their code they're probably generating code from that as well and so then you're running afterwards so you're already making them take longer than they would have otherwise have done so there's a couple of things we can do you can run asynchronously we've found it's really useful to say oh you're kicking off a build you're compiling you've got a ton of processors sitting on this machine or at least it's got the capacity to run a ton of threads we'll do the static analysis compilation at the same time as build well we can do it in the background if it takes a little longer and come back to you helps to do as much parallel execution as you can obviously incremental is very important you can't always do all the work each time the most important lesson we learnt is share the common work if you're going to go and construct things like pass trees and symbol tables reuse them the first time you try and roll these tools out at scale there's a tendency to say oh I'll just write out at all throw it in do them the tool does all the same word picks up whatever the compiler built and then it starts doing some analysis on the next one does the same thing you suddenly find you've got 6 7 8 9 10 copies of the same information being generated and used to provide the analysis and the time taken to get analysis results back is taking longer and longer and the value is correspondingly shrinking for the user so things we learned are have a common engine if you can and reuse that sort of infrastructure to do your checks and keep the check simple that also helps with false positives and accuracy there's two types of checks that we think are the best ones to try and focus on for performance one is what I call syntactic checks so these are super simple you can do these against a parse tree and this is a little like what I talked about before when we talked about enforcing api's right you're just looking for a particular function call maybe particular function call with a particular set of arguments it's pretty much pattern matching against a tree it's fair fast it's fairly reliable scale as well simple flow sensitive checks we looked at one of those examples before if you're going to walk through a function and track some values you want to track a tractable number of values you want to try and bound it so that your running time is small so you say simple but still flow sensitive let's have a look at how we might do one of those things so our first example with the bad member said so there's a good example of something that's really essentially just a syntactic check believe it or not that's a sort of a vague representation I've pulled out on the screen of what a syntax tree from our compiler might be Glee looked like by the time we see it instead of Canales us for that memset call it's pretty much permit got all the information you need to be able to flag the bug ok it tells you oh you're doing a size of type there that type is 8 bytes looks like I've got it right on this slide at least you know you're taking the address of thing and it's giving you the type of the result of that operation it's a pointer to a thing pointer all that looks like the wrong type doesn't it and in fact we can go to a symbol table which I haven't shown here you look up what's the size of that thing oh it's four bytes much smaller than the eight at the bottom there so we really have to do here is flag this go look up a simple table size for done a question over there oh they could be oh yeah that's true that is an error on my slide thank you there's many of those but you get the idea a quick syntax tree check and we can find the bug if we go look at the other the second example we did well so this is a good example of simple flow sensitive analysis so okay we can we can just track maybe for certain classes of variables currents just an enumeration variable it's like an integer so it's either unknown or it has a known value we can track it through the function and we can see what its value might be just as if tests it's a fairly simple flow sensitive check a lot of work being done but we can get a lot of value from it there's a popping up our warning again so the other thing I asked us to do is make defects obvious and actionable so one way to do that is to use heuristics to provide to restrict the number of false positives we spear out to narrow the surface area of what we gonna warn about a couple of things here one is trying to French shape between success and failure paths right do all roads lead to Rome is one question we often find ourselves asking and that's sort of what we saw in the last flow sensitive example right does every path leave this test with the same value of false if they all do all roads lead to Rome then that's more than likely a problem you want to tell the user about if there's a possibility it could be non false on one path well then maybe that test really is what they meant to do so you can use simple heuristics like that to be much more accurate in the sort of results you provide people with so context matters tremendously let's look at an example of where context matters when we're going to report errors to people so here's another interesting case somebody's initializing a system they're bringing up what sort of functionality am I going to provide to the user what sort of hardware maybe am I going to turn on and off so I'm gonna look at a Flags integer that's been set up somewhere with said bit flags I'm gonna say is it yeah do I want to enable the cool stuff if so I better bring up that hardware otherwise I'll just enable the standard stuff so that expression it turns out will evaluate to zero at compile time by design because this is a popular pattern people follow for making sure that they on purpose have some dead code maybe the cool stuff board isn't ready to run yet so I want the code paths in there I've been working on them and so on but I need to turn them off I turn them off because I didn't put enable cool stuff in G flags so it's dead code in there we warned them about the dead code people say your tools are noisy your tools are useless this is what I've always thought about your tools I'm turning them off and then we didn't scale we didn't get bring them the limit value proposition but we can use some important context in this example to help us narrow things down in this case which comes from real code unfortunately somebody forgot to set at least the first value of their enumeration when they wrote it out so that even when they did say yep the board's ready to go let's have the cool stuff enabled they couldn't work out why it wasn't working luckily our tool could and then we're able to say you know what you've got a zero valued flag there and we're able to use our knowledge of how the enumeration is defined which is just held for you in the symbol table and say well it was the first enumeration it was given zero and now they're testing it against a flag that looks kind of dodgy so that context helps to save this time we should omit the error much lower chance of being a false positive and people get less angry about it so you want clear warning messages okay that's a big part of making things actionable now if we go back I'd have to say that is not the clearest warning message and that is unfortunately the actual warning message in our shipping software today for that particular bug it would be nice if instead we could say and I should take a work item to go do this pop that on my backlog Andrew I know you're out there somewhere my PM thank you to say something like look you're using this particular flag and it's zero it's always going to be zero tell the person what the problem is point them back to where in the source something went wrong and suggest what they should have done instead so that's part of having good error messages together with good error messages with diagnostic traces so we saw a little bit of this in the demo I did with herb on today yesterday you can see a trace in visual studio when you get a warning message and we even pop up this nice box that tries to tell you what the analysis tools were thinking as they walk through a particular error so this is our way of doing it but you know I know there's other other equivalent sort of tooling and equivalent experiences from other vendors and I think places we see real innovation or with things like fixit's in plane tiling right this is a fantastic thing the ability to not only say hey this is what I think is wrong and this is why I think it's wrong but I can even fix it for you if you want it's really nice so enough about how to make it scale just from the nuts and bolts perspective what happens when we try and go further so far we've just looked at some trivial local examples we've talked about the things we all know oh I missed an ampersand in my men coffee or I mem said you know these are sort of your traditional limit problems but what happens when we try and find deeper bugs solve harder problems a bit like the lifetimes problem we talked about yesterday problem is that no functions in Ireland we immediately hid the fact that real software is composed of lots of lots of function calls and we want to understand what they're doing otherwise we can't understand what's happening to the values and the storage as we move through all those calls in a program so I've sort of got too broad of options for dealing with that so one is to do interprocedural analysis where you analyze all of the code whole program analysis look at what happens move the information from call to call simulate full execution of the whole thing that's at one extreme so that is a hard problem in P hard type problem so I was put in mind yesterday in herbs talk he kept referring to 42 the answer to life the universe of in everything and in the book that comes from obviously there's a great big computer built deep thought which churns away for millions of years and finally comes up with the answer being 42 it's very similar to the way interprocedural analysis tends to run it goes and turns for years and years and years does lots of deep thought and gives you a somewhat cryptic but sometimes very important answer hopefully it's often more important than 42 but it does take a long time to get there and it gives you a lot of our other information along the way it's useful but it's not in the inner loop and that's what we talked about with scaling before so when I think about global whole program interprocedural analysis the way I think that it is it's great to have in your organization it's incredibly invaluable - it's an incredibly valuable tool but it's not something you put in the developers in the loop because it can't run in there it just doesn't have the time and space it can act as an advisor maybe later it can act as a gate it can act as a final quality check but it can't really help them as they're compiling individual translation units and moving towards building some new piece of functionality the way even track procedural or function at a time local analysis camp which is what we've looked at so far in the talk so if we look at intra procedural how do we get into our procedural if we look at that how do we handle the fact that we've got all these function calls and we can't see what they're doing on the other side we're only looking at the current function we're in so we need to have a way to work out what the meaning of those functions is what are those semantics so we've got some options we can take a guess we can know some of them that are special you saw us doing that earlier with the memset call we know what mimics it does it writes to memory so we can find bugs based on it we can look at what the type system tells us about the functions we can annotate them to insource so let's use an example to drive some decision-making here and this process reflects pretty much what happened over the course of 10 or more years at Microsoft as static analysis evolved now the code you're looking at on the screen is a super simplified representative example of the heartbleed bug that was popular was it late last year I think and basically that bug was all about perhaps trusting your packet your incoming network data too much okay so if we look at this example we get again it's very C style but this is exactly the sort of code you often see in a code base as large and old as ours and especially and lots of other people's to be honest and especially in network type code you'll often see more C style code so here we've got a received packet as a packet of bytes giving us byte there to make things a little bit easier it's got a length it's got another size and that's the buffer we need to write out too with a response and so in this case it's a bit like a ping we're just going to take what we got write it back out to a buffer and then send it back to the person to say yep got your data saw it here it is so the code is pretty simple unsurprising the bug is that the way we're calculating how much data to copy into buffer is unrelated and untested against the actual length of that buffer so the input data comes in we read the header from it and it says this is how many bytes you should copy out of me and we just said ok we'll copy that many bytes whether or not that's actually the number of bytes that are accessible via the point of rec whoops and so that turns out to be an information disclosure type vulnerability so you could craft packets that go to this and get it to overrun the buffer and send you back more than you intended to send back and then people can see all sorts of interesting things like passwords but they're interested in picking up if we look at the call site for a function like this it's just pretty representative we've got a read and write going on so let's look at potential mistakes we could get here as well now remember if we go back to make packet for a moment and look at its declaration it's one of these tremendously useful declarations that I think be Ana's talk pointed out maybe herb highlighted as well a couple of pointers in a couple of integers really meaningful stuff to an analysis tool could be anything I don't know what the integers mean I don't know what the pointers are it's hard to look maybe with a lot of work infer their relationships from looking at the code but in this case it might be difficult given that you're not using the relationship in the mem copy call so there's a couple of pointers in a couple of integers so when you make a call where you read the documentation you look at the help maybe maybe you look inside the function you've got some buffers let's imagine they're vectors in this case to be a bit more C++ ish you read it into a vector you pass the contents of what you read in to make packet you give it another vector to write it - which presumably you gave some standardized size that should be big enough and then you write it out again and if everything went fine that's okay there's a problem with this code as well and so the problem is the we're just telling make packet hey here's the length of the buffer but that's not the length I read into the buffer from the network if we look at the comment above socket read it tells you how many bytes it read into the buffer we didn't ever use that return value except to see did you fail that's a very common bug again that we see in people's code so with static analysis tools we could try to catch either of these bugs with purely local analysis it would be incredibly noisy if we warned about either of them so we don't so that leaves us here it's a terrifying place to be a giant sad face giant sad face because we've got buggy software it's hard to call correctly exposes security vulnerabilities in the system and there's nothing the static analysis tools can do about it it's a sad day so what can we do to fix it well I mentioned annotations and that's where we went first we discovered this amazing concept which was add to the language add some macros that look just like language and there you are in baby-blue plop them all over your function declaration and they look a little bit esoteric but if you've worked with them for a few years they start to look really perfectly reasonable I promise so the first thing that annotation tells you there is it says hey this function succeeds if the return is greater than or equal to zero which is pretty much what the function agrees is the case and probably what a comment on the function would originally said and then the in reads there tells you that the buffer pointed to by wreck can be expected to be Reckling bytes long they're all initialized bytes you can safely read from them but you shouldn't write to it and the out rights too well that one's really nice and esoteric with all its different parameters in there it says look I expect you will be writing to this buffer that I that the annotation proceeds and I like I'll expect that there's a capacity for that buffer of buff size and that I will you will write to anywhere up to return which in this case is the return value of the function bytes into that buffer see it's easy as pie start reading them and after a while they just magic and natural but the good news is they do let the tools work out what went wrong now I will I will certainly admit that that warning message is anything less than easy to understand or read but just like the annotations if you've been reading them for a little while a couple of years maybe more then you can quickly see that what that annotation is telling you is the simple fact that payload length is not in any way related to rec lengths and so you're probably over reading the buffer which is in fact the problem so your annotations they help us catch the bug so now let's go look at the callsign what can annotations do here well we probably could now catch the read buffer size problem we could probably say hmm that's not the right length probably but remember when I said in reads just says that many buff that many bytes has been allocated and they're initialized to some valid value well that's what a victor does right so read buffer dot size still meets the criterion but the annotations on a bad packet which should be cool to make packet they're set out originally so this illustrates one of the problems with annotations is you think you're being precise but maybe not precise enough maybe not carrying enough information about being caught so how could we improve the situation and catch the caller involved in the whole heartbleed disaster here as well more annotations of course because they helped us before so having more of them can only help us more right and if you rinse and repeat and do this long enough and company I work for did then you end up with millions of annotations in your codebase just like this and they are really helpful they do help you catch bugs so in this case you know your socket read now says hey if it's less than zero then it failed otherwise it's going to write to this buffer this many bytes and the buffer has to be big enough to hold them it's going to be buff size and it's going to write somewhere between return and buff size bytes sorry some way up to return bytes which is less than buff size see they're really easy they just roll off the tongue and you can annotate right as well so now we can go find that bug we can issue a message it's still a little bit dodgy because we're still not a hundred percent sure that read initialize that data in any different way from Victor's default initialization of the data would have done but we can give people a warning with some confidence that something's going wrong here so annotations made things better so that's a fairly subtle bug it's a significant bug that cost a lot of people a lot of money and we can find it now with annotations annotations are good we did make the behavior of the functions much clearer assuming you know how to translate those annotations into some sort of mental model and you've spent a couple of years sort of loading all of that into your brain however you notice they were viral we put them on one function and suddenly we needed to put them on the next set of functions that called that function and you need to put them on the functions the function calls as I say you suddenly end up with millions or tens of millions of annotations everywhere and they're not really the source code so do people update them when they're maintaining the system they're like a language extension really and they're non portable as a result it makes it harder to exchange code other people and they tend to rot people tend to write meditation that's right once and then they leave it you know because if they're not getting a warning from a tool why would they change it so this is how we feel after do some annotations about our example me yeah we're not sad anymore but we're not super happy can we do better I think so so sorry I've moved my move my warning around the place there on the slide and what's happened thanks PowerPoint must have a bug probably needs my tools so in baby blue again we've moved on from annotations we threw them all away and change the return type while we were there because it seemed easier to do so and we decided to use some of the types that we're promoting in the GSL now there's one type that's not the geocell there today array buffer so type we've talked about while we were working on the geocell and we just haven't baked it enough and don't think it's quite general enough yet to pop in there I'm going to keep it here in this example because I think it just illustrates the more general idea which is types can replace annotations right and they're more useful when you do you can throw away your annotations note that I have barely changed the actual ugly code inside this function okay so buffer for ease of reference think about it it's just an array view with an extra size T unit an extra an extra integer in it that's saying okay just like a ray view I point to somebody else's data just like a ray do I know the data has length I also record another integer which is how much of it you've written to or read from depending on how you want to use that buffer we call that the used length and it's pretty useful in an example like this because you can look at the length and that tells you well am i big enough to take the payload that I want to write it do it yes I am good and then you can use it and you can write into the buffer and you can set its length at all why is that missing yeah so you can and you can set its length and you can return setting its length appears to be missing from this slide for reasons I don't fully understand but the point is we're using buffer data and buffer length down an arm M copy call so we can use these types with minimal annotation but the types a review and array buffer carry the same information as the previous annotations do an array of you we know has a size and length but now they're tied together we can know this type and we can know in the tools that it has those same properties as we had FranNet ations similarly an array buffer has a size a length and are used just like we saw in the outright sanitation so we can understand how those things tie together makes it much simpler for the tools to look through this code and give us the same warning as we had before but it's better because these types let us rewrite the code to become much simpler code so these types let us say that received buffer let's see if we can shrink it down as soon as we get it to match the length that the first few bytes told us it hare and if it can't it's gonna die it's gonna fail fast that's what that first function will do just like all the operations on a ray do its range checked so the first thing we do when we get that array view in that we're trying to read from is we say okay first three can I read the first three bytes well link that says yes good so I read the first few bytes I work out how long the payload is and then I just shrink down that array do I shrink it by taking the first n bytes of it if you like from here on now the problem is it fails you fail fast whether that's a good or a bad thing is up to you you could put any if test in there first you could if test the payload length against if against against rec dot length I chose not to because I was sort of running out of slide space but the point is whether you do the if test or not this software will fail when the heartbleed bug tries to manifest itself so the software's safer as well as easier to check the tools know this and the tools can leverage this we can issue a warning at that line saying oh I see you're calling first without testing the value you're passing it against length that could fail at runtime okay you might want to check that first in case you want to handle the error more gracefully than just failing dead and the programmer can make a decision there so the beauty of this is we were able to simplify the implementation code we're able to make it safer and the tools can continue to do the same good job they did before now this may not be so exciting if you don't currently have a code base filled with annotations that made it checkable by static analysis tools earlier but if you do this is a big leap forward because suddenly you can use these vocabulary types that are common and well-defined and portable and that you can exchange with other people you can get the same quality static analysis checking you got before and you can prevent bugs so let's see what happens in the caller we don't even need to change the caller if we don't want to so we can encrypt we can incrementally adopt these things right we don't have to go and change the annotations on every function in the system all at once I think over time we probably should because the types are clearly better than the annotations after all they're real sort of exact executable code they're not some macro that just sits on top and gets looked at by analysis tools they're real code that gets compiled people use them so they keep them in sync with what the software design is they're maintained over time but if you can't go modify your legacy os socket read now I stopped it right the tools can still do the same detection they did before you're calling code becomes slightly simpler to write because you simply construct an array buffer from your write buffer packet the read buffer there you can pass directly in because array view constructs straight off a vector it knows what the size of the vector is and it knows how to get the data vectors data so array you just constructs with the same simple a set of arguments as it did before and you can just call your same old function so we're much more compatible we don't need the annotations but we can work with them and these types don't have to be viral until you want them to and I think the point is you want them to because they carry this extra information so let's compare them to annotations so just like with annotation suddenly we can find these subtle defects we couldn't find before we've made our intentions clearer by using the types we've bound together the pointer and the length to make it clearer that these two things are related and we can use them to reduce false positives but the good thing is these types are also checked by the compiler so the type checker is doing some of the work for us and in fact some things you can't write anymore when you use the new times and so a good example of that was resizing that rereceive buffer to be the wrong size you can write it but it's never gonna run some of them you can't even write if you try and construct it from a buffer and say oh I'll use buffer data some random number that's bigger Arabia will just say I can't compile if you're going to give me a container that I recognize and a size that's not from that containers size I can't do that if you construct one from a static array or if a rave you can deduce the size of the static array and you're trying to tell it oh no no no use a bigger size and that static array world won't compile so some defects are no longer possible when we use these types as we said they're not viral you can preserve your legacy code and api's as you slowly move through the process of updating they are source code so they are kept up to date they're not language extension so they're portable and most importantly our tools don't have to interpret what they mean the way we used to with annotations what does in reads mean does every tool think it means the same thing well it better because otherwise we get really confusing messages for the users the semantics of a set of vocabulary types like the GSL are precisely defined the tool needs to know them but it needs to know them once so we made it to ring the happy place by using types so now I'm going to go uncor pura time going to break out from our value proposition and say there's a manifesto that we could have here which is we should try and prevent defects from being constructed rather than just find them during construction that's what the first half of the talk was about oh I can find your bug I can look at your old C code and find your Park what we want to do is stop you writing the bugs as you go because the cheapest bug to fix is the one you don't write at all and so we get two new bullet points here if we write less bugs we make you more productive if we add more information to your programs with types about what you're doing we make everybody more productive we improve your programs and make them clearer when you say I'm using an array view instead of passing a pointer and some integer somewhere in the parameter list you're saying this is a contiguous sequence of elements I intend to read from it's a much clearer statement so how does this relate to the the core guidelines that have been talked about this week so that effort shares these same principles to things sort of go together so a lot of the checks I've talked about a broader than that original you know they're they're talking about are maybe this is about a buffer overflow maybe this is a logic error right so taking us through the historical perspective but the core guidelines if it has the same principles underlying it we want clearer intent in programs we want more effective static analysis results and we're going to try and use types to do that to make your programs contain more information both to the reader and both the toolings to make them safer so I'm going to talk in the last few minutes here about CPP korchek which is just a name I've invented for the purpose of this slide or of the two and that's the name of the tooling that we're going to ship sometime in the next month that will contain some of these initial checkers for the CPP core guidelines and particularly for some of the profile rules that are identified in those guidelines so my team built to checkers really there's one still under construction waiting for the bounds profile one for the types profile so these were all mentioned in herbes talks hopefully they're familiar and the 3rd one that's under construction there is the lifetime profile which we demonstrated on stage yesterday so we build these things using a framework that we've developed over the years for doing static analysis against individual functions and that framework mean we could write downs and types and they're about 600 lines of C++ against a framework so they're super small and super simple checks and it's worth I don't know how many of you have but I encourage you to go look at the guidelines look at those profiles in particular they are very simple checks you can tell that from reading them that the checks simply the rules simply consist of you know don't make this sort of cast don't do pointer arithmetic these are very simple things to check very powerful once you adopt them let's talk about the framework we used to build these things for a moment so we build our own framework in C++ greatest language in the world bla bla bla it's evolved over the years and we found it very useful it was very useful in writing these checks so it's somewhat portable C++ becoming more portable by the day it is certainly compiler agnostic there's a little layer where we talk to the compiler where we want to get past hires trees and symbol information don't working yeah you switch this Mike I think is thank you perfect there you go one microphone became bored alright so we're trying we try to become Pilar eggnogs agnostic because unsurprisingly the company's large as Microsoft we've been through more than one iteration of C++ compilers in our time so we've had to build a intermediate representation for doing analysis so that's something like those of you familiar with compilers it's a bit like a SSA type form a CFG where you can get a control flow graph built for you off of the compilers internals and those representations the control flow graph the types the symbols all that sort of information were built to be agnostic to the actual compiler front-end that generates them for us we've got you know parts in the framework that let us report warnings and maybe potential fixes we know how to have things suppressed and how to one of those suppressions and tell you in expressions are broken and we have a single consistent output format and this is my chance to plug Seraph which is a attempt to provide a sort of a standard format from all our static analysis tools at Microsoft it takes ml based because it's an output format and everybody loves XML right so easy to read so we've got this framework it's got two levels of analysis it supports for local function so one is what I was talking about before which is simple checks simple fast scalable checks you can look at just the trees and work out it's somebody calling memset what are the what are the parameters do they look dodgy okay I'll report a warning you can also do very simple flow analysis you can get a flow graph you can walk the flow graph you can record your own values you can make your decisions about whether or not you want to report on something like dead code as we showed an earlier example there's a second level of API and I should just add that first level of API is all you need to implement any of the checks that we've defined so far in the CPP core guidelines they're all very simple flow sensitive but that's it checks even lifetimes so that simple base API is more than enough to get you a long way in terms of value there is a second level of API which does path sensitive walking through a function so it explores all the potential paths does full abstract execution it's got a memory model and it's super sophisticated as a result it produces more false positives and it takes longer to run so you don't need that sort of sophistication to be able to do nice deterministic checks like we want for the core guidelines but it's still a simple a useful thing to have in your bag of tricks if you're developing a static analysis checks comes so you write against a framework and then we get an execution engine that's what we're gonna be bundling next month with these two tools and that execution engine does all the hard work and doesn't let shared work I talked about before that helps you scale when you're running this thing all the time over a large code base so it does a shared construction of this intermediate representation of the program it's a mutable tool the clients it knows how to do all the warning reporting we talked about before and if you're doing a flow sensitive analysis it gives you some standard ways to traverse that graph and ask questions about it report if you're doing the path sensitive analysis and that engine does all the hard work for you it does an abstract interpretation of the program it maintains all the values that thinks it's seen valuates expressions and it's got path feasibility magic to it and constraint evaluation it does a whole lot of work for you and I like things that do work for me and it lets you track your state along as it does its evaluation ask questions make decisions issue warnings it's a fairly simple model to use but it's got a lot of power behind it and so we plan to progressively make share this I guess with the broader community is open source so we're gonna start with that first level and with the implementation of those cpp gawd core guideline checks that we're releasing next month in the months following we will start to open source this stuff I work at a very large company a large company comes with a very large cohort of lawyers turns out that any time you want to make anything available to the community you have to talk to the large cohort of lawyers and lawyers like to talk I guess its billable hours right so we're not finished the talking with the lawyers this stuff will start to trickle out and we'll share it with you and I encourage anybody who's interested to pick it up look at it improve it make it better just as I encourage people to do the same thing for the GSL and for the core guidelines and I also encourage people who are interested to help us make it more portable get it ported to other compilers if you think it's valuable if it has used to you pick it up use it make it better spread it round because as we saw in that originals a couple of slides with Lynde Vinson a new proposition is we all use it we write better software we go home earlier and we have better weekends which I don't know you that it's certainly my aim so join us is what I'm saying let's make everybody's in a loop better even if you just run the tools even if you help even if you run other people's tools I don't care join us and give people feedback on how to make things better I've included in the deck a bunch of these resources so Microsoft github io code analysis thank you very much Andrew for getting that organized at very short notice that's the sort of landing site we can find a bunch of pointers in information to what we are doing at Microsoft with static analysis and if you watched that space that's where more information about these checkers and the project that underlies them will become available over time I'll put a link on there to the seraphs standard which is the output format we use in case you'd be interested in either outputting it yourself from your own tools or consuming it to be able to do all sorts of interesting transformations on people's bugs there's a link to code analysis in vs because that's what pays my bills and there's a link to claim Tidy there as an example of something it doesn't pay my bills but is also a very useful and sophisticated static analysis tool that you should be using if you compile with claim so that's all I have I have five minutes remaining so if anybody has any questions I'm more than happy to answer more attempt to good sir so it seems like when you said use types instead of annotations that seems really straightforward that seems like it's a great question I'll just repeat it because that's what people will hold up a card and tell me to do I just do what I'm told right and the question was types rather than annotations seems blatantly obvious well why didn't we just go straight there why did we bother going to annotations in the first place there's a couple of good reasons one is changing types is changing the code that people generate and when we started using annotations a long time ago there was a lot of nervousness back then about C++ can it be performant can you really wrap things in a class and still get the same sort of performance as the R or C when I'm passing an integer and a pointer it's not possible this C++ thing will never take off believe it oh there was a lot of people who felt that where there's a SIL appear to be a lot of people who feel that way and they're wrong we all know they're wrong that's why we're in this room right there's a second reason and one was speed of adoption so you know without going into too much detail corporate history but I think it's true of almost everybody who existed 10 15 20 years ago there was a period of time where suddenly we realized that there were people who were not so nice out there in the world who wanted to attack the software that we make and people run and so on and when you realize that you suddenly and you've got millions of lines of code that are there software you've got something you need to do in a hurry changing all of the code that's being generated for that is a fairly risky project so annotations seem like a safe thing to do because they're just macros right they don't contribute to cogent you can put them on and not disturb the code base in a possibly harmful way but you can still get the analysis checking so part of its historical as well great question anybody else okay so it's a great question about interprocedural analysis and if I summarize it quickly you're saying that there are some linked timecode gen scenarios are very similar or do actually do whole program optimization it's the same sort of analysis some people have like an in-memory server to try and get around the big hard lots of work part of the problem and did we consider that yes so there there's lots of different into procedural efforts we've looked at over the years and you know we've got a research group are always looking at more of them but the main interprocedural engine we've used at Microsoft over the years called prefix is an extremely precise engine and it simply blows out memory on any machine like it blows out storage and it blows out execution time on anything that is like reasonably realistic commercial code sizes that we deal with so you know to give you some sort of an idea of the scale of the thing you know we were excited when we did a whole lot of work a year or two maybe more two or three years now back to really optimize it so that instead of taking two weeks to run it would take two days and that was a big effort to get it to that point so even if you ran it as a server you're looking at a really long sort of turnaround but you know there's lots of gray areas I mean I in this talk I do talk about local and global as extremes there's lots of positions in between where you can sort of turn the dial but yeah it's a one-hour talk somebody at the back yep okay so the question is yes if you have a bunch of developers who know they're smarter than the analysis tools and just want to silence them how do you operate in that environment I think if you're going to adopt these tools in a in a hostile environment if you want to call it that a great approach is the incremental adoption approach so you need your tooling to support being able to be turned on for any parts of the codebase or being able to baseline an existing set of warnings sort of like a hold the line position and say just tell me about new things the experience we've had and there's plenty of hostile developers to tools everywhere the experience we've had is you need to show people the value and so that goes back to that scaling proposition when you up thousands of engineers you have a large percentage of them who know better than the tools and so and each other and and so the best thing to do is you try and keep the noise very very low to begin with show people the real value and in my experience the day you help somebody find the bug they were just debugging or that just shipped and is causing them deep embarrassment is the day they stop being so critical of the tools and only mildly critical of the tools in the future and my session is over thank you very much

Channel: CppCon

Views: 12,271

Keywords: Static Program Analysis, C (Programming Language), Programming Language (Software Genre), Neil MacIntosh, CppCon 2015, Computer Science (Field), Bash Films, Conference Video Recording, Event Video Recording, Video Conferencing, Video Services

Id: rKlHvAw1z50

Length: 60min 38sec (3638 seconds)

Published: Sat Oct 17 2015