R-Evolution: A Story of Rust Adoption - Ryan Levick

all right so today I'm going to be talking about a revolution or an evolution at the company that I work for and this is a story of adopting rust at that company and so you all know me see me before I'm Ryan Levesque that's me right there and my co-presenter is Sebastian Fernandes he lives in London or actually doesn't live in London he lives outside of London and so he's not here tonight but he will be in Barcelona so if you're there say hello to him and basically we the story here today is about the company that we work at Microsoft and our slow and sometimes extremely painful journey into adopting the RUS programming language I mean hopefully after tonight you get an idea of what that journey has looked like so far maybe some idea of where it might be going in the future and then the interesting problems that adopting a programming language at any company let alone a huge engineering company like Microsoft and why adopting languages in general can be quite difficult but why adopting rust actually has made a lot of sense for us at Microsoft so to begin with that the first thing we can talk about is the problem the problem that Microsoft as a company was facing and ultimately where rust potentially could step in and help us with that problem so really what it comes down to is that there is a a problem in our industry some might say the billion dollar problem and no it's not no pointers although that has something to do with it actually and that billion dollar problem obviously affects Microsoft is a company quite largely because it's affecting everybody in the industry you can be sure that it's affecting one of the biggest companies in that industry and to be blunt about it the problem really is the language C++ and this well of course that's going to maybe shock some people in the room and we don't mean to say that if you use C++ you're stupid because that's certainly not the case C++ is an extremely powerful language it's also an extremely important language in the industry and has got us to the point that we're in for better or for worse and of course Microsoft is also has a huge hand in C++ we have a compiler so for for any troubles that C++ has introduced to the industry we're definitely partly to blame for that for sure and specifically what the the issue that C++ has is that it is not memory safe and what that means is that you can cause all kinds of issues quite easily in C++ by messing with memory in ways that are not sound and this might be common problems that you've heard of before like use after free or double free or something like that and ultimately what this leads to is not only buggy software that doesn't work right but also insecure software or software that others can exploit in order to take control over a computer that they shouldn't have control over and do nasty horrible things with it and here is a graph that we have produced and this is produced by a team at Microsoft called the the Microsoft Security Response Center and the Microsoft Security Response Center or nsrc which I'll probably refer to it from now on as is responsible for mitigating finding all the security related bugs in the software at Microsoft and going to the teams that are in charge of that code and getting them to fix it and that includes also spending a lot money to have people report the bugs to them instead of using those bugs for nasty nasty reasons and this graph with the show's over here is basically memory safety issues or many memory voter abilities security vulnerabilities by year and the good news is is that it's not going up the bad news is is that it's not going down so it's been at 70% basically since 2006 when this graph started and probably long before that as well and it has remained constant at 70% basically this entire time and we'll be talking later about ways that Microsoft has invested a ton of money in trying to get this number to go down and while that has been somewhat successful in some ways as you can see it has not been successful and actually reducing the number of CVEs or memory vulnerabilities that we see year-over-year and so of course you know Microsoft is a company and so we always ask ourselves well who cares this is actually costing us any money or costing our customers any money and the answer is absolutely it costs a lot of money in fact here is one figure a hundred thousand dollars but wait this figures from 2005 and at the time where when they reported this it was purposefully conservative so there are plenty of people who believe that this figure probably is now over a million and this is per issue found and there are several issues found per year certain issues can cost a lot upwards of a billion so we have in 2005 when this first report came out there was one issue that came out where the cost to Microsoft and to the customers where this where this vulnerability was being exploited was estimated to be over a billion u.s. dollars and this is one issue so if we have let's say it's a good year and have two or three that is a possibility of billions and billions of dollars being lost and of course this doesn't take into account like the pain and suffering that the engineers who had to fix this were or who knows maybe it was a bank system or something like that and people lost savings or something like that the these are cysts of critical systems that people are relying on and sometimes you can't even estimate what the actual damage is when they're exploited so this is not a trivial problem it's not a problem that we can just shrug off this is a problem that in some ways can be an existential threat to to us as a company and also to us as an industry everybody who works as a software engineer in the room so naturally we come to the like the easy question of how do we fix this this is not a good thing we should work on actually addressing this issue and there's been many many ideas of how to do this over the years and we'll go through some of them so the first bright idea that we've had was we need better programmers of course these dumb programmers keep on introducing these security vulnerabilities and this is what you see on hacker news all the time when when people talk about exploits they say oh I mean of course some junior engineer wrote this exploit if I were working in this company this would never happen I'm way too good of an engineer to introduce these issues and we as as Microsoft and as the Microsoft Security Response Center are here to say is that is not true there is literally not enough training that can be done to solve these issues even if you were able to hire very very skilled engineers only and give them enough time to write code as they expect to write code they will still make mistakes now let alone there's not enough engineers out there that are highly trained and C and C++ and you have deadlines and things of that nature these mistakes will happen unless you make them not possible okay okay that's fine we can't have better programmers but how about another idea we need better static analysis tools we need more tools to say when a programmer who's writing C or C++ actually makes a mistake and unfortunately while this does work to a certain extent and has actually reduced the the cost of a lot of mitigations that we've had before it still is not completely soundproof and as we saw before 70 percent vulnerabilities year-over-year this obviously does not fix the core of the problem and so you can throw a lot of static analysis tools at C and C++ it will find some issues but there are plenty of issues that cannot be found because basically if you had a static analysis tool good enough to find all these things it would be baked into the compiler in the first place so so then we come to our last and the probably the only good idea for actually addressing this and that is we need to make these issues impossible to actually produce we need to make it so that they just don't happen in the first place at all and there's one known solution for this that's been around for a very long time since the 1950s that addresses a large swath of these problems and just gets rid of them completely and that's garbage collection so thank you very much that was my talk we should all use garbage okay of course not obviously we've still got a few more minutes to go so and we're at a rust meetup here so garbage collection has its limitations right garbage collection is is slow at times garbage collection is hard to predict and if you're writing certain systems like operating systems databases browser engines things like that you can't afford this the the unknowns that come with the garbage collection you need a language that is more or less completely predictable ahead of time and this is why I C and C++ continue to exist today they continue to exist because the programmer can be more or less sure of what's happening with their program ahead of time modulo a whole bunch of things like caching and stuff like that but it makes the job of understanding the system a lot easier when you can get rid of automatic memory management through garbage collection but of course that comes with the issue that we've already been talking about where you have all these memory safety issues that get get introduced because of it and beyond this garbage collection doesn't solve all kinds of security issues there's still other issues that are possible even with garbage collection namely around data races and things like that and so Microsoft as a company long before I joined and long before my colleagues joined has been sitting around and wondering what can we do about this and there's been research on it there's been new programming languages developed internally but nothing has quite worked out but we think that we found a language that potentially could get us at least quite far into our journey of trying to get that 70% number reduced down and that language is rust and of course here what I'm preaching to the choir right you're at a rust meetup you've probably heard about a lot of these things before you understand probably that rust is a memory safe systems programming language it can do a lot of things if C++ can do but it can do it in a way that is memory safe so you can't have these exploits so that's great you're all more or less sold here right so we're gonna skip over that part and we're gonna get to the actual hard part of it which is not convincing people that that rust is a great language that should be at least looked at the real issue is the issue of adoption it's very different to go from rusts is a cool language that I like to use on the weekend too rust is a language that powers one of the most used operating systems in the entire world or rust is a language that powers our billion-dollar business these are questions that have to be answered over and over again with many different people so before we get into that we should talk a little bit about how languages get adopted and a problem that doesn't really get talked about when we talk about new languages and the way that you can think of it most clearly I think is this right here you have costs and can anybody read what it says down below yeah it's really hard right those are the benefits and you wouldn't notice but the benefits are much bigger than the costs but you can't read them you can't see them so what are you gonna do and that's really the fundamental issue that we have here you have cost-benefit analysis you can determine do the benefits of adopting this language exceed the costs that come with it but the problem is that the costs that come with adopting a new language are very easy to measure and very easy to understand you have to hire new people you have to bring that that language into your build system you have to interrupt that language with existing code you have to make sure that existing tooling works with that language these are very easy to say how long will it take us to get this language to to fill to to check off all those things but the benefits of adopting a language might not be so clear if you're adopting for instance let's say Elm which is a language that I really love to use inside of the browser and you want to use that over JavaScript now if you're an element easiest you might say that I'm much more productive than I am in JavaScript I write fewer bugs but these are really hard to actually quantify and so oftentimes what happens is you end up talking anecdotally talking about how you feel when you use the language which I know as a programmer these things really matter they're important but when you're talking to somebody who runs a business and who has to make kind of a reel at the end of the day dollar and cents actual decision about these things frankly and unfortunately they don't really care and you know what I can understand that right you can't really transport your feelings of how this language makes you feel when you write code to this business person who is making the decision and that's fair enough okay but this is what makes the rest more adoptable actually is that the problem that we saw before of memory memory vulnerabilities and things like that is a problem that rusts addresses we've already seen that we have kind of dollar and cents figures attached to this so if you can go into a company and say we're going to get rid of a thing that it's costing you and your customers billions and billions of dollars a year that's a pretty easy argument to make and because of that you end up with a picture more like that where the benefit is clear and it's much bigger than the costs are and that's roughly where we find ourselves today with rust at Microsoft although we're still at the very early parts of our journey so the conjecture we have is four security critical software C++ is no longer acceptable and the bet that we have is that rust allows us to write performance security critical components safely so then we're done right but of course what this means is not rewriting the world we have a whole bunch of things that we need to pay attention to as we go along if we started today trying to rewrite every single line of C++ that exists it inside of Microsoft we would probably be done in 20 years and of course in that time we're going to go out of business right so let's talk real quick about some of the challenges that that you're faced at this scale when you're when you want to adopt a new language the first one of them is build systems cargo for instance is a great tool that when you're using cargo when you're using rusts exclusively in a project or your you're a hobbyist who is writing their first project in rust cargo is amazing and actually is one of the best things about rust you get a built-in build system testing system right out of the box but if you have a build system that's thirty years old and is meant to build an entire swath of C++ code along with other languages bringing that bringing cargo into that system doesn't normally work and that's an issue that that we have to address the next one is tooling we have a whole bunch of tooling that is meant to address specific things inside of visual C++ code that might work with LLVM might work with rust but sometimes it doesn't and it costs time and money obviously to get those things to work and the next one is is interrupts you you have existing code you want to take that rest code and basically embed it inside of your existing code is your rest code able to even talk to your existing code what does that look like if it's a flat C API that you're talking to it's pretty straightforward pretty easy albeit it's unsafe and you have to worry about how are you going to mitigate your safety issues there you might end up in the same situation you're trying to get away from but if it's C++ it's much harder how do you in interact with a heavily templatized C++ class from rust these are questions that are not really answered and then of course there's they're just kind of weird and strange things that happen when you're working at a large company probably nobody in this room unless you work for Microsoft or Google have to worry about building the RUS compiler from scratch and when I mean from scratch I don't mean from source I mean literally proving from the beginning of computing that there is a safe and secure way to build up the rest compiler what does that look like that might mean trying to build rusts using C++ and there's the the Umrah c project that is a russ compiler built in C++ and if we have a trusted C++ toolchain you can build that that the Russ compiler with that trusted tool chain and then take it up to the latest version there have also been talks of building the original Russ compiler that was built in what was it Oh camel back in the day building that and then following all the way from 2009 or whatever all the way up until today because you have to go to the EU or the US government and say hey we've we built this thing and we can promise you that nobody has injected malicious code into it and that's that's something that you don't normally need to worry about but when you're adopting a new language at the scale you have to and that brings me to the the next and final point and that's adoption from the other perspective so yes I work at Microsoft but I also am an avid rust user and an enthusiast and somebody who enjoys being a part of this community and I don't want the adoption of rust at Microsoft to ruin what makes rust great today and what makes rust great today is things that we've talked about before with cargo a language that is user friendly that even when you're bashing your head against the wall it's still holding your hand there and trying to give you friendly error messages right so what are some of the challenges that we face right we face a challenge for instance of governance there's a real challenge of governance if you have large companies getting involved in here they probably want some sibilance of stability in the language and what does that mean well the temptation for some large companies might be to grab on and try to take control but that would in some ways kind of ruin what we've built up as the rest community an open community community that collaborates together we don't want large companies to come in at the exclusion of everybody else and then you have language changes we just talked about how at Microsoft would probably need to build the rest compiler from C++ or oh camel to do that we have these strange and esoteric needs that other people other users of Russ don't have and there might be a temptation to add changes to the language that would make our lives as as users of rust and Microsoft easier at the expense of other users of language and while the language might become better for us as a large user it could potentially come become worse for other users of the language and that's something we don't want the language is useful now because the community has carefully put the language together that way and bringing in an outside influence that doesn't have that context can frankly be dangerous and one of the last ones is what happens to the community when you have a whole bunch of what I call reluctant row stations coming and people that are using the language not because they love it not because it's fun not because they do it as a hobby in their free time or they enjoy working on the compiler or an open source library but because their boss told them to and they'd much rather be using some other language and they have to use this stupid language with a crab why how does that change our community when we have people who are are less willing to play along with the rules of the community how do we make sure as the community that we can safeguard what we've built up over time and continue and make sure that it continues to be what it has been in the past so that so there are challenges on both sides challenges to Microsoft as a large company trying to adopt a new language that can really have clear benefit and also challenge to us as as the rest community how do we make sure that we build up and continue to foster a community that plays well with these these software giants because we want them to be able to use it we want them to be able to contribute back we want them to be able to to participate in the community but we also want to make sure that with that power they don't come through and just ruin everything for us and so that's the challenge that we have ahead of us today if you're interested in this kind of challenge and what adopting rusts and rust actually be being used in large companies actually looks like we'd be happy to talk to you about it so thank you very much [Applause] [Music]

Channel: Rust

Views: 13,381

Rating: 4.9447002 out of 5

Keywords: rust-lang, rust, rustlang

Id: qCB19DRw_60

Length: 24min 55sec (1495 seconds)

Published: Fri Nov 15 2019