Ashley Williams - How I Convinced the World's Largest Package Manager to Use Rust, and So Can You!

all right so you may notice that I changed the title of my talk this is because I'm a terrible speaker and you should never invite me to your conference ever so I will just do this random stuff I'll just change it up but the gist of the talk is the same but like I think a lot of things I figured out a lot more about what I wanted to say when I actually wrote it which was a lot this morning and last night and before that too I promise I promise but I'm calling it then in the art of convincing your company to use rust because while I will be talking about trying to get your company to use rust and giving you some tips along the way it's less going to be about convincing and a little bit more about some Zen disclosure I don't really know that much about then and I'm probably the least then person out there well we'll see how it goes so my name is Ashley Williams you may know me as a g-dubs on Twitter I don't recommend you follow me but if you do I'm sorry I've been programming for a little while now I primarily write the language JavaScript and I usually write it on the server which means that I use nodejs in fact I'm so into nodejs that I actually sit on the board of directors for the node Foundation so I'm pretty pretty sold in on Don JavaScript but that being said I've written a whole bunch of different languages I've written rust I've written Erlang of written Ruby in general the thing is is I really really love programming languages and the reason I love programming languages is because I like to think about thinking and in particular the type of thinking that happens when people write code today is going to be a little different instead we're going to talk about the thinking that happens when you try and convince somebody else to write code just a little bit differently and by focusing on that thinking hoping we can hope we can be a little bit more successful in convincing them to try something new so I work at this little company called NPM which stands for nearly perfect machine ok that's not true at all like everyone's software our code is not anywhere near perfect but actually NPM stands for node package manager some people might tell you it that's not what it stands for but it is but we're basically the package manager for JavaScript kind of like your cargo as you might understand in fact cargo was at least in part inspired by and also a maybe negative reaction to NPM which is kind of interesting but we often describe ourselves as a package manager for JavaScript but people put all sorts of things into NPM packages from CSS to shell scripts to also rust and not only do people put tons of things in packages but they make lots of packages in fact as of this morning there were four hundred and seventy two thousand and five packages in the NPM registry which makes it by an order of at least three the largest package registry in the world now I'm saying the world because I don't know if aliens have package registries and theirs might be bigger I have no idea don't want to get too ambitious here but what am i doing all this talk about MPM right so you might be surprised at hearing that something like NPM which is really a JavaScript and node developer tool written by JavaScript and node developers knowing that you might be really surprised to have seen this PR come into the rustling web site last December I may just put our last December to add MPM to the rust friends page which is a friend a page on the website which lists companies that use rust in production so that means as of December last year all right MTM uses rust in production as part of the registry services so if you've ever typed NPM install there's a relatively good chance that at least somewhere along the way you got kid run a little bit of rust so the reason we're here today is you're like oh my gosh you did it right you got your company to write some rust how can I get my company to write some rust and I'm going to tell you right now that this answer to this is this as every good answer you'll get for any technical question which is it depends and I can't tell you for sure I'm sorry um but that being said there's this book called art zen and the art of motorcycle maintenance which is the namesake of this talk and I'll be quoting it a little bit throughout here but what they say is this and I think it really fits with asking about how to convince your company to use rust which is the number of rational hypotheses that can explain any given phenomena is infinite so if we say okay what are the reasons a company will use rust well if I think hard enough I could probably come up with a thousand answers and I could probably give a pretty interesting talk about that that being said what I'm going to do instead is I'm going to give you ten tips today that are kind of little stories about my journey with MPM and rust and I can't guarantee you that you'll be able to use these to convince your company to use rust but I do think that they will make each and every one of you a better relation and a better rust evangelist in general so with that being said let's get started the very first tip is to not rewrite your software and rust my job done CL just kidding okay so right oh you're like oh this talks going weird what I mean to say is don't completely rewrite your software and rust as someone who loves rust you might say yeah we've got a rewrite in rust rust is definitely the best language and I'm here to tell you not to do this because complete rewrites are very often unsuccessful and nothing sours the taste of the technology more than the memory of your own company's failure so before you think about doing a major rewrite and rust think twice because you don't want their first experience to be a complete failure in writing this so what do you do if you want to use rust you have this big complex system but you shouldn't rewrite it because you know complex realize are not a very good idea the way I like to think about this is so articulated using this amazing pediatricians law it's called galls law and so he loved studying systems and in particular he liked studying how systems fail and so this is what he says he said the complex system that works is invariably found to have evolved from a simple system that worked a conflict system designed from scratch never works and cannot be patched up to make it work you have to start over with a simple system so if you want to bring rust into your company there's a good chance that maybe you'll be writing a brand new feature and you'll get that like really awesome blank screen just like I will write this from scratch this is great chances are though that you are not going to have that option instead your company is going to be like we have a huge mess and we need to clean it up and we need to add more stuff can be like oh no how do you do this all right if you have a complex system you want to write it in a new language you don't want to design it from scratch because it's not going to work what do you do so the one of the very first engineers at NPM who is now our CTO her name is CJ Silverio she gives a presentation called cheating calls law and so this is basically how the law works and so it helps that we are a web based service if you are not your mileage may vary but this works out pretty well so what you do is you have your monolith or your complex system all right what you need to do is define a very specific interface put a proxy in front of your system and then break off a simple system so in this sense what you're doing is you're not rewriting the complex system you're just splitting off a little tiny piece and then you can start again from that simple system you can write a simple system and this is the architecture which is afforded MPM the ability to do a lot of experimentation and that includes adding rust so what you might see here all right you might go OK this is kind of like a classic microservices thing am i here to give a microservices start today no I'm not here to give a microservices talk today there's enough people in hacker news that are really excited about it go find one of them in fact my my opinion of micro services is that they are a useful tool but remember once you have a complex system and then you distribute it you have an even more complex system it gets more complex you don't get less complex you just end up with an agility to be able to try things out by having little systems that you change only one time but the nice thing about this is that if you don't try to do a complete rewrite if you're able to identify in your system some sort of generality something with a really concrete interface you're able to like maintain that interface and then write something in rust below it because at this point you've made it so that the implementation doesn't matter this might seem counterintuitive but when you want to try and convince your company to use rust you should make it so the fact that it's written in rust doesn't matter because that makes it extremely low risk and therefore why would they say no why not give it a try and so speaking of give it a try this brings me to tip number two which is to show up with code now there's this thing in the industry at least that we say in the United States which is this thing called skunk works which I don't even know why it's called that it just it sounds bad and I don't like it the idea is not that you should like go secretly write your entire product and rust and then show up one day and be like here look at this because that's going to be terrible you have no Buy in skunk works doesn't work well what I am saying is show up with code rust is one of those things that for people who aren't in this room it seems really far away it doesn't even seem possible so even showing up with even a small bit of code that does something and especially if that something is even related to what your product does shows that it's possible shows that it's real for many people rust is this thing that they hear a lot about on the internet but isn't terribly concrete so showing up with code helps and so for the story I didn't really know I thought rust was cool but it didn't even occur to me to even show up with code or even introduce rust to NPM because NPM has no notice fast and in particular node has async and that's like the thing for note is we have async and so suddenly I remember I got this like secret just one day via Twitter and it was these docks for Tokyo and it was like we've read written a sink for us it's going to be super awesome and I was like oh this is very neat and the reason I thought this was neat is because currently at the time I was learning one of the key patterns that we have in the NPM registry which is called a registry follower or the follower pattern and so in general this is what it looks like so we have all sorts of user behaviors you add some collaborators you star a package you unpublish you publish a package all of these things go into CouchDB in fact at the very beginning of NPM all of NPM was inside CouchDB we have done that splitting the monolith have broken it out into tons of little services but for the most part this is what it looks like and this was a huge problem because putting everything inside of CouchDB and then having you know two million people use it made it catch fire like really fast very very much on fire it was the most on fire thing ever so we had to split things out and so what we started doing was we created followers that watched for these events coming into CouchDB and so what happens is these changes enter the registry and then the registry is able to emit a stream of events and then each one of these followers listens for that event and for each one of that event is able to do something and so this is one of the key patterns that we have in the MPM registry and I thought hey we have this new async thing I want to play with the new async thing I'm learning about this follower pattern the follower pattern is awesome because you don't have to work at NPM to write an NPM follower so I have been writing followers and node to like do things like count the bad words that people put in their read maze so it's like this could be immediately just kind of interesting I could play with it and I could build something that could make it so that other people in rust could play with all of this data too and so to do this I ended up writing this code called changes stream which is a crate it's not on crazy i/o yet because Tokyo is not yet inside of hyper which is the HTTP library that I use to do this hoping soon that that will happen but I was able to write this changes stream rust and one of the coolest things about this was why I was able to play with a sink too I was able to collaborate with a lot of people in the rest community and like find out how easy it is to get help and how awesome it is for people you can just walk into like IRC and be like this isn't working and I'll be like I really want to help you let's do this in fact I was able to make that MPM friends PR from Hawaii because at the Mozilla work week I got to pair program with the person on hyper who writes hyper to write this which was really really cool now one of my main goals was writing this wasn't to do it in the rustiest way ever in fact maybe everyone's looking at this code right now and being like this is terrible why would you do it this way our node streams so good that you want to emulate them with rust I don't know maybe well my goal is to show up with code to show that I could write something that was NPM like in rust and in fact this is the original node code for when I was writing a follower and you'll see that the interface is nearly identical so is it very rusty I don't know I don't really care the fact is I was able to be like hey we can do this in rust it's like possible and that's the whole goal of showing up with code so let's move on to my next tip so you'd think oh wow you did all that work you showed up with your code that's like really selling it hard and I'm here to tell you actually don't sell it too hard and I feel like this is important to tell the rest community because you all love to sell you really do and so I want to say don't when I was first at MPM they were looking for another language they were kind of in the market because we have a bunch of op stuff and we needed to write in something other than node we also we were just kind of bored and so they were interested in go and they were tricking go out now if I had shown up and I have been like oh my gosh go go is freaking terrible why on earth what do you possibly consider using go which I see people smirking because you're like I wouldn't say that but you would you would I seem to say it okay it's real and here's the trick if I had said that it wouldn't have worked well it wouldn't have worked really well at all because bashing other languages makes everybody look bad and so while it seems to be this like really intense competition for which language is the best that's not the angle that you should take and so I think that the rest community has done a relatively good job in doing this but again as you start looking into your company the goal is not to make other languages look bad but to remind people of how awesome and cool rust is literally by just not bashing other languages you can actually stand out in the language force it makes you exceptional now you're saying okay Ashley you said don't sell but you're saying don't bash with the languages bashes bashing isn't selling I'm better than that it's like okay the other thing I want to caution you is as someone who's very fanatical or excited about rust you probably have a lot to say and you know a lot about rust and you want to make sure the other person knows so you might be tempted to be very controlling of somebody else's first experience of rust you certainly don't want them to have a bad experience so what you might do is try and like be sure that they have a good one and what I want to say is this don't do this if the language is mature if the language is good the language is going to speak for itself you don't want your company to adopt a programming language because you were the one who controlled it all because in the end having a language inside of a company that's dependent on a single person is really bad and so maybe you see that as a type of job security but it's also like pretty terrible tactic so the other reason that you don't want to control someone else's first experience is that everyone is going to love everything about rust right away obviously they'll they'll come to it but the thing is is that the things that you really love about rust might not be the things that somebody else does and vice versa and this is pretty important so let me tell you a small story who here knows what this does all right how many people think this is a nice thing don't raise your hand anyways so my CTO was learning rust uh and she has lots of cool ideas we both kind of feel like style is just something that shouldn't get in your way in programming but it turns out when you're learning something new sometimes you get a little bit more defensive and you need things to be more comfortable and so c.j really didn't want to have the compiler yelling at her about style because she just wanted to learn rust and then she was going to deal with file later but she didn't want to deal with it and so she was like Ashley what the heck is this style situation what is going on here and I had just I've used rust format and I just like I don't care what it does I'll just do it it tells me because I didn't have an opinion um so I had to actually look up and find out what do you do to turn the warnings and the compiler off about style and basically you have to put a name tag on your code that says you're bad so this is obviously up to the people who designed the compiler but what I would say is I could have been like CJ you got to do it the rough way you got it don't use this warning this is an anti-pattern I could have been really staunch about it I was like that's not very rough like a view but instead I was like here use this also I'm sorry that it's so passive-aggressive because fundamentally you know what she ended up doing is she actually ended up writing a bunch of rust and if we hadn't totally hung up on this one little thing it never would have happened and she would have thought that rust was super terrible and not a fun experience so again remember that people are going to have different experiences than you coming in and you should give them space to have it because criticizing something doesn't necessarily mean that they don't like it it actually means that they're engaging so I've told you not to sell and then clearly what I'm going to do now is I'm going to tell you to sell so rust helps $0.05 I almost wrote free rust help five cents and I was like doesn't make any sense but anyways so one of the really neat things that has had been happening at NPM for a while even before I got there was there was a lot of ambient noise about NPM and it was because people would rust and a lot of people would just kind of share neat things that they saw about rust so instead of selling super hard and trying to control people's experiences a really useful tactic is to just kind of ambiently share things drop things in slack and so this started happening a bunch and then it wasn't just me dropping things in is a whole bunch of people to the point where and I didn't even know this was the thing but if you use slack you can use something called a react G which means that if you respond to a slack message with an emoji a specific one it'll auto generate a channel with all of the things that have that reaction on it so we clearly use the crab emoji so everything that gets responded to the crab emoji lands in our slack channel and this is really cool because it starts getting people excited about contributing to this conversation about rust we don't even have to be writing any rust we're just kind of having an ambient conversation about it and this is really cool because then instead of just being you talking about rust a bunch of other people will start saying things and I thought this was a fascinating comment because this was when CJ found out about the habitat project using rust and learned some from it which you all heard about earlier today and that was something that I didn't even know so not only are you sharing things but then that starts encouraging other people to share things about rust and you end up learning things too additionally I know we love to talk about Twitter as being a giant trash fire because it kind of is but it does turn out that your tweets matter and part of the reason I want to talk about this is so this is Carol nickels she is a co-author on the new rewrite of the rust book and she was tweeting about like oh are you still having trouble learning life times and if you're anything like a beginner and rough answer is definitely yes definitely still having trouble and so she was sharing about some work and wanting some help and while she may not have gotten a lot of responses on Twitter via this not a lot of replies what she didn't know is that inside NPM CJ was sharing this to try and get other people who are interested in rust inside NPM up to speed so even if you're not seeing the engagement that you'd want to with your tweets about rust you'll actually be surprised to find that people are taking those and bringing them inside their companies and having those conversations there all right so now we're on to tip number five and this is something that I love to say all the time if you came to Russ bridge yesterday you would have heard me say it and it's that the hard part of programming isn't programming it's true it's not in fact it's all of the things around programming it's how to get started with a project it's documentation so rust has amazing docs and single-handedly if I was hold it by the NPN people why they ended up choosing rust over go it was because of the documentation there because there's documentation here and I can find it was a combination of having the API reference and also the narrative Docs alongside that to help that out but what I would say is like while they cited that the docs were really important I think that the tooling also helps M p.m. is a package manager so we clearly care a lot about package management the chances of us switching to a language that doesn't have a package manager is going to be pretty tricky probably because the threat of us trying to write one would be too great and we'd get very distracted but having all of this tooling around it really made getting up and running fast additionally having all this tooling also kind of assuage the fear of is this idiomatic rust at least you had some helpers along the way to kind of get you working that way and last but not least we didn't even say it because I think for us it was too obvious but most tech communities are terrible and the rest community has put a lot of effort into making the community really really awesome I'm not allowed to give specific examples of why I think certain other communities are really bad but what I can say is I've always had a really fantastic experience with the Russ community and they might be afraid to say it but reminding people that it's a very safe and welcoming community is a really huge selling point I can't tell you how many times I have to tell MPM people it's ok to go on the rust IRC it's not awful it's not awful I promise they have so much so much fear because experiences before on those types of platforms have been so bad but Russ has done a fantastic job of making their community something that you literally want to join not that you're just stuck with and deal with all right so duck's community and tooling you're like ok yeah does might seem easy in common but I want to remind you that they are neither they are actually incredibly rare and so these are some of the most key selling points although inside a community like Russ they might feel really obvious all right so tip number six as you may have anticipated the hard part of programming is programming so when I first was thinking about giving this talk I I thought I was going to maybe talk a lot about how all the technical aspects of the async i/o plus some of the other tooling stuff and I think this is really cool but then you know what I discovered I discovered that selling something on technical merit is really easy the thing that's hard is getting to the point where you get to sell it on technical merit which is what all of these tips that I'm giving you now are so I think you've heard a lot of really awesome technical stuff about Russ today and I think you'll continue to hear some so if you're wondering oh no where is all the technical element here what I'm trying to do is give you a platform so you can get to that position where then you can say oh well Russ is super fast Russ is super secure all of that but it turns out to getting to that conversation like really getting to it like you're really thinking about using that language takes a lot longer and the vast majority of that conversation up to that point isn't necessarily about technical merit all right and so this brings me to tip number seven which is that you need to be prepared for things to change like any good programming this is why we use version control this is why we use version managers things change and it turns out that dealing with change is hard and so during my experience of trying to walk through NPM thinking about Russ I ran into some interesting things that made me change my mind so I'll give you a problem all right so our node service logs are full of runtime errors if yours aren't I don't know what you're doing okay no just got runtime errors and so we've won one of the great ways to go in and refactor your code is to go take a look at all those errors and logs and you know parse them out and then find things to fix but it turns out that in particular if you're NPM with the number of services that we have and the number of users that we have the traffic we have parsing our logs is extremely slow because they're huge now my coworker Ryan was wanting to learn Russ and he was like you know what I'm going to do I'm going to write a log parser because we one and that seems like something real should be good at oh no just dropped into my slack at around 8 p.m. last Monday oh no my rust parser is 2 times slower than the Jay s oh gosh what and so I saw that and I was like oh there's probably some stuff you need to fix like you're new it's okay and so it turns out that yeah we made this one PR which throws that lock-on standard in so you don't basically make a Siskel every time you read in and that shaved off three minutes but it was still significantly slower than the JavaScript version and yes there were some stuff with strings it turns out that strings are really really hard but in the end all right node was actually kind of faster because of a default thing in node so if you don't know what this is this is a buffer and it turns out that the global buffer that standard an reads into a node it's just larger by default than rust and so it's faster and that was kind of a bummer but this was really interesting to both Ryan and I like it took a while for us to figure this out because Ryan just said this to me he was like I just fully expected to rust my way into 50% for over j/s sometimes you forget that v8 is pretty darn fast now at this moment you might be saying wait I thought this was about convincing your company to use rust it sounds like you don't like it what's going on and so this is another quote from zen and the art of motorcycle maintenance and it says you're never dedicated to something you have complete confidence in no one is fanatically shouting that the Sun is going to rise tomorrow they know it's going to rise tomorrow when people are fanatically dedicated to political or religious faiths or any kind of dogmas or goals it's always because these dogmas or goals are in doubt and so I'm not necessarily saying that rust is slower than node that would be a miserable thing to say on stage right now because I also don't think it's necessarily true but what I do want to say is I thought that I was going to be selling rust because of its speed it's what I thought but it turns out that I'm selling rust because of its safety remember I was talking about these huge piles of logs of runtime errors I wouldn't have those if the entire thing was written and rust so we were actually trying to solve a problem that was like caused by another problem again we're chaining this all together but again I thought ok speed might be the thing but now I kind of think it's safety and part of that reason is because making rust go fast is actually kind of hard and so if you don't know this is Sonic the Hedgehog whenever we talk about speed at NPM we say gotta go fast and we talk about Sonic so I mean yeah two tails anyways the thing is is that making rustico fast is pretty hard but making rust correct is actually pretty simple and being correct is just as cool as being fast if not cooler but it's just a little bit less flashy potentially and I just didn't really think that that would be how I was going to be selling it my mind changed and that was a really interesting thing to experience so when you're thinking about convincing someone to use rust the reasons that you might have are probably going to change and you need to be ready for that if I had responded to Ryan with there's no way like you just wrote really bad rust like rust is way better than nodes this again would not have gone over very well instead I was like huh that is really interesting I'm really glad that we can write rust and not have any more friggin runtime errors all right so kiss 8 of course as you may have anticipated is actually to be prepared for things to stay the same so things are going to change but also a lot of things are going to stay the same all right I have told everyone a bunch of times boring code is my absolute favorite code but a lot of people do not agree with this people like really flashy new things they don't like boring code I like boring code so for all the people who don't like boring code let me try and convince you here all right so just because you finally gotten your company to start using rust it doesn't mean everything will change in fact the best possible outcome is that nearly nothing changes at all and this is why first off you have the scientific method principle if you suddenly change everything with your start writing stuff and rust you're not going to have any way to measure what rusts impact is you're just going to have changed everything and so instead of baby being better you'll have no way to know because everything will just be different but more importantly the reason you should just change maybe one or two things so that you don't burn everyone out because it turns out that changing lots of stuff stresses people out and when people are changing things like you don't want to be stressed out you want to actually have a good experience so a little story here we deploy our rust and all of our production stuff using something called a git deploy system so we can just type git push origin and force push a branch and then write it to a branch called deployed production and then we have ansible scripts in the background that just push our stuff up it's super great I never have to think about how to like deploy a system all right and so when we thought about writing rust one of our first questions was oh geez how are we going to deploy it now rust has a super awesome thing like you can make a compiled binary you can deploy it that's not what we do all right we just deploy rust using nearly the exact same process that we deployed note is that good I don't really know but it works and everyone is really comfortable with it so we were able to change just one thing and people really enjoyed that all right so I know I'm running a bit out of time but I have two more tips for you this is tip number 9 and it's have a problem to solve it might seem a little bit obvious at first that if you want to use a tool you should have a problem to solve to solve but you might be surprised by how many people are like throw a rust on it it'll be good and like don't actually do the work of being like what should I actually do with this rust and so I always say figuring out what problem to solve should be easy because all software is absolutely terrible so figuring out where your specific terrible software is is probably a great place to find some rust and put it there all right so for us at NPM what I like to say is numbers are big this is our problem all right over the last year 70 billion downloads occurred and that's been ramping up so we're averaging about two billion downloads a week this is a large problem like literally it's large it's a large problem and so what we are really excited about and this is fundamentally what we decided that we wanted to adopt rust for is we want to be doing something that we call store file transforms which returns to this idea of the follower which is right now a lot of packaged data is dynamic but it's a lot faster to serve static data and so we want to write followers that take this dynamic data do a transform on it and then store it so we can serve it statically which is going to make it go a lot faster given the traffic that we have all right so I would urge you specifically if you really want to use rust find a problematic part of your product and then try rewriting that in rust don't just generally say oh let's use it because having something specific is going to give people a way to really envision what adding rust to the product would do and last but not least this one is my favorite so I know that the core team is currently focused on developer ergonomics because they're worried that rust has too large of a learning curve and if someone who's a teacher I also really care about that and I do think that rust has a pretty strong learning curve but I'd like to point out the other side of the coin which is that the learning curves are kind of a blessing in disguise NPM is a really small team we have 25 people working at NPM 13 of which are engineers most service engineers end up spending a lot of time working by themselves and that's kind of a bummer and so the idea of introducing rust it's it was for us it wasn't a silly oh no people have to now get up to speed they have to learn how to reuse rust no no it was actually a pleasure it was like up it was a benefit that we got to actually teach people new things so after this huge whole refactor of the log parser Ryan just said you know this is the really fun part about learning new things I still have the patience to do any number of silly things and to take somebody who's been working extremely on an extremely complex system and say learn a new thing and do some stuff that silly is actually really energizing so instead of thinking about the fact that teaching your team something new is going to take time and slow them down realize that learning something new is actually energizing and if it's hard that's probably even better because at this point the absurdity of learning something difficult it can be something that actually makes you more excited about getting stuff done in the first place so in summary have listed out a bunch of these tips as you've noticed a vast majority of these actually contradict each other that's because as I said before there's not really a surefire way to convince your company to use rust it depends and so you should try out all of these try out some of them figure out what works for your team but there should be something in here at least if it's not necessarily convincing your team to use rust it's convincing anyone to literally do anything because there's nothing special necessarily about convincing someone to use a programming language so I'd like to leave you with this if you are going to repair a motorcycle or convince your company to use rust an adequate supply of gumption is the first and most important tool if you haven't got that you might as well gather up all the other tools and put them away because they won't do you any good so gumption is kind of a funny word especially in English so I translated it and what it means is like a spirit of initiative and resourcefulness so fundamentally what you need in order to convince your company to use rust is the desire to do it and so if you're interested you should try it and you should start and figure out what works along the way so go forth and spread rust thanks you

Channel: Rust

Views: 58,125

Rating: 4.807065 out of 5

Keywords: rustfest, rustfest2017, rust, rustlang

Id: GCsxYAxw3JQ

Length: 37min 33sec (2253 seconds)

Published: Fri May 12 2017