Adopting Typescript at Scale - Brie Bunge | JSConf Hawaii 2019

Aloha my name is Bree and I work at Airbnb big changes at big organizations is hard there's a lot of people to convince and a lot of code to migrate I'd like to share with you how we brought typescript to Airbnb I appreciate you all being here with me I know you could be out at the beach with your snazzy new towel but I hope there's something in here for everyone whether you're navigating a big change at your company and this could be used as a sort of case study whether you're actively migrating to typescript now and there are tools and techniques that we'll talk about that can help or if you've heard a bit about typescript and want to learn more first we'll cover what typescript is what we mean by at scale the process of proposing typescript at this scale addressing questions and concerns along the way in response to the proposal how we gradually adopted typescript and migration strategies that we're using to move from JavaScript to typescript by a quick show of hands how many people have used typescript before oh cool that's a big number of you not everybody raise our hands so let's give a quick introductions so that everybody is on the same page suppose we had this greeter function it takes a name and returns hello that name so if we pass it in J's cough it would say hello J's cough how friendly here's what that code would look like in typescript notice that it's quite similar the only difference is that we've annotated the parameter with what type it is so if we use this function in our typescript project and we pass a string everything compiles fine but if we pass something that's not a string in this case an array of strings typescript will give us an error that a string array is not assignable to a parameter of type string we didn't have to refresh the page click through our flow look at the error in our console and trace back through where the error happened we got this era immediately in our editor right after we typed it we can express the type of other objects as well this interface describes a shape of a person object that has a first name and a last name and you can describe types for more complicated constructs typescript normally comes with a compiler that can tell you when something has gone wrong it also has a language server that editors can hook - for things like auto-completion find usages for factoring and more in this example we've hooked up the type for our with styles react higher cut order component so we get autocomplete for the hundreds of CSS properties it accepts including inline documentation amazing I didn't have to flip back and forth to the docs pages I got all of this in my editor with types with types throughout our code we can do more as it's being authored this just scratches the surface of what typescript is capable of but gives you a sense of the kind of areas that can help catch and the tooling it enables so that's the typescript part what about the at scale part Oh is there a problem oh thanks tied scripts I'm not actually making all this up if you did typo a variable that would give you an area that looked like that so it actually is real thing so let's fix that so the scale changes a conversation I've been on small teams before where it was just hey you wanna use typescript yeah sounds cool let's use it but as you scale up to hundreds of engineers and more and more code that conversation changes we would have changed that we were proposing is using typescript in our main repo the one that powers airbnb.com and making it a primary language for front-end development more people are affected by the change and more code would have to be migrated let's quantify what we mean by at scale an error maybe we have lots of JavaScript there's over two billion lines with JavaScript checked into our main repo and over 100 internal NPM packages these are separate repos that we've packaged up to an internal NPM registry that we can share across repos that's a lot of code we still got some backbone in there talk about Shaun's history of JavaScript we got some history over the the Airbnb decade so we we also have lots of Engineers interacting with this code there are over 1300 at the company 200 of which are fronting the majority of these front-end engineers contribute to that main mono repo those numbers paint a picture of the environment in which we were proposing typescript and what did it what did it look like at that scale every month we have a fun meeting where all the front-end engineers around the company get together and we geek out about new front-end technologies and patterns for things we want to consider in more depth we draft proposals that outline things like benefits trade-offs alternatives considered exit strategy and long-term owners people weigh in on the pros and cons of these proposals and we decided as a group whether it makes sense to move forward this ensures we make deliberate decisions about what we commit to as a broader team and avoid hopping on the hype train without legitimate technical justification we've been exploring typescript at Airbnb since 2016 on smaller teams and a static type system was a top request from a 20-17 front end survey given this positive signal Joe and I shout out to Joe second row there we drafted a proposal for typescript and presented it to the front end working group the proposal detailed why typescript made sense at Airbnb let's highlight some of the main reasons everybody's mission is to make it so anyone can belong anywhere every issue a user encounters with our product gets in the way of that mission the same is true of the products you all are working on typescript helps prevent bugs which in turn helps our end users typescript also enables a wealth of developer productivity benefits and tooling like editor auto completion and refactoring like we saw earlier with typescript engineers can move faster more safely we're investing in graph QL and Apollo at Airbnb which lets us generate typescript types from our graph ul schemas this means that we can get end to end size type safety because the types used by the back end and the front end share a single source of truth a back-end engineer can make a change to the API without unknowingly breaking clients and front-end engineers can be confident about what data will be coming back from the server type mismatches have been a major source of bugs for us so this end-to-end type safety was a major selling point sounds great right but there were lots of questions and concerns in response to our initial proposal let's look at a few of them in more depth our mono repo depends on our internal NPM packages do we need to convert those to typescript first in order to get autocomplete and type checking so this is the predicament we have we have our tiger project that depends on a jsm PM package how do we get types for that package it would seem like the package needs to be converted to typescript first but that's problematic because we be blocked on the maintainer converting it and maybe they're reluctant to do so because at the early phases of our proposal it wasn't sure we were going to move forward with it but on the flip side in order for developers to have a good experience with typescript we needed that type safety so how do we get around the seeming chicken and egg problem typescript has a feature called declaration files files with a DTS extension which we can define types for JavaScript files let's look at an example so we have the greeter function that we saw earlier and the corresponding DTS file on the top there are no implementation details here it only describes what the types are typescript stitches these together such that the Declaration file is used at compile time and the original JavaScript is used at runtime so let's go back to our original scenario and see how declaration files can help sure if that project had been converted to typescript we couldn't generate a DDS file as part of the typescript build but we were considering that that wasn't an option so instead we could put the declaration file in our typescript project another choice is that we can create a separate NPM package and put the declaration files there this is nice because now we can share the declaration files across several repos and this is how types for things like react work you have a you install react and you can install a types react package the types for react along with those for 5,000 other packages live indefinitely types a community maintained repository the vast majority of public dependencies we had in our main repo were already typed and definitely typed the active community around typescript was a major selling point and we've contribute back I'm sure they're people around this room that have contributed back thank you type definitely typed works great for public NPM packages but what about the internal ones we mirrored this definitely type setup internally by creating a separate NPM scope so instead of installing from types you did install from Airbnb types and the repo is set up similar to how definitely type is so we could add our types there and publish these types internally and we've open-sourced a starter kit if you are interested in setting up this sort of thing it doesn't have any types in it it's more just the setup for how you can add tests and publish your own types how many bugs can typescript help prevent a recent study to type or not to type show that 15% of bugs in a selection of github repos were preventable with typescript internally we have a process for documenting incidents that occur in production this is a blameless process the point is so that we can learn from our mistakes so we don't repeat them in the future so I literally sat down and read through six months worth of post mortems super-fun reading right so my favorites like tale of the uncut type error the peril of the per muted parameters okay maybe they didn't have such exciting names in any case I categorize them as JavaScript related or not and determine which of those could have been prevented with typescript let's look at example where typescript would have helped we made a change to our shared input component that broke a form on a settings page users were unable to submit the form because it no longer passed validation here is a simplified version of that shared and can put component before the change it takes an on blur prop and passes it straight through to the input element a change was made to add a new in on blur handler but there was a subtle bug can you spot it the event parameter was no longer being passed to the on blur prop this caused an issue several hops away in a different repo the input component was being consumed as part of a redux form which expects an event or value to be passed in order for the validation to work properly without the event the form no longer pass validation which meant the submit button was always disabled how could typescript have helped us here the types for redux form capture the constraint we saw in the docs the on blur prop must be passed an event value so if we had been using typescript with Redux form types hooked up we would have seen an error at the call site where the event wasn't being passed another common class of issues involved strict mill checks where you use a property to structure or try to invoke something that could be null or undefined you've probably seen that one before another was type mismatches typescript tells us when we try to use types that are not assignable to one another so now that we have a better sense of the common kinds of post-mortem bugs typescript can help prevent what's the overall percentage 38% we found that 38% of incident causing bugs that reach production bugs that have real effects on our users and bottom line could have been prevented with typescript this was a huge finding for us it helped make the impact real we saw light bulbs go off as we replicated some of the incidents and showed people the arrow type script would have given sure we could have written tests to catch these sort of things but static type checking adds an extra layer of protection so if you access to a similar history at your company it could be worth taking a look with someone who knows typescript to see what this percentage looks like for you do teams want to switch to typescript we pilot a typescript on several teams specifically targeting teams that hadn't used typescript before to gain additional perspective we help them get set up and gather feedback along the way after using it for a while we sent a survey to them asking whether we should continue investing in typescript and the feedback was overwhelmingly positive we'd recommend using this sort of trial period for testing out a new technology or pattern the front-end working group was on board with it because it was self-contained and it was easy to rollback should we not move forward with it and it helped the proposal because we could gauge whether teams actually liked using typescript there were concerns around build times we measured it and found no significant impact we have over 500 e eslint rules enabled in our main repo with the typescript eslint parser we were happy to discover that most of them just worked if we were to ever move away from typescript in the future we could strip off the types and end up with roughly the same JavaScript who would have written otherwise so one by one we documented considered followed up and came up with solutions to the questions and concerns raised it was important to us to work with detractors and hear out their concerns we converted many of them to advocates in the end and our proposal became stronger from their feedback after sufficiently addressing concerns we surveyed all front-end engineers on whether we should adopt typescript given the positive response we had enough evidence to move forward and the proposal was approved from there we gradually expanded adoption and at this point we had completed the pilot phase that was valuable for proving out typescript and getting the foundation in place we ironed out frictions early and improved tooling and documentation so later teams would have a smoother on-ramp we have ongoing contact with the typescript team and help surface issues like prioritizing better handling of default props as one example this phase also grew our internal typescript community most people had Airbnb don't know typescript yet so this meant more people could help and answer questions next we moved into a beta status and teams could opt into using it to help onboard teams we created internal documentation and a style guide and hosted learning sessions we set up a slack channel an internal Stack Overflow tag at Google email group of github group people could use on their PRS we wanted to make sure people could get the help they needed the final step is graduating typescript to general availability meaning it's stable and everyone should start using it we're actively approaching this at the moment the remaining steps are solidifying style guide documentation ramping up internal education efforts and migrating more code we're about 50% of teams using typescript and 10% of the files in our mono repo have been converted so far using this gradual approach made for smoother adoption if we've said from day one that everybody should start using typescript person after person would have run into the same issues or had the same questions instead we could figure things out on the smaller scale and carry forward and those answers would carry forward as we expand it out we've been exploring a couple options for migrating our code to type scripts our original migration strategy was a JavaScript type script hybrid let's see how this strategy works in the context of our mono repo this is a simplified version that I came up with by literally going to airbnb.com and giving reasonable names to the things I saw so no company secrets here let's zoom in on the homes project and see what it would look like to convert it using the hybrid strategy we added a typescript config file and renamed individual files from jst TS or JSX to TS x typescript errors pop up and we proceeded to fix them an awesome feature of typescript is that not all code has to be converted before it can compile and run the allow JSF configuration option allows javascript and typescript files to live side-by-side so at this point we can check in what we have so far and the site still works we don't have to pause development migrate the whole project and then my and then do that first we can migrate instead file by file we repeat this process over time until the whole project is migrated on the topic of migration I wanted to take a brief moment and share some tips we found helpful the first is TS fixed me we added a global type alias for typescript any type which means something could be anything we called it TS fix me to indicate that it should be revisited and fixed up later on best practice is to avoid any because it loses type safety but it can be helpful during the migration process the TS ignore comment silences the error on the following line properly typing a file might involve unraveling some deep chain of dependent types we try to avoid this by like converting leaf files first but sometimes it's inevitable so TS fixed me and TS ignore can help split things up such that incremental work can be checked in these are intended to be temporary and we're planning on adding type coverage tooling to help remove them as we refine the types over time in JSX we use prop types on a react components for runtime type checking when converting a JSX to tsx we can either remove the prop types in favour of a tight script type or keep the prop types and add a type script type in addition in our react component library for example we wanted to keep prop types so that JavaScript consumers could still get runtime checks but to avoid declaring a second type that would need to be kept in sync with those prop types we created a props type utility that is able to derive the type script type given the prop types and the default props in this case the prop types and default props get combined into this final type if you're curious how this works under the hood I created a gist that you can check out recently we've been experimenting with a revised all in typescript migration strategy so let's go back to that Holmes project we were looking at before and use The Awl end strategy instead to see how it works we start off with all j/s we convert everything over to TS get the project compiling perhaps we use some looser types than we'd like but we still have type scripts strictest options turn on and then we proceed to refine the types over time removing the TS fix means NTS ignores this has some advantages over the hybrid strategy it's easier to refine type I type rather than file-by-file if you're developing a feature and you only care about adding one type it's a much lower barrier to entry to just fix up that type in isolation rather than having to convert the whole file first fix up all the errors and then add the type you care about not having to rename the file also means that it's easier to review sometimes if a file is renamed in one commit then changes are made in another they show up separately in code review and the reviewer has to piece together what happens it also makes it clear what types are missing type scripts inference is powerful and we can leverage it for large portions of a file some files require a few to know TS fixed means in order to in order to compile type script can figure out the rest another is that developers can use a consistent mental model they don't have to contact switch depending on what file extension they're in like why can't I add a type here why am I not getting compilations errors there types can be added consumed and type checked in all files yeah that sounds great but how do we migrate all the code code mods are a powerful tool for large-scale code modifications in their simplest form a codemod could be a global find and replace across a project you might have done this in your editor before and there are code mod libraries that do use read exes but they are brittle because they might break depending on subtle code style variations alternatively we can use an abstract syntax tree which is something that the speaker's earlier brought up to so this is the ast representation of this code notice how for every piece of code on the left it has a corresponding node in the tree on the right so just for fun say we wanted to write a code mod that reverses all the identifiers in our code we'd start with our code as the input creating the ast out of it modify that ast and emit new source code out the power here is that we are making this change programmatically the alternative would be editing each file to make the change and this works fine if you only have a handful but once it gets to be more like thousand files the thought of manually changing them gets really sad Facebook's J's code shift is a popular tool we use for code mods at Airbnb this transform captures the modifications we just made to that ast to reverse the identifiers we find all nodes that correspond to identify errs replace those nodes with new ones with the name reversed and then return the new source code back out Missy Elliott would be proud of us so we flip the m reversed it I got laughs that's great I was debating goes like Joy Division I do it we took the code and we modded it found the identifier slip the member stump is your friend you oh yeah a tool that cannot help you with your rap skills but can help you with your code mods is ast xplornet which Sean pointed out earlier it has a source code that transform and the output and an interactive ast all in one place also I just landed a definitely type here to add types for Jaz code shift that makes it easier to author code mods and type scripts several patterns emerged while migrating JavaScript code to typescript we repeatedly moved static class properties into the class body for react components created a props type annotated react lifecycle methods we were encoded these as code mods so that we can run them repeatedly over more code and we package them up in the tool called TS migrate the goal is to send a JavaScript project in and get a compiling typescript project out now you you'd still need to find the types over time but it gives you a baseline to work from we applied this tool to our internal shared react components library which is used everywhere across the site we had types for this in our internal definitely typed but it was hard to keep those up-to-date with the rapid development of the shared react components library so instead we wanted to emit types directly from the source so it was our first target for TS migrate we converted over 30,000 lines of code to typescript and instead of devoting the whole team's time four weeks to do this we could run our suite of code mods and do it in minutes we use the type information from prop types and made use of TS fixme and we are continuing to refine from this baseline but even with this we generated meaningful typescript declaration files we can consume in other repos in case that PR line count looks a bit scary with the typescript compiler half for visual regression testing and our just test running in CI I felt confident that my changes wouldn't break anything and of course we made sure the site was still working no regressions whew we have since run TS migrate in other places and we're iterating on it and we're planning on using it across more and more code we plan to open source it so that you can use it to migrate your code to thank you for following along this journey with me I'd like to leave you some key takeaways that we can draw from this typescript transition that apply more generally effecting change at a large organization can be a challenge but strong evidence and addressing questions and concerns make a compelling case adopting change gradually helps reduce friction and prove out the value a clear migration path helps teams move over to new patterns and tooling can help expedite the transition I started this work while on a product team feeling dissatisfied with my tools when I learned that there were others in the company who also wanted to make this change I partnered with them and carried the initiative forward it's all too easy to complain about things accept the status quo positive change can come about through actions of people who care so I'd encourage you to pursue the things that you're passionate about your organization to make life better for you and those around you thank you all for listening and thank you to everyone at Airbnb who contributed to this effort shout out to Joe and Mohsen who are in the audience and miles and Portland's and there are a bunch of other awesome Airbnb engineers who would love to talk to you I also have some typescript stickers and some Balogh keychains for the like the first 30 people that come find me so just as an incentive yeah thank you so much [Applause]

Channel: JSConf

Views: 40,694

Keywords: TypeScript, JavaScript, Change Management, Software Engineering, JSConfHI

Id: P-J9Eg7hJwE

Length: 25min 56sec (1556 seconds)

Published: Sun Jun 16 2019