ElixirConf 2021 - Nicholas Henry - Preventing Brain Freeze: Onboarding with Living Documentation

[Music] [Applause] [Music] [Applause] [Music] thank you very much good day everybody my name is nicholas henry and i would like to welcome you to my presentation on preventing brain freeze and building better onboarding experiences for new developers with a living documentation i'm an independent software developer and i've been onboarded to a few software projects and some of these of course an ice cream headache let's talk about onboarding onboarding is typically thought as the process of introducing a newly hired employee into an organization in the context of software development we can consider onboarding to be the process of introducing a newly hired developer onto a project we can map the onboarding journey for a new developer to two main domains the problem domain where the developer learns about the business and the solution domain where the developer learns about the source codes architecture and conventions to onboard new developers these two domains require an effective knowledge sharing process there are a number of methods of knowledge sharing and in-person knowledge sharing such as peer programming can be the most personally rewarding as a new developer to a team we should always try to pair with our teammates if you're an existing team member who is on boarding a new developer and need to perform a code review perhaps consider asking the author to walk through the code with you you'll be amazed by how many corrections they can discover by themselves however in person knowledge sharing is not always possible so we need to lean on other methods especially when you're working with code that was perhaps authored in isolation or the author is no longer on the team as software developers we hope that the source code can tell us everything about the project but unfortunately it doesn't tell us the full story especially the design decisions made over the application's lifetime what other methods are available to us documentation is a method for asynchronous knowledge sharing when in-person knowledge sharing is not possible now before you quickly switch to another track to check out another talk we're not talking about the traditional documentation approach documentation does have a bad rap and rightfully so duplicated documentation may exist across multiple sources separate from the code base large documentation efforts slow down or put feature development on hold and documentation can often yield a low return on investment often we document the wrong things where naming conventions might have been a better tactic traditional documentation hinders sharing with new developers and it doesn't enable it so what is the alternative to traditional documentation we want to focus on documentation that is reliable low effort collaborative and insightful these are the four principles that cyril mate laid out in his book living documentation a book about continuous knowledge sharing reliable documentation is an accurate and in sync with the software being delivered that's why it's important to leverage augment and curate the project source code low effort documentation is the documentation you don't have to write or has little maintenance we leverage naming conventions and type specs to communicate in your code base collaborative documentation encourages everybody to contribute to it that at the very least gives everyone access to reader and insightful documentation embeds learning of the business domain and the architecture decisions it acts as a reality check on your software design just like testing if it's hard to document then perhaps you need to rethink your design and it will be much as concepts and patterns that we'll be discussing today and how these patterns can be applied to elixir to async knowledge sharing with new developers your code base is the authoritative source for knowledge of the problem and solution domains the two domain new developers are learning when onboarding to a new project we want to leverage code as the foundation for living documentation we want to augment code to add missing documentation to make it complete and we want to curate code to make the most important concepts prominent these are the three strategies will drive the structure for the rest of the presentation and how they be applied to the elixir project but first i want to talk about you where do you find the mode of motivation and time to invest in knowledge sharing i would like to frame this question in the concept of habit creation in the book atomic habits authored by james clear he describes the challenge of habit creation in terms of leading and lagging indicators he states your outcomes are a lagging measure of your habits your net worth is a lagging measure of your financial habits your weight is a lagging measure of your eating habits and your clutter is a lagging measure of your cleaning habits lagging indicators are the results of the habit over time where leading in indicators is the frequency and quality of practicing your habit these are the indicators that you can measure immediately for software i would say that your team's velocity is a lagging measure of your software development practices and habits or in terms of onboarding your team's onboarding experience as the lagging measure of your knowledge sharing practices but what are the leading indicators for good software development practices and more specifically not knowledge sharing habits let's take a look at a practice that i hope we all are familiar with test driven development is a software development practice for some kills the fear of getting started allows others to refactor a module without completely breaking the application and for most of us sleep soundly at night but let's look at another facet of tdd design pressure jason rainsberger describes design pressure as the guidance or pressure that test driven development places on your software design kelly sutton a software developer from gusto describes it this way design pressure is the little voice in the back of your head may manifest by a crappy test with too much setup the gains that you get from listening to your tests make your software more cohesive and reduce coupling these are your leading indicators for software velocity investing in onboarding and knowledge sharing is kind of difficult in that we may never benefit from that lagging indicator or we may only benefit from the lagging indicator in six months time when we return back to a model add a new feature and understand the context and the design decisions we have made so if tdd can help us with the velocity of a project is there a practice that can help with onboarding but still has a leading indicator that tdd gives us readme driven development may be one of those practices read me driven development was coined by tom preston winner a founder of github about 10 years ago in a blog post and it's always fascinated me tom states writing a readme is absolutely essential to writing good software until you've written your software you have no idea what you'll be coding i propose that we expand on a readme driven development might look like beyond a readme markdown file in the base of your source code repository readme driven development might describe the problem that we're solving in a module module doc attribute of the context module tom goes on to state write your readme first first as in before you write any code or tests or behaviors or stories or anything although i haven't practiced this consistently through my career when i have practice it hasn't always been tested to always been first but i can say that has an interesting effect on the design i code sorry excuse me on the design of the code i wrote and you may discover some of these benefits too you may discover that your names and the structure of your code changes as well read me driven development has the same result as rubber ducking a technique described in the book the pragmatic programmers rubber ducking can be described as a very simple but particularly useful technique for finding the cause of a problem and simply to explain it to someone else the other person should look over your shoulder at the screen and not his head his or her head constantly like a rubber duck bobbing up and down a bathtub they do not need to say a word the simple act of explaining step by step what the code is supposed to do often causes the problem to leap off the screen and announce itself readme driven development can help names leap off the screen and announce themselves the design will let off the sleep off the screen and announce itself and it's this leading indicator for the habit of knowledge sharing documentation will place a design pressure on your code that you write much like tdd does there's a different type of design pressure it injects humanity in your co and your project to improve async knowledge sharing living documentation consists of a number of patterns that help us leverage augment and curate knowledge to share with new developers there are close to a hundred different techniques and practices that matter writes about in his book on the subject but we'll look at a handful of these and see how they can be applied to our looks of projects i think it'd be rather boring to describe the depth catalog of these patterns it'd be much more fun to discuss these patterns and techniques in the context of an elixir project and how they improve knowledge sharing in the onboarding process to make the application of living documentation more concrete i've created a simple umbrella project for a wealth management platform under the apps directory i've stubbed out multiple child apps with names that you expect to see in a platform targeting a financial domain we'll be focused mostly on a specific module that calculates the rate of return for a client's financial portfolio so let's start with building the foundation for living documentation by leveraging your source code to onboard new developers the key to living documentation is automating documentation we aim to create living documents that evolve at the same pace as the project does in elixir xdoc enables living documents now this may sound incredibly basic but simply adding the x dot package elixirs documentation library to your project will result in some immediate benefits you've interacted with the beautiful interface when navigating a lecture in phoenix's documentation and other libraries leverage this for your in-house projects as well in in-house projects developers will be using the elixirs documentation facilities such as module dock and dock alex's documentation attributes but they're not actually generating the documentation with the mixed docs task running the documentation generator will actually pass and produce warnings if module references and file references are not correct treat the documentation you write with the same care as you would with your code on ci you're probably running mixed compile um errors as as warnings and mixed dialyzers as steps in your build to verify your code and your type specs add an additional step to verify the project's documentation with mix docs this mix task doesn't return a non-zero status thing if they're awarding so it won't fail the build so a little shall magic bash magic can fix this at the beginning uh at the beginning of one consulting engagement i was happy to see a consolidated effort to document key concepts in a project with references to other modules but when running mixed docs there was a slew of warnings that showed many of these references were broken and running mixed docs as part of ci would have prevented this okay so we have xdoc installed it is running part of our ci but to make this documentation a fast a first class member of the approach project it needs to be published and accessible by all members of the team you may ask yourself why publisher when read the document when i can read the documents documentation from the source code as discussed in the section on readme driven development i've discovered you'll learn something different about the way your application is structured when viewing it via the generated documentation generating documentation as part of your staging environment and pr review application allows you to include it in the peer review process add this to your pr template checklist as a reminder for your fellow team members to review a simple way to include it in your review application is to generate the documentation copy it into the public static directory and let the static plug server just make sure you do this based on the environment and not include it in production when i was first on boarding to a specialized publishing platform that there was this comprehensive overview of their ingestion pipeline well i was grateful for the overview of the problem solution domains that had some trust issues with the documentation although the documentation was included in the source code repository it was a separate directory called docs which was treated as this miscellaneous bucket for domain specific and operational documentation the domain specific documentation was not integrated in the source code at all i would have trusted documentation if the overview was included as part of the ingestion module itself and it referenced the actual sub modules within the module dock and all of this could have been validated using mixed docs i suspect most of you for your in-house applications are probably set up as an umbrella project a parent project with multiple child applications located in the abs directory xdoc with its support for our umbrella applications can kind of actually lead us down the wrong path our immediate instinct is to run mixed stocks in the parent directory of the umbrella project which will generate all the dots in one pass this has several downsides uh you're having to run the documentation generator for the entire umbrella project that is all the child applications which is time consuming all the modules end up being in a single table of contents in the sidebar which can be kind of difficult to navigate and organize then you have a single global configuration for your project that is applied to all of the child applications perhaps for uh one child application you would like to control the grouping of the modules but for another you would like to just use the default grouping is there a better way well let's think about our favorite web framework phoenix we don't have a single phoenix package we have multiple packages that provide the functionality for a complete web framework the phoenix framework publishes six packages in total sure each of these packages could have do have different concerns allowing us to opt in but it does make curating publishing of the documentation easier my recommendation is to configure the umbrella project to ignore all the apps and then generate the documentation for each child application individually we want to use the umbrella project as an entry point to explore the rest of the application and you can see in the example application on github that i provided to see the exact configuration on how to do this a template to follow is to have the umbrella project readme document the problem domain and then provide a catalog of the child applications in our solution domain the problem domain can be described in terms of a vision stakeholders and the business capabilities the application provides and then our child applications are the documentation for those are generated separately and we just link from the umbrella read me often source code doesn't tell the full story and there may even be an impedance mismatch between representations in the problem and solution domains inhibiting onboarding this is known this is where knowledge augmentation patterns can help us we can augment code to clarify our code bases one of my favorite examples of this is financial calculations let's take an example of the rate of return if we visit the investopedia and online resource for finance they can define a rate of return as a net gain or loss of an investment over a specified time period expressed as a percentage of the investments initial cost when calculating the rate of return you are determining the percentage change from the beginning of the period until the end while the formula looks like this in our problem domain an implementation in elixir may look like this in our solution domain when using the decimal module our definition in the problem domain looks very different to our implementation in the solution domain can we do better for our new developers fortunately when working with x-dog we can add additional functionality to help us communicate with our new developers xdoc after all is just html and it opens and is open to extensions by using additional tools such as javascript libraries in the case of financial calculations we can utilize ktx a math type setting javascript library to render formulas let's take a look at how to extend x-stop with k-tx we perform this con configuration in our next project module for nist for a nested application configuration i like to extract that out to a dedicated function hence the docs function the x doc before closing kit tag configuration allows us to insert any html before the closing head this is where we reference the ktx javascript library again i've extracted an attention revealing function describing the markup we're going to include before the closing head tag now that we have xdox set up with ktx let's see how this can be used in documentation for the rate of return module in the module dock we have included the definition of the rate of return from investopedia and the living documentation set of patterns this is an example of embedded learning putting knowledge into the code helps our new developers learn while working on it the second pattern this demonstrates is ready-made documentation often our problem domains the knowledge is already documented and we just need to link to it from our source code living documentation focuses on writing less documentation and utilizing existing documentation the final pattern that this demonstrates is stable documentation stable documentation has a great roi invest in this documentation because it's unlikely to change in our example here i think we can agree that the rate of return calculation is not going to change the module attribute formula is where i've included the ktex marker and we've included the module attribute between the double dollar signs this is an indicator by the ktx library to render the math email this working code is the working code based on the example for investopedia is also included and this is verified by our test suite with our rendered documentation using mix docs we are able to communicate the problem domain in this case the formula while elixir code is used to implement the solution domain within your project there is a massive amount of knowledge stored in the source code but not all knowledge is equally important or necessary to share this is where our third and final strategy comes into play curation curation allows us to select the relevant knowledge in our domain and share it with our team members a curation pattern we will review is the guided tour and sightseeing maps on boarding a new developer tour project can be a little bit like visiting a new city a guided tour can be valuable to explore the must-see attractions and sometimes introduce you to something some interesting unknowns although known for its application data science live book elixir's interactive and collaborative code notebook is a tool we can use to create guided tours and sightseeing maps for our application it's even pointed out in the live book readme under the section custom runtimes when executing elixir code you can either start a fresh elixir instance connecting to an existing node or run it inside an existing elixir project with access to all of its modules and dependencies this means live book can be a great tool to provide live documentation for your existing projects let's take a look at connecting live book for a guided tour after starting up live look at the command line we navigate to the tour directory of our umbrella project here we select the guided tour for financial ratios this is a subset of markdown so it's great if you want to just edit that within your editor you can or you can use the live bulk interface to edit that as well again this is included in the example application this is a simple example based on our rate of return calculation i've literally copy and pasted the markdown and for the live book file but unlike our static documentation we can execute the examples although we've selected our notebook we then need to select a runtime to evaluate our examples the runtime we will select is our umbrella project as a mixed standalone node once connected we can evaluate our examples we could expand this example further to provide a complete tour of the financial portfolio management application with the existing implementation i particularly like this example as live book has before support for ktx out of the box the methml language that we wired up to xdoc to render the formula one team i worked with included a manual verification steps as part of their review process manual verification steps are great for a ui but can be a little repetitive for back back end only features often included 15 different lines of code to copy into an iex how cool would it be to provide a markdown file that the reviewer could load up into live box could these verification markdown files even be committed to the project's repo now that we've learned the concepts and some patterns of living documentation applied to elixir how do we get started applying this to your project my recommendation is to always start with small experiments never large documentation efforts you want to avoid the trappings of additional documentation excuse me just try things of traditional documentation such as stalling feature development with these types of initiatives small experiments give you the opportunity to try some of these ideas without risk and help generate interest from your other team members here are a couple experiments you can try first with the next feature you can add with the next feature you add to your code base try readme driven development as a reminder this doesn't necessarily mean a readme markdown file located in the base of your project start with overview documentation for a feature located in a module using the app module doc attribute start with describing the problem domain and see how this improves naming of concepts and the overall design of the solution domain maybe even try iex examples that can be verified by the test suite these examples can be used as an entry point for new developers to experiment with the application second add x dock as a dependency to the application and try to generate the documentation if some modules and functions have already been documented you may receive warnings as the documents are generated this could be a great pr for fixes this is a great place to start and making documenting documentation a first-class member of your project from there start navigating the documentation and determine how easy it is to reason about do the names in the core documents make sense can i restructure the modules to make it easier for new developers to navigate finally you could start with your experiment finally you could start your experimentation with live book if you have a complex domain try to build an interactive tutorial for it this can result in an amazing onboarding asset remember onboarding is knowledge sharing practice knowledge sharing with living documentation to prevent the next ice cream headache for your new developers if you have any questions please ping me on the conference or social media platforms thank you so much for your time today and take care

Channel: ElixirConf

Views: 746

Keywords: elixir

Id: EMWWv6RyqMc

Length: 29min 54sec (1794 seconds)

Published: Sun Oct 24 2021