ElixirConf 2021 - Phil Toland - How to Build a Great (Elixir) Team

[Music] [Music] so um my name is phil tolland uh and today i'm going to be talking about uh building a great elixir team and this is basically an adoption talk but it's a little bit different uh from the usual adoption talk usually people are going into an organization that's not using elixir and the question is how do we convince them to use elixir in this case we knew we were going to use elixir it was an elixir organization it seemed like the right tool for the job but there were a lot of other decisions that were involved in that to make that team successful so i want to talk about just broaden the scope a little bit and talk about some of those other things that go into making a successful team beyond just the technology so to get started uh let's talk a little bit about who i am uh so you know why you should listen to me or not as the case may be uh so first of all i live in new mexico i used to live in austin and when i moved to new mexico and i would meet people who were from there they would say like why would you move here from austin right like there's just inconceivable um well if any of you have stepped outside today that's one of the reasons right right now it is 60 degrees and 30 humidity in albuquerque so uh i thought that was a pretty good reason the other thing is i'm a big hockey fan go avalanche if you have an hour or two you want to uh you don't have anything else to do with just catch me in the hall and ask me you know like what do you think the app's chances are this year uh you won't regret it i promise uh so i've been a software developer since the mid 90s which is kind of a scary long time ago now but if any of you were in software development back then those were the days of like 90 of projects failing everything was waterfall this was before agile was even a thing and it was a pretty traumatic experience quite frankly i worked on way too many projects where we were just not set up to succeed and as a result for a long time i've been very interested in this idea of how do we build teams in a way that they can succeed and i think we've you know i think we've been successful um mike there is on my team as well so he can he can tell you afterwards say you know is that guy full of it or not i'm sure he'll be more than willing to tell you uh the other thing is i've been an early developer since about 2007 i was very fortunate to be able to uh to get into that area and pretty much stay there and then since about 2017 i've been working with elixir uh and really enjoying that i was working for a startup when the pandemic happened and they folded it turns out that the founder was using investment income to fund the company and when the stock market took a dip it was bad news for all of us uh we we had these thursday morning calls because we're all just you know distributed and we get on the call one thursday morning and he's like well guys it's been a great run but that's it monday is everybody's last day oh boy not the kind of meeting you want to have but fortunately i landed at pepsico e-commerce in in june uh and that i think has been a pretty decent thing um so i'm gonna talk about that a little bit uh and currently at pepsico i oversee two teams two squads however you want to say it with about three major projects uh so i've had quite a bit of of time to think about the things i'm going to talk about today and i'm really passionate about teams and culture i don't because of my role i don't get to write code as much anymore although it's still something that i really enjoy uh but my you know my passion as a leader is figuring out how to create the culture that you know makes people want to come to work in the morning so let's talk about building teams i think it's important to approach building teams with the same care that we use when building software when we when we build a software product we have a very we're very intentional about it we follow a methodology or a process and i think when we build a team and a team culture that we should approach it with that same sense of rigor and intentionality and i often say you know i build the team they build the product uh and and i think it really helps me to have that mindset of this team is something that i'm building as if i was you know building a web app or whatever so today i'm going to present a case study of building a team and i hope that at the end of this that each of you has learned something that you can take away take back to your own team and make it a little bit better so i'm particularly interested in this intersection of culture technology and process and by culture i mean the values decisions and actions that we take every day technology of course is our tech stack which for most of us here is going to be elixir and if it's not elixir this is going to be a good argument for you to adopt elixir but in processes our ways of working right like the meetings the ways that we interact with each other uh and we should always be asking these questions right does our process and our tech stack should be built on the foundation of our culture those values that we have and ask ourselves do does the the process respect our values right um does the does the technology work with the process work the way that we want to work and if you don't ask those questions you can create a team where it's kind of miserable to work and i'm sure everybody's worked on one of those teams we want to find a situation where those three things are in balance so i'm going to give you a little bit of an overview of the first project that we worked on i'm not going to go too much in depth because quite frankly the details of the project aren't super important it was it was an internal application mostly going to be used by people within pepsico but it was i suppose this is one of those situations where the phrase big data is actually appropriate of course being at pepsico we have a ton of consumer propensity data and we have data scientists who spend a lot of time figuring out how to tease meaning out of that data and that's basically the group that i'm in that's what we do we have data scientists data engineers and application engineers and we all work together to create a consistent product from from the bottom step of analysis all the way up through the end user and when i joined the company they didn't have that application development capability yet so that was my first task we need to build this capability of putting user interfaces on top of this big data analysis so they there was a manual process where people would fill out a form and send it to a team who would then annotate it and send it to data science data science would send them the results in an excel spreadsheet they would take those results paste them into a powerpoint is this sounding familiar right then they would send the powerpoint on to the business user and we're trying to automate all of that so that they can just go into a web app plug in the information that they want and get back the results that they're looking for with all without all the rigor role i'm you know of course we have to figure out how to insert powerpoint and excel in there because this is a corporate application but um we want to make those we want to minimize the use of powerpoint excel to the roles in which they are useful um so quite a lot of the data science and data engineering work had already been done when i started and there was an existing ui prototype that was built in elixir and phoenix but it was included in a larger code base because that was just the easiest way to get it bootstrapped so one of the first things i had to do was pull that code out into a separate repo and then start building a new application on top of that for those of you who are interested the the photo here is a uh is an abbey a medieval abbey and in swaziland france it is i imagine that was quite a project uh to put that up so i thought it was appropriate um so this is a rough timeline of of the you know the way things went down i joined in june of 2020. um there was sort of requirements and ui ux design going on i was extracting the code from that uh from that larger application uh and then late summer august ish we start recruiting um our designers sort of finishing up the the final ui designs we hired two developers in september but there was a three-week lead time before they joined the team so they're both joining the team in sort of mid-october and then by the end of the year and by the end of last year the ui work was done the problem was that the data engineering and data science infrastructure that we were relying on was not as we start integrating with this stuff we find out that data scientists are data scientists they're not software engineers and so um we had to we had to fill the gap there and step in and and clean up a lot of those things and work on those coordination back end automation but by spring of of this year we were sort of into that feature cycle where you know the the users are coming to us with bugs and new feature requests and you know we're sort of getting into that that nice little groove and at that point we added two new developers on this particular project and then began looking at what the next big thing was so this is sort of the context for the the stuff i'm going to say next so i want to start off by talking a bit about culture so culture's one of these things that we talk about a lot on software teams and uh it seems to be there's a lot of confusion about how to build culture right people always say we want a good culture okay great but it's not as simple as just hiring a bunch of nice people and putting them in a room together culture requires being grown like a plant right you gotta feed it you gotta fertilize it and more importantly you gotta make sure everyone agrees on what we're working towards what is the culture that we want it's one thing to say we want a good culture but what is it what is a good culture right in fact what does culture even mean well i'll tell you how we defined it so culture is creating an environment of trust and safety where creativity can flourish that's what we want we want people to feel safe when they come into work to make mistakes to express their opinion to you know break outside of the box and that's the thing that we have to be intentional about in order to create it and i like to say culture is defined in the moment by each individual decision and interaction that's what culture is it's the way we treat each other it's the decisions that we make it's the values that we as a team hold and the values that guide our decision making so it's fine to say that right great that's wonderful how do we get there what do we do to get there well the first step is setting expectations early so i put together a slide deck of sort of my expectation for the team and i sat down with each of the developers when they started showed them that slide deck and said this is what i expect and this is what you can expect from me and it's really it's a simple thing it was a 30 minute conversation but setting those expectations early makes a big difference uh it gives people sort of a framing a context to think about the way they interact with other people and there's no awkwardness of well i was expecting this why didn't you do it and to this day i have the same discussion with everyone who joins the team the other thing is we have a clear set of guiding principles for our team and you might say well what are guiding principles well guiding principles are just us sitting down and writing down those values that i mentioned earlier these are the things that make our team hours these are the things that we really value and so this is what our team wiki says about it our guiding principles provide a road map to build a culture that we want so when we have to make a hard decision we fall back on those guiding principles so and the other interesting thing here is that these guiding principles give us a vocabulary to talk about the behaviors and the values that we want to see so when we say when you know somebody comes to me and they say we're having an issue how do we solve this what are our guiding principles what values do we need to rely on in order to make this decision so here are the guiding principles that we have and that's that's kind of small i apologize but i try to fit them all in one slide we're one team learn rapidly you'll notice that a lot of these are based very heavily on the lean principles that's not a that's not an accident results matter done is better than perfect be accountable quality and excellence constantly improve empathy and innovation trust and respect be the leader assume positive intent and fill the gap and these are the things that to me when we sit down and we talk about what i expect from our team members this is what i expect right learn rapidly get better deliver results be accountable for what you're doing remember that we're one team and fill the gap and i think assume positive intent is actually a really big one right it is really easy to think those other guys man they don't know what they're doing but really they're probably in the same situation we are they're doing the best they can with the information that they have and it helps to be patient so i know i tell you there's a moment that comes with the team when i hear somebody say you know i think i'm going to fill the gap on this and that's why i'm like yes we did it we're there the other thing uh that sort of feeds into this culture is the it's okay to list and this is something i borrowed from from an article in harvard business review that i thought was really great and you know part of the the guiding principles is writing down those things that we value so we can go back and refer to that list and we have this vocabulary to talk about it the it's okay to list is something very similar it's it's all of those things that we that we want the team to feel comfortable doing but we're making it explicit right and i tell i tell everyone who joins the team like you are empowered and i get a question a lot like what does that even mean right that's great but what does that mean well this is one way we answer that question it's like go look at the it's okay to list you can do anything that's on there no questions asked so these are some of the things that we have on the it's okay to list right it's okay to have a bad or unproductive day it happens to all of us don't beat yourself up over it it's okay to reach out for help if you're stuck because after all we're one team it's okay to block distractions during focused work right nobody's gonna yell at you if you don't answer a slight message for an hour or two hours or however long uh it's okay to be wrong even really wrong right uh you know this is sort of a a you know sort of a mutually uh mutual disarmament process here right i'm not gonna yell at you if you're wrong and you're not going to yell at me if i'm wrong and we're all going to get along and be happy and there's going to be rainbows and unicorns and ice cream afterwards it's okay to punt things until later right we don't have to make every decision right now it's okay to figure out what you want to work on if the top task on in the up next column is something that you don't feel comfortable with take the next one it's okay um it's okay to disagree i mean not that i mean we're software developers right we never disagree but if it were to happen that would be okay uh and it's okay to socialize especially in an environment where we're all working remotely uh it there's this mindset that can creep in that if we're having a conversation we need to be focused about business get things done but we also need to be okay every once in a while stepping back and saying you know what let's talk about oh i don't know hockey or you know whatever i'm sure there are other things people are interested in um so that's the culture part right and i talked about a couple of different ways in which we build the culture i think the important thing to take away from this is that you're intentional about building your culture that you document your values that you document the ways in which you create your culture the next thing i want to talk about is process i don't know why i chose the eiffel tower for that one um i'm sure there was a there was a complicated process to make putting that thing together so the core of our process is lean software development so the interesting thing about lean is that it's really not something you do if anyone says we do lean i give you permission to tell them that they're full of it right lean lean is a set of principles it's a set of values lean is something you are it's not something you do you can say our process is lean and ours is um and so for those of you who aren't aware so there are seven lean principles the first one is eliminate waste this seems obvious but what i found is everyone sort of has their thing that they think is super super necessary that we have to do that nobody else is convinced of and we never get rid of those things right i always say that if if you are not uncomfortable with eliminating something then you haven't cut enough yet you have to cut till you're uncomfortable and the amazing thing is nine times out of ten when you cut that meeting or that artifact or that deliverable that you think is absolutely necessary it's not so that is the first one eliminate waste the next one is amplify learning i think this is a big one right we should always be learning i'm okay with people making mistakes because most of the time the team ends up better afterwards the next one decide as late as possible this is a big one most of the decisions we make we make too early mary and tom poppendick who wrote the book about lean software development say that you should decide at the last responsible moment there is a moment in software development where if you delay that decision any longer it's bad right decide right before that the next one is deliver as fast as possible i don't think i need to convince anyone here that this is important hopefully not if so meet me afterwards in the hall but delivering as fast as possible is what makes deciding as late as possible possible because that quick delivery cycle means that we can put off decisions and if we make a mistake or we decide wrong it's okay because we can push out another release just like that right so these two are really they they are really interdependent the next one is empower the team the people who are closest to the work being done should be making the important decisions and that to me that is not something that always that business stakeholders are always comfortable with but this is really important build integrity in so integrity is it's not quite quality right it's not like quite bug-free integrity is a software system that is high quality fit for purpose and easily maintainable right so when we talk about integrity it's bigger than bigger than just quality it is all of the things that we think of that we want a software system to be when we deliver it and the last one is see the whole this one means don't optimize a piece of your system in a way that makes that is detrimental to the whole system when you think about like our application we've got an elixir app on the front end we've got some sqs cues we've got some s3 buckets we've got some airflow dags and it would be really easy to like zero in on one piece of that and optimize it until it is just as fast or as efficient as possible and throws the rest of the system into chaos right you should always look at the whole system and there's something that that's kind of magic that happens when you integrate these principles into your work one thing is you start focusing on flow right we want to sort of create this feature production pipeline where work starts at one end as a request and comes out the other end as running tested code and it needs to flow through there as smoothly as quickly and as effortlessly as possible and when that happens developers love coming to work it is great it is a great feeling it allows you to find a sustainable pace and keep it up indefinitely it allows you to measure your capacity and throughput in a way that's meaningful uh and it gives you a sort of platform that you can build on to always keep moving and to never stop learning and i think though that is probably my thoughts on software development boiled down to one sentence always keep moving never stop learning so our process our ways of working like i said we build it like we build our software so we start out with the simplest thing that could possibly work right all of those extra columns all of those extra steps the extra sign-offs the requirements traceability all that stuff you ain't gonna need it so we started out with a kanban board with four columns on it up next in progress in review done that's it this is the this is the setup we've been using since day one and i always said when we need another column we'll add it and we never needed it i think it's hilarious that when i took the screenshot in review was over the whip limit uh because it's never over the whip limit and it just so happens like the one day i take the screenshot it's over and uh mike pointed it out and i briefly considered taking another screenshot and i was like nah i'm gonna leave it in so uh we have a very simple kanban board um we have team and project stand-ups daily and retros every other week we have a process improvement meeting with the whole team every other week and we have a wiki page that we treat as a backlog for our process improvement and retrospective so when somebody says wait a minute what actually are our expectations for a pr review put in the backlog we'll talk about it in the next meeting otherwise our slack channel gets chaotic so i'm going to talk about metrics briefly so i gather metrics once a month from jira and i track them in excel and i just there's just four metrics that i that i'm interested in we don't do planning poker we don't do sprints we don't do any of that so to measure our capacity i look at total tasks completed and tasks completed per developer and that gives me a rough idea for the month of what our capacity is and then in terms of making estimates right so people do occasionally ask me how long is this going to take and i want to tell them it's going to be done when it's done unfortunately they don't like that so i track the average cycle time and the median cycle time and the cycle time is just the amount of time that a task is either in the in progress or in review columns and you might say why would you need the average and the median well let me tell you because what happens is every once in a while a big rock comes through that drags the average up and by looking at the difference between the average and the median i can get a better feeling for how the team's really performing right so if we have a big task that comes in and takes two weeks to complete for some reason mike's grinning so for some reason something comes in takes a long time like that that's going to drag your average up right and i typically use the average when we're making estimates for for our business stakeholders but the median is probably closer to what the team can actually do so for this year we have four developers they do about 37 tasks per month average four day cycle time meeting time two days so when somebody comes along and says we have a new feature we want you to do here's the mock-ups for it how long is this going to take we do a feature breakdown where we break those mock-ups down into tasks and we're pretty good about getting the task about the right size i mean you can people say well wouldn't it be easier to do planning poker i don't think so right like you're you're estimating either way whether you're sort of estimating whether the task is going to be the right size or you're putting a number on it either way you're doing the estimate and planning poker just takes a while and you know when you have a small team you've usually got one person who's going to be really vocal about their estimates and everybody else is going to say what he said and then that's not helpful right you might as well just have that guy sit down and write all the estimates himself so this i think works out better for us we've gotten pretty good about sizing the tasks and then if one starts to run long we just break it up right break it down you've reached a good stopping point great we'll we'll call this one done create another ticket for the next part uh and so like we can break those down into uh we can do the feature breakdown we look and say okay this is 30 tasks times four days 120 work days add some on because stuff happens and now we've got an estimate we can go to the business with that we're fairly confident about but we haven't spent a ton of time getting to that point the other thing that i don't like about planning poker is that it tends to give the impression of more precision than is really there right more precise but not more accurate is not better so i feel like be having a very rough estimate is very useful uh because we don't give the impression that we know more than we do so with that let's talk about technology right this is what we're all here for so elixir was always going to be the foundation for our tech stack and very early on as i'm getting ready to hire developers we have this question do we want to do a spa do we want to do server side rendered and here's the thing we're building an internal application we don't know how much the team is going to be allowed to grow we're starting with two open head count do you really want to hire one javascript developer and one elixir developer that seemed really risky to me right because what happens if your javascript developer is out sick and you find a bug i mean hopefully hopefully your elixir developer can fill the gap on that one but i like the idea of of having a team where the developers are fungible right if this something's broken over here we just find the guy who's got some time and he goes and works on it or gal um so when and and and i know what you're thinking right which is but there's a third option i'm getting there um so i decided let's start with two full stack developers right two developers that can work top to bottom that way we've got fungible developers we've got a higher bus number this is better for everything and of course view right so um the app was originally all server-side rendered and then as the developers came on board as we started to get more comfortable with what the app was supposed to do who was who was going to be working on it those sorts of things then we start converting to live view let's do a little experiment let's see how it works okay we're going to do this on one view that worked out great now we're going to do it on another view and we just keep going and today the app is i think all live view mike's nodding it's all live view did i mention i don't code much anymore so live ut not only solves this spa versus server-side rendered dilemma for us it also was a huge boon for us being a small team because the developers were able to stay in the same language and focus more on delivering value than on the technical details of the you know the architecture right and i think this is really important to keep in mind for a small team and we're very pragmatic right i feel like this is probably one of the best things that a small team can do i don't know why i chose vegas for being pragmatic um maybe because it's like the opposite so like you know the paris is maybe not pragmatic i don't know um but some examples is like we we went with bootstrap at first well i should say i went with bootstrap at first before the other developers came on board because i'm like i have a limited amount of time i'm dealing with all the stakeholders we need to get something in front of them bootstrap is a path of least resistance so that's what we did once we had more developers on the team once they were able to get ahead of the curve a little bit we ended up moving to an alpine tailwind stack because honestly i think it's just better it's lighter uh it you know it doesn't have some of the gnarly dependencies that bootstrap has and also i'm not sure how much this should go into the decision making but bootstrap's not like really popular and we want to be in the cool kids club so um we went pedal right but that's the kind of pragmatism we're talking about you know like being focused on using the technology in a way that moves us forward and being willing to you know get get out in front of things a little bit and i summarized this as i said earlier earlier as be curious stay agile and keep moving we keep exploring new technologies and approaches um so as an example we recently started using vega light and nx for some of our graphing and we tried it out on some isolated graphs in a new part of the application if it works out great we can replace the old code incrementally if not well you know it's one screen in a new part of the app we can rewrite that easily enough um and it's okay to fail right like sometimes these explorations don't work out uh early on that we decided as a team to adopt surface and then when it looked like live view was sort of moving closer to surface's functionality we backed off a bit and it's like okay let's let the dust settle here and see what happens but you know we learned a lot about structuring live views when we were doing that and i think the team is much better for having had that experience for experimenting a little bit uh and you know that there was a developer who had to spend some time rolling that stuff out but i feel like it was very much worth the cost so the key takeaways from all of this i would say the first one is lead intentionally and i know some people are going to say that it's like i'm not a manager what does this mean to me honestly everyone on the team has the opportunity to lead whether you are you know like the expert in a particular area of the code whether you know more about a particular technology whether you've worked with the business longer and understand the business domain everyone has the opportunity to lead in some way and i think it's important that we're intentional about that right not just oh hey you know here i am in this position of of influence and you know this seems cool to me right think about those decisions and this goes back to the what i said earlier about being explicit about the culture that you want to create but i think it also plays into the way we make decisions sometimes it's easy to fall back on defaults and if it's not an important decision that's fine right but for the important stuff let's think about it let's think about the impacts the consequences of the decision we're about to make and you know and and of course for the unimportant stuff don't spin your wheels right like fall back on the default who cares most of the time having a decision is better than or even if it's a bad decision it's better than not having a decision right you need to be able to move forward you need to be able to learn and everyone needs to understand and agree on the team's mission and this goes back to that intentionality don't assume that everybody's on the same page as you are we're all software developers we've all sat with our business users and they've all told us like we're all absolutely on the same page and then you talk to them individually and they are absolutely not so let's not do that with our teams right let's make sure everyone knows why we are coming in what we're doing and what we value so this is going to sound super corporate but a little while back we sat down and put together a mission statement for our team like i said i've got two squads working with different tech stacks working with different parts of the business and we realized that we needed to have something that we could point to that said this is what we're about and i want you to notice that it doesn't say anything about software development because we are in the business of solving problems right and it's also sort of really tailored to the problems that we experience that we are trying to solve within the business so i don't know if i mentioned it but i work for pepsico it's kind of a big company it's been around a little while uh and in that situation silos tend to come up right and uh we have to really break down those barriers in order to do our jobs effectively and so that's why our mission statement says we develop refine and evangelize sustainable ways of working that transcend boundaries throughout the global organization we didn't put that in there to sound good we put that in because that is one of the core things that we have to do to be successful and making sure that everyone from our stakeholders to our co-workers to our teammates understand that that's what we're doing when we come in in the morning has been a really good thing for the team and when we put this together we got the team together took a lot of feedback took a lot of ideas we got together with our key stakeholders we workshopped this a lot and then i sat down with with my leadership team and we came up with the final version and having that input from our stakeholders from our team members was a really sort of i don't know i can't think of a better word than bonding type experience right like it it gave everyone a sort of single frame of reference and put us all on the same page uh and yeah i also felt icky about like we're making a mission statement but it turned out to be much more meaningful than i thought the other thing is making the right call at the right time the right decision always depends on context and you know live view could have turned out to not be the right answer for us if it was less mature or if our domain was a little bit different but it was the right decision at that right time you know and the when i mentioned earlier about bootstrap right that was the right decision in that moment a few months later it wasn't the right decision and we corrected and that's okay you don't want to make your decisions too early you don't want to make them too late and you want to make sure that you're taking the context into consideration when you make those decisions i remember hearing years ago and i can't find corroboration for this but somebody told me that in the early days the ietf's unofficial motto was rough consensus in running code and i love that so much and i still use it right like look we're never going to agree on everything we're software developers but uh if we can like get all into the same ballpark and just you know make something work that's a pretty that's a victory that's a win in my book and then build the team for the long term we build our software knowing that at some point in the future we're going to have more users we're going to have more traffic we're going to have more data so you know we don't want to start out with you know we're going to build something that is you know can handle billions of records if we don't have a record yet um but we want to make sure that that the app can grow right like that we have um that we're not going to have to rewrite it if we need to scale that sort of thing and i think we want to build the team the same way and so again we approach it like we approach software so we make changes incrementally we don't make big changes to our ways of working if we come up with something if if two people come up with a problem that they have in one of our process improvement meetings it's like okay let's vote on the one we're going to address this time and we'll address the next one next time make those changes incrementally ensure each change is meaningful right if if we do something we want to make sure it moves the needle for us so before we make the change we have the conversation how will we know if this is successful test after each change right so we make the change when we come back together in two weeks time we say did that work out and if the answer is no we stop doing it i know right it's crazy so we test after each change and then we drop the stuff that doesn't work lather rinse repeat right so we just like we incrementally build our software we incrementally build our team we incrementally build our process and that's created a team that has been able to scale very nicely so remember earlier when i was talking about the timeline and i said that we were hiring and we had our new developer starting in mid-october i think it's basically exactly a year ago today that the first developer other than me joined the team and this is what the team looks like today and this is there are a lot of people smiling here um which as the manager makes me feel good we've been able to scale pretty well and today we work on three major projects there are two squads here and just in case anyone was thinking that i was making the argument that elixir was the only way to do this well half of our team uses python and javascript and has a spa but it was the right decision for them in the context that they were working in so that is the end of my presentation uh if you want to go in depth in any of this please just find me in the hallway uh i can talk right so and i enjoy i really enjoy talking about this i i will talk about my team until you run away hopefully not but you know it's been known to happen thank you [Applause]

Channel: ElixirConf

Views: 655

Keywords: elixir

Id: oVnJ-iK-h2A

Length: 41min 58sec (2518 seconds)

Published: Fri Oct 22 2021