MALE SPEAKER: Thank you for
coming to Edmond Lau's talk. He will be talking about how to
be a more effective engineer. I met him about four
months ago or so, and it was also during
one of his talks. And I was just blown away by it. And I thought it would be so
useful to share his experience and knowledge with
the rest of Google. He is actually
also an ex-Google. So, he worked at Google before. He also worked at
various startups. And this talk will be based
on his experience as well as everyone he
interviewed with, and what makes for an effective engineer
and for an effective team. So without further ado, let me
introduce to you Edmond Lau. [APPLAUSE] EDMOND LAU: Yeah,
it's great to be back. I think it's been maybe 10
years since I first joined Google right out of school. I joined the search
quality team. And since leaving
Google, I've worked at a bunch of
different startups, including Ooyala,
Quora, and Qup. And two of those were actually
founded by former Googlers. And so, even though
I've been away from Google for quite
some time, Google culture has actually been a large
part of the various startups that I worked at. And earlier this year, I did
publish a book, "The Effective Engineer." And what I like to
do during this talk is share with you some of
the stories and lessons that I've collected
during these experiences. And so my promise to
you during this talk is that you sort of walk away
with a framework and a set of actual strategies that
you can apply to your day to day job on how to become
more effective as an engineer. But, before we dive into
the meat of the talk, I want to do a quick poll
and get a show of hands. Who here has pulled a long night
on a project for work before? Who here has done that-- who's
done work on the weekends before? A large number. Who here has worked multiple
months on a project, only to see maybe it not
actually launch? What about working multiple
months on a project and then see it launch, but
not actually sort of seeing the
impact or the effect that you actually wanted? And let's do one more. Who here has to do pager duty? And how many of you have been
paged in the middle of night or over the weekends before? Yeah, so it's a good
number of people. And when this stuff happens, a
lot of times we sort of wonder, was all this time and
energy, was it worth it? Was this actually the
best use of my time? And this isn't a question that
you just sort of ask at Google. It's also questions that we
ask at other places as well. I remember when I
left Google, the week immediately afterwards, I
jumped-- dove head first into my first startup. I didn't even take
a week of break. And this first
startup was Ooyala. It was an online video
company focused on building a platform for the enterprise. It was founded by
a former Googler. I remember my very first
week, the CTO told me, you're responsible for
building this feature that's already been promised
to a paying customer. And the video player has
written an actionscript, because it was a Flash
based video player. And it was a language
that I didn't know. The servers were
all written in Ruby on Rails, which I had
no familiarity with. And there was not
a single unit test across the entire codebase. And so, I was really scared
of just accidentally breaking something production. And so, my first
two weeks there, I ended up pulling 70 to 80 hour
weeks to try to get that done. And when I want to
talked to my CTO about how intense this felt,
his response was sink or swim. There's no life
preserver coming. You're just going to
have to work hard. And that was a very
intense and very stressful time in my career. And so, the next
startup I joined wasn't all that different. When I join Quora, which
was a question answer site, the jobs page actually
read, you should be ready to make this startup
the primary focus of your life. That's sort of how
intense the culture was. And that time, all
of these narratives about how you need to work
hard to succeed sort of really lined up with how
we were thinking. My team and I, we really
wanted to succeed. But at the same time, we
knew we were the underdogs. While we were building
an online video platform for the enterprise,
YouTube was free. Our biggest
competitor, Brightcove, dominated most of the market. And so we thought, one
of our secret weapons was that we're just going
to outwork everyone else. We're going to work
so hard that we're going to overcome all these
obstacles and actually succeed. And it took a while before
I started wondering, was this actually the
best use of our time? Because we worked
on projects where we'd spend multiple months
building a future for a paying customer, but then they
would never use it. Or at Quora, we'd
build content tools. And it just wouldn't get
the adoption from users that we actually wanted. There were all these things
where we spent so much time on these projects,
by the end of day, it didn't really
lead to any impact. And so, we started
sort of wondering, can we actually work smarter? Can we deliver
significantly more value while working fewer hours? And it's sort of like after
doing years of actually working these crazy hours before I had
this key insight that effort doesn't really equal impact. You may have good intentions,
but those good intentions don't actually lead to results. Think about a staff
engineer at Google. They probably produce
more than twice the impact or twice the output of
a more junior engineer, but they're probably not
working 2x the hours. Or think about Jeff Dean. So, I heard Google had to sort
of invent some new engineering level for him, like
senior Google fellow. He probably produces more than
10x the impact of most of us in this room. But he's not working
10x the hours because that's
physically impossible. There aren't that
many hours in the day. And so, the notion
that effort is sort of what
determines our impact isn't really a correct one,
and we need a better framework for the really understanding
and thinking and reasoning about impact and effectiveness. And so, that's sort of
where I sort of came upon this principle of
leverage, where leverage is defined as impact that
you produce for the time that you invest. It's the rate of return on your
investment of time and energy. It's the ROI. It's the rate of impact. If you think about the Pareto
Principle, the 80-20 Rule, it's the 20% of
work that actually produces 80% of the results. Those are the tasks, those are
the activities that actually are extremely high leverage. Now, sort of a central thesis
that I want you to take away from this talk is that this
central concept of leverage is really the guiding metric
that effective engineers should use to determine how and
where to spend their time. Now, it might seem a
little obvious and maybe a little simple to some of us,
but the same time, oftentimes we're so engrossed
in what we're doing, we're so focused on the
project that we're working on that we don't really
take the time to ask, are we working the right thing? We're attending meetings. That might not be the
best use of our time. Or we're fixing the
next urgent bug, or we're just fighting
the next fire. Or we're working on projects
that end up not shipping or not going anywhere. And we don't really take a
step back to think about, was our time on these
projects actually well spent? Were those activities
actually high leverage? And you can think about if
you're walking along the road and you see a boulder, that
boulder is really hard to move. But if you can find
the right lever, then you can apply just
a little bit of force and move that boulder
out of the way, because that lever
amplifies your output. In a similar way, in
software engineering, we're looking for
those levers where we can amplify our energy, our
time and produce massive gains. And so, the next question
we might ask ourselves, given that we have
this framework, is how do we actually
apply it to engineering? What are the highest leverage
activities for engineers? This is a question that I really
personally wanted to answer. I knew that working
60 to 80 hour weeks simply wasn't sustainable. It wasn't going to help us
win in our different markets. Also, I'd spend a lot of
time on engineering hiring and recruiting. And I'd screen thousands
of resumes or interview maybe 500 candidates. And I really want to
know, how do we actually identify the most effective
engineers to hire for our team? I'd also spent about maybe
a year and a half building the on boarding and
training programs for engineers at Quora. This was basically what
every new engineer at Quora would go through. And I wanted to know,
how do we actually train engineers and teach
them to be more effective? And so, I really wanted to know
the answer to this question. And so, that's what
started me on a quest where I quit my job at Quora and
then basically took two years to explore this question. And I went around
Silicon Valley. I interviewed a bunch
of engineering leaders from different companies. So, VPs, directors,
managers, tech leads, people from larger companies
like Google, Facebook, LinkedIn, Twitter. A few were from more medium size
companies like Dropbox, Square, Airbnb, Box, Etsy. And even smaller startups:
Instagram, Reddit, Lyft at the time was pretty small. And I grilled them. I asked them really
hard questions. Asked them, what separates
the most effective engineers you've worked with
from everyone else? What are the most
valuable lessons that you've learned
in the past year? What investments have you
made for your team that have paid the highest returns? I really wanted to know what
from their experiences proved out to be the highest leverage
activities that engineers everywhere should
be focusing on. And now 22 months
later, I basically had a collection of stories,
lessons, and actual strategies on how to be more
effective engineer. Now, everyone's
story was different, but there were a lot
of common themes. And in this book,
this is still a lot of those common themes
through actual strategies that you and I can apply in our
day to day jobs as engineers. And so, what I'd like to do
for the rest of this talk is actually share with you
five of those high leverage activities for
engineers that I think would be really useful for
us to incorporate in our day to day jobs. The first high leverage
activity really starts in our own
mindset, how we think about our own
development as engineers. And that activity is
optimizing for learning This Mantra is something that
has guided all of my own career decisions. Every change I made from company
to company or from company to writing a book was
because I saw an opportunity to actually increase
my learning rate. And this is really
important because learning is something that actually
compounds over time. What you learn today sets you
up for other opportunities and other learning that you
might have in the future. And when I say that it
compounds over time, I actually mean a few things. One, this curve of learning
is actually exponential. Just like if you were investing
in your financial investments, your investment portfolio,
your financial investments compound over time
exponentially. In a similar way,
investments in yourself also do the same thing. So, the second implication
is that the earlier on in your career
that you invest in yourself, the more
time that learning has to compound the long run. And thirdly even small
deltas in your learning rate can have tremendous
impact on how much knowledge and how much learning
you have in the long term. And so, it's really
hard actually to quantify how much we're
learning, but suppose you actually could. Suppose you could actually
improve yourself by 1% per day. What would that mean? Well, it would mean by the end
of the year, after 365 days, you'd actually be 37
times better than you were at the being of the year. That's a huge difference. And the best engineers and
the best engineering leaders actually aggressively
and relentlessly optimize in their own learning. One of the pieces of
advice Tamar Bercovici, who is a senior engineering
manager at Box, tells all of the
engineers he manages, is to own your own story. What she means by that
is that you need to take control and take ownership of
your own learning and growth, rather than waiting for
opportunities to come to you and find opportunities to
really invest in yourself. Read books, take classes, work
on side projects, attend talks. These are all ways that
you can invest in yourself and then those
investments will compound in your career in the long run. And so, when you think
about growing your career, ask yourself how might
you improve yourself every single day? And commit to making
a habit out of that. Now learning is one
thing that compounds. But another thing
that also compounds and that also is
very high leverage is actually investing
in your own iteration speed, how quickly you
can get things done. The faster we can
get things done, the more impact we'll
have in the long run. And because we're
all engineers, one of the best ways that we can
invest in our iteration speed is actually by
investing in tools. When I was talking with
Bobby Johnson, who's the former engineering
director at Facebook, he made the observation
to me that he found that people who are
successful, almost all of them wrote a lot of tools. He said the strongest
engineers and his team spend probably a third of
their time working on tools. Tools to do monitoring, tools
to make debugging easier, tools to just glue
the system together. But the surprising
thing was that he said this wasn't actually
obvious on his team. Because a lot of
engineers, we like working on the new shiny tool. We want to be the author
of some new system. We want to build this
new data storage system and be the author of that. When in reality, even
more mundane tasks, such as investing in tools and
investing in your own iteration speed, can be extremely high
leverage and have huge payoffs. This is sort of the reason why
a lot of big companies, Google, Facebook, Twitter,
LinkedIn, Dropbox, they all have teams the
focus on development tools. Because if you can decrease
bill times by, say, even one minute per day
and engineers are building, say, 10 times per day, and
you have 1,000 engineers, that's a one engineering
month saved per day. I remember when I joined
Google back in 2006 and I was compiling in a Google
web server on search quality team. It was something that you kicked
off and then you went home and it compiled
itself overnight. That's how long it took. And by the time I
left in 2008, sort of when Blaze was getting
introduced, bill times I think dropped to around 20 minutes. And that was huge. I'm sure it's probably dropped
down even further since then. I'm not sure how fast it is now. I see I see some
shakes of the head. But all of these
investments in bill times are extremely high leverage. Because it means that you
can, instead of doing things in large batches,
you're doing them more iteratively over time. One of the proudest
accomplishments that we did while
at Quora was we had a system where
we could actually deploy code production
40 to 50 times per day. For every time an engineer
pushed to commit to get, it would automatically kick off
a suite of a few thousand unit tests. And if all those tests
passed, that code would then ship to a canary
machine, that would then run another battery of tests. And if that passed, the code
would be automatically shipped to all our webmachines
production. And that entire process only
took six or seven minutes, which meant that we could push
code production 40, 50 times a day. And that changed
deployments which, from any team to
this one off event that they have to do maybe every
week, maybe even every day, to something that was a very
normal part of a development. And it meant if we had
a question, how often is this feature that's on our
web page actually being used, it means that an engineer could
just add a simple log line, push that code to production,
and then start getting some data back within minutes. So, a lot of these
questions that are impossible with
a slower workflow, we were able to do
because of a system of continuous deployment. And you might wonder, when
does it make sense for us to invest in tools? And a good rule of thumb that
I got from Raffi Krikorian, who grew the infrastructure
team at Twitter from about 50 engineers
to 450 engineers, used to tell his team that
if you have to do something more than twice manually,
you should build a tool for the third time. And that's a good rule
of thumb, because we tend to underestimate how
often we're going to need to manually do certain tasks. Sometimes we get started,
we do things manually because it seems to be simpler. And then requirements
change, or we mess up during our manual steps. And then we end up
spending a lot more time than we actually could have. Instead, if we had invested in
tooling and automation sooner, we could save ourselves
a lot of time. And so, when you're
working on a project, you should also always ask
yourself, what are the events? What are the bottlenecks that
you face during development? How can you speed
those things up? Because all of
those improvements will sort of
compound in how fast or how quickly you
can get things done. You can also ask same question
when you're debugging, as well. Say you're building
an Android app and you're debugging
this photo sharing flow that's a few clicks
away from the home screen. Could you wire up your flow so
that when you start the app, you land right on that screen? Simple tweaks like to optimize
your own debugging flow to really optimize
your iteration speed can have a huge difference
in the long run in how productive and effective you are I've talked a lot about how
to get things done quickly. But another important
question to think about is not only how to get
things done quickly, but how to get the right
things done quickly. And so, that's another
key high leverage activity is actually validating
your ideas aggressively and iteratively. And a good example of how to
do this right as well as how to do this wrong
actually comes from Etsy. Etsy is a company that
sells handmade goods online, and last year they hit
almost $2 billion in revenue. And they had this
one project where they were trying to
build infinite scrolling on the results page. So, when you type
a query at Etsy, you see a bunch of
product products. And the product page is paged. And they were
exploring, maybe we should add infinite scrolling. Similar to how on Google Images,
if you scroll down on the image search page, results just sort
of keep loading onto that page. And so they'd spent many
months building this out, ironing bugs, and just when
they were about to ship, they decide, we should
probably test this. And so, they ran an AB test. And they found that click
through rates actually dropped by 10%. Purchase rates actually
dropped by nearly 25%. So, there was no way they
were going to actually launch this product. Then they actually
spent some time trying to figure out
why this wasn't working. And they realized that
for this product change to actually work, it sort of
relies on two assumptions. One is that if we show
more results to users, they'll be more likely
to buy products. And the second is that
if we show results faster to the users so that they
don't actually have to page, they would also
choose to buy more. But the key insight is that
each of these assumptions could have been independently
validated with much less work. If the assumption is that if
we show more results to users, they will buy more, they
could have just increased the page size. And in fact, when they
did that after the fact, they found that that really
had no impact on purchases. An assumption that
maybe a faster page will lead to more purchases,
that's something a little bit harder to test. But they got a little creative. They artificially
added some latency to some fraction of users,
made the page slower. And when they tried
that test, they found out that also didn't
have much of an impact. Each of those tests
were much easier to run. Very little engineering
effort required. And if they had run
those tests first, then they would have realized
their basic assumptions behind infinite scrolling didn't
even prove out in the wild. And so, there was no real
reason to actually invest all that time and effort to
build infinite scrolling. They sort of took
these lessons to heart when they worked on a
different project, which was rebuilding the product
page when you click through on a search result. They actually went through
14 different iterations of this page when they
were redesigning it. And they broke down
the redesign project into a number of different
testable assumptions. Does showing more related
products on the page actually decrease bounce rate? Does showing the price
in your native currency, does that increase purchases? Does showing a default
shipping option and the price of
a default shipping option make a difference? Does swapping sides of the
page make a difference? They broke down each of
these assumptions, each of these hypotheses,
and tested each one. And there were a bunch
of ones that didn't work. They failed. But, all of the information that
they collected from these 14 different tests helped
them build confidence in theories about which
changes did matter, which ones did make an impact. And so with that, they were
able to launch a redesign that when I talked
to Mark Hedlund, who was the former VP of
product engineering at Etsy, he said that this was
actually the single largest experimental win
in Etsy's history. It was the most
successful project that they launched in terms
of purchasing performance. And it was only
possible because they learned that experiment
driven product design is a very powerful tool. Now, this is something
that I think, at least when I was
in search quality, Google did pretty well. They run a lot of AB tests. They sort of validate
a lot of changes to prove these changes are
actually improving searches and improving the bottom line. But another take
away from Etsy is that incrementally
validating your assumptions is a very high
leverage technique. If you can break down a large
problem into smaller testable hypotheses, and then
evaluate each one, you can really build
confidence in what works and what doesn't work. Or if a test proves that
your assumption is correct, then you have more confidence
that the path you're going down is the right path. If it proves your
assumption is incorrect, then that means you maybe need
to revise your project plan. At the very least, this could
save you months of effort. And in startups,
there is this idea of focusing on the
minimal viable product. What's the smallest
version of the product that takes the least amount of
effort to build that you can show to real users
and get feedback and validation that what you're
building, what you're designing is actually the
right thing to do? That's a very powerful idea. Now something, while
Google, especially search, it does a good job
with their AB testing, I think the idea of building
this minimal viable product is an idea that they can
definitely leverage more. Because a lot of
times at Google, we sort of build
products for speed. We optimize for performance,
we optimize for scale rather than really focusing
and asking a question. Like, is this product
actually the right thing to build in
the first place? Because it doesn't matter if
something is really performance if it's not the right thing
that users actually want. One of the good rules of thumb
that I got from Zach Brock, who was an engineer
manager at Square, is that he would constantly
ask his engineers, what's the scariest part of the
project that you're working on? That's the part of the most
unknowns, the most risks. That's the part you should
actually tackle first. Because you want to
reduce and tackle the risk head on so that if it proves
that these risky areas are there aren't doable or don't
impact your bottom line, you can cut your losses
short and move on. And so, when you're
working on projects, you should really
ask yourself, can you break this down into
smaller testable hypotheses? Can you use an inexpensive
test to validate that what you're doing
is the right thing? How might you expend 10%
of your effort upfront to validate that the
project you're working on will actually work? Because that 10% is a very
high leverage use of your time. It can save you months of
wasted effort further on. Validation definitely
is very important and can save you wasted effort. But another powerful technique,
another high leverage activity that can help you
reduce wasted effort, is this idea of minimizing
operational burden. A lot of times, we spend so much
time maintaining and operating our software that we
don't really have time to actually build new things. And if we can minimize
the operational burden that we actually have
on a day to day basis, we can spend a lot more time
actually focusing on impact. And a great story which
really illustrate this point comes from Instagram. When they were acquired
in 2012 by Facebook for a billion
dollars, they actually only had a team of 13 employees. And of those 13 employees, only
five of them were engineers. They had a really
small team supporting over 40 million users. So by any metric, this was
a pretty effective team. And I was curious, what was
it that allowed the Instagram team to be that effective? And so, I sat down with
Mike Krieger, the co-founder of Instagram, and I
asked him, was there any secret sauce to this? What was your key technique? And he said one of the most
important mantras that they had on Instagram was to
do the simple thing first. They actually had this
on posters in the office. During design
meetings, when someone was proposing a new
feature, they would actually challenge each other, is
this the simplest thing that we can do? If not, why are we adding
all this complexity? The key insight here was that
every feature they added, every system they added,
was another potential fire that they might have to
put out in the future. And with such a small
team, they couldn't afford to spend all that
time just putting out fires. And they were very
aggressive and very stringent about cutting sources
of complexity. And this actually a
very common theme. And I mentioned in the beginning
of how I went around and asked a lot of leaders, what's the
most valuable lesson you've learned the past year? And it turns out that a
lot of engineering leaders, including Mike Curtis, who
is the head of engineering at Airbnb, or Chris Lambert,
who's the CTO of Lyft, and a bunch of
other leaders, all said that they wished they
had made things simpler. They wish they hadn't
added so much complexity in the last year. And the reason why this
was such a common theme is because we often ignore
the hidden costs that are associated with complexity. When you introduce
sources of complexity, we're actually
introducing an ongoing tax that we have to pay as we
develop software in the future. And this tax actually comes
in a variety of forms. On the most basic level,
there's code complexity. That when we're dealing
with a very complex code, it's hard to ramp up on. It's hard to understand. It's hard to reason about. And as engineers, sometimes
we see complex code and then we tiptoe around it. We decide, we're not going
to touch this code base because it's a little hairy. And then, we miss out on
opportunities to actually build things that might be impactful. Or we do things in
a roundabout way just so that we can avoid
the complex piece of code. Besides code complexity,
there's also system complexity. So, how many moving pieces are
there actually in your system? I was teaching a
workshop at Pinterest. It was a five week
workshop for engineers on how to be more effective. And one of the stories
I gathered from them was that back in 2011 when
they were scaling their site, they actually had six
different data storage systems. They were running Cassandra,
MySQL, Membase, Memcached, MongoDB, and Redis. And they actually only had
a team of three back end engineers. That's insane. There were, like, two
systems per engineer. Each system on paper
set claims that they would solve some scalability
problem that they were facing. But in reality, each system just
failed in its own special way. It was just another fire
that they had to set out. And so, when you
make choices that really sort of fragment
our infrastructure, that has a lot of ongoing
taxes that we have to pay. It means that we're
spending-- we're thus able to pull together
resources to really strengthen libraries and abstractions
for a particular system. It means we have to take time
to understand the failure modes of each system
that we introduce. It means that every new
engineer who joins a team not to ramp up on one
additional system. And so, it took a
while, but in the end, Pinterest finally learned
that it's much simpler to have a system, a
scheme, where you can just add more machines to scale. And so, they cut out a
lot of the other systems that it didn't really
need and ended up with a much simpler system that
they could actually maintain. So, we've talked about
code complexity and system complexity. Another such complexity
is product complexity. Now, that comes when there
isn't a clear vision for where the product is going or there
isn't enough product focus. Product complexity leads
a lot to code complexity and simple complexity
because it means you have to write more code
and build more systems in order to maintain and support the
features that you're building. And when you have a wide
surface area in your product, every new feature
that you want to add becomes harder to
add because you think about how does this fit in
to the existing context of all the other features. There's more code branches
you have to look over. There's more issues and
bugs that are filed. There's more context switching
that you have to deal with to go back and forth
from feature to feature. And all of the time that
you spend doing this context switching is time
that you're not investing in
abstractions, [INAUDIBLE] paying technical debt. And so, those are all things
that end up slowing you down. [INAUDIBLE] last
soft of complexity, organizational complexity. Because when you have all of
this complexity in your code and your systems
and your product, it means that you need a
larger team to actually deal and manage with them all. And that means you
have to spend more time hiring and interviewing. You spend more time training,
onboarding, mentoring new engineers. And so, there's another tax
on the organization as well. Now alternatively,
some teams decide OK, let's just split our
team to smaller teams and each team sort of
manage a separate component. Or maybe even have a one
person team for each component. And there's cost
for that as well. It's less motivating
to work on your own. If you get stuck, it can
become more demotivating. The quality can go down
because it's harder to get feedback
from your coworkers if you don't the
same shared context. And your bus factor
also goes down. I remember when I was
working at Ooyala, there was one
point in time where I was the only person who was
knowledgeable about this piece of software called
the logs processor. So, the logs processor was
responsible for processing all of our customer's data
and then produce the reports that they could actually
read and sort of understand what their viewers were doing. And I was taking a much needed
vacation down in Hawaii. I was hiking around one
of the volcanoes there. And then, I suddenly get a
text, and it's from the CTO. And it says, logs
processor is down. I said, well that sucks. Unfortunately at that
time, I was the only person who knew how to fix it. And also unfortunately,
there's no Wifi in volcanoes. And so, that wasn't
a great situation. It was bad for me because
my vacation was interrupted. It was bad for the team
because they depended on me. And it was bad for our
customers because they couldn't get the reports and the
data that they needed to operate their business. And so, there are all
these costs of complexity on these different levels. On the level of code, systems,
product, organization. And they all come
from this core thing where we added more
complexity than we needed. And so, that's why this
theme of asking ourselves what's the simplest solution to
this problem is so important. Because it allows us to
stem off this complexity explosion at its bud
at the very beginning. And so, minimizing operational
burden is super important. And the last high leverage
activity I like to close on is actually building a
great engineering culture. I mentioned earlier on how
I've spent a lot of time on engineering hiring
and recruiting. And one of the
questions that I'll ask a lot of the
candidates I interview is, what's one thing you like
and one thing you dislike about your previous company? And over time,
this really helped paint a picture of what
exactly attracts an engineer to a company. What are the things
that turn them off? And it should actually
come as no surprise that engineers, they like to
work in environments where they can focus on high
leverage activities. They want to work
in environments where there's a chance to focus
on and optimize on learning. They want to work at
places where everyone is very productive and
getting a lot of things done, and there's good tooling
and good iteration speed. They want to work at places
where products aren't going to waste because assumptions
and hypotheses weren't being tested earlier on. They want to work at places
where they can actually build new features as opposed
to just maintain and pay off taxes for old existing ones. Now, Google is a
sort of great place in that there's a very strong
sense of engineering culture. But this is also something that
you can focus on your team, as well. On a team level,
you can really ask, what are the highest
leverage activities that you can start working on? You can think about how can
you invest in your own learning and in your own
growth to really sort of further your own career? You can think about
what tools can you build or what workflows can you
improve for both yourself and for your team to really
improve your iteration speed? Or think about a project
that you're working on that's a multimonth project. Are there ways that
you can break it down into testable hypotheses that
you can validate to make sure that you're actually building
the right thing for customers, for users, or other
members of the team? Or think about if you're
working on something that seems a little
bit too complex, is there a way to
actually make it simpler and pay off these future
taxes on complexity? Or think about, how can you
improve the engineering culture in your own team so that new
engineers who join the team can ramp up much more quickly? All of those high
leverage activities will make us more
effective engineers. I'll be happy to do some Q&A.
Thank you all for coming. [APPLAUSE] I also a few limited
copies of books that we're going to hand out. May be Cindy or [? Chang ?] can
sort of hand them out as people ask questions. AUDIENCE: Unit tests. I kind of read in your
book and it kind of presented unit tests as a high
leverage kind of activity. But at the same time,
kind of like intuitively when I think about
writing unit tests, I feel like it's kind of slow. I got to write all these
to cover all this code. And if I want to refactor it,
I got to change this unit test. So, iteration speed
kind of slows down. Maybe you could
talk a little bit about exactly why
unity tests are really high leverage in spite of that
kind of feeling that I get? EDMOND LAU: Sure. Great question, [INAUDIBLE]. So the question is, how do
you reason about unit tests with this framework of leverage? And really, I think
it's important to not be religious about unit tests. I think there are some
teams that focus on, OK, we need 100% coverage of our code. And I don't think that's
necessarily the best approach. It's a lot better think
about which types of tests would be high leverage? Which types of tests would
be easy to write, and yet pay off high rewards? And the area of
code bases that tend to benefit from a
lot more tests are code paths that either
see a lot of traffic, code paths that a lot
of engineers touch, or code paths that
are a lot riskier. And so, if something were to go
wrong, data would get corrupted or the cost would be large. In each of those
areas, it can be really beneficial to actually
have a set of unit tests. On the other hand, if you
have a piece of code that's sort of off to the side,
not many people use it, it's not touched that
often, in those cases, writing a unit test there might
not have that strong a payoff. And so, I would sort of
break down unit tests into different priority buckets
and sort of focus on the ones where there is more
traffic, people are changing it a lot more
often, or it's a lot riskier. Because those will have sort
of a better rate of return on your time. AUDIENCE: So, it seems to me
you have a lot of experience with working with startups
and small companies. And I wonder actually
how does your advice translate to a bigger
organization such as Google? As you were talking
through strategies, I was thinking in my head
about a particular example that illustrates this strategy. And for a big
company, for example, for the recent problems,
organizational complexity kind of becomes unavoidable. So, it seems to me
from your presentation, you give advice
for the more bottom up approach, which
Google is a lot like. But for a big
organization, actually maybe some top down
strategies are needed as well. For example, for using product
complexity and organization complexity. Do you have any
thoughts on this? EDMOND LAU: Yeah. So the question, to
rephrase a little bit, is how do these ideas
of focusing on complex-- or addressing organizational
complexity apply to larger companies like Google? And I think it's
a great question. I think some of
the clarity comes from-- some of the strategies
that an individual contributor, an individual engineer where
you can sort of influence the complexity in
large organizations comes I think a lot from data. So, I know Google is a
very data driven company. And a lot of times, if you
can ask the right questions and back that up with any data
from users, data from activity, you can sort shift organizations
or products in a way to sort of streamline it. So, if you see a source of
complexity from a feature that you gather data and it
seems like it's not actually being used, I think
that sort of forms a more compelling case
for decision makers that, maybe this is an area of
the product that isn't really worth all the engineering
effort that's being spent on it. And so, I think data
is was definitely one area that can be used
to amplify your voice when you are making arguments
of that nature. I think another way that
you can sort of reduce complexity in these situations
is to talk to the decision makers and get clarity
on what the focus and what the mission is. And make sure that the
projects that you work on are ones that sort of align with
that mission and that focus. Because if you're working on
something that seems peripheral to that mission or
focus, then you're sort of introducing additional
sources of complexity to that mission, that focus,
that the organization has. So, you want to
at least make sure that your own work is aligned
with the goals of the rest of the organization has. AUDIENCE: So, I have
two questions having to do with the transition
between from engineer to engineer manager,
so maybe they are within the
scope of the book. One is about mentoring, the
other is about technical data. So, first would be, what are
your thoughts on mentoring in terms of leverage? And the second is,
how do you recommend convincing an
engineering manager that the code complexity
in your project is worth taking a sprint break
from delivering to actually refactoring something thing
from scratch with a much simpler version? EDMOND LAU: Good question. So the two questions. One is, is mentoring a leverage? Two is how does refactoring
and paying off technical debts [INAUDIBLE] into this? So with regards to the
first one mentoring, mentoring is definitely I
think extremely high leverage. I spent a year and a half when
I was at Quora building out the onboarding and
mentoring programs because I realize that even if
an engineer would have spend and one hour a day
training a new hire, that only represents like
1%-- it's like 20 hours. That's like 1% of the new
hire's total hours in that year. And yet it can have a dramatic
output, a dramatic impact on that engineer's output. And so, it's super
valuable, I think, to invest money in training
your teammates and making sure that you tell them
and teach them what the best practices
are before they spend a lot of time
writing code that's not following the
right guidelines or following bad designs. And so, mentoring
is super valuable. And the second
question is, how do you convince engineering
managers to actually focus on or to budget time
for technical debt? And I think some of the best
strategies I've seen for teams is to rather than just
to have that conversation and spend all that energy
convincing your engineering manager after, like
say, every project, to instead sort
of just set a sort of consistent budget for that. So some teams, for
instance, every month will have one week where
it's code cleanup week. Or after every sprint, they'll
have some number of days where they spend refactoring
and cleaning code. That way, it's-- you don't
have to have this sort of conversation every
time after every project. And in addition, you can
use that time and prioritize which areas of the code
actually need to be refactored. Which areas of code
need to be cleaned up. And so, I think it's
more useful to have a more general
conversation where to hit the goals
for the sprints, we're going to make
some trade offs. Those trade offs will
lead to some debt. And we want to have a
consistent schedule where every so often, we budget
some number of days to basically clean up that debt. Yeah, it's a little tricky
if you're a new engineer. I would say that even
as a new engineer, the way that you should
approach technical debt is to focus on the areas
where reducing that debt allows you to be
more productive. So, if you're reducing
debt on errors of the code that you have to
deal with yourself, a lot of times
doing that making, that investment actually
pays off on your own project because you can get your
project done sooner. That's sort of, I
think, a very easy way to justify that time spent
because you can provably demonstrate that it actually
accelerates your own project speed. And so, I would start there. AUDIENCE: Some of
the advice mentioned tackling unknowns early in
the project lifecycle and some of the other advice was
focusing on the simple things. I wonder, do you have
any advice on how to identify unknowns that
could be removed by focusing on simplicity
versus unknowns that deserve their own investment
and investigation? EDMOND LAU: So, the
question is, how do you differentiate between unknowns
that need to be investigated and unknowns that you can
potentially cut by focusing on a simpler solution? I don't have a simple
answer for that but I think some of the
approaches that you can take is to think about what are your
goals for this certain project? And if you didn't have to
design a grand system, what would be the simplest
solution that you could think of that could
maybe hit most of those goals? Because a lot of
times, the complexity comes from trying to address
the last 10% or the last 5%. And if you would take a
more pragmatic approach, maybe you can ask
instead, what covers 80% percent of the use cases? Can I come up with
a solution for that? And then maybe for
the other cases, can we do something different
rather than making the core solution more complicated? And I think that's
something that-- that's a theme that I've
definitely seen other teams do with success. Because then for the
most part, they're dealing with something that
is simple to reason about. AUDIENCE: So I have
a question, a lot of cases we have
dependencies on other teams. For example, my
team, sometimes I go, there's a problem in the
[INAUDIBLE] Bluetooth stack. Then so I have to make a
talk with them or either I'd have to look into the
Bluetooth stack by myself. Which sometimes can be
harder because I don't really have a lot of expertise
on the low level code. So, can you talk a
little your experience about how do you deal
with team dependencies? How to talk and actually
cooperate with other teams? EDMOND LAU: So the
question is, how do you deal with interteam dependencies
where you're sort of blocked on someone else? So a lot of times when you're
blocked on someone else and they're not actually
working on this thing that you really care about,
that happens because there's a misalignment in incentives. Somehow your team
has a certain goal. The other team has
a certain goal. And those goals, maybe our
priorities don't really align. And so, I think
structurally, a good approach is to try to make
those goals align. To see if you can agree
that a certain focus is a priority for both
teams this quarter and sort of agree
from that perspective. If they don't align,
I think at least having that conversation
tells you then where-- the part that
you're blocked on, where that fits on
their priority list. And so, at least having
that conversation, you know that maybe this is only
like a medium to low priority for the other team. At that point, then you can
make this decision of well, do I wait for them
to work on that? Or do I then-- maybe I ramp
up on that Bluetooth stack to actually make progress. But I think it
really sort of starts from that communication
about goals and incentives. If they align, then you can
sort of rely more on that team. If they don't align, you do
have to make that trade off of, maybe it's worth my time to
learn this Bluetooth stack so that I can get the thing
out the door sooner rather than waiting for them to
get to it on the bottom of their priority list. AUDIENCE: So, you
mentioned the 60 to 80 hour work weeks not
being sustainable. I was wondering what your
typical work week is like? And if you ever work
weekends anymore? And you also mention learning. I was wondering how
much time per week do you set aside for learning. Is it an hour on the
weekend that you spend or is it five minutes a day? How do you incorporate that
into your daily routine? EDMOND LAU: So nowadays,
I still work at a startup. I work at a startup called Quip. And we actually work
very normal hours, so we work maybe 40 hour weeks. I do spend a lot
of personal time on just sort of investing
in things like the book or these talks or
other ways I can teach engineers to be more effective. Mainly because that's
something I'm excited to do. In terms of how much time
I invest in learning, I personally love
to read, so I end up just reading a lot of books. And I think books are a
really large source and wealth of information for me. I don't know how much
time I spend reading, but I [INAUDIBLE] spend
a lot of time reading. MALE SPEAKER: Thank you very
much for coming [INAUDIBLE]. [APPAUSE]
