JAKE: So-- SURMA: This is exciting
because this is a new episode. I don't know this one. JAKE: I know. How many times will we
have to record this one? Let's find out. [MUSIC PLAYING] So I want you to cover
a less technical topic. There we go. We just lost 50% of our viewers
right there and then, but-- SURMA: Goodbye. JAKE: --thank you. Thank you to everyone
who stayed, because I do think it's an important
one for most developers. I want to talk
about interviewing, like particularly interviewing
for technical roles. SURMA: That's a controversial
topic, isn't it? JAKE: Oh, well, I'm sure
we'll find out in the comments if it's controversial or not. But, yeah, I think interviewing
is really difficult. As a system, it's unusual. Imagine that after a
speed dating session, you had to send one of the
people you met a marriage proposal there and then,
because that's kind of how interviews work, right? You spend-- SURMA: You do know they made
TV shows out of that concept, right? JAKE: What? The job thing or
the marriage thing? SURMA: The marriage thing,
speed dating then marriage. JAKE: That's
interesting, isn't it, because, yeah, shows
like "The Apprentice," they spend the
whole series on it. But some of these dating
shows, it is just an hour. So I guess that is
more realistically like an interview. SURMA: I think
interviewing well as an-- I mean, interviewing
can mean so many things. It is you want-- you're
looking for someone to add to your business,
to your company. And I guess the
purpose is figuring out if that person that you
are considering a candidate is actually good for
the role and if you are good for the candidate. JAKE: Yeah, exactly. Yeah, you get an hour or two
with them to decide that. And then you pick
one and propose that you enter into a
long-term relationship backed by a contract. There you go. Job done. So I thought I would give
my tips for interviewing. I don't claim to
be perfect at it, but I've been doing it
for a number of years. So these are just
my reckons really. But at Google, I
was given an award for being one of the
top interviewers. SURMA: Which means the
most interviews, not the best interviews. JAKE: Let's-- yes,
OK, fair enough. It was more the number. But they get me to do them
because I'm good at them. That's what I tell myself. Can you remember what the
award was, by the way? SURMA: Do you mean the
yellow helium balloon star? JAKE: That's right. I got a balloon. So-- SURMA: And it
lived on your desk, I think, for a considerable
amount of time. JAKE: I had that balloon
for nine months, I think. It was a good balloon. It was a high-quality balloon. And even once that balloon
started to drift down and hit my co-workers in
the face, and they said, "Can you get rid of that?" I said, "No, that's my balloon. I got it for doing interviews. I'm keeping it until
it hits the floor." SURMA: That sounds like a
little jokey embellishment, but that actually happened. We were all observing the
balloon hover slightly lower every day as it slowly
fell to its death. JAKE: And people thought I got
rid of it because it was gone. But what I'd actually done is
put it in the recycling bin, knowing that when someone
would open the recycling bin, the balloon would come back out. I'm fun to work with. Anyway, if you're watching
this at home and you think you know better than
me at interviewing, where's your balloon? Show me your balloon,
and then we'll talk. That's what I'm saying. But, yeah, a lot of
this is subjective. And I'm looking forward
to the discussion in the comments around this and
hearing other points of views, provided the moderators
don't delete everything as they've been doing recently. Sorry about that. But here are my reckons. First up, I want to talk
about the general attitude. And you hit upon this before. You're trying to find
the candidate who's going to be the best employee
for your company, the best teammate for you as well, if
that's what you're hiring for. My first piece of
advice is don't try to defeat the candidate. And this sounds obvious,
but I see this a lot. The candidate is competing
against other candidates indirectly. They're not competing
against you. You're not there to
try and catch them out. You should try and be
on the candidate's side like you would with a teammate. And I say "should" because we
know from Twitter and YouTube comments that there
are a lot of people who have the personality
of the Riddler from Batman. It's all of that kind of, I
am more superior than you. Therefore, I win. And I just say to people
like that, grow up. Pack that in, and
definitely do not take it into a job interview
because it's a horrible thing to do. It might seem like I'm
laboring this point. SURMA: Can you implement
inheritance just with functions and manual
prototype manipulation? You don't? You fail. JAKE: I've definitely got that
stuff wrong before as well. Like what you just
said there, I've used that question
in interviews. And I am embarrassed about it. So I'm saying stuff. But the advice I'm giving here,
like I say, I'm not perfect. It's based on the stuff
that I've got wrong as well. And it might seem like I'm
laboring some of these points, but you watch TV and films
and the way interviews are dramatized there. A lot of people think that's
how it's supposed to be. Let's take a tech example. A friend of mine interviewed
for an agency in London. And as part of the
interview, they asked her to take a look at
their website, their agency site, and suggest things
that could be improved, that kind of thing. And they had this 404 page
that was this big, cute image. You know cute 404 pages, right? You've seen them before. SURMA: Sure. JAKE: But she pointed out
that their website has this consistent nav at
the top and at the bottom except their 404 page. And she was like, well, hang on. This could be a
problem because the 404 is a dead end for the user,
that they might have had a misspelled URL or something. And you're not
giving them that nav, that way of that they
could help themselves out. So there's an improvement. And this didn't
sit well, clearly, with, well, the head of
design at the company-- and during the Q&A
section launched in with, I've got a question for you. Why do you hate delight? And I was told this
story five years ago, and it still makes me angry
because it's exactly that. My friend did learn
from that experience that there was someone senior
in the company who was toxic. So kind of dodged a bullet. So it was useful in some ways. But seriously, the candidate
is nervous in an interview situation, and they're
in an unfamiliar place. So if you're the
kind of person that takes advantage of that to land
some punches, you're a bully. There's no pride
or honor in that. There's no benefit in doing it. SURMA: It's a suggestion. And I also don't think those two
things were mutually exclusive. You could have still left the
404 page cute with the head bar and the bottom bar. And you could-- I mean, it's your
opinion to disagree with a candidate's suggestion. But you have just enough
to justify your opinion as they have to justify theirs. So in the end, it
should be actually a constructive dialogue. And I think if
anything, as you said, the company failed that
interview themselves then. JAKE: Yeah, absolutely. My next piece of advice is
don't judge the candidate on unrelated things. And again, it's not-- SURMA: I mean your shirt
is questionable today. JAKE: Yeah. I've gone quite bright today. I actually bought this
for doing these recordings because every time
we've been filming, I keep getting told to
wear something brighter. So I actually bought this
with recording in mind. SURMA: Which is unusual
for you because you usually rep the conference
shirts exclusively. JAKE: I usually wear shirts I
got for free at conferences. But turns out conferences
aren't happening right now. So I've had to start
buying clothing, Surma. Think of me during the
pandemic having to buy clothes. But, yeah, don't judge
people on unrelated things. And, yeah, clothing
is one of them. But I saw this
Twitter feed recently where someone in the web
development community was saying they judge candidates
on the setup of their code editor. Like an example they gave was
the kind of syntax highlighting they use. And this is similar to
judging people by clothes. And I do have
friends in old tech who will judge people
by the suit they wear to the interview. And in both cases, like syntax
highlighting, the suit thing, folks will defend that
attitude by saying, well, it tends to
correlate with ability, or it tends to correlate
with professionalism in terms of the suit thing. But these things are just
feeding into your biases. And if you interview
properly, you can actually test for ability. You can actually test
for professionalism. You don't have to crystal
ball these things. You don't have to read
tea leaves or overthink, because it's inaccurate,
and it's unfair. SURMA: Yeah, statistics rule
101 is correlation is not the same thing as causation. Just because things
correlate doesn't allow you to look at the
effect and imply back that the cause is also true. I remember that the core team of
Go, the programming language-- pretty famous people with
Rob Pike and Russ Cox and some of the people. And they all are advocates of
not using syntax highlighting. Are you telling me that
those people are not highly productive, good engineers? I am not so sure. And as you say, don't-- the main problem is
don't use proxy metrics. Don't-- if you are in
an interview, you can, as you just said, actually
measure directly whether the capabilities that you need
are present in the candidate. There is no need to
rely on proxy metrics like this person is not
using the same header as me. Therefore-- JAKE: Bad. Absolutely. To give a couple examples
of that, 10 years ago, I was doing some job interviews. And I took along one of those
tiny little notebook laptop things that were a
trend at the time. And I had some blind text on it. I didn't use it
often for coding, but it was sort of set up. It wasn't set up well,
but it was possible. And I brought that along with
me for coding interviews. And the reason I was bringing
that along with me is it was the only portable
device I had. It was the one that
I could afford. I had a desktop machine at
home, but this was the one portable device I could afford. SURMA: Could have
brought that, I guess. JAKE: I could have. I could have brought
the whole desktop unit, slammed that down on the desk. And it's like, can
I get a plug socket? Plug socket, anyone? But, yeah. But it is kind of
horrible to think that someone would
see this tiny laptop and think that that's
how I normally did things or that was the best
I could set up an IDE. And the same goes with clothing. To most job interviews
I've been to, I'm wearing what I was wearing
for my job at the time, because if I turned up
for work with a suit, there's only so many times that
I can say wedding or funeral. It's like, well,
what is it today? Oh, it's a funeral this time. Yeah, it's a funeral
for the person that got married yesterday. It was a very eventful wedding. You know? It doesn't work. People are just going to
wear what they were wearing or what they're
comfortable with. SURMA: Yeah. JAKE: But I would say
there are exceptions to some of these
things, like maybe there are cases where the
role does involve seeing how the candidate
handles someone who's being aggressive and argumentative. Maybe the role does
involve familiarity with some of the IDE
trends, plug-ins. Or maybe the role does
involve someone being dressed in a particular way. Again, you should
test for these things, but also tell the candidate that
that's what you're testing for. Days in advance,
tell them that's what you're going to be assessing,
so they should prepare for that. If you're going to
have a coding test, let them know you're going
to have a coding test. If they need to bring their
own laptop, let them know. Yeah. It sounds simple, but it goes
wrong in practice quite a lot. SURMA: I agree. And I think there
could be more clarity here as well in many
instances that I've seen. But it's just like what
you're going to test and how you're going to test it. We often talk about
whiteboard interviews, which have gotten
a lot of criticism because that's not
how you do your job. You don't write code
on a whiteboard. And I know how-- I remember how painful it
is to write code, write curly braces with a big
marker on a big whiteboard. It's just not how your eye has
been trained on a daily basis to perceive your own
code, and it will just lower your abilities. And even to this
day, I often end up in a situation where I
have to tell an interview candidate that
they expect people to write the code in
a Google Doc, which is better than on a whiteboard. But a Google Doc is not what
we are used to writing code in. So Google Docs doesn't
take care of indentation, or syntax highlighting, or
any of those sort of things. And so I always try to
be very vocal about like, I understand this
is not your IDE. Don't worry about syntactical
errors, misspelled API names. We all know that IDEs
take care of that for us. It's about talking
about the problem, knowing that you have
the right solution. Often, we also know
that we google a lot. We get help from the internet,
which sometimes in interviews for some reason is frowned
upon, which technically I also don't understand. And all these things I
think should, as you say, be laid out before. And what are the criteria? Why are these criteria there? And what is allowed,
what's not allowed? JAKE: And most of
what you just said on when it comes
to coding questions has just taken care of a
big chunk of my content that's coming up later in this. SURMA: Apologies. JAKE: No. SURMA: We can just
skip my comments. JAKE: No, no. We'll keep those in because
I can just skip to later on. And I do want to talk more
about testing for coding. But first I want to just
start with the very beginning of the interview or even just
slightly before the interview, because I would say make sure
the candidate is comfortable. Do they need a bathroom break? Do they need a break
for any other reason? Do they need a drink? Some of these are
more important when it comes to on-site
interviews than VC interviews. But to give an example,
we had a candidate coming to Google who they had
to rush because their train was delayed. So they'd sort of
run to the office. And as such, they were sweating. And so I just said,
hey, do you want to take 20 minutes to cool down? And they said yes. And I don't know if
they would have asked for that if I didn't offer it. But it felt important
because I know how I get in that situation. I end up in a sweat loop because
I'm sweating because I'm hot. Then I start getting
embarrassed that I'm sweating. And eventually just shame ends
up siphoning all of the water out of my body. And I am not going to interview
well when I'm like that. SURMA: No, and that's the thing. If you're running,
evolutionarily speaking you are in fight or
flight, which means your blood is anywhere
in your muscles but not your brain,
which will significantly hurt your chances of
using it correctly. And those things interviewers,
I think, need to be aware of. JAKE: Yeah. You want the candidate
at their best. So enable them to be that. All right, let's talk
about the questions. I would say, maybe
this is controversial, but use the same questions
for each candidate. Some say they prefer more
flowing, natural conversation. But I prefer the same questions
because it's a stronger basis for comparison. It also avoids some
unconscious bias. You can branch
off your questions with follow ups and
that kind of thing, but I would say stick
to the same core questions for each candidate. SURMA: I actually
do the same thing. I have the same
question that I've been using for a long time,
which actually is helpful because I know what my level
of expectation should be, like what is the bell curve
of people at different levels? How far do they get? What kind of traps
do they run into? How can I recognize early what
holotype of interview candidate this is? And how can I help them get
over a hump that is potentially just a red herring to
actually evaluating their actual performance
and capabilities? And so I think it's
really, really good advice. And, yeah, you can
refine it over time. You can find different
branches over time for people with different skills. Some people care about
memory optimization. Some people care about
runtime optimization. Some people care about
the visualization. And all of these things, you
can usually take your problem down different roads
while still sticking to the original question,
so you know what to expect. JAKE: Absolutely. But I would say for
the first question, start open-ended
and uncontroversial. Don't make it a big,
technical deep dive. Try and ease the candidate in. I usually go for
something like, in terms of work, what do
you enjoy doing, or what do you want to focus
on in this particular role? SURMA: Apologies. Yeah, I was jumping to
technical coding questions, but you're right. Obviously, the
interview doesn't-- shouldn't start
with that actually, because it is very important
to get to know your candidate. And just what kind of
person are you talking to? JAKE: Yeah, and it just
gives them an easy question that they can answer as well. And so it kind of
puts them at ease. I would say,
especially if you're saying what do you
want to do in the role, or what do you enjoy doing, I
have seen a lot of candidates-- and I would say this correlates
with younger candidates, but it doesn't matter,
but it does tend to-- will say something
like-- or they'll take an attitude of like, I'll
do whatever it takes, sir. They're really eager to please. And if that happens, I try
and nip that in the bud and say, no, here's the things
that I like to focus on. Here's the things I don't like. You're, by example, saying,
look, it's OK to have a focus. It's OK to have an
opinion on this stuff. And with this
question, yeah, it's mostly about easing them in. But I'm also looking to see that
they've thought about the role, that they have enough experience
to know the things they enjoy. SURMA: I mean, and this
is specific to developer relations, but I always
think developer relations has so many different facets. You can do the stupid stuff
that we do by doing videos. You can write articles. You can write libraries. You can go to conferences. You can communicate more
with the engineering team and help them out set
the right roadmap. There are so many different
facets of developer relations that are part of the role,
that actually nobody can do all of these at the same
time at the same level. You will be better at some,
and you will enjoy some more than others. And that's exactly what
is interesting actually about candidates. Where do they see
their strengths? And what part do they actually
know is part of the role? Which part they maybe didn't
know was part of the role. These are all interesting
conversations to have. JAKE: Yes. But no matter what
the first question is, there's something else
that happens quite a lot. And that's that
the candidate has been thinking about this
interview for a while. They've been thinking of
what they're going to say and the points they
want to get across. And sometimes the question
you ask first doesn't quite match what they were expecting. But they're not going
to let that stop them. They're going to give the
answer to the question they wish you asked instead. And it's tempting to get
sarcastic at this point. But, again, you're
not there to win. Avoid telling them off. Just let them say their bit. And then rephrase your
question and ask it again. SURMA: Yeah. And we have-- you have
to be careful that people are going to be nervous. Potentially, they're
scared to bits that they're now,
depending on the company, have one interview, two
interviews, five interviews. And they probably
want to get the job. Otherwise they wouldn't be here. And that might just inhibit
them from parsing the question correctly. And some people need to prepare
for an interview to feel calm and to be able to
get through it. And so, as you say, it is
tempting to be like, huh, candidate can't even
listen to a question. But be careful about
those kind of assumptions. JAKE: And I would say,
if that's something they do throughout
the interview, it's noteworthy that they're
not great at listening maybe. And that could be an
issue for an employee. But if it's with
the first question, I wouldn't worry about it. All right, let's do a little
bit of-- let's do a role play for this next point. Surma-- SURMA: Oh, boy. JAKE: --I want you to ask me the
question that is on the screen. SURMA: All right, Jake. JAKE: Hello. SURMA: How would you
defeat a giant robot army? JAKE: Well, good
question, by the way. I would build an even
bigger robot suit. And I would punch all the evil
robots in their robot dicks until they were very sorry
about the whole thing, and the planet was saved. SURMA: Well done. 10 out of 10. You're hired. JAKE: OK, let's try that again. Another question for you. SURMA: Jake, so
tell me about a time you helped defeat
a giant robot army. JAKE: I've never done that. SURMA: Fired. JAKE: And if you are looking for
someone to fight giant robots, you just learned
something useful. I don't have the experience. Now, aspirational questions are
OK, especially as an opener. But if there's a way to
phrase your question that taps into actual
experience, I would say that's usually the better way. If you ask the candidate, how
would you handle a disagreement with a co-worker, you're asking
for a theoretical answer. You'll get a fantasy answer. And that's not as useful
as if you said, tell me about a disagreement
you had with a co-worker and what you did to resolve it. Now you're getting
something real. SURMA: I punched
him in the face. JAKE: Well, that's an
important thing to learn. You want to learn that. If you've got someone
violent on your hands, that's something you want to
learn as early as possible, hopefully before hiring them. But, yeah, if you're
asking for evidence, you learn about
their experience. You learn about their attitude. You also learn about
their communication skills through their
telling of the story. If the candidate
doesn't have an answer, offer them thinking time. If they can't think
of anything, then say, sure, OK, you can give me
the theoretical version. What would you do? Not very useful to
you, but at least the candidate gets through
the question, which brings me to my next point. Try to avoid making a
candidate feel like a failure, even if they kind of are. If they flunk a question, try
and make it so they didn't. Interviews are stressful
and high pressure. You're trying to get the
best out of the candidate. If they struggle with a
question, help them through it. Even if it becomes useless to
you in terms of the interview, at least they feel like
they got through it. And they're not taking
negative baggage into the next question
or the next interview, because it could have just
been that one question that they struggle with. Due to nerves, they got
in a mess or whatever. SURMA: Yeah, or you tapped
into their weak spot in terms of topic. These are all like it could
just be that you, as the one interviewer, got unlucky
with this candidate in a specific constellation. And on that note, I would
also avoid the typical phrases where you go like, oh,
that's an interesting answer, because even though
it is sugarcoated, interview candidates
can pick up on that. And then you have just
increased the pressure without even realizing it. So try to be genuine about
helping them and actually wanting them to succeed. JAKE: Also, I'd say that if
you're expecting the candidate to say something negative about
themselves or the company, make sure the candidate
knows that's OK. A lot of candidates
are nervous about this, and it will work against them. What they tend to do is
actually say something positive but wrap it as a negative like,
my problem is I'm too helpful, or I think the problem
with the company is there's no way
it can be improved, or something like that. So just let them know, hey-- say something negative
about yourself, or say that you're looking for
people who will change things about the company. So you're looking for people
with those kinds of ideas. So it kind of puts them-- you're letting them know it's
safe to say negative things, both about themselves
or about the company. OK, it's time that we talked
about the coding question, which we've sort of mentioned. SURMA: Duh-duh-duh. JAKE: You ruined all my content. So no, this-- your
points were very good because it's
some of the stuff that I wanted to say as well. Coding tests are controversial. I think it's good to assess
the candidate's ability. Whether a coding exercise
is the right way, I don't know for sure. I tend to do a 20
to 30-minute coding exercise in an interview. As such, the
questions I use tend to be simplified compared to
real world equivalents, which is a weakness of that system. There are alternatives. I've seen companies do homework
assignments or longer pair programming sessions. It's a difficult
balance to get right. Something short is
short, which is good. Something longer can
be more realistic. But then it might not be
respectful of the candidate's time. Remember they've
got a full-time job. So if you're taking them in for
a whole day of job experience, essentially, one, I hope
you're paying them for that. But also, they've
got a full-time job. Do they actually-- you're
taking a lot of their time away to do this. SURMA: On the one hand,
the coding exercise during an interview
is inherently unrealistic of an
engineering job because you will spend the
entire day on coding, most likely, and not just
20 minutes on a problem you have just discovered
where you are not allowed to google things. It is inherently unrealistic. Some companies have tried
to make it more realistic by taking a bigger
realistic problem and giving the candidate
a week to take it home. But as you said, that is not
only disrespectful potentially of their time because
they might have a job, so they now have to
sacrifice their evenings. But also, companies
are giving them work that they
could actually use, like that is useful to them. And now they have interview
candidates do work for them, which is also not fair. So it is an extremely
delicate balance to strike. JAKE: Yes, and I'm going to
stick to my experience here. So it's the small
coding exercise. In terms of that
stuff, I would say avoid "write this
particular algorithm." SURMA: The infamous
invert the binary tree. JAKE: Yeah. So instead of write code
to invert this binary tree, present a more
real world example which could be solved by
inverting a binary tree. If the candidate realizes
that inverting a binary tree is the right way to
solve the problem, that's like 85% of
the problem solved. In the real world,
they could look up how to invert the binary tree
if they couldn't remember it. The important bit is
being able to connect the theory with the practice. A lot of candidates, they cram
learn all of these algorithms. But if they can't see
a problem and realize which algorithm
they need to use, that's a gap in their learning. And it's something that's
useful to you as an interviewer to find that out early,
because those connections are the most important thing. SURMA: Yeah. As a little tip from me as
an interviewer to people who take interviews, who
interview at companies, for me-- and actually, I only
can speak for myself obviously. But if you are
solving a problem, and you know there
exists an algorithm, or you even know
the name but not how it works, for me it
is a 10 out of 10 points answer if you say, let's assume
I have the function called invert binary tree. And you just call it without-- like you just use
it in your code without having the
implementation. Knowing an algorithm
off the top of your head and how to implement it to
me is not part of the job, unless you are working on-- I don't know-- the library
that is inverting binary trees, and you are interviewing
for that job. Then, yes, then that
is part of the job. But that is the niche. In the general case, if you
know there are solutions and you can even name them, you
get full points from my end. JAKE: Yeah. I would say, when I'm
interviewing as well, it doesn't matter if they
can't name the algorithm. If they write what is
inverting a binary tree, but they never say the
words "binary tree" or "inverting," doesn't matter. I don't need someone
to name those things. I would say, yeah, try and find
something real world as well, like maybe a problem that
you solved as part of the job that they're applying for. But, of course,
if it's something that took you hours
to solve, it's not fair to expect them
to solve it in 30 minutes. So you'd really
sort of boil it down to maybe the
interesting parts that are solvable in a
short amount of time. Also, try and find something
with multiple phases, like something that has
like a simple solution that maybe isn't optimal. And then you can build on
top of it to make it better, or make it faster, or
make it hit edge cases. SURMA: I strongly
agree with this. I like problems with
multiple solutions, with different not
only complexity, but it's a trade-off. And even just having
a conversation about these
trade-offs can be very interesting to assess how
much the candidate has thought about trade-offs, memory
trade-off, runtime trade-off, code complexity
maintenance trade-off. These are all things that
matter usually in real cases. And it helps you ensure that,
as you said earlier, feel-- make the candidate feel
like they're succeeding, because it is the more
solutions that are possible, the more likely it is
they will find one of them and can help you or can
explain them to you. And something that
actually is really nice once they found a solution
is something that I often do is ask them to write one
or two unit tests, which-- JAKE: Oh, that's nice. SURMA: --makes them think
about, can they actually abstract the interface? Can they figure out edge
cases that are worth testing? How do they test? And all these little things
can be very, very interesting to talk about. JAKE: Yes. And if the candidate
struggles as well, if you've got a
multiphased question, you can just help them to one
of the phases, the next phase. And then call it a day. And the candidate feels like
they've achieved something. They're not taking negative
energy into the next interview. I would say if you're
doing that kind of thing, be careful you don't scare
the candidate with simplicity. Let them know it's
multiple phases. I've seen candidates freeze up
because the start of a question is relatively simple. They feel it's simple, and
they're looking for the gotcha. They're looking for the
complicated bit they haven't spotted, but it's not there. SURMA: Yeah. JAKE: And obviously-- SURMA: I've definitely had that. One of my old questions
or one of my old series of interview questions
for the coding started with, OK,
let's make the tools we need for the bigger question. How do you generate a random
number between A and B? And exactly as you said, people
froze up and were wondering like, they had a solution, but
surely it can't be that simple. And so I've learned
over time to be like, this is not a trick question. We are building simple tools
that we can put together later on for something way bigger. So just roll with me. JAKE: Yeah, I would
say definitely avoid using the word "simple" as
well in case the candidate does struggle with [INAUDIBLE]. SURMA: True. No, that's very good advice. JAKE: I usually say
something like, if you think of a solution, but you're
unsure whether it's optimal or complete,
don't worry about it. Write it down. It's better to write
something down, and then we build upon that
rather than try and get it perfect first time and
end up with nothing, which I have seen happen
in some interviews as well. During these questions,
helping the candidates fine. Just make a note
of how much help they needed, because that's
part of your assessment. Again, it means they leave
the question hopefully feeling positive as well. Here's something that you
mentioned earlier as well. Allow them to use
their own laptop. Don't make them code
on a whiteboard. If they want to use a
whiteboard, that's fine. But don't make them. Don't make them code
in a Google Doc. And this is me apologizing
for every Google interview where this still happens. I refuse to take part in this. Even if the interview
advice I'm given is, say, telling me
to use a Google Doc, I don't because I just
don't think it's right. So, yes, that does still happen. And I'm sorry about it. No, people should be
using the environment that they are comfortable with. Also, try not to assess
too much at once. If you're testing their
algorithmic problem-solving, tell them you're not worried
about variable naming because that probably
doesn't matter. You said some of
this stuff before. If you do care about
variable naming, maybe present them with
some badly written code, and ask them to
refactor it to be more maintainable, because then
the algorithm's already there. They don't have to
worry about that. They just have to read the
code and make it better. Also, don't test memory. You said this as well. Allow them to look stuff
up like method names. Remembering how slice works
versus substring in JavaScript, it doesn't matter. I usually tell candidates to
use me as a search engine, and they can ask me questions. And I will tell them, or I'll
look it up myself because it's good to sort of hear where-- what they're thinking. And that's a good
way of doing that. SURMA: Whenever I-- I've
done the same thing, but I often offer candidates
to use their editor instead of the Google Doc. And some candidates still
choose the Google Doc, but it's their
choice at that point. But in the editor,
I just ask them to share the screen with me,
so I see what they're doing. I don't think autocomplete
is cheating, not at all. It is a tool we all
use when we work. Why would it be cheating? I've seen this a couple
of times where people-- you are not allowed. It's more like an
exam where they expect you to know everything
on the top of your head, which is just wildly
unrealistic and not in line with what the
job actually looks like. JAKE: We all make mistakes, more
so under interview pressure. And if the candidate
has an off-by-one error, is it worth making
them sweat over it? I tend to ignore it or
say, hey, on line 12, you've got an off-by-one error. And they'll spot it. It's not worth it. It's something they would
see in the real world when they run the code. I'm not going to
show you the coding question I use because then
I'd have to pick a new one. And I like my coding
question too much. But here's the example
of the kind of thing I'm talking about. So this would be aimed
at web developers who are expected to write JavaScript. It's kind of showing,
can they fetch some json and append some
stuff to the page? I wouldn't expect them
to remember the DOM APIs. I'm just looking to see if
they know how the DOM works and how async code works,
because that is something that they will have to
do in the real world. And then in a second
phase of the question, I'd say, OK, our API returns
10 results at a time. We'll fix that one day. But in the meantime, we'd like
to display 50 results at once. Can you make that
happen with this API? So now they have
to do five fetches. It tests more
advanced async stuff. Can they return the
stuff in the right order? Do they fetch it one by one,
or do they fetch it altogether? Do they wait until they've
got all of the results before putting any
of them on the page? These are all little phases. They might choose they fetch
the first set of results, and then they start
fetching the next one. And then you could
say to them, hey, is there any way you could
make this faster by-- well, I wouldn't tell them
by fetching it all at once, unless they needed the hint. But see if they could
figure that out themselves. There are many layers
to something like this. Ask them if there are certain
user interactions that might cause bugs here. Could they spot
if the user enters submit multiple times before
the previous results have come back? You might end up with those
sort of things interleaving. Do they know how to handle that? Do they know how to handle
that in an efficient way? It's a really basic question. It's something that we
tend to do in our jobs quite a lot, fetching some
data and displaying it. But even to something
like this there are so many layers
that you can add to see where the candidates at. SURMA: Yeah. And as we mentioned
earlier, you can take it down many different
paths depending on where the candidate's
strengths are, which hopefully, at this point,
you've talked to them about. If they are more
into visualization, and you want to see-- and that's something that you
actually want on your team as well. Maybe you want to change it
up and lead it down that path. Maybe they can visualize
the in-progress requests or something like that. It's nice to have these
questions that are flexible. And you can bend
them a little bit to fit more towards
the candidate, so you can make the candidate
shine at their strengths. JAKE: Absolutely. And once all that's done, you
say goodbye to the candidate. You've still got work to
do because you need to rate the candidate's performance. I would say, make
sure you stick to what happened in the interview. This comes into play
when the candidate is someone you know,
either a friend or someone with an online
profile that you're aware of. Let's say that candidate
A performs better than candidate B. But
you know candidate B, and you know they didn't show
their best in the interview. And you decide, oh,
they're actually better than candidate A based
on some of the stuff you know. But maybe candidate A wasn't
showing their best either. So it's kind of unfair that
you're offering that leeway to candidate B and
not candidate A. I would say if you're unsure,
do another round of interviews. Get them both back in again. Also, write your assessment
straight afterwards. Make notes during the interview,
but write up your conclusions straight away because your
memory doesn't get better with time. SURMA: No, it really doesn't. And I would also
say that's something that I have experienced
and therefore do in my-- when I interview as well. Tell the candidate that you
are taking notes because, especially right now
in a virtual setting, it can often seem like
you're not paying attention. You're just chatting
with your work friends while you're waiting
for the candidate to come up with a solution,
because maybe you're touch typist. So you can look the person
in the eye while typing. I can that to an extent. But often, especially
when it's coding things, I do look at my notes. Yeah, just make sure
the candidate knows that you are being respectful
and you're listening, even if you're not looking
at them and typing. JAKE: I've been on both sides
of the table on that one. I've felt like I was being
ignored because they were just looking away going, uh huh,
tap, tap, tap, tap, tap. Uh huh, tap, tap, tap, tap, tap. Yeah, but they're making notes,
which is the right thing to do. Yeah, also agree
on a rating system, so each interviewer is
scoring the same way. That kind of thing really helps. Also, come to your
conclusion before speaking to anyone else who is doing
the interview, because group and actually
polarization is a thing. Even without you
knowing it, by talking to someone else, your
opinion, it might even change. But it's more likely
to become more extreme. So record your own viewpoint,
so when you discuss it with other interviewers, you
can be persuaded, of course. They can say, well, I felt this. And you could adjust
your own viewpoint. But make sure you have
your initial viewpoint, and make sure that's heard. SURMA: Yeah. I think in general, biases
are a thing to be aware of. And it can be biases
introduced after the interview, as you say, by talking
to other people without somehow writing
down a conclusion first. But also make sure
in the interview to be aware that we as
humans suck at being neutral and open-minded. Usually, we see a person. We have an opinion somewhere in
the subconscious of our brain. And it can and most
likely will affect how you perceive what they are doing. Their agenda, their race, their
earrings, all of these things can and probably will
affect your evaluation. And you need to work hard to
not let them influence you, and use fair criteria JAKE: Yeah, and that's
really at the root of all of these tips
and rules that I've put into this presentation. It is to just try and avoid
bias and avoid unconscious bias as much as possible. It can't be eliminated,
but you can avoid it through some of these methods. But that's all I've got. But it did get me a
balloon, and it also got me some good teammates as well. SURMA: Well, I'm
not sure about that. [MUSIC PLAYING] Focus on the documentation
and the naming. [CLEARS THROAT] I squeaked
while I said that. Make notes during the interview,
but write up the conclusion straight away because you're
[INAUDIBLE]---- oh, [BLEEP].. I can't speak today. We're so close to the end.
