[MUSIC PLAYING] NANDINI STOCKER: If you've
been hearing a lot about voice technology, artificial
intelligence, and how it's the next big
thing, but you're not quite sure what that means for
you, you're not the only one. Whether you're a
developer trying to create a
conversational interface, a business that wants
to know what this means for the future of commerce, or
just a human being wondering why in the world we need
more robots when we're trying so hard just to
connect to each other, you've come to the right place. Hey, Sachit. How's it going? SACHIT MISRA: Not too bad. How about you? NANDINI STOCKER: I'm
so glad you're here. SACHIT MISRA: Glad to be here. NANDINI STOCKER:
Seriously though. You've been doing this
for quite a while. SACHIT MISRA: It's
definitely different. And it's fairly new. NANDINI STOCKER: I'm Nandini. And welcome to this
new video series where we'll explore all
things related to voice tech and the rich elements
of human communication that will be the
key to its success. Hey, Google. Let's talk to Number Genie. NUMBER GENIE: I'm thinking
of a number from 0 to 100. NANDINI STOCKER: We're
looking for great apps out there to
highlight how you're using these elements to
create your own conversational experience. SACHIT MISRA: What you're
building is a conversation. NANDINI STOCKER: What was the
biggest challenge for you? SACHIT MISRA: You realize
you have so many options. NANDINI STOCKER: How do you
maintain that variability? SACHIT MISRA: So there's
a lot of opportunity here, actually, for the
developer to be clever. NANDINI STOCKER: Because
cats rule the internet. SACHIT MISRA: Right. NANDINI STOCKER: So let's talk. It's kind of where we're
starting today is hello, world. [MUSIC PLAYING] [APPLAUSE] NANDINI STOCKER: I'm Nandini. And I enable conversations
between humans, among businesses, and any
combination in between, some of which include some machines. Who's excited about
this series, developers? [APPLAUSE] We just launched this
as a teaser on YouTube. But it's coming,
and you're going to hear all about that today. But I want you to stop
and take a moment. Take a picture in
your mind right now of why voice is interesting. Why is it? I wonder what you're thinking. We're living in a
time when innovation is shifting our outlook
so rapidly of our future. In fact, it's so
rapid that it's worth taking that moment to stop and
think about where you are now and how different it could be in
a week's time, a month's time. And next year, I
guarantee you will be a different landscape, a
different ecosystem than we are today. You're part of that, by the way. So why is that? Enabling machines to
speak on our behalf, to speak to us out loud,
is so deep to our context as a culture and
as human beings. It makes us question
where we are. Makes us question where
we are with technology. It really is putting us at
the intersection of technology and the human condition. More than ever
before, we're actually confronting a reckoning between
science and culture right now. So I have a lot more to
say about that topic. But today is not
the forum for that. Today we're just
starting a conversation. And the hardest part about
any conversation is listening. So we want you to know,
developers, designers, businesses, we've heard you. We've seen the feedback
from I/O and conferences like this, that the most
valuable part of the experience is interacting with Googlers
and with other developers and people creating
experiences out there. So we decided to
create a different kind of content feedback loop for
you and with this series. So in our first few
episodes, for example, we're going to start by
helping you understand just how to get your arms around the
Zero-d interface and then how you supplement that with
other elements like visuals. We'll show you what palette
you have to work with, and how to create
your landscape, how to build your world for your
users in your own unique way. So in our first season,
we just wrapped filming. It's very exciting. We're going to have six
episodes we've already filmed. We're going to have things
like greetings and goodbyes. How do you start and
end a conversation? Including, how do
you do reengagement? How do you identify users? How do you authenticate them? And how do you
make money at this? Transactions are a
big part of this. We'll also show you how
to create custom voices to brand your experience. What does it take to create
character and persona? And also, what are sounds
involved in all this? There's audio
landscaping you can create that's an
entirely new world, that will be the gateway to
things like virtual reality technology. So from there, we'll explore
timely topics and deeper concepts and feature
and highlight examples from the ecosystem. We want to showcase
what good looks like, what are you guys creating out
there, and what kinds of people it takes to make this work. And then on each
episode, we'll always feature guests that will
help us dig a little deeper. And they will be people
from the community. Sometimes they'll be Googlers. And come on up, you guys. [MUSIC PLAYING] So we're going to do a
little live show today to talk a little bit about what
this means and hopefully get a little bit of insight. And we'll actually not
answer all your questions. But we have a few questions that
I've gathered from social media and just from our advocacy work
that we've done in this space. So to start out, I want
to introduce our panel. And I'm thrilled to have
this great set of panelists here that I personally selected. Because I feel like they're
most representative of where people who have come
at this and pivoted to this field from
other areas to give us some insight into their
journey of how they got here. Because I really think that
there are a lot of experts in the field, but
actually very few that are considered experts. But there's this tapped in
muscle that you don't even realize you have, that
you can design and build for this space. And that's what
I'm hoping you'll get out of this today
and the whole series. So we'll start by-- first
we have Marc Paulina, who together with Peter
Hodgson in the audience, they've created a new rapid
prototyping technique that involves-- it's a methodology for
interaction design. And in fact, to give you
an idea how we're really taking the audience and building
the series with the audience and our users in mind
and our viewers in mind, we've built this panel
also directly from feedback from you guys. So, for example, Marc just
ran a sprint with Peter with a small group
across the street with this new methodology. But it has to be done
in a small scale. So we want to scale that big. So we're going to show it on
camera so that you can all learn from it. That's an example. Another is Kimberly Harvey, who
has pivoted with deep research experience. Recently pivoted to design. She'll tell you about that. But Kimberly on the
panel is a direct request from the GDE community. Yes, Shuli. Shuli from the Tel Aviv
area, a UX research expert in children's research,
asked about having more representative
from UX research. And Kimberley offers such a
great point of view for that. So she's here. And then finally, if you
don't already know him, Sachit, he has found
the decoder ring between design and engineering. And I can credit Sachit
as fully responsible-- him and his team--
for pulling me out of my shell of where I was
working and doing more advocacy work. So thank you. And also, another example is
the G+ community, I'd asked, what else do you want
to see in our videos? And the immediate answer was
authentication, transactions. How do we do those? And so we reached
out to David Wang, who's going to be on one
of our very first episodes talking about that, who is from
the Product Management team. So this team are all
advocates for you. And we're really excited
to dig into some questions. So welcome, everybody. So Sachit, let's start with you. Voice. What's all the noise about? SACHIT MISRA: Sure. First off, Nandini, thank you
for having me on the panel. And thanks to all
of you for coming. It's really exciting
actually to be here. Just Nandini gave me
a great introduction. But just to add to
it a little bit, I'm an engineer at
Google working mostly on developer tooling. And I was working on the Living
Room team before on Android TV and Cast. And Google Home became
part of the Living Room. So that's how I got sucked
into this, thank god. And so about a year
ago, Nandini and I were working on some
of the first samples. Which by the way, she helped. If you ever have
the chance to have her help with an
app you're building, she has an extraordinary
talent for both the design and the technical portion. She'll really help the-- NANDINI STOCKER:
Please continue. SACHIT MISRA: --the
code you write. Seriously, she saved
me a lot of time. Trust me. In any case, to get back to your
question, Nandini, about what's all this noise about. I think that this noise
that we're hearing is the low, low hum
of the Sci-Fi promise, to take words
directly from Nandini. And what I mean
by that is we are at the very, very,
very early stages of the next mode of interaction
between us and machines. And this is something we've
dreamt about for decades. And it's finally happening. And so far, it's
still a little rough. A lot of these interactions
are still a little frustrating sometimes. But what's important
is to listen through the noise for the
signal of those magical moments that we have with machines. And it's happening today. And we're going to be
telling future generations about these days, years
from now, when it's working as they expect it to. NANDINI STOCKER: Great. Thanks, Sachit. So how about you, Kimberly? What's all the noise about? KIMBERLY HARVEY: God. So I guess noise, meaning
what's exciting about voice. For me, what is
exciting about it is I feel like we're getting closer
and closer to a direct thought download. I studied communication and
cross-cultural communication in grad school. And everyone knows humans have
a lot of different methods of communicating. Someone might be
watching this on video. I'm using gestures. I'm using words. We have systems of writings
all around the globe. But I feel like having humans
interact in a way that's been natural-- that's been one of
the first skills that we learned as babies-- can get rid of some of
the challenges we have-- we're having-- can get rid of some
of the challenges we have getting one thought from
one person's mind to another. NANDINI STOCKER: Great. And Marc, you've been a
designer for a long time. And you bring some
unique perspectives on interaction design. But you started in voice,
actually, but way before we had the Google Assistant. So how about you? What's all the noise about? MARC PAULINA: Yeah. I think it's a really
interesting time at the moment. So talking about like the
journey that we've had, I wasn't a voice
designer before I started working on voice actions
in the Google Search app. I was a interaction
designer working on mobile, working in automotive and TVs. But voice for me was
quite a stunning journey. It was so different
than what I was used to. So for me, it's been
really enlightening thinking about voice, and
conversation, and natural user interfaces. And how there's such an
emergence right now of all of these technologies, like
AI, Internet of Things, robotics, and so on, where
the technology is really trying to take the burden and
the friction away from people. And so we've got all of
this, these new natural user interfaces that set people's
expectations so much higher than we've had before. So as a designer,
it's been really exciting to try and create
these new design methodologies where we can understand
people a lot more and try and meet
users' expectations. As Sachit was saying, in the
future, it should just work. But right now, with the
technology how it is, there's a lot of really
interesting design problems. NANDINI STOCKER: Yeah. What we've been
seeing that there is some kind of forgiveness there. People are at this point
where they're like, OK, we get that it doesn't work
perfectly all the time. And it's great to see that
people are working on that. But it's like people get
the possibility finally, which is so exciting. Well, let's start. I have a few questions. I polled online. And then we have a few others
that we got from just asking around at the conference. We do user-centered research. Very not last minute at all. So this one comes from Twitter. Bob Stoltzberg. Is the Google
Assistant a Girl Scout? KIMBERLY HARVEY: I
can take this one. Well, the first
thing I would ask Bob is, why do you think
it's a Girl Scout? What is it that lends
itself to that stereotype that you have in your mind? That's just from a
research perspective. But to answer his
question, I would say that the Assistant
is not a Girl Scout. But it can facilitate
relationships with characters
like Girl Scouts. We have an entire team at
Google that just concerns itself with the personality. Make sure that the
interactions we're designing are on par with
that personality, make sure the wording is right,
and also shows different parts of the Assistant's character. NANDINI STOCKER: That's great. And Ryan Germick, who heads the
Doodle team and the Personality team, has a great analogy, that
you ask personality questions. You poke at things
and ask it things that you can know what to expect
an answer from or a little-- like what color-- what's
your favorite color? Things like that. And it's like, when you-- it's a way of
establishing trust. So when you establish trust
with your neighbor, for example, you might borrow a cup of sugar. Like Ryan says,
borrow a cup of sugar before you go and ask
for the lawn mower. So one of our episodes
features how to write. How do you write for personality
questions, that are just about the character
you're interacting with, as opposed to the
deeper back and forth interaction about whatever
that is the task you want to accomplish? So that's a great question. And then, so we'll switch
to a-- oh, a zinger from the developer community. This one via LinkedIn,
probably for Sachit. But maybe one of the
others can pipe in. So there's a common
sentiment among developers, who are leery of starting
projects found in newer services and tools,
as they've been burned numerous times in the
past by deprecation and abandonment of support. So given the inherent
community-based nature of conversation design, can
developers find assurances that Google recognizes this pillar
and won't be abandoning support for the Assistant SDK,
API.AI, the speech reco API, or other related projects to-- this is amazing--
allow innovators to create new products and
even found businesses upon? Well, I will say one thing. If we were doing that,
none of us have any jobs. So we're here. So just to say that. But Sachit, from the
developer perspective, you guys work with the tools
and all the APIs and that. So what would you say to that? SACHIT MISRA: So first, I
would empathize with the person asking the question. NANDINI STOCKER: It was
Brandon Hunter, by the way, from LinkedIn who asked
amazing questions. Thank you, Brandon. SACHIT MISRA: Thanks, Brandon. So I've felt that burn before. So I certainly wouldn't want
to inflict that on anyone else. I would also say in
this space that I think a little bit extra
of an adventurous attitude is actually called for here from
the developers point of view. Because early investments now,
in terms of just picking up the knowledge around these
APIs and around even just the terminology. Just figuring out
what the design looks like for these types of things
will pay off exponentially in the future. But to directly
answer the question, in terms of Google's
investment in this space, I think it's very
clear, just even using the list of
services and APIs that the person asked
in the question. It's very clear
that we're trying to push an entire suite of
products on developers here. And I think that gives
evidence to the idea that Google's really trying to
push heavily into this space. And I can tell you from
personal experience that, I mean, just
internally, we are seeing this as the next step. And this also is shown
through the consumer side. So the Google Assistant, as
an entire company initiative, reflects the developer APIs and
services that we're releasing. So I think what's
clear, if you look at the scope of what Google
is actually putting out there, is that we see this really as
the next step for both users and developers. And we will be supporting
this moving forward. So I wouldn't worry too much
about the sort of platforms going away or these
services going away. Even if there are changes
and that kind of thing, clearly the platform as a
whole, the conversational push as a whole, isn't
going anywhere. And we're sticking to that. NANDINI STOCKER: Great. Thank you. OK, let's pivot to design. Marc, can you talk about
some of the design methods you've used for
voice interaction? MARC PAULINA: Sure. So, as I said, learning the
design process for voice, being new to the field
is quite a challenge. As I said, we can't use a
lot of the same assumptions and the same design
process and methodologies that we used for
other experiences. For example, designing apps,
websites, and so on, for voice. So it helps to keep
learning new methodologies. So I personally take a lot
from service design theory. One of the principles
of service design is that we democratize
the design process. So that means-- for me, it means
empowering the whole product team to be able to come up
with powerful experiences and come up with
user-centered designs. And so the way to
do that is, just as a designer, or a researcher,
or as anyone in the product team, just to be aware
of who the user is and what the user needs. And keep asking the question,
what's the motivation? What's their goal? Even what are their
anxieties and their fears? The more that you understand
about the user, the more user-centered your
designs are going to be. Like you said, people have
got much higher expectations of these natural use
interfaces than they have of other types of interface. And then the other
thing is best practices for conversation design. One principle we have
conversation design is that we focus a lot
on designing for failure, because with the technology
how it currently is, there can be a lot of misrecognitions. And also, just with language,
there's a lot of ambiguity. So we focus a lot-- the happy
path is fairly straightforward. But the unhappy path
can be quite complex. So we spend a lot
of time focusing on designing for failure. And then there's also
investment in prototyping. So that's another aspect
of user-centered design, is being able to validate what
you're designing with users. NANDINI STOCKER: Yep. Sachit, you have a term for
it instead of happy path. It's-- SACHIT MISRA: Yeah. I have something I
call the happy tree. NANDINI STOCKER:
It's a happy tree. Yeah. There's a Bob Ross reference
in there somewhere. Happy little trees. OK. Well, and then, I mean,
you mentioned research. So Kimberly-- so
we can all talk. We can all speak, or gesture,
or we can all communicate. It's the interface we learned
first and the one we know best. So it's not like we need
any help understanding that. Finding a button. So why do we need UX research? KIMBERLY HARVEY: Why
don't we need UX research? Just because we
know how to speak doesn't mean we understand the
intention of what's going on. For example, backstage,
I was saying, let's say Marc was outside
in the center of the hall. And I was talking to someone. And I said, what
does Marc look like? They would describe Marc
in a certain way like, he's wearing white shoes. He's got a
multicolored shirt on. But if we were working
in an office, and Marc had been out with
the flu for a week. And this was his first day back. And I turned to my boss, and I
said, what does Marc look like? My boss might say, well,
he looks a little better than he did last week. Same question, completely
different answers based on the context. And a lot of times,
while we are sitting and we're developing, even
designing an experience, we really don't know, until
we actually see it in motion, whether we've ironed
out all of the kinks and whether there are
surprising behaviors that are emerging from the designs
that we didn't expect. So it's a great way to-- it's almost like QAing
your human condition to make sure that you've
got everything covered. NANDINI STOCKER: So
Sachit, what about you? I mean, you--learning other
disciplines as you get into this. But just from a
developer perspective, getting just the ramp
up into this, what can a developer bring from
past platforms into this space? SACHIT MISRA: So I think the
great thing about our platform in particular is it's mostly
driven through the cloud and through the web. So what that means is for
developers who have already been developing on the
cloud, or like web apps, or even APIs they've been
building for mobile apps, that kind of thing, you'll be able
to bring all that experience in. And the tools that we provide,
like in particular API.AI, to abstract some of
the harder problems, like the natural language
understanding, those tools are fairly easy to
use for anybody. You don't need a computer
science degree really to understand that
kind of stuff. So first learning
that is fairly simple. There's a low barrier to entry. And then secondly, with
your existing cloud and web knowledge and just using
basically efficient coding practices around things
like string manipulation, those kind of things,
you can already build apps pretty
effectively on our platform. And one of the things I'm
actually really looking forward to hopefully, that I want
to see from the community, is more tooling and
frameworks that we have similar to front-end apps
on other platforms, like web apps and mobile apps. There's a lot out there if
you just want to build the UI. And so I'm really
looking forward to seeing how people
build frameworks for building conversational
UIs and voice UIs. I think that's going
to be really exciting. NANDINI STOCKER: Cool. Great. And then usually, once
you start building, you're early in the process. You want to start prototyping. So Marc, you have some really
unique perspective here. So do you have suggestions
for how to prototype in voice interaction design? MARC PAULINA: Yeah. When I think about
prototype, I'm looking at something that I
can use early in the design process, that I
can learn quickly, and I can iterate
really quickly. So I think it's really important
that we invest in prototyping, especially if we don't have
the years of experience working on voice use interfaces,
and that we're really learning on the go. So there's a few
methodologies that we've used with some success designing
conversational UIs at Google, such as Wizard of
Oz prototyping. So this is the idea
about Wizard of Oz, is that you remotely control the
device that the participant is using in a study environment. But they think that they're
speaking to the Assistant. But really, they're
speaking to me as the puppet master
pulling the strings-- NANDINI STOCKER:
It's like, you-- MARC PAULINA:
--behind the strings. NANDINI STOCKER:
--could literally create the whole scene. You could go behind the-- yeah. MARC PAULINA: You
could basically emulate what they would
believe is like the end user experience. But basically there's no AI. There's no cloud. It's just me on the other side. NANDINI STOCKER: Because
talking is as scrappy as it gets in terms
of role-playing. MARC PAULINA: And it's
incredibly-- conversation is-- it can go anywhere. You never really know. There's no set path. And so to feel
natural, you need to be able to pivot in real time. So really, unless you had
the AI built for that, the only person who can
do that is a real person listening in real time. So Wizard of Oz is
really powerful. And it can be as
simple as having a Bluetooth speaker
and the audio files ready to go on your laptop. And you're just
basically playing them as you're
responding to the user. It takes a bit of practice,
but it's really good. And then the other
methodology that we use early in the design process is just
saying out loud role-playing. And in the Google
Design Sprints, when we're actually got
the whole product team designing their
conversations, we'll actually do
investigative rehearsal. Which is a methodology created
by Adam Lawrence of "This is Service Design Doing." And it's basically-- the
idea is that you rehearse the conversation in real time. And then you investigate
that conversation. And you ask questions about it. So it is the most scrappiest,
most lowest fidelity prototype you can imagine, but
incredibly powerful for the first draft
of the conversation before you invest in
any coded prototyping. SACHIT MISRA: It's funny
you mentioned that you-- for the sprints you
were just talking about, that you have the
whole product there. Because basically what
you're facilitating is a conversation to
build a conversation. And it actually creates
better conversations in terms of the actual apps. Yeah? MARC PAULINA: Yeah. Having the whole
product team there, it democratizes, as I
said, the design process, that everyone shares the same
understanding of the problem. Everyone understands
the user requirements. Everyone understands their goals
and maybe the business goals as well. And everyone's thinking
through that at the same time and trying stuff out. So it really is like a
platform approach for design, which is pretty powerful. NANDINI STOCKER: Yeah. In fact, that's
probably a good place to close and just
wrap up, and is to get everyone
thinking about this. I've been doing this for a
really, really long time. So hopefully, if you feel
like we raised more questions in this panel than answers,
then good, because there's a lot to talk about. As I said at the
beginning, this is just the start of a conversation. It's like the inception
of conversations. There's so many different kinds. There's worldwide conversations. There's conversations between
one person and a group. There's two-person
conversations. This is just one of them. And it's a much larger one. So a conversation about
the future of technology, and where we are today, and
where we're going from here is going to take a
new kind of ecosystem. This is called the
conversation economy. You guys are part of it. And with your help, we need
to build this new ecosystem. We can actually, at this
point, take a quantum leap past this AI first world. In fact, words really matter. It's actually quite unfortunate
that we call it artificial at all, when we need things
like authentic and advanced intelligence more than ever. We're all starving to
amplify our own intelligence. So maybe amplified intelligence,
hashtag fix the glitch. So we can-- but with
all of your help, with the ecosystem of the
developer community on all platforms, on all devices-- it doesn't matter--
users come first. And if we build a world using a
people first creative approach to building
solutions that people can use in their
daily lives, it's more than just talking and
getting answers anymore. People want insight. And they actually want
to be able to do things in their world. We want to find those micro
moments that are assistive. Well, there's another A-word. And we just need
the whole alphabet. And we need all
disciplines at the table. Diverse voices, diverse
approaches, diverse thinking. This is really a time
when, it's like I said, the intersection of science
and culture, human interaction, human solutions to a
global problem of like, we built these machines. Let's make them
work on our behalf to help us connect more
with each other in our world and the things we
want to get done. So personally, it's my-- I mean, I've been
doing this a long time, working with this technology. But really, it's my personal
mission to give voice to others and to help create
a culture that we can build experiences
that illustrate a shared vision of our future. We need to
collectively recognize that good means
going beyond, don't be evil, or do what's right,
to actually create experiences that are to show how it's done. And so hopefully,
with this series, we want to show what
good looks like. We want partner experiences,
developer experience to showcase what you're
creating and for your users and for the common good
of users everywhere. So we want to give insight
to the kinds of people inside Google who are building
this and also changing the landscape of the
future of technology. And also, this is the power of
spoken language and discourse. What kinds of people does it
take out there in the ecosystem to change the landscape? It's a disruptive environment. It's a really,
really good thing. So we're all living our
legacies in real time. What are you going
to be known for? What kind of world are you
creating for your kids? Have that conversation with
yourself and with each other. And we're going to create
some amazing things. And I can't wait to
see what you create. And we will look forward to it. Thank you so much to our panel. And thank you to the
audience for being part of this exciting,
creative moment for us. Thank you. [MUSIC PLAYING]
This is really neat to learn about how Googles departments are moving into the world of Voice UI.
Panelists
Sachit Mishra, Developer for Assistant/Actions on Google
Kimberly Harvey, Conversation Designer
Marc Paulina, UX Designer for Google Assistant on Wear
Nandini Stocker, (moderator) Head of Conversation Design Advocacy & Partnerships.