[MUSIC PLAYING] JEANINE BANKS:
Welcome to Google I/O. [APPLAUSE] I enjoy this time when the
developer community comes together from all
around the world, literally millions of people
from different backgrounds, who all share one thing in common-- the joy of solving problems
with technology to get more done and make life
better for everyone. Technology is
constantly evolving. And it's having a profound
impact on our everyday lives. As developers, you're
at the forefront of this sea change, the
ones creating experiences that shape the world. Because of you,
billions of people can uplift themselves, their
communities, their families. That's why I love how Google is
making deep investments to help you leverage the largest
mobile platform on the planet, the most popular web browser. That's Cloud Services, hundreds
of open source projects, and breakthroughs in AI. Remember 25 years
ago when the web just became a programmable
platform and it allowed us to bring the world's
knowledge to our desktops? For users, it was like magic. But for developers,
it was complex. I still remember the
first time I wired up a database to a website. It was not magical,
but it was determined. I was so determined
to make it all work. And I did get it to work. Now, that complexity has only
grown in depth and scope. No matter what kind
of developer you are, we're here to help you. Earlier today, you
saw many of the ways that Google is using generative
AI to improve our products from search offering
relevant follow-up questions, to Workspace assisting
with drafting emails, and to Photos letting you
edit images in whole new ways. And as Sundar shared, PaLM
2 is the foundation model that powers all of
these experiences. We want to help
you take advantage of that same technology to
build your own innovations. Now, the question
is, how to make generative AI
accessible to everyone who wants to build powerful
experiences in products, but in the most productive
and responsible way possible? At Google, we've been thinking
a lot about that challenge. I hope you'll see
today that we're committed to providing
open, integrated solutions so you can deliver
amazing experiences that your users love. So let's start today by
looking at how you can access the PaLM 2 model for yourself. Now, you may have seen
the new game, I/O Flip. Who's seen it? Everybody's seen it? [APPLAUSE] It's fun. I've been watching,
seeing how so many of you have already started
playing with it. How about we go under
the hood for a look at how I/O Flip is made? I've asked Simon to join me. Simon, are you ready? SIMON TAKUMINE:
Absolutely, Jeanine. Hi, everyone. [APPLAUSE] So I've always been a gamer. But building games requires
some creative skills that I wasn't born with. But with generative AI and
all these amazing Google tools working together, it was easy
to make this delightful game. JEANINE BANKS: So the game
has some pretty unique and quirky characters. For all the collectors watching,
how many cards are there? SIMON TAKUMINE: Well, we
created millions of unique cards for the game. First, the pictures
on the cards were created using a
technique pioneered out of Google Research
called Dream Booth. It enables you to
fine-tune image models with a specific subject,
transporting them to anywhere you can imagine. JEANINE BANKS: So the picture
is just one half of the card. Each one also has a fun
description from the keywords that I select, right? SIMON TAKUMINE: Yes. Those keywords are used
by us, behind the scenes, to build more complex prompts. Now, prompting models,
it's a bit of an art form. You never know exactly
what you're going to get. But experimenting with them
is really part of the fun. Now, this is where
MakerSuite comes in. It's an easy to
use tool that runs in your browser and
a place to experiment and craft your prompts. Let me walk you
through how you can create these cool
descriptions in minutes. OK. So here we have MakerSuite. Jeanine, let's make you a card. What class and power would
you like your dash to wield? JEANINE BANKS: OK. So now, I enjoy a good meal. Let's go for a
culinary wizard who's capable of making
anything delicious. SIMON TAKUMINE: Coming right up. And in the great
tradition of TV chefs, here's one I prepared earlier. First, as you can
see, we provide the model with some context. You're a writer for descriptions
of a trading card game. And then we give the
model an example. Here's a pretty
cool one for Sparky. Finally, it's time for Dash. Let's get it pasted in. Oops. Write a short description in
less than 30 words for Dash, who's a culinary wizard capable
of making anything delicious. JEANINE BANKS: So because
large language models are non-deterministic,
how do we make sure we're doing all of
this in a responsible way? SIMON TAKUMINE: Yeah,
that's a good question. We build our models with
our AI principles in mind, so you have a responsible
foundation to start from. And we also want
these to be playful. So let's add, keep it upbeat. And hit Run. Let's see what it generates. Ah, that's pretty cool. But if you don't like
that description, you can just try again. Awesome. And since we're all
developers here, let's have it output in JSON
and another kind of a schema to give an idea of what we want. Cool. And since I want to play
it with the family later, let me ask it to add a
Japanese description too. Cool. JEANINE BANKS: Oh, that's slick. [APPLAUSE] You don't need to be
an AI expert to iterate and tune prompts in MakerSuite. And now that we
have the prompt, you can easily copy it to
VSCode or open it in Colab. SIMON TAKUMINE: That's right. Here we have it in Colab. You can easily access the model
directly through the PaLM API. And when you're ready to shift
the development and build, you can access the model
directly through our PaLM API libraries that we built in
Python, Node.js, and Java. JEANINE BANKS: Thanks, Simon. SIMON TAKUMINE: Thanks, Jeanine. [APPLAUSE] JEANINE BANKS: Let me
show you some examples of how some developers
are using the PaLM API. Take Got It AI's enterprise
customer chat product that uses the PaLM API to
create summaries of customer conversations, or one of
my favorite from Play Labs, where they're using
the PaLM API to build an agent that suggests
ways to play and engage with your friends. You can even use the PaLM
API with popular frameworks like LangChain, to build
intelligent agents that use ReAct to navigate
complex tasks. As you can see, we're
giving you access to our large language models,
the ones powering so many of Google's products. And starting today, we're
rolling out new features in MakerSuite, so you can
create synthetic data and easily customized safety settings. So sign up now to try out
MakerSuite and the PaLM API at g.co/palm. [APPLAUSE] We're also thrilled to release
several new Firebase extensions that use the PaLM API. [APPLAUSE] So you can integrate and extend
from Google and other API providers. Take the new chat bot
with PaLM API extension, so you can easily
add a chat interface for continuous dialogue,
text summarization, and more. Last month, we added
coding capabilities to Bard, an experiment
that's now powered by PaLM 2. Bard can act as your
collaborator in over 20 languages, helping you to
generate code, find bugs, add annotations, and more. My favorite thing to
do is play with Bard to help me learn
something entirely new, like asking for examples of
code in a different language. Over the next hour, you
will see more exciting ways that AI can help you to be more
productive with all the tools and platforms that you rely on. Next, let's get
Matthew on stage so he can tell us more about
how we're helping you to be more productive
in mobile development. [MUSIC PLAYING] MATTHEW MCCULLOUGH:
Thanks so much, Jeanine. [APPLAUSE] It's exciting to see
how the world of AI is transforming all types
of developer workflows. It's making traditional
tasks easier and the previously impossible
entirely within reach. We want to share three
things from the world of mobile and Android
development with you. First, how we're
bringing the power of AI directly into the Android
developer workflow to help you be more productive. Next, how you can build
for the multi-device world that your users are asking for. And finally, how our Language
Toolkit and tool improvements seamlessly converge in the
modern Android development stack. As you saw earlier, AI
is the next breakthrough that will define mobile. And I'd like to
invite my colleague Jamal to share how we're
putting AI directly into Android Studio. Jamal, tell us more. [APPLAUSE] JAMAL EASON: Thanks, Matthew. I'm going to share with you
an example that underscores not just how you approach AI,
but how we approach everything we build for you as
mobile developers to focus on helping
you stay productive, become more efficient,
and get the most out of Google technologies. Now, if you didn't
know, 10 years ago, we launched Android
Studio to focus on delivering innovative
Android developer tools. Today, let me show you how we're
continuing to help accelerate your development with
demo innovations in AI called Android Studio Bot. [APPLAUSE] Studio Bot is a highly
integrated AI-powered helper in Android Studio designed
to make you more productive. We are in the early days. But I want to give
you a sneak peek. OK, let's jump in. Now, you can find
the pre-release of Studio Bot and
Android Studio Hedgehog, which you can find in
the Canary channel. OK. Here, I'm working on
the Jet Stack app. And I'm excited to get the app
ready for the all new Android hardware, especially tablets. Now, my tablet UI, it looks
OK, though it could be better. But you know, I don't remember
all the best practices in creating tablet layouts. But this is where
Studio Bot comes in. Now, Studio Bot is found
in a new Assistant window that you can find
in the toolbar. And what's unique
about this chat setup is that you don't need
to send your source code to Google, just a
chat dialogue between you and the bot. OK, I've already typed
the prompt to create a layout for an Android tablet. All right, nice. I have a quick response
on how to add this layout. Now, what's nice
about this integration in Android Studio is that we can
recognize the response as code with the right syntax
highlighting per your custom ID settings. However, Studio Bot does more
than just generating code. It's a place to ask
questions and contexts. So. for instance, I
don't need this layout in XML, but in Kotlin
since I'm doing this app and Jetpack Compose. So let's ask Studio Bot. How do I do this
in Jetpack Compose? And submit. And perfect, the
code makes sense. [APPLAUSE] And it gives me additional
guidance and documentation. And you know how
important tests are. Let's also ask Studio Bot to
create a unit test as well. So let's see. Can you create a
unit test for this? And submit. And note, I'm just having a
conversation with Studio Bot. And it remembers the context
between question to question. And great, Studio Bot created
the unit test right in context. Pretty cool. [APPLAUSE] Now, as you look at the
code for a Compose layout, there are steps such as
importing various dependencies. Studio Bot recognizes this. And with the smart actions
that Studio Bot generates, I can directly
convert this snippet into my project with the right
imports and dependencies. So let's try it out. So let me click on this
Explorer and Playground, Expand. And there we go. [APPLAUSE] We've moved from code submit
to actual code with Studio Bot. Pretty cool. Lastly, we have
added entry points to Studio Bot and key
points in the IDE, both in the Code
Editor and in the logs. So, for instance,
earlier today I was debugging a problem
in my Android manifest. I can open up Logcat,
right click on a crash, and send it to Studio Bot,
and just press Submit. And there we go. Studio Bot explains the crash. And it knows that I forgot to
add the internet permission. And it also gives
me a quick action to add the missing line of
code to my manifest, which I can fix in one click. [APPLAUSE] So I'm excited. But we're just at the
beginning of this AI journey. And we invite you to download
the latest version of Android Studio with Studio Bot, try it
out, and share your feedback. [APPLAUSE] Now, once you have
built your app with the help of
Studio Bot, you're ready to publish to Google Play. And here, we're also
bringing the power of AI. So today, we're launching a new
experiment of the Google Play Console that will generate
custom search listings for different types of users. You will also have control
of what you submit. But Google Play is there to
help you be more creative. So from start to finish,
develop to publish, we're deploying AI to
help you move faster and to be more creative. [APPLAUSE] Back to you, Matthew. MATTHEW MCCULLOUGH:
Thanks, Jamal. That was awesome. Now, let's dive in to
our multi-device world. As people add more connected
devices to their lives, they want their apps
to work across them and to adapt to each form
factor's unique abilities. Let's take a look at how a
developer, Peloton, responded to their users' needs
and is delivering an immersive,
multi-device experience. [VIDEO PLAYBACK] - Scene 101, take 1. [MUSIC PLAYING] - The Peloton app enables
members to work out at their convenience. - You can literally
work out anywhere. It doesn't matter if you're on
vacation, if you're at home. At Peloton, we want to
meet members where they are and on the devices they own. We heard consistent
feedback from our members about wanting a Wear OS app. - Our members wanted to be
able to use their watches to work out with Peloton. And we took our learnings
from the original bike, built on AOSP, to build
a great mobile app across many different
form factors. - We built a completely
new experience so that users have
even more options. They can now track
their heart rate in real time on their
wrist, on the phone, and on our connected
fitness devices. - After building
the Wear OS app, we wanted to improve the Peloton
app in the Android ecosystem. Great tablets, like the
ones from Samsung, Pixel, and Lenovo, are a big
reason why we're continuing to invest in large screens. [MUSIC PLAYING] With the Android SDK
and Jetpack libraries, it's really easy to create
a flexible UI to adapt to the different screen sizes. We use the Jetpack
WindowManager to support foldable specific use
cases, like tabletop mode in our video player. Now, we're positioned
to be even more forward-thinking by investing
in exciting new form factors. - Heart rate monitor
usage in our app is correlated with higher
engagement, better workout experience, and more
workouts a month. Since launch, we've seen
a bump in total workouts taken on the Android platform. We are very confident that Wear
OS was a considerable driver. - In the Android
ecosystem, there are so many different devices
with varying capabilities like phones, watches,
tablets, and TVs. At the end of the day, we want
the Peloton app to be awesome wherever our members use it. [END PLAYBACK] [APPLAUSE] MATTHEW MCCULLOUGH:
It's incredible seeing an investment
like that pay off. We're seeing that happen
not just for Peloton, but for all of you
who are working in this multi-device space. We're seeing a robust
market continue to grow for large screen devices. Tablets, foldables,
and flippables provide a huge opportunity
to reach premium consumers. Our hardware partners
are all in with flippable and foldable devices
from Samsung, Motorola, and Oppo. Google is all in. Earlier, you saw the Pixel
Fold and Pixel Tablet. We're optimizing Android
OS for large screens. And we're tuning over
50 of Google's own apps to look great on these devices. App developers are all in and
getting rewarding results. Apps like Concepts and U-Next
are seeing dramatically more installs and more engagement
on tablets and Chromebooks. WhatsApp, eBay, and Canva
are also giving users richer experiences on these
screens and, as a result, are seeing their Play
Store ratings increase. If you're excited
about enhancing your app for large screens,
here's how you can get started. First, start with Jetpack
WindowManager and window size classes and create
layouts that take advantage of the large screen. Version 1.1, available now,
has activity embedding, the fastest way for
multi-activity apps to provide great
large screen layouts. Next, leverage
posture detection-- leverage posture detection
for foldables and flippables with Jetpack WindowManager and
poses in the device emulator in Android Studio, like
Google Meet did here. Third, grow your
installs on Google Play. To help users discover
your optimized apps, we're featuring high quality
apps more prominently in the Play Store
and showcasing those with device-specific
screenshots. Finally, test out
your experiences. We're adding Pixel Fold and
Pixel Tablet configurations to the Android Studio Emulator. And I'm excited to share we're
announcing physical device streaming for these Google
devices in Android Studio. This lets you see how your app
runs on Google hosted Pixel Fold and Pixel Tablet
devices right from your IDE. You can sign up to join
the waitlist today. [APPLAUSE] Now, let's talk about
the screen on your wrist. Wear OS active
devices have grown 5x since Wear OS 3 launched. And it's the fastest
growing smartwatch platform. Now is a great time
to jump on Wear OS. And as developers, here
are the two big updates you should take advantage of. First, watch faces--
they're a beautiful way to engage with your users. And with Wear OS 4, it's
easier to create customizable and power efficient ones. We're making that possible by
introducing a new watch face format developed in
partnership with Samsung. This is a new
declarative XML format, meaning there'll be no code
in your watch face APK. This will make it easier to
bring your great watch face ideas to market. And second, we're
releasing new APIs to bring rich animations to
tiles, one of the fastest ways to get things done or
launch an app on Wear OS. We're launching the
first developer preview of Wear OS 4 today with
an updated emulator image featuring new Bluetooth
connectivity to other devices. But it's not enough to just
make these form factors easier to build for. The tools and services
you use should be purpose-built to take
advantage of these new form factors from the start. We've done just that. And we call it Modern
Android Development. We're building
our tools and APIs with the future of
the platform in mind. To show you what
I mean, I'd like to invite my colleague,
Florina, to the stage. [MUSIC PLAYING] [APPLAUSE] FLORINA MUNTENESCU: We know
your to-do list is long and your time is precious. We want to help
you do more faster. Modern Android Development
gives you the APIs, tools, and guidance to
speed up your flow and help you write
safer, better code so you can focus on building
amazing experiences. We aim to reduce the complexity
and the cost of both building and maintaining rich
cross-device apps, so everything you learn applies
across the entire ecosystem of Android devices. I want to share three
of my favorite updates that help you save
time and set you up for the future of the platform. First, Jetpack Compose, which
makes it faster and easier to build high
quality UIs. Today, over 24% of the top
1,000 Android apps take advantage of
Compose's productivity boost, double since last year. The Google Drive team told
us that Compose, combined with architecture improvements,
cut their development time nearly in half. And Clue increased their
development speed three times after rewriting
their app in Compose. That's pretty amazing. [APPLAUSE] The latest updates to
Compose make it easier to build rich UIs.
We're bringing Compose to even more
surfaces with Compose for TV in alpha and home screen
widgets with Glance in Beta. Flip through content
with a new pager APIs. Build flexible layouts
with FlowRow and Column. And take advantage of the
latest Material 3 components. And we're still focused on
improving Compose performance out of the box. Second, Kotlin-- one
of the best things to happen to Android development
since switching from Eclipse to Android Studio was making
offline an officially supported language all because of you. I still remember
that announcement. I was sitting right here in
the Shoreline Amphitheater. But I had a way
worse seat back then. Since then, more and
more apps have adopted it and have seen what an
expressive, concise, and fun language it is. And now, 95% of the top 1k
apps use it and love it. Kotland is at the core of
our development platform. And we keep expanding the scale
of Kotlin support for Android apps. We're collaborating
with JetBrains on the new K2 compiler,
which is already showing significant improvements
and compilation speeds. We are actively working on
integration into our tools, such as Android Studio, Android
Lens, KSB, Compose, and more, and also leveraging
Kotlin code bases-- our large Kotlin code bases
to verify compatibility of the new compiler. We now recommend using
Kotlin for build scripts. So with Kotlin in your build
and in your UI with Compose, you can use Kotlin everywhere
throughout your app. And finally, great languages,
these are great tools. Android Studio keeps you
productive and in the flow. Our Quality Insights is
a great example of this. In the App Insights tab,
you can see critical issues from Firebase Crashlytics,
navigate from stack trace to code, all without
leaving the ID. Today, we're adding Android
vitals crash reports. You can now access
your crash reports from the Play Console, no
need for any additional SDKs or instrumentation to your app. [APPLAUSE] From making debugging better
to updating Compose and Kotlin, these were three of
my favorite updates in the world of modern
Android development. We hope they help you fly
through your to-do list and build amazing apps. And now, back to Matthew. [APPLAUSE] MATTHEW MCCULLOUGH:
Thanks so much, Florina. Enabling the productivity gains
that Florina just mentioned, that's something that
teams all across Google are thinking about as we build
tools and services for you. For example, Flutter is
Google's open source framework for building multi-platform
applications from a single code base. When Betterment wanted
to shift their strategy to a mobile-first
approach, they turned to Flutter to share their code
base across iOS and Android, which allowed them to
achieve 100% feature parity across both platforms. [APPLAUSE] And with the latest
mobile update to Flutter, you can tap into Impeller for
enhanced graphics performance. Additionally, with
Flutter 310, we now include JNI bridge to Jetpack
libraries written in Kotlin. So you can call a new Jetpack
library directly from Dart without needing to use
an external plugin. Whether you want to
shift your strategy to be mobile-first or expand
your app to new form factors, our mission is to give you
simple, helpful tools that save you time, so you can
build great mobile experiences. Next, to talk to you about what
Google is doing on the web, I'd like to invite Matt
Waddell to the stage. [MUSIC PLAYING] [APPLAUSE] MATT WADDELL: Jeanine
talked about the importance of being open by design. And that's one of the things
that makes the web so critical. It's a platform that's
owned by none of us and shared by all of us. And with Google's contributions
to Chromium, Angular, Interop and many other efforts, we want
to keep moving the web forward to keep on keeping open. We also want to make sure that
the web's tools and APIs are solving real world problems
and doing so efficiently. So today, I'm going to
share a few challenges and opportunities that we're
trying to address together to deliver a web that's both
more powerful and way easier to work on. The first opportunity is really
about reaching new customers across platforms. The web is a key part
of this conversation. It's often called the
universal runtime. At the same time,
nearly all of us are also making investments
in client and server apps. Now, this is not a
web or a native thing. Honestly, that debate
is pretty boring. This is a web and native thing. [APPLAUSE] The key is figuring out how
to take your investments in desktop or mobile and make
it easy to target the web. Webssembly, or Wasm,
makes this possible. Since it arrived a
couple of years ago, we've seen companies like
Figma, Unity, Snap, VLC, and others use Wasm to bring
code from their C++, C+, even their Swift apps
and run it on the web. Today, we have some really
cool updates for the community. For starters, WebAssembly
now supports managed memory languages. [APPLAUSE] This extends the benefits of
Wasm to even more workflows. For example, if you're writing
your app in Dart and Flutter, Wasm can now run
your browser code more than three times faster
than compiling to JavaScript. I'm also excited
about what this means for Android developers,
most of whom are writing their
apps in Kotlin. Starting today, thanks to
some early work by JetBrains, Kotlin also runs on Wasm. [APPLAUSE] This means you'll be able
to write your Android features just
once, then use Wasm to deploy your app to the web. You get to reach new customers
with native performance. And it's a great example of
how web and native can really scale your efforts. Another opportunity that's
certainly top of mind right now is AI and machine learning. And there's a critical role
for the web to play here. And it all starts
with GPUs, the stuff you need to train and run the
models that you saw earlier from Simon. Today, most of these workloads
are served in the cloud. And there's a complementary
set of GPUs in the devices we use every day. The problem is it's not
very accessible right now. And we can change
that with the web. That's why we're excited
to highlight WebGPU. It's a-- AUDIENCE: Woo! MATT WADDELL: --new API-- yes. [APPLAUSE] It's a new API that unlocks
the power of GPU hardware and makes the web AI ready. After all, the web is already
on phones, desktops, laptops. WebGPU just gives you access
to all of that local compute, so you can save
money increase speed, and build
privacy-preserving features. Let me share a few details. In terms of performance, ML
libraries like TensorFlow.js now run over 100 times faster on
WebGPU than vanilla JavaScript. [APPLAUSE] These kinds of gains
make a big difference when you're running AI models
or graphics-heavy apps. You can really see
the difference here in this diffusion
model running on WebGL on the left versus WebGPU
on the right, both of which are running inside the browser. Now, I'm going to ask
the model for an image of a cat next to a window with
the sunlight streaming in. Fun fact, lots of Chrome
folks are cat people. So this example is
pretty on brand. As you can see, WebGPU finishes
in just under 10 seconds, while WebGL takes over
three times longer. And until recently, WebGL was
the fastest implementation on the web. AI, ML, LOMs, look, there are
a big deal for a good reason. And by using WebGPU, you
can now build your app with compute that's both in the
cloud and on the local device. I also just want to
acknowledge the many new APIs and components that we're
discussing at I/O to deliver a more capable platform. I'm not going to do them
justice in just five seconds. But there are more than 100
new updates in the IO sessions that we'll discuss
from storage solutions, like SQLite, to UI features,
like View Transitions. We do this work as an ecosystem. So please do check them out. Now, so far, we've talked
about a more powerful web. These APIs and these
features, they're cool. What's really cool, though,
is when they're easy to use and they seamlessly
match your workflow. So you can spend
more time actually building in less time
as a systems integrator. Let me show you some
updates that will really enhance your productivity. The first is about
Web Frameworks. And yes, it's easy to
romanticize about vanilla JavaScript, especially if
you started your own journey with text editor, View
Source, maybe Stack Overflow, and sheer determination. The thing is, more
than half of us use front-end frameworks
to build our apps. This abstracted web is the web. So today, we're
introducing some new tools to reflect this reality. First up is Firebase
Hosting, which many of us already use to deploy apps. Last year, we added
support for frameworks like Next.js and Angular. [APPLAUSE] Yes, I see you. And starting today, we
are expanding the support to include Astro,
Flutter Web and others. [APPLAUSE] In addition, you can
now serve back-end code in hosting preview channels. So you can verify
changes to your app without affecting production. For all you Flutter
fans, we continue-- AUDIENCE: Woo! MATT WADDELL: I see you. For all you Flutter
fans, we continue to address your top asks. And version 3.10 significantly
reduces low times and integrates Flutter Web
into existing web content. In fact, what you're
going to see here is an example of Flutter
Web running inside of an Angular app. It's like a big framework hug. [APPLAUSE] Finally, regardless your
framework of choice, you'll eventually need to
debug some of your code. It happens. Many of us debug
with Chrome DevTools. And starting today, it does
a much better job of code that's generated by modern
frameworks from cleaner stack traces, to a new Show
Your code option, to more breakpoint reliability. It just works. So you can focus on
your app experience. Another area that really
impacts productivity is performance tuning. Have you already reduced
your load times by 3x? Probably yes. Do you still want
it to be faster? Also probably yes. And too often, that work
feels like you're trying to do it with your eyes closed. We want to help. So today, we've
got a few updates that offer insight into
real world performance and make it easier to actually
achieve your performance goals. In terms of metrics,
Core Web Vitals has emerged as a key yardstick
for quality and performance with an initial focus
on page loading. At the same time, we know that
responsiveness is a key part of the overall experience. Is your app reactive or
responsive or more, ah, snap, in a given session? Today, we're announcing
that Interaction to Next Paint, or INP, is
graduating from experimental. And by next year, it'll
become the vital metric that captures that
dimension of responsiveness. The lower the number,
the more users are getting immediate
feedback in your app. And that kind of
Intel is crucial when it comes to app quality. To help you actually reach
your performance goals, we've been working with
Frameworks to integrate Vitals into measurement directly. And as of today, that set
includes Next.jx, Nuxt, and Angular. And if you use
Angular, version 16 now includes a new reactivity
model and improvements to server-side rendering and
hydration that significantly improve your vital scores. Look, we get that at performance
isn't just an optimization. It's one of your most
important features. And today's updates are designed
to really speed things up, both in terms of
the benchmarking and the actual tuning. OK, one more thing, and I
really do mean one more thing, both because I'm going
to hand off after this, and because it's something that
I'm especially excited about. So far, we've talked
about APIs, capabilities, quality improvements,
tools, all the things that you can do on
the web platform. And yet, all too
often, the WWW actually stands for Wild Wild West. Is this API supported
in this browser? Can I depend on
the CSS transition? Am I getting the
latest and greatest when I use this framework? Trying to make sense of all
that is like that moment-- it's like just
before the orchestra starts, when everyone's
tuning their instrument. It's big, and it's
loud, and it's noisy. And then the musicians
come together around a single, shared note. And it feels great. We want to bring that level of
clarity to a platform changes. So we've been working
in the WebDX community group with browser vendors
like Apple, Mozilla, Microsoft, framework providers like
Angular, Nuxt, Next.js to establish a stable and
predictable view of the web. And it's called Baseline. Baseline captures
the evergreen set of features that are
supported across browsers. And it finally takes
the guesswork out of can I actually use
this in my app or not? [APPLAUSE] Every year, we'll also
introduce Baseline 23, 24, 25 which will be a cut of
everything's new across browsers and compatible. It's going to be an annual
release for the entire web ecosystem. [APPLAUSE] MDN has been a close
partner in this work. So here is Hermina
Condei to share more. [VIDEO PLAYBACK] - Thanks, Matt. Hi, everyone. I'm Hermina Condei. And I'm the director
of MDN at Mozilla. At MDN, we provide documentation
for modern web development to more than one million
developers every day. However, we understand
that keeping up with the constantly
evolving web platform is a major challenge
for developers. More importantly, there's
tremendous innovation happening in the browser space. And developers don't
have a common language to talk about the features that
are generally available for use on the web. To address this
issue, we've partnered with Google and the
WebDX community group to provide a transparent view
of stable and well-supported features that drive the
platform's growth, giving developers a simple and
clear signal of what's the baseline for
the web platform. Starting today, we'll be rolling
out this labeling on our site, as you can see here, and hope
to cover all relevant features in the coming months. Back to you, Matt. [END PLAYBACK] [APPLAUSE] MATT WADDELL: Thanks, Hermina. Browser interop
and compatibility really get to the core of
what makes the web great. It's this ability to
collaborate and federate across companies, stacks, and
endpoints, where all of us is truly better than
any single one of us. And with Baseline,
we can still provide a consistent experience. No more wild Wild West,
just the world wide web. And we're grateful for
the chance to contribute. Next up is Lawrence
to talk about how we're bringing this approach
to AI and machine learning to enable your creativity and
to solve some really important problems for customers. [MUSIC PLAYING] [APPLAUSE] LAURENCE MORONEY: Wow,
Matt's a hard act to follow. So thank you, everybody. And we've been seeing a lot of
amazing and very cool AI stuff today. And we've explored
tools that are designed to make your life
easier as a developer. And we want to
empower you to be more effective on mobile, on the web,
and, of course, in the cloud. But AI isn't some kind
of magic pixie dust that you can just sprinkle
to get a solution. It requires smart
engineering work from developers, just like you. And as a developer,
you are a dreamer. You're a maker. You're a thinker. And you're a shaper. And it's our goal
and our passion to make it easy for you to use
your skills to take advantage of all of these new
opportunities afforded by machine learning. So let's take a look at how
coders just like you were able to engineer a solution
for my friend Lance, who was able to reconnect with the
world after he lost everything. [VIDEO PLAYBACK] - This is my smart guy room. This is my dog, Wicky and Sammy. And this is my sick
gaming computer. I live on the plains
of Eastern Colorado. This is my new house. My old one burnt down. But we'll get back to that. This is me, Lance Carr. I've got a rare form
of muscular dystrophy. So I can't really move my body. For me, playing video games
is my link to the world. But I had to stop gaming because
this house burnt down along with my adaptive equipment. And I had absolutely no
way to play "Warcraft" or any other computer game. I definitely had a low point. So when I got connected with
some cool folks at Google-- Hey, guys. - Hey, Lance. - --I was pretty stoked. We teamed up to develop this
thing called Project Gameface. - Yeah, the name is pretty cool. - It allows me to precisely
control my mouse with my face. It's so precise, I can
even write in cursive. - Project Gameface uses
MediaPipe, which links together a number of different AI
models to give you an output-- and in this case,
the output is a mesh of 468 points on your face-- and convert that into telemetry,
like mouse movement or clicks. It runs natively on devices. And it only needs an
input from a webcam, which, to me, is one of the
most amazing things about it. - Controlling my computer
with funny faces, it's pretty awesome. - The face is what we show
to the outside world anyway. So why not bring these
two things together? There's a beautiful
symmetry there. - I was stronger when I was
six months old than I am now. Muscular dystrophy takes. And this actually
added an ability. So it's the first time
I've gained something, in a physical sense. My hope is to definitely give
this technology to everybody who could use it. - There are many
people out there who have ideas, who
have creativity. So we're open sourcing
this technology for folks to create miraculous
solutions like this one. - I just want to make
a lot of people's lives better and easier. Well, we're going
to get back to it. Thanks for stopping by
the plains of Colorado. - Oh, you saw me. - Look at the edges
of the cliffs. - Right. - Whoa, oh, whoa. - Wait, what was that? - He's a warlock. - Oh. - I should've known. [END PLAYBACK] [APPLAUSE] LAURENCE MORONEY: Thank you. And this, this is just
one of the many reasons that I do what I do. And I believe that
building AI solutions can help us all make the
world a much better place. And as you saw in that
video, Lance's mobility is quite limited. And solutions that can help
him interact with the world can be very expensive and
very difficult to maintain until AI and machine
learning came along. And this is where good
software engineering, combined with these new technologies,
can help us solve problems that just were not feasible before. So let's follow some good
engineering practice. And we'll decompose the
problem into smaller, solvable solutions. And in this case, that meant
taking multiple ML models and orchestrating them
together with traditional code. There was one that
detected a face, which then fed codes that
cropped the image to just contain that face. And then there was
another ML model that detected
landmarks on the face before we finally
wrote code that turned the location of those
landmarks into mouse telemetry. And now, you have ML models
and traditional code working side by side to solve a
problem quickly and easily. Solutions like this
are what we delight in. And we're really excited
to announce and show you the work that we've been
doing with MediaPipe, where we've taken-- [APPLAUSE] Thank you. Where we've taken common
problems that we as developers needed to solve. Now, some are relatively
simple, like image or text classification. And some are much more complex,
like facial landmarks or hand poses. But the goal was
to encapsulate them all into a single,
modular solution that you can drop into
your apps or your sites. But we haven't stopped there. For you to be successful in
building revolutionary apps like Project Gameface, you also
need to go beyond the solutions that we thought of. So MediaPipe offers
customizable solutions that can easily be integrated
into your platform of choice, ready to solve
those hard problems, like the way Project
Gameface grew out of a head tracking
pipeline of models to one that gave us finer control
over facial landmarks. And of course, the models can be
customized through retraining. So if the model's
close to what you want but not quite
there-- for example, the hand pose model
might be great. But it can't identify
rock paper scissors. So we're making it easy
for you to fine-tune the pipeline for that with
very little code, like this. And then when it comes
to integrating ML models into your Android, iOS,
or your web application, you don't need to worry
about tensors or any of those other complex,
underlying machine learning constructs. We're making it easier for you
because the MediaPipe framework will tightly integrate
with the native data types in your chosen language. And all of this is
available for you to use today, including
nine new solution types. So I think you're going to have
lots of fun playing with them. I know I did. And I think you're going to
build some amazing things with MediaPipe. But not only that,
Project Gameface, that we created using
this technology, is being open sourced today. So Lance, you're
going to get your wish of making this available
for everybody to use. So now, I'm going to invite-- [APPLAUSE] Oh, thank you. So now, I'm going to invite my
friend and colleague, Sharbani on stage. And she's going to
tell us more about how we're powering end to end
workflows for you to build on. Sharbani. SHARBANI ROY: Thank
you so much, Lawrence. LAURENCE MORONEY: Thank you. SHARBANI ROY: I just
loved those examples. I never get over how ML is
such an exciting and impactful space. Our machine learning
products are underpinned by deep customer needs-- be it for diverse technical
customers and partners across the globe or
right here at Google. We're focused on providing a
unified ecosystem of tools, services, tutorials,
and open standards across machine learning
workflows from access to data sets and pre-trained
models with Kaggle, to cutting edge
research with JAX, to powerhouse model deployment
with TensorFlow and Keras. We're sharing the
same innovations that power all the stuff you
heard about earlier today. To help show you how we
work to pass this on to you, let me walk you through
a fun example inspired by our littlest of users. My family loves getting outside. My kids are especially
into birds, which are easier to hear than spot. My kids often ask, Mama,
[SPEAKING SPANISH] which is Spanish For, mom,
what bird is singing? After our last
adventure, I remember that the Google Research
Team had recently opened sourced a new bird
vocalization classifier model on Kaggle models. So I fired it up in Colab. It's a great place to write and
execute code from your browser, offering free access to powerful
Google Cloud GPUs and TPUs, perfect for machine learning. And just like that, I've
got some simple code to identify birds by their song. I used a recording from
the previous outing. And here, you can see,
it identified the sound that I captured as a
broad-tailed hummingbird. Hummingbirds are fun to watch. But they're so fast
and hard to see. So the next day, when Lawrence
and I were chatting-- yes, we actually were together. And no, we're not
paid actors or models. He reminded me about the
new large language model integration in Colab. Now, let's see what happens
when we add in some Colab magic. We can get the model to describe
what the bird will look like by simply typing %%llm or by
writing a function in Python that uses this. This is a simple way to access
large language models that is almost as easy as
using a chat interface, but backed by the flexibility
that you want as a developer. They integrate beautifully into
Python and, as you can see, run like lightning on Colab,
thanks to the free compute. OK. So the model has given me
a description of the bird. Now, let's take another
look at another fun tool that our team has
been working on, KerasCV, which is the new
library of state of the art computer vision
models for Keras, the high level modeling
framework in TensorFlow. So taking the
description that we just got with a few
additional lines of code, it can help me take a
complex diffusion model and give me a beautiful
picture of the bird. Gorgeous. [APPLAUSE] KerasCV and Keras
NLP are toolboxes that solve many common
ML developer problems-- avoiding overfitting? Check. Augmenting data? Why yes, we've got you covered. Transformer based models? Yes. All within the
familiar Keras API. Also, KerasCV and
Keras NLP models are part of the
TensorFlow ecosystem. So you can run
seamlessly inference on device, in the browser, or
on almost any compute surface. Now, I know this is a fun
demo I built for my kids. But think about
what we just did. We went from a
sound to a picture through a process
of orchestrating multiple complex models
with some simple code in a matter of minutes. And what I showed you
was a process for how we solve problems, whether
it's a bird, or a plane, and detecting flight contrails
to mitigate global warming. By the way, check out that
competition on Kaggle. We want to enable you to solve
all these problems with machine learning, wherever you
are in your workflow and wherever you
are in your journey. There's a lot to take in. But we can help you today. Visit g.co/ai/build to learn
more about the new technologies and techniques I just shared,
including Keras, TensorFlow, Kaggle, Colab, and more. [APPLAUSE] We've also shared a
really cool walkthrough that lets you train a
generative text model and shrink it to run
on an Android phone, taking you end to end
from data to deployment. And if you want to take your AI
application to the next level, to share how you can do
that with world class infrastructure and
tooling from Google Cloud, I'd like to invite Chen
to tell you all about it. [MUSIC PLAYING] [APPLAUSE] CHEN GOLDBERG:
Thank you, Sharbani. When it comes to
cloud, generative AI is opening the door for
professional developers with all different skill
levels to be productive. All of this is made
possible because of Google's industry-leading
AI-optimized infrastructure. With our new cloud capabilities,
along with managed services, you can build
enterprise-ready applications without needing
expertise in areas like security, scalability,
sustainability, and cost management. Like you heard from
Thomas, we are transforming Cloud development with Duet
AI, a new generative AI powered collaborator that acts as your
expert [INAUDIBLE] programmer. We believe Duet AI
fundamentally changes how developers of
all skill levels can build cloud applications. It provides collaborations
where you need it, within Cloud Console,
Chat, and your IDE. With Duet AI, you will
also have the power to not just call
Google-trained models, but also custom code models
trained directly on your code. Well, instead of me telling you
about it, let me show you how. Let's say I'm running a
shopping website called Sybol. And thanks to Google Cloud,
we have a global footprint. We have lots of
customers in India. So we want to give them
a better experience. And the first step is to
support the Hindi language. So I head over to
Cloud workstation-- my secure, fully managed
development environment, which is now available in GA. And all I have to do
is to create a function and add a comment. And now, thanks to Duet AI,
I can see a code snippet for using Cloud Translation API
is suggested to me immediately. Generated code is a good start. But good software
engineering practice, like ensuring that my
dependencies are up to date, is essential. So before I go to
production, I can check these and ensure they work. Hm, it looks like I have an old
version of a telemetry library. So let's click
Upgrade to fix it. And my website now
supports Hindi. [APPLAUSE] [SPEAKING HINDI] What would have taken me a
long time-- not to mention, I might not have been
able to do it on my own-- just took me minutes. And one of my personal
favorite ways to use Duet AI is making the work
of maintaining large code bases simpler. I've come across this code,
which I'm not familiar with. Well, now, instead
of pinging the owner, searching for related, code
and spending a long time reviewing it, I can
just ask Duet AI to help me understand
this piece of code. And here it is. If you want to give it a go,
sign up for Duet AI today by joining our Trusted
Tester Program. [APPLAUSE] So that was really cool. But how awesome it would be
if it had a similar experience for your code base? Vertex AI lets you do just that. You can tune and customize
foundation models from Google with your own code base. No ML expertise is required. And you can call your
custom code models directly from the Duet AI. Here's an example
where we created a fine tuned model using
Symbol's code base. We wanted to understand our
site's performance on mobile. So we added a comment. Next, Duet AI generated
code for performance tracing from our own library. That's extremely useful, right? And while having the ability to
tune and customize foundation models is awesome,
let's also look at how Vertex AI can help
create new content like images and text, so we can continue
to grow our business. In Vertex AI, you can
easily access a full suite of foundation models from
Google and open source partners with enterprise-grade data
governance and security. And you don't need to worry
about all the work needed to set them up. Let's give it a go. So we want to add a new
product to our catalog. And handbags are always popular. In Vertex AI, I can simply
choose Imagen, our text to image foundation model. I will then enter
my desired prompt. And ta-da, I now have multiple
variations to work with. Cool, right? [APPLAUSE] I think I'll pick this one. It's almost perfect. And if I want to
tweak the design, I have more power than
simple in painting. I can use mask-free editing. So it will work, regardless of
the complexity of the image, giving me the freedom to
easily iterate and explore different options without
the complexity of hosting my own model or figuring
out a type of parameters. So let's change the material
to iridescent blue with a scale texture and at a tassel. Now, this is a bag I would buy. Typically, we would now create
a physical prototype of the bag to take it to market. But we only have a few minutes. So let's use an
existing tote bag, so I can show you how we can
generate creative options with Vertex AI. I took a few pictures of
the bag using my phone and was able to fine-tune
the model with them. And in just a few
minutes, I can now see my handbag literally
anywhere from the Grand Canyon, to the beach, to the city
without getting on a plane-- very carbon friendly. This shot is my favorite. And I think I'll use it. With Vertex AI, I can quickly
and easily upscale it, so it looks consistent on
high resolution displays, in my online store,
and in print. It's almost ready to
be added to my site. As I'm expanding globally,
the power of Vertex AI will let me to generate text
captions for accessibility and localize them into
more than 300 languages. Maybe my next market
will be Arabic speaking. So let's translate
the copy and-- hm, that was easy. [APPLAUSE] Also with the Embeddings API,
I can calculate the embedding vector for my
generated text and use that to do lots of cool
stuff, like ensuring I have the right style of
description for my brand or other powerful use
cases like Semantic Search, or Questioning Answering,
or content recommendations. We love how generative AI
makes workflows simpler. And the power of Vertex allows
you to do it at global scale easily, securely, and safely. Just imagine what ideas you
can now turn into reality and reach a global audience. Speaking of global reach,
over three billion businesses from across the world rely
on Google Workspace daily. They can use it to create,
connect, and collaborate. What if you could leverage
the power of Duet AI to build apps on
Workspace without even knowing how to code? Let's say I'm asked to create
an app to better manage traffic requests for our team. I head over to AppSheet,
a no-code platform used to build apps integrated
with Google Workspace. I describe in natural language
the travel approval app I want to build. Next, Duet AI walks me
through the process step by step, asking a
simple set of questions like, how would they
like to be notified? What are the key
sections of my app? And most importantly,
what's the name of the app? Let's call it Simple Travel. Once the questions
are answered, Duet AI creates the app
with travel requests from my team within
Google Workspace. [APPLAUSE] I love how Duet AI
empowers everyone to get things done quickly
and more effectively. And for those of us
who do love to code-- and I'm sure there are a lot
of people like that here-- we are announcing new chat
APIs in Google Workspace today, which will be generally
available in the coming weeks. With these APIs, you
can build chat apps that provide link previews
and lets users perform actions like creating or
updating records. Atlassian used these APIs to
build their Jira app for chat. New extensibility features
for Workspace make it you can easily build
workflows across Workspace and third party apps, creating
connected experiences. And coming to preview
in the next few weeks are new Google Meet
APIs and two new SDKs that allow you to bring
your app into Google Meet or bring Google Meet data and
capabilities to your apps. As you can see here,
we have the Figma app, [? InMe, ?] with multiple
people working together. [APPLAUSE] All of these AI investments
we show you today can fundamentally
transform the way all cloud users, whether you
are a developer or a business owner, build new experiences. From our enterprise-grade cloud
products and large language models to flexible
open frameworks that Lawrence talked
about earlier, there has never
been a better time to explore what AI
and ML can do for you. Next, I'll pass it back to
Jeanine to wrap us up here. [MUSIC PLAYING] [APPLAUSE] JEANINE BANKS:
Today, we're seeing what happens when software
engineers and new technologies like AI work together. We can unleash creativity and
build things that simply never existed before. What's really
amazing is the impact that we can all make
together as a community. And I stress that we're really
only just getting started. The opportunity to be a
part of something wonderful is right there for all of us. Let's explore what happened
when over 1,000 developers came together on Kaggle to
solve a problem experienced by the parents of deaf children. [VIDEO PLAYBACK] - We're a family of
four, the two of us and our twin boys,
Arin and Ishay. Arin is deaf, and
Ishay is hearing. - Every single day,
I would be like, can I just download
ASL in my brain today? Can I just become proficient? I know what I want
to say to this kid. Can't I just find the
language that he understands? - No one told us it
was important for us as parents to learn ASL. How would we communicate
with our daughter if we didn't learn ASL? - Learning any language
is pretty difficult. But if you put in the
perspective of learning a language to communicate
with your child, it's something that you
know that you need to do. - What we really want to do
is make sign language more accessible to the hearing
community, especially these hearing parents
of deaf infants. PopSign is a game that uses
AI to help anybody be more confident in their signing. - I hold it with one
hand and aim with hand, and then sign dad, and then-- oh, yeah, cool, got it. - PopSign was built
by a team of students at Georgia Tech and RIT. The game tracks
your hand gestures and confirms that they are
the correct signs in real time on your device, using
MediaPipe and TensorFlow lite two powerful cross-platform
tools for building machine learning apps. - You're not just
watching, but you're doing. So the learning that happens by
doing is a lot more powerful. - This is just the
beginning of a journey. And there are going to be lots
of positive ripple effects-- new innovations, new
technologies, human connections so that people can understand
different people's lives, both for the deaf and hearing. - As a mom, I always felt
connected to her, even without language. But now that we're learning ASL,
our connection is blossoming. And our relationship
is better than ever. [MUSIC PLAYING] [END PLAYBACK] [APPLAUSE] JEANINE BANKS: Using technology
to help parents communicate with their children-- now, that's stories like that
that I've been waiting all year to be able to share
because it shows what's possible when technology
is accessible to everyone. Before I wrap up, I'd like to
share one more example of this. Last year, we launched
the ARCore Geospatial API. And it's enabled
many of you to build location-based,
immersive experiences. With ARCore now available
on over 1.4 billion devices, we wanted to go a step further
and share this opportunity with more types of creators
than just the hardcore coders in the house. Starting today, you can
easily design and publish with our new Geospatial Creator
powered by ARCore and Google Maps platform. [APPLAUSE] It's available today in
tools that creators already know and love-- Unity, and Adobe Aero,
and geospatial prerelease. Anyone can now create
engaging AR experiences with just a few clicks. To give you an idea of what you
can achieve with these tools, we're partnering with Taito
to launch the "Space Invaders World Defense" game
later this summer. [APPLAUSE] It's inspired by the
original gameplay. And it turns your city
into a playground. Today, you saw how
we're making it easier for you to cut
through complexity and make your apps
work even better in this multi-device,
multiplatform world. We're bringing the power
of AI to help guide you throughout your entire
development experience with assistance built right
into Android Studio and Google Cloud. You can access the
same foundation models to build your own AI solutions
with MakerSuite, Firebase, and Vertex AI. And Google's commitment to
the open web hasn't changed. We're still collaborating
with the community to push forward
innovations like WebGPU to make the web API-ready, like
WebAssembly to accelerate app performance, and Baseline, so
you can build with confidence across browsers. And now, you can head to
the Google I/O website to find 200 sessions and
other learning material to go deeper into
everything you heard today, all the I/O announcements. [APPLAUSE] And I know I speak
for all Googlers when they say they're super
excited to join you online as well and other
developers from all around the globe
on Adventure Chat. It's all on io.google. I look forward to chatting
with you all too online, later. We've all heard
from you that you're eager to get back with
your community, together in person, as well as
with Google experts. So we're coming to you with four
new flagship events this year. And we're calling
them I/O Connect because it's all
about connecting with the experts behind
all the I/O announcements. We'll be bringing along hands-on
demos, co-labs, office hours, and more, so you can get up to
speed on all of this new tech. The first one is in Miami. It's going to be on May 24. And then we have Amsterdam,
Bengaluru, and Shanghai to follow soon. And if you can't make
it to one of those, you can still join us at one of
the more than 250 I/O Extended meetups that are happening
all around the world in the next few months. Thank you for joining us today. This is your moment. Go create. [APPLAUSE] [MUSIC PLAYING]
