[Gabriela Surita speaking]
We built Gemini from the ground up to be natively multimodal,
including something quite important for both
of us, programming code. Gemini is able to consistently understand, explain and generate code that is correct, and well-written in most
programming languages. That includes Python, Java, C++ and Go. It substantially improves coding abilities over previous PaLM 2 models. From a benchmark of around 200 programming functions in Python it consistently solves about 75% of them in the first try, versus around 45% on PaLM 2. If you allow Gemini to check
and repair its own answers, this number jumps to over 90%,
which is a huge step forward. It can help you create and
prototype new ideas in seconds. Let's give it a try. I really like trains, and if I wanted to create a
train-spotting location web app, I can simply ask and
get a working prototype in less than a minute.
[light clicking] While the code isn't perfect, it's really helpful to have a first draft. Gemini, on its own, has
the ability to transform software development as we understand it, but it can also be
deployed as a key component of more sophisticated systems. [Rémi Leblond speaking]
Gemini is great at coding, but we've been able to
take it even further, creating a specialized version that performs remarkably well
at competitive programming. Now, why do we care about
competitive programming? Well, it is one of the
ultimate litmus tests of algorithmic coding abilities. So, you have thousands
of talented programmers from all over the world that come together to compete and try to solve
incredibly complex problems that require not only coding,
but also math and reasoning. Two years ago, we presented AlphaCode, and it was the first AI system that could compete roughly at the level of the average human competitor. Today, I'm delighted to
introduce AlphaCode2, a new and enhanced system with massively improved
performance, powered by Gemini. When we evaluate AlphaCode2
on the same platform as the original AlphaCode, we solve almost twice as many problems. While AlphaCode broke through the top half of human competitors, on average, we estimate that
AlphaCode2 performs better than 85% percent of
competition participants. Let's have a look at our model in action on one of the hardest
problems that we faced, and I say "hard" because
in the original contest in which the problem appeared, less than 0.2% of participants
actually solved it. The problem is quite difficult. It's very abstract, so I can't
get into too many details, but the basic gist of
it is that we are tasked with computing aggregate statistics that account for what appears to be an impossibly large
amount of random arrays. The really cool thing is that to solve it, AlphaCode2 makes use
of dynamic programming. Dynamic programming is an
advanced algorithmic technique, which basically simplifies
a complicated problem by breaking it down into easier sub-problems again and again, and what's really impressive is that not only AlphaCode2 knows how to properly implement this strategy, but also when and where to use it. What the example shows us is
that competitive programming is not just about implementation, it's also about understanding, maths, computer science and indeed coding and that makes it an
extremely hard reasoning task. So, it's not very
surprising that up 'til now, generally available large language models have scored very poorly on this benchmark. These models are really, really good at following instructions, but AlphaCode needs to do more than that. It needs to show some
level of understanding, some level of reasoning,
designing of code solutions, before it can actually get
to the actual implementation to solve the problem, and it does all that on problems that it's never seen before. Another thing that is
great about AlphaCode is that it performs even better when it collaborates with human coders who can provide grounding. Basically, developers
can specify properties that the code samples have
to obey and when we do that, we see performance increase significantly. We think of this, this kind of interaction
between programmers and AIs, as the future of programming, where coders will not
just give instructions, but actually collaborate
with highly capable AI models that can reason about their problems, that can propose code designs and that can even help with
the actual implementation. AlphaCode2 was built for
competitive programming, but we're already working on bringing some of its unique capabilities right into the general Gemini models, as a first step towards making this new programming paradigm
available for everyone.
