Google Keynote (Google I/O ‘24)

[Cheers and Applause]. >>WOMAN: Google’s ambitions in 

artificial intelligence. >>MAN: Google launches Gemini, 

the generative AI. >> And it's completely changing 

the way we work. >> You know, a lot has happened 

in a year. There have been new beginnings. 

We found new ways to find new Ways to find new ideas. 

And new solutions to age-old problems. >> Sorry about your shirt. 

We dreamt of things -- >> Never too old for a 

treehouse. >> We trained for things. 

>> All right! Let’s go go go!

>> And learned about this thing. We found new paths, took the 

next step, and made the big leap. Cannon ball! 

We filled days like they were weeks. 

And more happened in months, than has happened in years. 

>> Hey, free eggs. >> Things got bigger,  

like waaay bigger. 

And it wasn’t all just for him, or for her. 

It was for everyone.

And you know what? 

We’re just getting started.

>>SUNDAR PICHAI:  Hi, everyone. Good morning. 

[Cheers and Applause]. welcome to Google I/O. 

It's great to have all of you with us. We have a few thousand 

developers with us here today at Shoreline. 

Millions more are joining virtually around the world. 

Thanks to everyone for being here. 

For those of you who haven’t seen I/O before, it’s basically 

Google’s version of the Eras Tour, but with fewer costume 

changes. [Laughter]. 

At Google, though, we are fully in our Gemini era. Before we get into it, I want to 

reflect on this moment we’re in. We’ve been investing in AI for 

more than a decade, and innovating  at every layer of the stack: 

Research, product, infrastructure We’re going to talk about it all today. 

Still, we are in the early  days of the AI platform shift. 

We see so much opportunity ahead for creators,  for developers, for startups, for everyone. 

Helping to drive those opportunities  is what our Gemini era is all about. 

So let’s get started. 

A year ago on this stage, we first shared our plans for 

Gemini, a frontier model built to be natively multimodal from 

the very beginning, that could reason across text, images, 

video, code, and more. It’s a big step in turning any 

input into any output. An I/O for a new generation. 

Since then we introduced the first Gemini models, our most 

capable yet. They demonstrated 

state-of-the-art performance on every multimodal benchmark. 

And that was just the beginning. Two months later, we introduced 

Gemini 1.5 Pro, delivering a big breakthrough in long context. 

It can run 1 million tokens in production, consistently. 

More than any other large-scale foundation model yet. 

We want everyone to benefit from what Gemini can do, so we’ve 

worked quickly to share these advances with all of you. 

Today, more than 1.5 million developers use Gemini models 

across our tools. You’re using it to debug code, 

get new insights, and build the next generation of AI 

applications. We’ve also been bringing 

Gemini’s breakthrough capabilities across our products 

in powerful ways. We’ll show examples today across 

Search, Photos, Workspace, Android and more. 

Today, all of our 2-billion user products use Gemini. 

And we’ve introduced new experiences, too, including on 

Mobile, where people can interact with Gemini directly 

through the app. Now available on Android and 

iOS. And through Gemini Advanced, 

which provides access to our most capable models. 

Over 1 million people have signed up to try it, in just 

three months. And it continues to show strong 

momentum. One of the most exciting 

transformations with Gemini has been in Google Search. 

In the past year, we’ve answered billions of queries as part of 

our Search Generative Experience. 

People are using it to Search in entirely new ways. 

And asking new types of questions, longer and more 

complex queries, even searching with photos, and getting back 

the best the web has to offer. We’ve been testing this 

experience outside of Labs, and we’re encouraged to see not only 

an increase in Search usage, but also an increase in user 

satisfaction. I’m excited to announce that 

we’ll begin launching this fully revamped experience, AI 

Overviews, to everyone in the U.S. this week. 

And we’ll bring it to more countries soon.

[Cheers and Applause]. There’s so much innovation 

happening in Search. Thanks to Gemini we can create 

much more powerful search experiences, including within 

our products. Let me show you an example in 

Google Photos. We launched Google Photos almost 

nine years ago. Since then, people have used it 

to organize their most important memories. 

Today that amounts to more than 6 billion photos and videos 

uploaded every single day. And people love using Photos to 

search across their life. With Gemini, we’re making that a 

whole lot easier. Say you’re at a parking station 

ready to pay, but you can’t recall your license plate 

number. Before, you could search Photos 

for keywords and then scroll through years’ worth of photos, 

looking for the right one. Now, you can simply ask Photos. 

It knows the cars that appear often, it triangulates which one 

is yours, and just tells you  the license plate number.

[Cheers and Applause]. And Ask Photos can help you 

search your memories in a deeper way. 

For example, you might be reminiscing about your daughter 

Lucia’s early milestones. You can ask photos, when did Lucia learn to swim? 

And you can follow up with up with something more complex. 

Show me how Lucia's swimming has progressed. Here, Gemini goes beyond a 

simple search, recognizing different contexts from doing 

laps in the pool, to snorkeling in the ocean, to the text and 

dates on her swimming certificates. 

And Photos packages it all up together in a summary, so you 

can really take it all in, and relive amazing memories all over 

again. We’re rolling out Ask Photos 

this summer, with more capabilities to come.

[Cheers and Applause]. Unlocking knowledge across 

formats is why we built Gemini to be multimodal from the ground 

up. It’s one model, with all the 

modalities built in. So not only does it understand 

each type of input, it finds connections between them. 

Multimodality radically expands the questions we can ask, and 

the answers we will get back. Long context takes this a step 

further, enabling us to bring in even more information, hundreds 

of pages of text, hours of audio, a full hour of video, or 

entire code repos. Or, if you want, roughly 96 

Cheesecake Factory menus. [Laughter]. 

For that many menus, you’d need a one million token context 

window, now possible with Gemini 1.5 Pro. 

Developers have been using it in super interesting ways. 

Let’s take a look. >> I remember the announcement, 

the 1 million token context window, and my first reaction 

was there's no way they were able to achieve this. 

>> I wanted to test its technical skills, so I uploaded 

a line chart. It was temperatures between like 

Tokyo and Berlin and how they were across the 12 months of the 

year. >> So  

I got in there and I threw in the Python library that was 

really struggling with and I just asked it a simple question. 

And it nailed it. It could find specific 

references to comments in the code and specific requests that 

people had made and other issues that people had had, but then 

suggest a fix for it that related to what I was working 

on. >> I immediately tried to kind 

of crash it. So I took, you know, four or 

five research papers I had on my desktop, and it's a mind-blowing 

experience when you add so much text, and then you see the kind 

of amount of tokens you add is not even at half the capacity. 

>> It felt a little bit like Christmas because you saw things 

kind of peppered up to the top of your feed about, like, oh, 

wow, I built this thing, or oh, it's doing this, and I would 

have never expected. >> Can I shoot a video of my 

possessions and turn that into a searchable database? 

So I ran to my bookshelf, and I shot video just panning my 

camera along the bookshelf and I fed the video into the model. 

It gave me the titles and authors of the books, even 

though the authors weren't visible on those book spines, 

and on the bookshelf there was a squirrel nut cracker sat in 

front of the book, truncating the title. 

You could just see the word "sightsee", and it still guessed 

the correct book. The range of things you can do 

with that is almost unlimited. >> So at that point for me was 

just like a click, like, this is it. 

I thought, like, I had like  a super power in my hands. 

>> It was poetry. It was beautiful. 

I was so happy! This is going to be amazing! 

This is going to help people! >> This is kind of where the 

future of language models are going. 

Personalized to you, not because you trained it to be personal to 

you, but personal to you because you can give it such a vast 

understanding of who you are. [Applause]. 

>>SUNDAR PICHAI: We’ve been rolling out Gemini 1.5 Pro with 

long context in preview over the last few months. 

We’ve made a series of quality improvements across translation, 

coding, and reasoning. You’ll see these updates 

reflected in the model starting today. 

I'm excited to announce that we’re bringing this improved 

version of Gemini 1.5 Pro to all developers globally.

[Cheers and Applause]. In addition, today Gemini 1.5 

Pro with 1 million context is now directly  available for consumers in Gemini Advanced,  

and can be used across 35 languages. One million tokens is opening up 

entirely new possibilities. It’s exciting, but I think we 

can push ourselves even further. So today, we are expanding the 

context window to 2 million Tokens.

[Cheers and Applause]. We are making it available  

for developers in private preview. It's amazing to look back and 

see just how much progress we've made in a few months. 

This represents the next step on our journey  towards the ultimate goal of infinite context. 

Okay. So far,  

we’ve talked about two technical advances: 

multimodality and long context. Each is powerful on its own. 

But together, they unlock deeper capabilities, and more 

intelligence. Let’s see how this comes to life 

with Google Workspace. People are always searching 

their emails in Gmail. We are working to make it much 

more powerful with Gemini. Let’s look at how. 

As a parent, you want to know everything that’s going on with 

your child’s school. Okay, maybe not everything, but 

you want to stay informed. Gemini can help you keep up. 

Now we can ask Gemini to summarize all recent emails from 

the school. In the background, it’s 

identifying relevant emails, and even analyzing attachments, like 

PDFs. And you get a summary of  

the key points and action items. So helpful. 

Maybe you were traveling this week and couldn’t make the PTA 

meeting. The recording of the meeting is 

an hour long. If it’s from Google Meet, you 

can ask Gemini to give you the highlights.

[Cheers and Applause]. There’s a parents group looking 

for volunteers, and you’re free that day. 

So of course, Gemini can draft a reply. 

There are countless other examples of how this can make 

life easier. Gemini 1.5 Pro is available 

today in Workspace Labs. Aparna will share more later on.

[Applause]. We just looked at an example with text outputs. 

But with a multimodal model, we can do so much more. 

To show you an early demo of an audio output in NotebookLM, 

here’s Josh. >>JOSH WOODWARD: Hi, everyone! 

Last year, at I/O, we introduced Notebook LM, a research and 

writing tool grounded in the information you give it. 

Since then, we've seen a lot of momentum with students and 

teachers using it. And today, Gemini 1.5 Pro is 

coming to Notebook LM, and it's great. 

Let me show you. So here we are in Notebook LM. 

You can load it up with all of the materials here on the left. 

In this notebook, I've been using it with my younger son,  

and I've added some of his science worksheets,  a few slide decks from the teacher, and even an  

open source textbook full of charts and diagrams. With 1.5 Pro, it instantly creates this notebook  

guide with a helpful summary and can generate  a study guide, an FAQ, or even quizzes. 

But for my son Jimmy, he really learns  best when he can listen to something. 

So we've prototyped a new feature with  Gemini, and it's called Audio Overviews. 

Notebook LM is going to use all of the materials on the left  

as input and output them into a lively  science discussion, personalized for him. 

Let's take a listen. >> So let's dive into physics. 

What's on deck for today? >> Well, we're starting with the 

basics. Force in motion. 

>> Okay. >> And that, of course, means we 

have to talk about Sir Isaac Newton and his three laws of 

motion. >> Ah,  

yes, the foundation -- For understanding how objects move and interact.

>>JOSH WOODWARD: Oh, yes, this is  where multimodal really shines. 

Now, it generated this audio  discussion based on that text material. 

And what's amazing is that my son  and I can join into the conversation. 

And steer it whichever direction we want. 

When I tap "join." >> Hold on. 

We have a question. What's up, Josh? 

>>JOSH WOODWARD: Yeah, can you give my son Jimmy a basketball 

example? >> Hey, Jimmy! 

That's a fantastic idea! Basketball is actually a great 

way to visualize force in motion. 

Let's break it down. >> Okay. 

So first, imagine a basketball just sitting there on the court. 

It's not moving, right? 

That's because all the forces acting on it are balanced. 

The downward pull of gravity -- >>JOSH WOODWARD: Pretty cool, right?

[Cheers and Applause]. I gotta say, the first  

time my son heard this, you should  have seen how big his eyes got! 

Because he was gripped. They were talking to him. 

He was learning science through  the example of basketball. 

His favorite sport. Now, what's interesting  

is under the hood, you saw that Gemini had used  some of the concepts of gravity, Sir Isaac Newton,  

but nothing in there was about basketball. It connected the dots and created that  

age-appropriate example for him. And this is what's becoming  

possible with the power of Gemini. You can give it lots of information in  

any format, and it can be transformed in a way  that's personalized and interactive for you. 

Back to you, Sundar. [Applause]. 

>>SUNDAR PICHAI: Thanks, Josh. The demo shows the real 

opportunity with multimodality. Soon you’ll be able to mix and 

match inputs and outputs. This is what we mean when we say 

it’s an I/O for a new generation. 

And I can see you all out there thinking about the 

possibilities. But what if we could go even 

further? That’s one of the opportunities 

we see with AI agents. Let me take a step back and 

explain what I mean by that. I think about them as 

intelligent systems that show reasoning, planning, and memory. 

Are able to “think” multiple steps ahead, work across 

software and systems, all to get something done on your behalf, 

and most importantly, under your supervision. 

We are still in the early days, and you’ll see glimpses of our 

approach throughout the day, but let me show you the kinds of use 

cases we are working hard to solve. Let’s talk about shopping. 

It’s pretty fun to shop for shoes, and a lot less fun to 

return them when they don’t fit. Imagine if Gemini could do all 

the steps for you: Searching your inbox for the receipt, 

locating the order number from your email, filling out a return 

form, and even scheduling a pickup. That's much easier, right?

[Applause]. Let’s take another example 

that’s a bit more complex. Say you just moved to Chicago. 

You can imagine Gemini and Chrome working together to help 

you do a number of things to get ready: Organizing, reasoning, 

synthesizing on your behalf. For example, you’ll want to 

explore the city and find services nearby, from 

dry-cleaners to dog-walkers. You will have to update your new  

address across dozens of Web sites. Gemini can work across these 

tasks and will prompt you for more information when needed, so 

you are always in control. That part is really important. 

as we prototype these experiences. We are thinking hard about how to do it in a way  

that's private, secure and works for everyone. These are simple-use cases, but 

they give you a good sense of the types of problems we want to 

solve, by building intelligent systems that think ahead, 

reason, and plan, all on your behalf. 

The power of Gemini, with multimodality, long context and 

agents, brings us closer to our ultimate goal: Making AI helpful 

for everyone. We see this as how we will make  

the most progress against our mission. Organizing the world’s 

information across every input, making it accessible via any 

output, and combining the world’s information with the 

information in your world in a way that’s truly useful for you. 

To fully realize the benefits of AI, we will continue to break 

new ground. Google DeepMind is hard at work 

on this. To share more, please welcome, 

for the first time on the I/O stage, Sir Demis.

[Applause]. >>DEMIS HASSABIS:  

Thanks, Sundar. 

It's so great to be here. Ever since I was a kid, playing 

chess for the England Junior Team, I’ve been thinking about 

the nature of intelligence. I was captivated by the idea of 

a computer that could think like a person. 

It’s ultimately why I became a programmer and studied 

neuroscience. I co-founded DeepMind in 2010 

with the goal of one day building AGI: Artificial general 

intelligence, a system that has human-level cognitive 

capabilities. I’ve always believed that if we 

could build this technology responsibly, its impact would be 

truly profound and it could benefit humanity in incredible 

ways. Last year,  

we reached a milestone on that path when we  formed Google DeepMind, combining AI talent  

from across the company in to one super unit. Since then, we've built AI systems that can  

do an amazing range of things, from turning  language and vision into action for robots,  

navigating complex virtual environments, involving  Olympiad level math problems, and even discovering  

thousands of new materials. Just last week, we announced  

our next generation AlphaFold model. It can predict the structure and interactions  

of nearly all of life's molecules, including how  proteins interact with strands of DNA and RNA. 

This will accelerate vitally important  biological and medical research from  

disease understanding to drug discovery. And all of this was made possible with the  

best infrastructure for the AI era, including  our highly optimized tensor processing units. 

At the center of our efforts is our Gemini model. It's built up from the ground up to be natively  

multimodal because that's how we interact  with and understand the world around us. 

We've built a variety of  models for different use cases. 

We've seen how powerful Gemini 1.5 Pro is,  but we also know from user feedback that some 

applications need lower latency and a lower cost to serve. 

So today we’re introducing Gemini 1.5 Flash.

[Cheers and Applause]. Flash is a lighter-weight model 

compared to Pro. It’s designed to be fast and 

cost-efficient to serve at scale, while still featuring 

multimodal reasoning capabilities and breakthrough 

long context. Flash is optimized for tasks 

where low latency and efficiency matter most. 

Starting today, you can use 1.5 Flash and 1.5 Pro with up to one 

million tokens in Google AI Studio and Vertex AI. 

And developers can sign up to try two million tokens. 

We’re so excited to see what all of you will create with it. 

And you'll hear a little more  about Flash later on from Josh. 

We’re very excited by the progress we’ve made so far with 

our family of Gemini models. But we’re always striving to 

push the state-of-the-art even further. 

At any one time we have many different models in training. 

And we use our very large and powerful ones to help teach and 

train our production-ready models. 

Together with user feedback, this cutting-edge research will 

help us to build amazing new products for billions of people. 

For example, in December, we shared a glimpse into the future 

of how people would interact with multimodal AI, and how this 

would end up powering a new set of transformative experiences. 

Today, we have some exciting new progress to share about the 

future of AI assistants that we’re calling Project Astra.

[Cheers and Applause]. For a long time, we’ve wanted to 

build a universal AI agent that can be truly helpful in everyday 

life. Our work making this vision a 

reality goes back many years. It's why we made Gemini multimodal  

from the very beginning. An agent like this has to 

understand and respond to our complex and dynamic world just 

like we do. It would need to take in and 

remember what it sees so it can understand context and take 

action. And it would have to be 

proactive, teachable and personal, so you can talk to it 

naturally, without lag or delay. While we’ve made great strides 

in developing AI systems that can understand multimodal 

information, getting response time down to something 

conversational is a difficult engineering challenge. 

Building on our Gemini model, we’ve developed agents that can 

process information faster by continuously encoding video 

frames, combining the video and speech input into a timeline of 

events, and caching this for efficient recall. 

We’ve also enhanced how they sound, with a wider range of 

intonations. These agents better understand 

the context you’re in, and can respond quickly in conversation, 

making the pace and quality of interaction feel much more 

natural. Here’s a video of our prototype, 

which you’ll see has two parts. Each part was captured in a 

single take, in real time. >> Okay. Let's do some tests. 

Tell me when you see something that makes sound. 

>> I see a speaker, which makes sound. 

>> What is that part of the speaker called? 

>> That is the tweeter. It produces high frequency 

sounds. >> Give me a creative 

alliteration about these. >> Creative crayons color 

cheerfully. They certainly craft colorful 

creations. >> What does that part of the