Michigan

Exclusive: How the Tensor G4 chip in the Pixel 9 could redefine the ‘AI phone’

Caradoc Wyn August 16, 2024

There’s a reason why Google moved up the date of its Made by Google event to August, although the company never made an official statement about it. It’s clear that Google wants to steal the show from the upcoming iPhone 16 by launching the Pixel 9 early.

To that end, Google demonstrated a wide range of Pixel 9 AI features, including Add Me to add someone to a photo after the fact and Call Notes to record and summarize phone conversations. And the Google Pixel Screenshots feature can help you figure out details using natural language queries.

Powering the whole thing is the new Tensor G4 chip, which is specifically designed to run Google’s most advanced AI models. In fact, it’s the first processor that can run Gemini Nano multimodally, meaning the Pixel 9, Pixel 9 Pro, Pixel 9 Pro XL, and Pixel 9 Pro Fold can all understand text, images, and audio.

To get a deeper look at Google’s secret AI weapon, I spoke with Jesse Seed, Group Product Manager for Google Silicon, and Zach Gleicher, Product Manager for Google DeepMind, about what the Tensor G4 chip can do and what sets it apart from others.

EXCLUSIVE: Inside the Google Pixel 9’s TENSOR G4! – YouTube

Watch on

What makes the Tensor G4 chip stand out from the crowd of smartphones?

Google Tensor G4 chip

(Image credit: Google)

Jesse Seed: I think the biggest innovation we made this year was the first silicon and the first phone to run Gemini Nano with multimodality. And that opens up some very cool use cases, one of them is pixel screenshots. That’s very handy when you’re trying to remember things.

Another feature that isn’t related to the Gemini Nano model but that I really love is the “Add Me” feature. Those of us who are the photographers of our family or crew will definitely appreciate being able to dynamically add the photographer. And that’s something we’ve worked on a lot, optimizing over 15 different machine learning models and also using the Google Augmented Reality SDK.

How did you manage to put something as advanced as the Gemini Nano on a phone?

Twins Live

(Image credit: Google)

Zach Gleicher: At DeepMind, we work with a whole bunch of teams at Google and we want to make sure that we build Gemini models that meet the needs of all Google products. When we were building Gemini in collaboration with Android and Pixel, we realized that there was a need for on-device models. We saw this as a challenge because on the server side, everyone was pushing for more powerful models that were potentially bigger. And we had all these interesting constraints on the other hand that didn’t exist before, like memory constraints, power consumption constraints, etc.

So working with the Tensor team and Pixel, we were able to come together and figure out what the key use cases are for these device models, what the constraints are for these device models, and we actually co-created a model. That was a really exciting experience and allowed us to build something that was so powerful and could support those use cases.

What will someone who hasn’t updated their phone in 3-4 years notice about the G4 chip?

Seeds: Improving what we call fundamentals, like power and performance, is really important to us. The Tensor G4, our fourth-generation chip, is our most efficient and most powerful. And we think users will see that in everyday experiences like web performance or browsing the web, as well as launching apps and the overall speed of the UI. I think it’s a really smooth experience. You’ll see it in the fact that web performance is 20% faster on average and app launch is 17% faster.

And what about gaming performance, because that’s really important for people buying a new phone these days?

Seeds: In our testing, we saw improved peak and sustained performance in both gaming and mainstream games running on the platform.

How does the Tensor G4 contribute to battery life?

Seeds: We’ve improved power efficiency in many everyday use cases. Things like recording video, taking photos, or scrolling through social media use less power than the previous generation.

This all contributes to the 20% longer battery life that you mentioned in the keynote. So the Tensor G4 contributes almost 20% longer battery life and achieves that goal.

What AI features are you most excited about with Gemini enabling on Pixel 9 phones?

Pixel screenshots are shown on a Pixel 9 Pro XL

(Image credit: Future)

Same: We believe one of the main reasons the Tensor team and the Pixel team turn to us for on-device use cases is because of the increased reliability. Because you don’t have to rely on an internet connection, the experience can be reliable and work no matter where you are.

Another point we consider is privacy. If developers don’t want the data to actually leave the device and be processed entirely on the device, this is possible with an on-device LLM.

In terms of AI features that I’m excited about, Pixel Screenshots is really great. I think that really shows how we can get these multimodal features working on devices that can work, as you can see in the demos. It was really zippy, had low latency, but it’s also a super powerful model. And all of that information and data is stored locally on your device and can be processed locally. So we’re really excited that Gemini Nano can enable experiences like that.

I think we’re seeing some traction for summarization and intelligent response use cases.

How is Pixel Screenshots different from Windows Recall, which ran into trouble over privacy concerns?

Seeds: One way to protect user privacy is to have a powerful on-device model. This way, nothing leaves the device when analyzing the screenshot. This way, we can solve this privacy problem.

I think the other thing is giving users the ability to decide what they want to do, how they want to use Gemini, for example. And what use cases they’re comfortable with and what they’re not comfortable with. So I think it really comes down to user choice. But particularly in the case of Pixel Screenshots, it’s a completely on-device use case.

We’ll be running all the usual benchmarks on the Tensor G4, but the AI era is changing things. What do you think about the performance of this chip?

Seeds: I think it really comes down to real-world use cases. How does the thing perform in your hand? So I think things like response time when browsing the web, how quickly apps launch, the speed and responsiveness of the UI are all real-world use cases. Those are good standard things to look for.

What about the AI perspective? When does a Pixel phone pass your test in terms of performance?

Google Pixel 9 held in hand.

(Image credit: Future)

Same: When we think about benchmarks for LLMs and Gemini, and Gemini Nano in particular, we see that the industry places a lot of emphasis on academic benchmarks. And academic benchmarks like MMLU are great because they provide a common metric. But they could be gamified and people could tweak them. And they might not capture what you really care about.

For a device-based model, we don’t care if it knows history questions. We think this is probably a better use case for a server-side model. What we care about are use cases like summarization.

We also need to think about constraints like battery consumption. We need to make sure that the model performs well and doesn’t use too much battery. And that the latency is good as well. So we actually work with the Tensor team to profile our models as we design those models together to make sure that we get an architecture that performs well.

Seeds: It’s not just about traditional performance metrics, it’s about quality. So when you look at things like the quality of the answers that the model provides, or even things like the quality of the photo. That’s what real users care about more than a number on the side of a box.

Yiamastaverna

Yiamastaverna

Exclusive: How the Tensor G4 chip in the Pixel 9 could redefine the ‘AI phone’

LEAVE A RESPONSE Cancel reply

Caradoc Wyn

Chappell Roan dedicates her VMA victory to drag artists

Moto G45: Is this the best 5G phone of 2024? Powerful battery and fast charging at a low price

Unintended consequences of limiting phone charges in prisons

What about technology? When to update your phone

ads

Recent Posts

Recent Comments

Archives

Categories

Exclusive: How the Tensor G4 chip in the Pixel 9 could redefine the ‘AI phone’

LEAVE A RESPONSE Cancel reply

Caradoc Wyn

You Might Also Like

Chappell Roan dedicates her VMA victory to drag artists

Moto G45: Is this the best 5G phone of 2024? Powerful battery and fast charging at a low price

Unintended consequences of limiting phone charges in prisons

What about technology? When to update your phone

ads

Recent Posts

Recent Comments

Archives

Categories