Google made two new TPUs, AI performance up by more than 100% to take on Nvidia

Google has unveiled its latest generation of its tensor processing unit. These chips are not for Pixel smartphones but AI, the buzzword in the tech world these days. These new AI chips obviously come with performance improvements over the previous generation, but Google is trying to do more with these chips, which are available in two variants—TPU 8t and TPU 8i.

These chipset aims to make AI more power efficient. See, AI technology is very capable, from responding to you in your natural language to writing code—it can do it all. But at the end, it is a system which sifts through massive amounts of data very fast to make connections and establish patterns that can be represented mathematically. It works in two steps—first, it is trained on data to find patterns. In the second step, it makes predictions based on those patterns to solve a problem.

In the whole process, the system consumes large amounts of electricity, which has emerged as a big constraint for data centres and this is where new Google chipsets come into picture. The new chipsets aim to make AI cheaper and less energy-intensive by making these two processes more efficient.

The new TPU 8t, which is for creating artificial intelligence software, delivers 124 per cent more performance per watt than the preceding generation. TPU 8i is for running AI services—a process technically known as inference—and provides a gain of 117 per cent.

And there’s more these new TPUs can do.

Faster data access for quicker responses

When AI works, it seeks information to find patterns in problems. Faster access to data makes the process quicker, which involves multiple steps. Google’s new TPUs also store more information and provide rapid response, so the system doesn’t have to seek information stored elsewhere. As a result, end users will get faster responses to their queries.

“AI systems built on the chips will be ‘generally available later this year,’ Google said in a statement.”

Scaling AI while lowering costs

“It’s about how you deliver the lowest possible latency of the response at the lowest possible cost per transaction,” said Mark Lohmeyer, Google’s vice president of compute and AI infrastructure, told Bloomberg. “The number of transactions is going way up, and the cost per transaction needs to go way down for it to scale,” he added.

But we have heard that Nvidia chipsets are widely used in AI?

Google still uses Nvidia chipsets, but these new chips are here to supplement the Nvidia-based systems it offers in its infrastructure. In fact, Google intends to be among the first to deploy gear based on a new design, Vera Rubin, from Nvidia coming in the second half of the year.

The company has said that it will continue to offer services based on Nvidia chips to customers who want to use the systems that currently dominate AI computing.