NVIDIA Is On a Different Planet

Gamers Nexus
21 Mar 202431:44

Summary

TLDRNvidia's GTC event unveiled the Blackwell GPU, emphasizing the company's shift from a gaming focus to AI and data center dominance. The Blackwell architecture, which combines two large chiplets as a single GPU, promises significant advancements in chip-to-chip communication and multi-chip modules. Nvidia also introduced Envy Link and Envy Link Switch for improved data center connectivity and a new AI foundation model for humanoid robots, highlighting its commitment to pushing the boundaries of AI technology.

Takeaways

  • 🚀 Nvidia unveiled its Blackwell GPU at the GTC event, marking a significant advancement in AI and gaming technology.
  • 📈 Nvidia's growth in the AI market is impacting its consumer and gaming sectors, with the company now functioning as a major AI Foundry.
  • 🔗 The Blackwell GPU combines two large dies into a single GPU solution, improving chip-to-chip communication and reducing latency.
  • 🧠 Nvidia's focus on AI extends to humanoid robotics with Project Groot, showcasing a future where AI-powered robots could perform complex tasks.
  • 🤖 The introduction of Nvidia's Thor and its multimodal AI models like Groot indicates a shift towards AI integration in various industries.
  • 🌐 Nvidia's Envy Link and Envy Link Switch technologies aim to improve data center communication, with the new Blackwell GPUs offering increased bandwidth.
  • 💡 Nvidia's RAS engine is designed for proactive hardware health monitoring and maintenance, potentially reducing downtime in data centers.
  • 📊 Nvidia's NIMS (Nvidia Inference Machine Software) is a suite of pre-trained AI models for businesses, emphasizing data utilization and IP ownership.
  • 🔄 Multi-chip modules are highlighted as the future of high-end silicon, with Nvidia's Blackwell architecture being a notable example of this trend.
  • 🎮 Despite the technical focus, gaming was not heavily discussed during the event, but the impact of Nvidia's AI advancements on gaming is expected to be significant.
  • 🌐 Nvidia's market dominance is evident, with its AI and data center segments driving significant revenue and influencing the direction of the GPU market.

Q & A

  • What was the main focus of Nvidia's GTC event?

    -The main focus of Nvidia's GTC event was the unveiling of its Blackwell GPU and discussing its advancements in AI technology, multi-chip modules, and communication hardware solutions like NVLink and Envy Link Switch.

  • How has Nvidia's position in the market changed over the years?

    -Nvidia has transitioned from being primarily a gaming company to a dominant player in the AI market, with its products now being used in some of the biggest ventures by companies like OpenAI, Google, and Amazon.

  • What is the significance of the Blackwell GPU for Nvidia?

    -The Blackwell GPU represents a significant technological leap for Nvidia, especially in AI workloads. It combines two large dies into a single GPU solution, offering improved chip-to-chip communication and potentially setting the stage for future consumer products.

  • What are the implications of Nvidia's advancements in chip-to-chip communication?

    -Improvements in chip-to-chip communication, such as those introduced with the Blackwell GPU, can lead to more efficient and high-performing multi-chip modules. This could result in better yields for fabrication, potentially lower costs, and the ability to handle larger data transfers crucial for AI and data center applications.

  • How does Nvidia's AI technology impact the gaming market?

    -While Nvidia has emphasized its AI capabilities, its advancements also have implications for the gaming market. The company's influence in game development and feature inclusion is significant, and its GPUs are often designed to support the latest gaming technologies.

  • What is the role of the Envy Link and Envy Link Switch in Nvidia's announcements?

    -The Envy Link and Envy Link Switch are communication solutions that Nvidia announced to improve the bandwidth and connectivity between GPUs. This is particularly important for data centers and multi-GPU deployments, where high-speed communication is essential for performance.

  • What is Nvidia's strategy with its new inference platform, Nims?

    -Nims is a platform of pre-trained AI models designed for businesses to perform various tasks such as data processing, training, and retrieval. It is CUDA-based, meaning it can run on any platform with Nvidia GPUs, and allows businesses to retain full ownership and control over their intellectual property.

  • How does Nvidia's project Groot and the Thor GS platform contribute to the development of humanoid robots?

    -Project Groot is a general-purpose foundation model for humanoid robots, and the Thor GS platform is designed to run multimodal AI models like Groot. The Thor GS has a Blackwell-based GPU with 800 Tera flops of FP8 capability and a built-in functional safety processor, making it suitable for AI-powered robotics applications.

  • What is the significance of the multi-chip module approach for the future of high-end silicon?

    -The multi-chip module approach is considered the future of high-end silicon as it allows for higher yields and potentially lower costs. It also enables better performance by overcoming limitations in communication links between different pieces of silicon, which is crucial for complex AI and data center applications.

  • How does Nvidia's market position affect its competitors, Intel and AMD?

    -Nvidia's dominant market position influences trends and game development, forcing competitors like Intel and AMD to keep up. Nvidia's substantial revenue from AI and data center segments gives it significant power in the GPU market, which can impact the pricing and development of consumer GPUs.

  • What is the potential impact of Nvidia reallocating assets from AI to gaming?

    -If Nvidia reallocates resources from its successful AI segment to gaming, it could further widen the gap between itself and its competitors in terms of performance, features, and market share. This could lead to Nvidia driving more trends and developments in game technology.

Outlines

00:00

🚀 Nvidia's GTC Event and the Unveiling of Blackwell GPU

The Nvidia GTC event showcased the unveiling of the Blackwell GPU, marking a significant advancement in GPU technology. The presentation highlighted Nvidia's shift from being primarily a gaming company to a major player in the AI market. The event emphasized the importance of multi-chip modules and the challenges of chip-to-chip communication, showcasing Nvidia's innovations in this area. The discussion also touched on the impact of Nvidia's growth on the consumer and gaming markets, and the company's role in the AI sector.

05:01

🤖 Advancements in AI and Multi-Chip Technologies

This paragraph delves into the technical aspects of Nvidia's advancements, particularly in AI and multi-chip technologies. It discusses the potential of multi-chip modules to increase yields and reduce costs for consumers, as well as the focus on improving chip-to-chip communication. The summary also mentions the debut of the B1000, expected to be a multi-die product, and the significance of Nvidia's Blackwell architecture. The paragraph highlights the company's efforts in democratizing computing and the anticipation surrounding the impact of these technologies on both the enterprise and consumer markets.

10:02

🌐 Nvidia's Positioning in the AI and Data Center Markets

Nvidia's strategic positioning in the AI and data center markets is the focus of this paragraph. It discusses the company's branding as an AI foundry and the potential for its technology to influence consumer parts. The summary covers Nvidia's multi-chip module technology, the Blackwell GPU's impressive execution, and the implications for the future of high-end silicon. It also touches on the company's partnerships and the concept of digital twins, which are digital representations of real workspaces used for training robotic solutions.

15:03

🔍 Nvidia's Blackwell GPU and Its Impact on Software Development

The Blackwell GPU's impact on software development and its seamless integration as a single package solution is the central theme of this paragraph. The summary explains how Nvidia has worked to minimize the challenges of chip-to-chip communication, allowing the Blackwell GPU to behave like a monolithic silicon chip. It details the technical specifications of the Blackwell GPU, including its transistor count and memory bandwidth, and discusses the potential for the technology to be integrated into future consumer products.

20:06

🤖 AI and Robotics: Nvidia's Project Groot and Nims

This paragraph focuses on Nvidia's venture into AI-powered robotics with Project Groot and the introduction of Nims, a suite of pre-trained AI models for businesses. The summary covers the potential applications of humanoid robots in various industries and the cultural appeal of AI-driven robotics. It also discusses the capabilities of the Thor platform, which supports multimodal AI models and has a built-in safety processor. The paragraph highlights Nvidia's efforts to create a foundation model for humanoid robots that can understand human language and navigate the world.

25:08

💡 Market Dynamics and the Future of Consumer GPUs

The final paragraph discusses the market dynamics in the GPU industry and the potential future of consumer GPUs. The summary explores Nvidia's dominant position and its influence on game development and features. It also considers the roles of AMD and Intel in the market and their pursuit of AI technology. The discussion touches on the challenges of providing affordable entry points into the gaming market and the potential for multichip modules to become more prevalent in consumer GPUs.

Mindmap

Keywords

💡Nvidia

Nvidia is a multinational technology company known for its graphics processing units (GPUs) and artificial intelligence (AI) technologies. In the video, Nvidia is discussed in the context of its advancements in GPU technology, particularly with the unveiling of the Blackwell GPU at the GTC event, and its significant role in the AI market.

💡Blackwell GPU

The Blackwell GPU is a new graphics processing unit developed by Nvidia, which is expected to be a significant technological breakthrough. It is described as a multi-chip module that combines two large chiplets into a single GPU solution, offering improved performance for AI applications.

💡AI

Artificial Intelligence (AI) refers to the development of computer systems that can perform tasks typically requiring human intelligence, such as learning, reasoning, problem-solving, and perception. In the context of the video, AI is a key market for Nvidia, with the company's GPUs being used in various AI applications and the new Blackwell GPU being particularly aimed at AI workloads.

💡Multi-chip modules

Multi-chip modules (MCMs) are electronic circuits that consist of multiple chips or 'chiplets' integrated into a single functional unit. In the video, this technology is significant as it allows for the creation of powerful GPUs like the Blackwell, which combines two large chiplets to act as a single GPU, improving performance and efficiency.

💡Chiplets

Chiplets are smaller, modular semiconductor chips that can be combined to form a larger system on a chip (SoC). They are used to increase the yield, reduce costs, and improve the performance of integrated circuits. In the context of the video, chiplets are a crucial component of the Blackwell GPU's design, allowing for high-density integration and efficient communication between the chips.

💡PCI Express

PCI Express (Peripheral Component Interconnect Express) is a high-speed serial expansion bus standard for connecting a computer to one or more peripheral devices. It is used for various components such as graphics cards, SSDs, and networking cards. In the video, PCI Express is mentioned in the context of the technical specifications of Nvidia's GPUs.

💡Envy Link and Envy Link Switch

Envy Link and Envy Link Switch are Nvidia technologies related to high-speed communication links and switches between GPUs. These technologies are designed to increase the bandwidth and efficiency of data transfer between GPUs in multi-GPU configurations or within data centers.

💡Digital Twin

A digital twin is a virtual representation of a physical entity or system, used for simulation, analysis, and control. In the context of the video, digital twins are part of Nvidia's technology portfolio, used for training AI models, particularly in the domain of robotics and automation.

💡AI Foundry

An AI foundry is a term that refers to a company or facility that specializes in the design, development, and manufacturing of AI technologies, including hardware and software. In the video, Nvidia is described as branding itself as an AI foundry, indicating its focus on providing comprehensive AI solutions.

💡Project Groot

Project Groot is an initiative by Nvidia that involves the development of a general-purpose AI foundation model for humanoid robots. The project aims to create robots that can understand human language, sense, and navigate the world autonomously.

💡Nims

Nims, or Nvidia Inference Models, are pre-trained AI models provided by Nvidia for businesses to perform various tasks such as data processing, training, and generation. These models are CUDA-based, meaning they can run on any platform where Nvidia GPUs are present, offering businesses the ability to retain full ownership and control over their intellectual property.

Highlights

Nvidia's GTC event unveiled the Blackwell GPU, marking a significant advancement in GPU technology.

Nvidia's shift from being primarily a gaming company to focusing on AI and data center markets has been profound.

The Blackwell GPU is expected to be the first multi-die product with larger tech designs integrated into smaller chiplets.

Nvidia's growth to unfathomable heights in the AI market is impacting its consumer and gaming market behaviors.

The Blackwell architecture announcement was the main focus of Nvidia's GTC event, showcasing its capabilities and interconnects.

Nvidia's advancements in chip-to-chip communication, such as NVLink and Envy Link, are set to be crucial for future high-performance computing.

The Blackwell GPU combines two large die into a single package, potentially offering a significant leap over the Hopper architecture for AI workloads.

Nvidia's project Groot introduces a general-purpose foundation model for humanoid robots, aiming to navigate and interact with the world autonomously.

The Nvidia Thor system, with a Blackwell-based GPU and 800 Tera flops of FP8 capability, is designed to run multimodal AI models.

Nvidia's NIMS tool is a selection of pre-trained AI models for businesses, allowing them to retain full ownership and control over their intellectual property.

Nvidia's RAS engine is designed for monitoring hardware health and identifying potential downtime before it happens.

The Envy Link and Envy Link Switch technologies from Nvidia aim to improve communication between GPUs within data centers.

Nvidia's focus on AI has led to a democratization of computing, making advanced computing more accessible to the masses.

The Blackwell GPU supports up to 192 GB of HBM3E memory, offering a significant increase in memory bandwidth for data-intensive tasks.

Nvidia's GTC event also covered the importance of digital twins in software, where companies use digital representations of real workspaces for training robotic solutions.

Nvidia's branding shift positions it as an AI foundry, with the expectation that its technology will influence consumer parts in the future.

The discussion around Nvidia's Blackwell GPU and its implications for the future of high-end silicon and consumer multi-chip modules.

The transcript highlights the commentary on the technology sphere and the absurdity of the current world situation, particularly in relation to technology advancements.

Transcripts

00:00

there's so many companies that would

00:01

like to build they're sitting on gold

00:03

mines gold mine gold mine it's a gold

00:05

mine gold mine gold mine and we've got

00:07

the

00:10

pickaxes nvidia's GTC event saw the

00:12

unveil of its Blackwell GPU and uh

00:15

generally speaking as far as Nvidia

00:16

presentations go this one was fairly

00:18

well put together there were still some

00:20

memeable quotes God I love

00:23

Nvidia if you take away all of my

00:26

friends it's okay

00:28

Hopper

00:34

you're you're very

00:36

good good good

00:39

boy well good girl PCI Express on the

00:44

bottom on on uh

00:48

your which one is M and your left one of

00:51

them it doesn't matter but M as a side

00:54

Jensen was absolutely right when he said

00:55

that Nvidia is not a gaming company

00:57

anymore and it's clear why companies

00:58

like open Ai and goog and Amazon get a

01:01

little bit nervous when considering they

01:03

have functionally a sole GPU source for

01:05

some of their biggest Ventures that they

01:07

working on right now and Nvidia at this

01:09

point has grown to actually unfathomable

01:12

Heights it's insane to think that this

01:15

was basically a a largely gaming company

01:19

up until more recent years it always had

01:21

professional and workstation and data

01:22

center was growing but gaming was the

01:25

bulk of a lot of nvidia's revenue for a

01:28

long time and that's changed and it's

01:30

clear and how it performs in the AI

01:32

Market will impact how it behaves in the

01:35

consumer and the gaming markets but at

01:37

this point yeah we We Knew by the

01:40

numbers that Nvidia was gigantic it

01:42

didn't really sink in though until I

01:46

made myself sit through this from

01:48

mainstream news coverage Nvidia still is

01:50

a center of the universe uh huge

01:52

performance upgrade now and I had to

01:54

Google what a a pedop flop was but

01:56

please please stop he'll timly

01:58

democratized Computing give code that

02:00

for or Java or python or whatever else

02:02

the vast majority of us never learned

02:04

making us all Hostage to the autocratic

02:06

computer class he's busting up that clue

02:09

what but I think that what they a lot of

02:10

people are wanting to hear about is the

02:12

debut of What's called the

02:14

b1000 that's not that's not the name

02:17

that's it's not even the technical part

02:20

but it's expected to be the first what's

02:21

called a multi-dye product basically

02:24

larger Tech Designs put into really

02:26

small uh they're called chiplets sounds

02:29

really uh kind of cute in a way what you

02:33

said software they're also uh yeah

02:36

talking about Enterprise digital there

02:39

there was more than just the the

02:41

Blackwell

02:42

U that new technology that was uh

02:46

introduced wasn't there what else that's

02:47

right uh and actually guy's name is

02:50

David Harold Blackwell uh a

02:53

mathematician it wasn't uh Richard

02:56

Blackwell the fashionista but um just

02:59

just just shut up just please

03:03

shut like a host to a parasite gaming

03:05

has finally done something productive in

03:08

the eyes of the massive conglomerate of

03:10

non-technical media as they scramble to

03:13

tell everyone that bigger number better

03:16

and uh try to understand literally

03:19

anything about the stock they're pumping

03:20

they they don't understand what it is or

03:23

why it exists but they know that money

03:25

goes in and money comes out and so you

03:28

can speak to it in English and it would

03:30

directly generate USD you do have to

03:32

wonder though if the engineers watching

03:34

this who designed and developed all the

03:36

breakthroughs are pained by their work

03:39

being boiled down

03:41

into make investor more money now please

03:44

but the cause for all this as you would

03:45

expect was AI so we're be talking about

03:47

some of that today uh the technolog is

03:49

really interesting that Nvidia discussed

03:51

some of the biggest takeaways for us

03:53

were advancements in uh chip to chip

03:56

communication multi-chip modules and

03:59

components like Envy link or Envy link

04:00

switch where uh the actual communication

04:04

link between the different pieces of

04:06

silicon starts become the biggest

04:08

limiting factor it already was but uh

04:10

that's going to be one of the main areas

04:12

and additionally we're going to be

04:13

spending a good amount of time on just

04:14

commentary because it's we live in an

04:17

absurd world right now and at least in

04:20

the technology sphere and it deserves

04:22

some some discussion some commentary

04:24

about that too so we'll space that

04:26

throughout and in the conclusion we'll

04:27

get more into uh our thoughts on it okay

04:30

let's get started before that this video

04:32

is brought to you by thermal take and

04:33

the tower 300 the tower 300 is a full-on

04:36

showcase PC case built to present the

04:38

computer straight on with its angled

04:40

tempered glass windows or on a unique

04:43

mounting stand to show off the build in

04:45

new ways the tower 300 has a layout that

04:47

positions the GPU fans against the mesh

04:49

panel with ventilation on the opposite

04:51

side for liquid coolers and CPUs there's

04:54

also an included 2 140 mm fans up top

04:56

the panels use a quick access tooless

04:58

system to be quickly popped in and out

05:00

for maintenance and you can learn more

05:02

at the link in the description below so

05:04

whether or not you're into all of the AI

05:06

discussion this is still an interesting

05:09

uh set of

05:10

technological breakthroughs or at least

05:12

just Technologies to talk about because

05:14

some of it will come into consumer

05:16

multi- chip modules are definitely the

05:18

future of large silicon uh making it

05:21

more higher yields for fabrication

05:23

hopefully lower cost that gets at least

05:25

partially pass to Consumers but I have

05:26

some thoughts on that we'll talk about

05:28

later but generally speaking speak for

05:30

AI that's kind of what got all the buzz

05:32

and despite being a relatively technical

05:34

conference and relatively technically uh

05:36

dense keynote as far as Nvidia Keynotes

05:38

go they knew a lot of Financial and

05:41

investment firms and eyes were watching

05:43

this one and so there was some appeal to

05:46

some of the chaos that those

05:47

organizations like to observe making us

05:49

all Hostage to the autocratic computer

05:51

class moving on let's start with a

05:52

summary of the two hours of Nvidia

05:54

announcements is a lot less fluff this

05:55

time than they've typically had there

05:56

was still some fluff okay so 3,000 lb

05:59

ton and a half so it's not quite an

06:02

elephant four

06:08

elephants one

06:13

GPU and for our European viewers that's

06:16

actually an imperial measurement it's

06:17

pretty common here we use the weight of

06:19

elephants to compare things for example

06:21

one of our mod mats weighs about

06:24

0.00

06:27

4165 of an African bush elephant adult

06:31

as fast as possible pretending to

06:33

Blackwell Nvidia made these

06:34

announcements the Blackwell

06:35

architectural announcement took most of

06:36

the focus Nvidia discussed its two

06:38

largest possible dieses acting as a

06:40

single GPU Nvidia showed the brainup

06:42

board that Juan only half jokingly uh

06:46

noted as being1 billion cost to build

06:48

for testing and relating to this it

06:50

spent some time on the various

06:52

interconnects and communication Hardware

06:54

required to facilitate chipto chip

06:56

transfer nvlink switch is probably one

06:58

of the most important an M of its

07:00

presentation its Quantum infin band

07:02

switch was another of the communications

07:03

announcements a lot of time was also

07:05

spent on varying configurations of black

07:07

well like multi-gpu deployments and

07:10

servers of varying sizes for data

07:11

centers dgx was another of those

07:14

discussed outside of these announcements

07:17

Nvidia showed some humanoid robotics

07:19

programs such as project Groot giving us

07:22

some Tony Stark Vibes including

07:24

showcasing these awfully cute robots

07:26

that were definitely intended to assuage

07:29

all concerns about the future downfall

07:31

of society from

07:33

Terminators five things where you

07:38

going I sit right

07:44

here Don't Be Afraid come here green

07:47

hurry

07:50

up what are you

07:53

saying as always the quickest way to get

07:55

people to accept something they are

07:56

frightened of is by making it cute and

07:59

now after watching that my biggest

08:01

concern is actually if they'll allow

08:03

green to keep its job it was stage

08:05

fright it's new it still learning

08:08

doesn't deserve to lose its job and its

08:11

livelihood over that one small flub on

08:14

stage and wait a

08:17

minute it's working and during all of

08:20

this robotics discussion they also spent

08:21

time speaking of various Partnerships

08:23

and digital twins software is a huge

08:26

aspect of this uh companies are using

08:28

digital representation of their real

08:30

workspaces to train robotic Solutions

08:32

and a modernized take on Automation and

08:36

the general theme was that Nvidia is

08:37

branding itself differently now and so

08:40

we are

08:41

effectively an AI Foundry but our

08:43

estimation is that this technology will

08:45

still work its way into consumer Parts

08:47

in some capacity or another multi-

08:50

Solutions are clearly the future for

08:51

high-end silicon AMD has done well to

08:53

get their first in a big way but Nvidia

08:55

published its own multi-chip module

08:57

white paper many years ago and it's been

08:59

work working on this for about as long

09:01

as AMD AMD went multi-chip with its

09:04

consumer GP products the RX 7000 series

09:06

Nvidia has now done the same with

09:08

Blackwell but with a more impressive

09:10

execution of the chipto chip

09:12

communication which is maybe made easier

09:14

by the fact that companies spend

09:15

millions of dollars on these looking at

09:17

the Blackwell silicon held up by Juan

09:20

during the keynote despite obviously

09:22

limited sort of quality of footage at

09:24

this vantage point we think we can see

09:26

the split centrally located as described

09:28

with additional splits for the hbm you

09:31

can see those dividing lines in this

09:33

shot so now we're getting into recapping

09:34

the Blackwell Hardware side of things

09:36

Blackwell combines two of what Nvidia

09:38

calls the largest possible dieses into

09:41

basically a single GPU solution or a

09:44

single package solution at least in

09:46

combination with ony memory and uh Juan

09:49

described Blackwell as being unaware of

09:53

its connection between the two separate

09:55

dyes and this is sort of the most

09:57

important aspect of this because

10:00

as described at least on stage this

10:02

would imply that the Silicon would

10:05

behave like normal monolithic silicon

10:07

where it doesn't need special

10:09

programming considerations made by the

10:11

software developers by uh those using

10:13

the chip to work around things like

10:15

chipto chip communication uh chipto chip

10:18

latency like you see just for example in

10:20

the consumer World on ryzen CPUs where

10:22

uh Crossing from one CCX to another

10:25

imposes all kinds of new challenges to

10:27

deal with and there's not really been a

10:29

great way to deal with those if you're

10:31

heavily affected by it other than trying

10:33

to isolate the work onto a single piece

10:35

of silicon in that specific example but

10:38

Nvidia says that it has worked around

10:40

this so the total combined solution is a

10:43

208 billion transistor GPU or more

10:47

accurately a combination of two pieces

10:49

of silicon that are each 104 billion

10:52

transistors for contacts the h100 has 80

10:55

billion transistors that's the previous

10:57

one but they're still selling it they

10:59

still have orders backlogged to fulfill

11:01

Nvidia had various claims of how many

11:03

X's better than previous Solutions the

11:06

new Blackwell solution would be with

11:08

those multipliers ranging depending on

11:10

how many gpus are used what Precision it

11:12

is if it's related to the power uh or

11:15

whatever but at the end of the day

11:16

Blackwell appears to be a big jump over

11:19

hopper for AI workloads there was no

11:21

real mention of gaming but it's likely

11:23

that gaming gets a derivative somewhere

11:26

it's likely in the 50 Series we'll see

11:27

Blackwell unless Nvidia pull of Volta

11:29

and skips but that seems unlikely given

11:32

the current rumors Blackwell supports up

11:34

to 192 GB of HPM 3E memory or depending

11:38

on which slide you reference hmb 3E

11:41

memory the good news is that that error

11:44

means that the slides are still done by

11:45

a human at Nvidia the bad news is that

11:49

we don't know how much longer they're

11:50

going to be done by a human at Nvidia

11:52

Blackwell has 8 terab per second of

11:54

memory bandwidth as defined in this

11:56

image and as for derivative

11:58

configurations or Alternatives of this

12:00

the grace Blackwell combination uses a

12:03

Blackwell GPU solution and Grace CPUs

12:06

which is an Nvidia arm collaboration

12:08

previously launched these combinations

12:10

create a full CPU and GPU product with

12:12

varying counts of gpus and CPUs present

12:14

andv video noted that the two Blackwell

12:16

gpus and one Grace CPU configuration

12:19

would host 384 GB of HPM 3E 72 arm

12:23

neoverse V2 CPU cores and it has 900 GB

12:26

per second of nvlink bandwidth chip to

12:28

chip Nvidia says it's gb200 so-called

12:32

super chip has 16 terby per second of

12:34

high bandwidth memory 3.6 terabytes per

12:37

second of NV link bandwidth and 40 pedop

12:40

flops of AI performance depending on how

12:43

charitably you define that and turning

12:45

this into a Blackwell compute node

12:47

pushes that up to 1.7 terabytes of hbm

12:50

3E which is an obscene amount of memory

12:54

uh 32 TB pers second of memory bandwidth

12:56

and liquid cooling most of the

12:58

discussion Beyond this point focused on

13:00

various Communications Hardware

13:02

Solutions including both inip and

13:05

intranode or data center Solutions we

13:08

previously met with Sam nafziger from

13:10

AMD who is an excellent engineer great

13:13

uh communicator and presenter uh

13:15

engineer is actually kind of

13:16

underserving his position at AMD he's

13:19

considered a fellow which is basically

13:20

the highest technical rank you can get

13:22

there uh but anyway we talked with him

13:24

previously about AMD moving to multichip

13:26

for RX 7000 although it's a different

13:28

prod product it was a different era a

13:30

different target market a lot of the key

13:33

challenges are the same for what Nvidia

13:35

faced and what AMD was facing and

13:37

ultimately those challenges largely uh

13:40

Center on the principle of if running

13:43

multiple pieces of silicon obviously

13:45

they can only be as fast as the literal

13:47

link connecting them the reason for

13:49

bringing this up though is probably

13:50

because a lot of you have either

13:51

forgotten that discussion or never saw

13:53

it uh and it's very relevant here so

13:55

this is going to be a short clip from

13:57

our prior interview with AMD talking

14:00

about some of the chipto chip

14:02

Communications and uh chiplet um

14:05

interconnect and fabric limitations so

14:08

that we all get some good context for

14:10

what Nvidia is facing as well the

14:12

bandwidth requirements are just so much

14:13

higher with GBU because we're

14:14

Distributing all of this work the you

14:17

know terabytes of of data we loved the

14:21

chiplet concept we knew that the wire

14:24

counts were just too high in graphics to

14:26

do to replay what we did on CPUs

14:30

and so we were scratching our head um

14:32

you know how can we get significant

14:34

benefit um and we were aware of those

14:37

scaling curves that I showed and and the

14:40

observation was you know there actually

14:42

is a pretty clean boundary between the

14:44

infinity cache um and out and we we

14:49

recognized that these things weren't

14:51

didn't need 5 nanometer and they were

14:53

just fine for the product and in six we

14:55

were hardly spending any power you know

14:57

and the and the the G gddr6 itself

15:00

doesn't benefit at all from technology

15:02

so that's where we came up with the idea

15:04

you know we already have these gddr6

15:06

interfaces in insect technology like I

15:08

talked about the cost of porting right

15:10

and all the engineers and we already had

15:12

that and we could just split it off into

15:15

its own little die and um I mean you can

15:18

see see the results right so we were

15:19

spending 520 millim squ here we

15:21

increased our um our compute unit count

15:24

by 20% we added a bunch of new

15:26

capability but we so this thing was

15:29

would be like over you know it be

15:31

pushing 600 550 millimet squared or

15:35

something right um but we shrank it down

15:37

to 300 the rest of that discussion was

15:39

also great you should check it out if

15:40

you haven't seen it we'll link it below

15:41

it's in our engineering discussions

15:43

playlist where we've actually recently