The Race For AI Robots Just Got Real (OpenAI, NVIDIA and more)

fascinating crazy maybe a little scary I

mean it sounds like we're talking about

not just using something to help humans

do their jobs but just replacing it

entirely could this be a new normal on a

on a large scale hi welcome to another

episode of Cold Fusion this scene is

from the popular animated American

sitcom the Jetson this episode titled

Rosie the robot first aired on September

23rd 1962 the main character in the

scene is presented with a few variations

of a robot one that's responsive to

commands will do housework and basically

work as a personal assistant 61 years on

and we're one step closer to seeing that

reality this week a tech startup called

Figure launched a humanoid robot called

figure1 you can have conversations with

it thanks to their integration with open

AI now at this point you might be

thinking we've already seen robots like

this over the years whether it's from

Boston Dynamics or Tesla how is this any

different well first let's take a look

at the demo of the robot before we get

into the weeds keep in mind that this

demo as claimed by the company is

unaltered and in real time but for the

purposes of brevity I've shortened some

parts all right so let's watch great can

I have something to

eat sure

thing great can you explain why you did

what you just did while you pick up this

trash on it so I gave you the Apple

because it's the only uh edible item I

could provide you with from the

table so based on the scene right now

where do you think the dishes in front

of you go

next the dishes on the table like that

plate and cup are likely to go into the

drying rack next great can you put them

there of

course

after seeing how well it did for those

tasks and the human style conversation

some of you will be impressed some

excited and some even a little bit

scared according to the company this bot

isn't being operated by anyone it's able

to carry out full conversations with

people and respond to external stimuli

and that right there is one of the

primary differences between figure 01

and the other Android robots we've seen

so far but let's back up for a second

how did a startup only founded in 2022

make such Headway who is backing figure

and what are their

intentions given what Nvidia just

announced in the past week are we moving

towards an aid driven robotics race and

most importantly how do these advances

in robotics and AI impact all of us

we'll take a look at all of that and

have a Roundup of all the latest robots

in this episode so sit back relax into

your docking station as we figure out

the future of robotics

you are watching cold fusion

TV it all started in the wake of Co at

this time robotic sales began surging

hitting a record higher in 2022 as the

workforce retreated these robots were

the kind that were used in warehouses

and assembly lines the exact sort of

predictable environment that was the

lowest hanging fruit to train and deploy

robots and amongst this backdrop comes

figure AI founded in 2022 figure AI Inc

is a California based Tech startup with

a focus on general purpose humanoid

robots within just 1 year of its

Inception the company raised $70 million

but now they've recently raised a

whopping 675 million and now have a

valuation of 2.6 billion officially a

unicorn and you know it's serious

business when the likes of Microsoft

Nvidia Jeff Bezos Intel capital and many

others have become investors figure's

Mission according to their website is to

quote expand human capabilities through

advanced AI they aim to do so by

specifically targeting the labor force

according to figure the labor force is

shrinking they mentioned that quote

there are 10 million unfilled jobs in

the United States 7 million of those job

openings are for essential roles such as

Warehouse transportation and Retail and

there are only 6 million people

available to fill these open positions

while key Warehouse suppliers predict

they will run out of people to hire by

2024 quote as we reach the upper limits

of our production capability humanoids

will join the workforce with the ability

to think learn and interact with their

environment safely alongside

[Music]

us figur founder Brett Adcock is no

stranger to the tech industry he's

previously founded two other companies

vety and Archer Aviation veter is an

online Talent Marketplace and in 2018

they were acquired for 100 million

Archer Aviation is now publicly traded

and they're working on building electric

vertical takeoff and Landing aircraft so

with his experience Adcock has built a

team of Industry veterans including

professionals who have worked at Boston

Dynamics Google Deep Mind and

Tesla perhaps their biggest breakthrough

has come in the form of their

partnership with open AI in a press

release in February 2024 figer announced

their collaboration and it will help

quote accelerate figure's commercial

timeline by enhancing the capabilities

of its robots to process and reason from

natural language using open AI research

and large multimodal models in simple

terms it allows for one to have full

conversations with the robot and I

suspect will also make it better at

computer vision so ultimately the idea

is simple build autonomous humanoid

robots to address the global labor

shortage and to do just that figures

first r robot is

[Music]

here let's start by looking at how the

robot actually works first of all every

single action and behavior in the video

was learned and not teleoperated the

robot is equipped with cameras and

microphones so when someone speaks and

asks it to do something the transcribed

texts and images from the cameras are

captured to a large multimodal model

which is trained by open AI to

understand both text and images the

model processes the full conversation

including previously shared images it

then decides which specifically learned

Clos Loop action to take by selecting

and loading the relevant neural network

configurations onto the GPU and then

executing it the mechanical action of

figure 01 is rapid at 200 HZ which

basically means that after it knows what

to do it sends back instructions at 200

times per second that's 5 milliseconds

of reaction time for all those micro

adjustments humans are in the order of

250 milliseconds and fun fact a cat's

reaction time is around 30

milliseconds so essentially they are

using a combination of different neural

networks to sort through audio and

visual inputs understand them and then

do the appropriate actions it's actually

kind of amazing all right so how do you

think you did I I think I did pretty

well the apple found its new owner the

trash is gone and the table wear is

right where it belongs did you notice

the speech hesitation there it's the

little humanlike imperfections that

allows us as viewers to break the

illusion that it's an in anate robot

we've already seen this in chat GPT

voice hey chat GPT can you describe a

nice kitchen scene but make it very

casual and breathe and say um and and

things that sound very natural while

you're

describing sure

thing picture this you're in this cozy

kitchen right the sun's peeking through

the curtains giving that warm glow you

can almost smell the aroma of fresh

coffee brewing um there's this gentle

hum of the fridge in the background kind

of soothing you know and uh maybe

there's a bit of chatter as someone's

flipping Pancakes on the stove it's just

this laidback Vibe like time's taking a

chill pill to see it play out in a real

robot was only the next

Frontier as aside in an episode I made

back in late 2021 I talked about how it

was only a matter of time before a

company cracked the code of mixing

dexterity mobility and natural language

into a single robot and through 3 years

later I think we're getting close but

all that being said as you can see

there's still some latency between

talking and getting a response and I

think that's from open AI side mainly

although we don't know the intricate

details of the integration voiced

instructions probably still have to go

up to a server and come back I've

noticed the same delay issue when

playing around with chat GPT voice so

there's still some work to be done if

these robots are going to be used in

large scale commercial applications

imagine an emergency response robot just

standing there for 10 seconds while

someone's having a heart

attack others are simply not impressed

months ago figure showed a demo of the

robot making coffee some people back

then asked if the robots are also

learning bad habits from the training

data the figure robot is also not the

best at walking but we can't be too

harsh it's still an early stage product

as I always say criticizing is easy but

building is hard with also such an

incredible amount of investment and very

few demo videos to show some have even

drawn the comparison to a certain Trevor

Milton if you're a regular cold fusion

viewer you'll know what that's all about

but obviously this is way too soon for

that

[Music]

conversation broadly speaking it's an

exciting time for the robotics industry

so the playing field for figure AI isn't

going to be Barren by any means let's

take a quick look at the latest in the

industry first up this digit which is

made by agility Robotics and backed by

Amazon before now robots could only

replace part of a human job enough to

make coffee or an omelet but this is

digit a robot it's maker say is ready to

do what human warehouse workers do and

they can lift about 35 lbs you know the

the robot weighs about what a person

would weigh it can reach about the size

of what a person can reach you know

we're really going for that kind of like

OSHA regulations and requirements for

the robot agility robotics is funded in

part by Amazon which is investing a

billion dollars in new Industrial

Technologies the fact that it can enter

places that were designed for humans um

and solve those challenges that exist

without having to rebuild uh the

environment for Automation and it

doesn't quit and it doesn't quit there

are about

$250,000 so they're not inexpensive but

you know over just a few years they're

worth it the digit robots will be

deployed in Amazon warehouses and the

company aims to produce 10,000 units per

year so Jake fascinating crazy maybe a

little scary I mean it sounds like we're

talking about not just using something

to help humans do their jobs but just

replacing it entirely could this be a

new normal on a on a large scale I mean

this is the thing right as a tech

correspondent for years I've had

companies tell me you know well this

technology is for augmenting a human or

making a human's job a little bit easier

in this case it is truly right out and

out there to replace a human worker

Amazon warehouse workers you know that's

one of the largest employers in America

people really go for that job and so you

know we're really I think at this sort

of historic turning point where suddenly

it's no longer the case of of of you

know when could technology replace a set

of Javas but this is truly something

that's built to do that next up abtronic

is designed to move Goods around a

warehouse and can work for 22 hours per

day all you need to do is swap out the

battery every few hours deliveries start

in 20125 the company's also working with

NASA on a robot called Valkyrie and that

aims to work in dangerous environment

ments and perhaps even future space

walks Toyota and hunde are also

currently working on robots of their own

but I'd say less seriously than the

others compared to other robots as

mentioned earlier figure o one's key

difference is the ability to directly

Converse in natural language in order to

get tasks done for you it's not a new

concept as about a year ago on Cold

Fusion we covered a project from Google

that did the same thing at that time we

saw that the concept was still in its

infancy and the robot was quite slow

earlier this year here we saw Elon Musk

sharing a video of Tesla's Optimus robot

folding some clothing at first glance it

was impressive but the curtains came

down later when Elon added that Optimus

couldn't do it autonomously but since

then there's been footage of the Tesla

bot carrying boxes watering plants

walking crouching handling delicate

objects such as eggs and Performing

other domestic activities interestingly

these abilities come from the knowledge

and benefits from Tesla's self-driving

efforts which is pretty cool but again

as far as we know it lacks that natural

language

aspect perhaps the atlas robot by Boston

Dynamics is the most agile robot we've

ever seen we've seen it organized

objects and even perform back flips but

is this and the figure1 robot really the

same thing Boston dynamics's Atlas is

definitely more focused on Mobility

whereas figure seems to be suited for

intricate tasks there's more of a focus

on finger dexterity alluded to by the

five fingers with three joints on them

and for factory deployment this this

could make more sense in a lot of cases

assembly line work is done by sitting

down or being stationary at a specific

location for long periods of time and as

we were working on this video Nvidia

announced project Groot a general

purpose foundational model for humanoid

robots it's meant to help humanoid

robots quote understand natural language

and emulate movements by observing human

actions quickly learning coordination

dexterity and other skills in order to

navigate adapt and interact with the

real world

[Applause]

about the same

size the intersection of computer

Graphics physics artificial intelligence

it all came to bear at this moment well

I think we have some special

guests do

we little Jetson robotics computers

inside they learn to walk in Isaac Sim

let's

go five things where you

going I sit right

here the system removes some of the leg

work from training robots in the real

world via simulation the lessons and

skills learned can be put into a

knowledge base and transferred to other

Robots part and video wants Groot

machines to be able to understand human

language sense and navigate the world

around themselves uh and Carry Out

arbitrary tasks the training part of the

stack starts with video so the model can

sort of intuitively understand physics

then accurately modeling the robots in

Virtual space in order to teach the

model how to move around and interact

with terrain objects or other robots

Jensen described this as a gym here the

model can learn Basics far quicker than

in the real world as we've covered in

another episode on the history and size

of Nvidia the already turning heads with

their stocks growing at unprecedented

levels this year I'll leave a link to

that episode in the description as well

the company has also mentioned that

they're working with Boston Dynamics

agility Robotics abtronic and of course

figure Ai and just thinking about this I

can already see an industry standard

forming open AI provides the

foundational models Nvidia provides some

of the processing hardware and software

packages and finally there's going to be

a plethora of different robot bodies by

different vendors so why is this all

happening now well the reason that a lot

of this stuff didn't work in previous

decades was because the instructions for

the robots had to be laboriously hand

coded now with generative Ai and the

latest generations of models robots are

beginning to be able to dynamically

understand their environment and adjust

seamlessly with no hard coding the

amount of change we're going to see over

the next 5 years 10 years will dwarf

everything that's happened over the last

30 we went from automating pen and paper

right with spreadsheets to connecting

PCS into loc Network to connecting

networks to get the internet to getting

the network effect of communicating

people globally but now we're

introducing machine intell you know

artificial intelligence machine learning

deep learning neural networks Etc so and

what that enables is the automation of

automation right and so the people who

were writing software particularly at

the lower end unless you are doing these

Advanced things they're gone right the

people that s is writing itself in doing

itself right it's just math programmer

back in the day so when I was writing

code it was the algorithms were if this

than that right ex or whatever it may be

right but you had to guess right you had

to use your your best instance then it

got smarter and smarter and smarter and

libraries to do bigger and better things

all that is being

automated the progress of automation has

been a funny story to watch unfold if

you've been following the tech World

closely for long enough you'd remember

all of those discussions over a decade

ago about how manual labor jobs would

eventually be taken by robots but as

was' seen since the AI boom a couple of

years ago it was the cognitive jobs that

were the first to be impacted in 2017

only one in five companies used AI

according to McKenzie now half of all

companies

do we've already seen many companies

laying off thousands of employees in

part due to technological advancements

like AI is completely the opposite of

what most people thought would happen

Goldman Sachs last year reported

reported that over 300 million jobs

could be lost or diminished by AI most

at risk are legal architecture and

engineering business and financial

operations management sales Health Care

Art and Design Goldman Sachs also thinks

that the price of robots and their

constitute components will drop

drastically and they've already dropped

drastically since last

year unlike humans robots don't need to

take a break they don't need to sleep go

on holidays strike and don't complain so

as soon as these robots are feasible and

cheap enough companies will be chomping

at the bit to install them so what do we

do about all of this well it's your

lucky day because I've already done an

episode on life after automation it's a

huge topic and will just be way too long

to get into in this episode so I'll

leave another link to that

below okay so back to figure one at this

point you might think all right the

figure demo looks cool but surely it's

going to be a long time until these

human robots are in production well it

seems like figure didn't want to waste

any time even before the demo earlier in

the year figer signed an agreement with

BMW manufacturing they're going to test

the figure 01 robot at the car makers

plant in Spartenburg South Carolina but

we're yet to get reports on how the 01

is performing and what tasks it's going

to do exactly but I still think that's

pretty

[Music]

interesting humanoid robots like the

figure one are a future conundrum will

they replace humans completely or just

alleviate us from menual tasks leaving

the more important stuff for humans but

on the other hand of course the military

would probably want to have a sniff at

humanoid robots in the future too

looking at consumer products though

we're pretty far away from getting a

highly functional robot that's cheap

that you can buy at your local store but

the day will probably arrive figure 01

is the worst robot from this company so

there's only going to be Improvement

aside from that the competition is

heating up so if it's not them it's

going to be another compy that's going

to crack the code from the broadest

perspective possible I'm sure opinions

from all of you will be split for the

engineering types this is an excellent

display of human Ingenuity for the

Skeptics they'll say it's all pointless

and we should focus our efforts

elsewhere the optimists will see this as

an exciting New Era for Humanity free of

drudgery regardless whether that future

turns into the Jetson the Terminator or

Utopia will just have to let Humanity

collectively figure that one out all I

know is that I'll be watching this space

closely and that's where we are with

figure one and Robotics today I hope you

enjoyed that or found it interesting and

if you did feel free to share it with

someone who would find it interesting as

well all right so that's about it for me

my name is toogo and you've been

watching cold fusion and if you're new

feel free to subscribe if you like it's

free cheers guys have a good

[Music]

one

[Music]

cold fusion it's new

thinking