The Race For AI Robots Just Got Real (OpenAI, NVIDIA and more)
Summary
TLDRThe script discusses the advancements in robotics and AI, highlighting the launch of Figure1, a humanoid robot by tech startup Figure AI. Unlike previous robots, Figure1 can engage in human-like conversations and respond to stimuli without human operation, thanks to its integration with OpenAI. The robot's capabilities, the company's rapid growth, and the potential impact on the labor force are explored. The script also touches on other recent developments in the robotics industry, such as Amazon's Digit and ABB's YuMi, and contemplates the future of human-robot interaction in various sectors.
Takeaways
- ð€ The tech startup Figure launched a humanoid robot named Figure1, capable of human-style conversations and tasks.
- ð Figure1's demonstration showcased its ability to perform tasks like picking up trash and organizing dishes without human operation.
- ð¡ Figure AI, founded in 2022, focuses on general-purpose humanoid robots and has quickly become a unicorn with significant investments.
- ð Key investors in Figure AI include Microsoft, Nvidia, Jeff Bezos, and Intel Capital, highlighting the industry's interest in advanced robotics.
- ð Figure's mission is to expand human capabilities through advanced AI, targeting the global labor shortage and essential roles like warehouse and retail.
- ð€ Figure1's advanced capabilities come from a partnership with OpenAI, enhancing its natural language processing and computer vision.
- ð The robot's actions are learned, not teleoperated, and it processes information at a rapid pace, though there's still latency in conversation.
- ð Figure1 is set to be tested in a BMW manufacturing plant, indicating a move towards real-world application and potential integration into industries.
- ð The robotics industry is experiencing a boom with companies like Agility Robotics, ABB, and Toyota also developing advanced robots.
- ð® Nvidia's announcement of Project Groot suggests a push towards a foundational model for humanoid robots, indicating a potential industry standard.
- ð The impact of AI and robotics on the workforce is significant, with predictions of job displacement and a shift towards automation of automation.
Q & A
What is the main topic of discussion in the Cold Fusion episode?
-The main topic of discussion is the launch of a humanoid robot called Figure1 by a tech startup named Figure, its capabilities, and the implications of such advancements in robotics and AI on the workforce and society.
When was the episode of The Jetsons, featuring Rosie the robot, first aired?
-The episode of The Jetsons featuring Rosie the robot was first aired on September 23rd, 1962.
What are some of the tasks that Figure1 is shown to perform in the demo?
-In the demo, Figure1 is shown to perform tasks such as picking up an apple, disposing of trash, and placing dishes on a drying rack.
How does Figure1 process and respond to human interactions?
-Figure1 processes human interactions by capturing transcribed text and images from its cameras and microphones. It then uses a large multimodal model trained by OpenAI to understand both text and images, decide on the appropriate action, and execute it.
What is Figure AI's mission according to their website?
-Figure AI's mission is to expand human capabilities through advanced AI, specifically targeting the labor force to address the global labor shortage.
Who is the founder of Figure AI and what is his background?
-Figure AI was founded by Brett Adcock, who has previously founded two other companies, Vety and Archer Aviation. Vety is an online talent marketplace, and Archer Aviation is a publicly traded company working on electric vertical takeoff and landing aircraft.
What is the significance of Figure's partnership with OpenAI?
-The partnership with OpenAI helps accelerate Figure's commercial timeline by enhancing the capabilities of its robots to process and reason from natural language using OpenAI's research and large multimodal models. This allows for full conversations with the robot and potentially improves its computer vision.
How does Figure1's reaction time compare to human and animal reaction times?
-Figure1's mechanical action operates at 200 HZ, which means it has a reaction time of 5 milliseconds. In comparison, humans have a reaction time of around 250 milliseconds, and a cat's reaction time is approximately 30 milliseconds.
What is the significance of Nvidia's announcement regarding Project Groot?
-Nvidia's announcement of Project Groot introduces a general-purpose foundational model for humanoid robots. It aims to help robots understand natural language and emulate movements by observing human actions, quickly learning coordination, dexterity, and other skills to interact with the real world.
What are some of the other companies and robots mentioned in the script?
-The script mentions Boston Dynamics, Agility Robotics (with their robot Digit), ABTronic, and Toyota and Hunde's robots. It also discusses Tesla's Optimus robot and Nvidia's involvement with robotics through Project Groot.
What are the potential future implications of widespread humanoid robot adoption?
-The potential future implications include the replacement of human labor in various sectors, the possibility of alleviating humans from manual tasks, and the potential military applications of humanoid robots. There are also concerns about the impact on the job market and the need for societal adaptation to these technological changes.
Outlines
ð€ The Emergence of Humanoid Robots
This paragraph introduces the concept of humanoid robots becoming a part of everyday life, drawing parallels to the animated sitcom 'The Jetsons'. It discusses the launch of 'Figure1', a humanoid robot by tech startup Figure, capable of conversing with humans and responding to stimuli. The segment also raises questions about the future of robotics and AI, and their impact on society and the workforce.
ð Figure1's Technological Breakthroughs
The second paragraph delves into the technical aspects of Figure1, highlighting its ability to process natural language and perform tasks through its collaboration with OpenAI. It explains how the robot learns and executes actions using neural networks and multimodal models, emphasizing the impressive speed and precision of its movements. The paragraph also touches on the human-like imperfections in the robot's speech, adding to its realistic interaction.
ð Addressing the Global Labor Shortage
This section discusses the mission of Figure AI to address the global labor shortage by deploying humanoid robots in the workforce. It provides background on the company's rapid growth and funding, as well as the vision of its founder, Brett Adcock. The paragraph also compares Figure1 with other robots in the industry, noting its unique capabilities in natural language processing and intricate task handling.
ð The Future of Robotics and AI
The fourth paragraph explores the broader implications of advancements in robotics and AI, discussing the potential for automation to replace manual and cognitive jobs. It mentions the investment and interest from major tech companies like Nvidia and the development of industry standards. The segment also highlights the role of generative AI in enabling robots to understand their environment dynamically.
ð The Implications for Society and the Workforce
The final paragraph reflects on the societal and economic impacts of humanoid robots, questioning whether they will replace humans entirely or just alleviate us from manual tasks. It mentions the potential military interest in such technology and acknowledges the split opinions on the future of robotics. The author expresses a commitment to closely monitor the developments in this field.
Mindmap
Keywords
ð¡Humanoid Robots
ð¡Artificial Intelligence (AI)
ð¡Labor Shortage
ð¡Natural Language Processing (NLP)
ð¡Investment
ð¡Automation
ð¡Machine Learning
ð¡Generative AI
ð¡Future of Robotics
ð¡Ethical Considerations
Highlights
The Jetsons episode from 1962 depicted a future with humanoid robots, and now we are one step closer to that reality with the launch of Figure1 by tech startup Figure.
Figure1 is a humanoid robot capable of having human-style conversations and performing tasks autonomously, thanks to its integration with OpenAI.
The robot's demonstration video, while shortened for brevity, claims to be unaltered and in real-time, showcasing its ability to understand and respond to commands.
Figure AI, founded in 2022, has rapidly gained traction, raising $675 million and achieving unicorn status with investments from major tech companies like Microsoft, Nvidia, and Intel Capital.
Figure's mission is to expand human capabilities through advanced AI, targeting the labor force to address the global labor shortage.
The company was founded by Brett Adcock, who has a history of successful tech startups, including an online talent marketplace and a company working on electric aircraft.
Figure1's abilities include understanding natural language, processing visual inputs, and performing actions with impressive speed and precision.
The robot's reaction time is 5 milliseconds, significantly faster than human reaction times, making it capable of complex tasks with minimal latency.
Despite the impressive capabilities, there are still challenges to overcome, such as the latency in voice command responses and the robot's walking abilities.
The robotics industry is experiencing a boom, with companies like Amazon and Toyota investing in warehouse and logistics robots capable of performing human tasks.
Nvidia's Project Groot aims to create a foundational model for humanoid robots, enabling them to understand language and learn skills quickly to interact with the real world.
The advancements in AI and robotics are expected to lead to significant changes in the job market, with estimates suggesting over 300 million jobs could be affected by AI advancements.
Figure has already signed an agreement with BMW to test Figure1 in their manufacturing plant, indicating a move towards real-world applications of the technology.
The future of humanoid robots raises questions about their role in society, whether they will replace humans entirely or just assist with manual tasks.
The development and adoption of humanoid robots are likely to accelerate, with continuous improvements expected from companies like Figure and others in the industry.
The potential applications of humanoid robots extend beyond domestic and industrial use, with possibilities in areas like the military and space exploration.
The robotics and AI advancements represent a shift towards the automation of automation, changing the landscape of jobs and the nature of work.
The future impact of these technologies on society will depend on how we collectively navigate the challenges and opportunities they present.
Transcripts
fascinating crazy maybe a little scary I
mean it sounds like we're talking about
not just using something to help humans
do their jobs but just replacing it
entirely could this be a new normal on a
on a large scale hi welcome to another
episode of Cold Fusion this scene is
from the popular animated American
sitcom the Jetson this episode titled
Rosie the robot first aired on September
23rd 1962 the main character in the
scene is presented with a few variations
of a robot one that's responsive to
commands will do housework and basically
work as a personal assistant 61 years on
and we're one step closer to seeing that
reality this week a tech startup called
Figure launched a humanoid robot called
figure1 you can have conversations with
it thanks to their integration with open
AI now at this point you might be
thinking we've already seen robots like
this over the years whether it's from
Boston Dynamics or Tesla how is this any
different well first let's take a look
at the demo of the robot before we get
into the weeds keep in mind that this
demo as claimed by the company is
unaltered and in real time but for the
purposes of brevity I've shortened some
parts all right so let's watch great can
I have something to
eat sure
thing great can you explain why you did
what you just did while you pick up this
trash on it so I gave you the Apple
because it's the only uh edible item I
could provide you with from the
table so based on the scene right now
where do you think the dishes in front
of you go
next the dishes on the table like that
plate and cup are likely to go into the
drying rack next great can you put them
there of
course
after seeing how well it did for those
tasks and the human style conversation
some of you will be impressed some
excited and some even a little bit
scared according to the company this bot
isn't being operated by anyone it's able
to carry out full conversations with
people and respond to external stimuli
and that right there is one of the
primary differences between figure 01
and the other Android robots we've seen
so far but let's back up for a second
how did a startup only founded in 2022
make such Headway who is backing figure
and what are their
intentions given what Nvidia just
announced in the past week are we moving
towards an aid driven robotics race and
most importantly how do these advances
in robotics and AI impact all of us
we'll take a look at all of that and
have a Roundup of all the latest robots
in this episode so sit back relax into
your docking station as we figure out
the future of robotics
you are watching cold fusion
TV it all started in the wake of Co at
this time robotic sales began surging
hitting a record higher in 2022 as the
workforce retreated these robots were
the kind that were used in warehouses
and assembly lines the exact sort of
predictable environment that was the
lowest hanging fruit to train and deploy
robots and amongst this backdrop comes
figure AI founded in 2022 figure AI Inc
is a California based Tech startup with
a focus on general purpose humanoid
robots within just 1 year of its
Inception the company raised $70 million
but now they've recently raised a
whopping 675 million and now have a
valuation of 2.6 billion officially a
unicorn and you know it's serious
business when the likes of Microsoft
Nvidia Jeff Bezos Intel capital and many
others have become investors figure's
Mission according to their website is to
quote expand human capabilities through
advanced AI they aim to do so by
specifically targeting the labor force
according to figure the labor force is
shrinking they mentioned that quote
there are 10 million unfilled jobs in
the United States 7 million of those job
openings are for essential roles such as
Warehouse transportation and Retail and
there are only 6 million people
available to fill these open positions
while key Warehouse suppliers predict
they will run out of people to hire by
2024 quote as we reach the upper limits
of our production capability humanoids
will join the workforce with the ability
to think learn and interact with their
environment safely alongside
[Music]
us figur founder Brett Adcock is no
stranger to the tech industry he's
previously founded two other companies
vety and Archer Aviation veter is an
online Talent Marketplace and in 2018
they were acquired for 100 million
Archer Aviation is now publicly traded
and they're working on building electric
vertical takeoff and Landing aircraft so
with his experience Adcock has built a
team of Industry veterans including
professionals who have worked at Boston
Dynamics Google Deep Mind and
Tesla perhaps their biggest breakthrough
has come in the form of their
partnership with open AI in a press
release in February 2024 figer announced
their collaboration and it will help
quote accelerate figure's commercial
timeline by enhancing the capabilities
of its robots to process and reason from
natural language using open AI research
and large multimodal models in simple
terms it allows for one to have full
conversations with the robot and I
suspect will also make it better at
computer vision so ultimately the idea
is simple build autonomous humanoid
robots to address the global labor
shortage and to do just that figures
first r robot is
[Music]
here let's start by looking at how the
robot actually works first of all every
single action and behavior in the video
was learned and not teleoperated the
robot is equipped with cameras and
microphones so when someone speaks and
asks it to do something the transcribed
texts and images from the cameras are
captured to a large multimodal model
which is trained by open AI to
understand both text and images the
model processes the full conversation
including previously shared images it
then decides which specifically learned
Clos Loop action to take by selecting
and loading the relevant neural network
configurations onto the GPU and then
executing it the mechanical action of
figure 01 is rapid at 200 HZ which
basically means that after it knows what
to do it sends back instructions at 200
times per second that's 5 milliseconds
of reaction time for all those micro
adjustments humans are in the order of
250 milliseconds and fun fact a cat's
reaction time is around 30
milliseconds so essentially they are
using a combination of different neural
networks to sort through audio and
visual inputs understand them and then
do the appropriate actions it's actually
kind of amazing all right so how do you
think you did I I think I did pretty
well the apple found its new owner the
trash is gone and the table wear is
right where it belongs did you notice
the speech hesitation there it's the
little humanlike imperfections that
allows us as viewers to break the
illusion that it's an in anate robot
we've already seen this in chat GPT
voice hey chat GPT can you describe a
nice kitchen scene but make it very
casual and breathe and say um and and
things that sound very natural while
you're
describing sure
thing picture this you're in this cozy
kitchen right the sun's peeking through
the curtains giving that warm glow you
can almost smell the aroma of fresh
coffee brewing um there's this gentle
hum of the fridge in the background kind
of soothing you know and uh maybe
there's a bit of chatter as someone's
flipping Pancakes on the stove it's just
this laidback Vibe like time's taking a
chill pill to see it play out in a real
robot was only the next
Frontier as aside in an episode I made
back in late 2021 I talked about how it
was only a matter of time before a
company cracked the code of mixing
dexterity mobility and natural language
into a single robot and through 3 years
later I think we're getting close but
all that being said as you can see
there's still some latency between
talking and getting a response and I
think that's from open AI side mainly
although we don't know the intricate
details of the integration voiced
instructions probably still have to go
up to a server and come back I've
noticed the same delay issue when
playing around with chat GPT voice so
there's still some work to be done if
these robots are going to be used in
large scale commercial applications
imagine an emergency response robot just
standing there for 10 seconds while
someone's having a heart
attack others are simply not impressed
months ago figure showed a demo of the
robot making coffee some people back
then asked if the robots are also
learning bad habits from the training
data the figure robot is also not the
best at walking but we can't be too
harsh it's still an early stage product
as I always say criticizing is easy but
building is hard with also such an
incredible amount of investment and very
few demo videos to show some have even
drawn the comparison to a certain Trevor
Milton if you're a regular cold fusion
viewer you'll know what that's all about
but obviously this is way too soon for
that
[Music]
conversation broadly speaking it's an
exciting time for the robotics industry
so the playing field for figure AI isn't
going to be Barren by any means let's
take a quick look at the latest in the
industry first up this digit which is
made by agility Robotics and backed by
Amazon before now robots could only
replace part of a human job enough to
make coffee or an omelet but this is
digit a robot it's maker say is ready to
do what human warehouse workers do and
they can lift about 35 lbs you know the
the robot weighs about what a person
would weigh it can reach about the size
of what a person can reach you know
we're really going for that kind of like
OSHA regulations and requirements for
the robot agility robotics is funded in
part by Amazon which is investing a
billion dollars in new Industrial
Technologies the fact that it can enter
places that were designed for humans um
and solve those challenges that exist
without having to rebuild uh the
environment for Automation and it
doesn't quit and it doesn't quit there
are about
$250,000 so they're not inexpensive but
you know over just a few years they're
worth it the digit robots will be
deployed in Amazon warehouses and the
company aims to produce 10,000 units per
year so Jake fascinating crazy maybe a
little scary I mean it sounds like we're
talking about not just using something
to help humans do their jobs but just
replacing it entirely could this be a
new normal on a on a large scale I mean
this is the thing right as a tech
correspondent for years I've had
companies tell me you know well this
technology is for augmenting a human or
making a human's job a little bit easier
in this case it is truly right out and
out there to replace a human worker
Amazon warehouse workers you know that's
one of the largest employers in America
people really go for that job and so you
know we're really I think at this sort
of historic turning point where suddenly
it's no longer the case of of of you
know when could technology replace a set
of Javas but this is truly something
that's built to do that next up abtronic
is designed to move Goods around a
warehouse and can work for 22 hours per
day all you need to do is swap out the
battery every few hours deliveries start
in 20125 the company's also working with
NASA on a robot called Valkyrie and that
aims to work in dangerous environment
ments and perhaps even future space
walks Toyota and hunde are also
currently working on robots of their own
but I'd say less seriously than the
others compared to other robots as
mentioned earlier figure o one's key
difference is the ability to directly
Converse in natural language in order to
get tasks done for you it's not a new
concept as about a year ago on Cold
Fusion we covered a project from Google
that did the same thing at that time we
saw that the concept was still in its
infancy and the robot was quite slow
earlier this year here we saw Elon Musk
sharing a video of Tesla's Optimus robot
folding some clothing at first glance it
was impressive but the curtains came
down later when Elon added that Optimus
couldn't do it autonomously but since
then there's been footage of the Tesla
bot carrying boxes watering plants
walking crouching handling delicate
objects such as eggs and Performing
other domestic activities interestingly
these abilities come from the knowledge
and benefits from Tesla's self-driving
efforts which is pretty cool but again
as far as we know it lacks that natural
language
aspect perhaps the atlas robot by Boston
Dynamics is the most agile robot we've
ever seen we've seen it organized
objects and even perform back flips but
is this and the figure1 robot really the
same thing Boston dynamics's Atlas is
definitely more focused on Mobility
whereas figure seems to be suited for
intricate tasks there's more of a focus
on finger dexterity alluded to by the
five fingers with three joints on them
and for factory deployment this this
could make more sense in a lot of cases
assembly line work is done by sitting
down or being stationary at a specific
location for long periods of time and as
we were working on this video Nvidia
announced project Groot a general
purpose foundational model for humanoid
robots it's meant to help humanoid
robots quote understand natural language
and emulate movements by observing human
actions quickly learning coordination
dexterity and other skills in order to
navigate adapt and interact with the
real world
[Applause]
about the same
size the intersection of computer
Graphics physics artificial intelligence
it all came to bear at this moment well
I think we have some special
guests do
we little Jetson robotics computers
inside they learn to walk in Isaac Sim
let's
go five things where you
going I sit right
here the system removes some of the leg
work from training robots in the real
world via simulation the lessons and
skills learned can be put into a
knowledge base and transferred to other
Robots part and video wants Groot
machines to be able to understand human
language sense and navigate the world
around themselves uh and Carry Out
arbitrary tasks the training part of the
stack starts with video so the model can
sort of intuitively understand physics
then accurately modeling the robots in
Virtual space in order to teach the
model how to move around and interact
with terrain objects or other robots
Jensen described this as a gym here the
model can learn Basics far quicker than
in the real world as we've covered in
another episode on the history and size
of Nvidia the already turning heads with
their stocks growing at unprecedented
levels this year I'll leave a link to
that episode in the description as well
the company has also mentioned that
they're working with Boston Dynamics
agility Robotics abtronic and of course
figure Ai and just thinking about this I
can already see an industry standard
forming open AI provides the
foundational models Nvidia provides some
of the processing hardware and software
packages and finally there's going to be
a plethora of different robot bodies by
different vendors so why is this all
happening now well the reason that a lot
of this stuff didn't work in previous
decades was because the instructions for
the robots had to be laboriously hand
coded now with generative Ai and the
latest generations of models robots are
beginning to be able to dynamically
understand their environment and adjust
seamlessly with no hard coding the
amount of change we're going to see over
the next 5 years 10 years will dwarf
everything that's happened over the last
30 we went from automating pen and paper
right with spreadsheets to connecting
PCS into loc Network to connecting
networks to get the internet to getting
the network effect of communicating
people globally but now we're
introducing machine intell you know
artificial intelligence machine learning
deep learning neural networks Etc so and
what that enables is the automation of
automation right and so the people who
were writing software particularly at
the lower end unless you are doing these
Advanced things they're gone right the
people that s is writing itself in doing
itself right it's just math programmer
back in the day so when I was writing
code it was the algorithms were if this
than that right ex or whatever it may be
right but you had to guess right you had
to use your your best instance then it
got smarter and smarter and smarter and
libraries to do bigger and better things
all that is being
automated the progress of automation has
been a funny story to watch unfold if
you've been following the tech World
closely for long enough you'd remember
all of those discussions over a decade
ago about how manual labor jobs would
eventually be taken by robots but as
was' seen since the AI boom a couple of
years ago it was the cognitive jobs that
were the first to be impacted in 2017
only one in five companies used AI
according to McKenzie now half of all
companies
do we've already seen many companies
laying off thousands of employees in
part due to technological advancements
like AI is completely the opposite of
what most people thought would happen
Goldman Sachs last year reported
reported that over 300 million jobs
could be lost or diminished by AI most
at risk are legal architecture and
engineering business and financial
operations management sales Health Care
Art and Design Goldman Sachs also thinks
that the price of robots and their
constitute components will drop
drastically and they've already dropped
drastically since last
year unlike humans robots don't need to
take a break they don't need to sleep go
on holidays strike and don't complain so
as soon as these robots are feasible and
cheap enough companies will be chomping
at the bit to install them so what do we
do about all of this well it's your
lucky day because I've already done an
episode on life after automation it's a
huge topic and will just be way too long
to get into in this episode so I'll
leave another link to that
below okay so back to figure one at this
point you might think all right the
figure demo looks cool but surely it's
going to be a long time until these
human robots are in production well it
seems like figure didn't want to waste
any time even before the demo earlier in
the year figer signed an agreement with
BMW manufacturing they're going to test
the figure 01 robot at the car makers
plant in Spartenburg South Carolina but
we're yet to get reports on how the 01
is performing and what tasks it's going
to do exactly but I still think that's
pretty
[Music]
interesting humanoid robots like the
figure one are a future conundrum will
they replace humans completely or just
alleviate us from menual tasks leaving
the more important stuff for humans but
on the other hand of course the military
would probably want to have a sniff at
humanoid robots in the future too
looking at consumer products though
we're pretty far away from getting a
highly functional robot that's cheap
that you can buy at your local store but
the day will probably arrive figure 01
is the worst robot from this company so
there's only going to be Improvement
aside from that the competition is
heating up so if it's not them it's
going to be another compy that's going
to crack the code from the broadest
perspective possible I'm sure opinions
from all of you will be split for the
engineering types this is an excellent
display of human Ingenuity for the
Skeptics they'll say it's all pointless
and we should focus our efforts
elsewhere the optimists will see this as
an exciting New Era for Humanity free of
drudgery regardless whether that future
turns into the Jetson the Terminator or
Utopia will just have to let Humanity
collectively figure that one out all I
know is that I'll be watching this space
closely and that's where we are with
figure one and Robotics today I hope you
enjoyed that or found it interesting and
if you did feel free to share it with
someone who would find it interesting as
well all right so that's about it for me
my name is toogo and you've been
watching cold fusion and if you're new
feel free to subscribe if you like it's
free cheers guys have a good
[Music]
one
[Music]
cold fusion it's new
thinking
5.0 / 5 (0 votes)
ChatGPT Can Now Talk Like a Human [Latest Updates]
AI Agents: The next generation of AI-powered bots | Zendesk
Stephen Colbertâs Cyborgasm: Giant AI Rat Penis | Tesla Cybertruck Canât Offroad | Google AI Fail
Bots, Scams, The Internet, And You - SOME MORE NEWS
AI Agents Take the Wheel: Devin, SIMA, Figure 01 and The Future of Jobs
Did AI Just End Music? (Now itâs Personal) ft. Rick Beato