Why it Was Almost Impossible to Put a Computer in Space

between the radiation the micro gravity

and the extreme constraints on Power and

cooling space is just about the worst

possible place to put a computer can you

guys let me

in that's better now if you follow space

research and exploration you probably

know that space is just full of

computers like this one so how do they

do that well the short answer is by

replacing them a lot the ISS takes

regular shipments of dozens of laptops

at a time which get and this is great

this is a direct quote absolutely

destroyed but not every computer can be

disposable and in 2017 the madlads at

Nasa HP Enterprise and kokia who

sponsored this video collaborated to

create spaceborn one the first Edge

Computing server that was intended to

run for an extended period of time on

the International Space Station

of course it being their first attempt

some uh let's say learning took place

and it turned out that the super

capacitors in the ssds were prone to

radiation related failure who knew but

since all it was ever meant to do was

run benchmarks anyway the mission was

considered a huge success and in 2021

they launched spaceborn 2 whose purpose

was to move Beyond proof of concept and

explore practical applications for onst

compute especially data analysis using

AI but the story doesn't end there

behind me is new space born 2 for

administrative reasons it has the same

name and Core specs as last time but it

just took off earlier this year and it

features more storage than ever over 130

tabt which is an incredible feat when

you consider the design challenges I

mean where do they even install these

things look up oh right I guess they

don't really need a ladder do they no

but you might

[Music]

to illustrate why Edge Computing is

needed on the ISS let's look at a use

case that's focused on astronaut safety

these are the Eva gloves that the crew

members wear during space walks and

according to this article from 2016 they

were responsible for half of all space

suit injuries so to ensure their

integrity between you es NASA requires

the crew to take hundreds of photographs

of them from every angle and then beam

them back to Earth where machine

learning is used to analyze them for

scratches or other hazards except for

one small problem that data transfer

takes five days but with spaceborne oh I

don't know how about 45 seconds not only

is this a huge time sa but with only a

handful of pictures needing to go to

Earth for further analysis spaceborn can

free up a significant amount of the

crew's limited network bandwidth for

other more interesting things with such

obvious benefits then you got to be

wondering why did no one ever try to put

a server on the ISS before the short

answer is after seeing how the crew

laptops fared many people thought that

they just plane wouldn't work and even

if they did there were a host of other

hurdles to clear like the launch okay

this is really cool rocket companies

like SpaceX and Northrop Grumman have

Shak test machines that are programmed

with profiles that will simulate the

launch conditions of their respective

rockets and if you've seen that viral

video of the machine that disassembles

hard drives by vigorously shaking them

you're going to know that surviving that

kind of treatment is no mean feed well

these machines Managed IT both in the

simulation and in the real world they

actually lift it off at the end of this

January every piece of equipment must

also pass an acoustic chamber test and a

US user friendliness evaluation to

ensure the station crew can install and

manage it and uh oh here's a good one

apparently all equipment sent up to the

ISS goes through what's called a white

glove test which uh thankfully is not

what it sounds like basically you put on

a pair of white gloves and then you just

manhandle the crap out of it if the

gloves snag or tear on

anything yeah that's a potential source

of injury I'm going to need you to file

that down which f fun fact they actually

do onsite and then repeat the test I

just hope they weren't filing any RAM

sticks speaking of let's take a closer

look at these machines machine Z cuz

it's not just one in here I know it was

kind of the point of this whole

experiment but it still weirds me out

that these are just bog standard HP

Enterprise systems that you could order

on their site today they don't even have

lead armor or anything in this case

we're looking at an edgeline 4000 which

is a multi-blade system and a dl360 dual

socket server we asked why these

specific machines and the answer we got

was shockingly relatable we sorted the

HP server catalog by depth power draw

and GPU support and these were the ones

we were left with all right fair enough

as for why two different machines well

here's the thing in a perfect world

multiples of the same machine would have

been better but due to power constraints

they chose to have one with more CPU

cores for more additional scientific

applications and one with pure CPU cores

but with a GPU for deep learning and AI

one thing they needed for both however

is Ample Storage kokia generously

sponsored this and brought us out here

so let's take a look at the let's call

them unique choices that they made for

their storage configurations first up

obviously gone is any trace of super

capacitors so kokia can proudly say that

their ssds are space ready I guess but

but what's less obvious is why they

chose a SAS interface Drive rather than

nvme for their high-speed bulk storage I

mean you would think this is Space Age

Technology they'd want the fastest thing

possible but these drives were selected

for their balance of performance

reliability and especially power

efficiency when you're looking at a

shared power budget across two servers

that is less than a typical gaming rig

every single lot counts oh right and

that's even under ideal conditions at

any given time to conserve power for

other priorities on the station the team

can be asked to operate in half power

mode or even to shut down entirely for

large operations like docking so the new

DL 360 server this guy right here gets

four 3.72 TB pm6 Enterprise drives

totaling

120 tabt of raw bulk storage for

scientific data and for backups then for

application drives we've got a really

wild config again they went with four

drives but this time it's their RM sixs

again they're using SAS for lower power

but this time two of the drives are

operating in a data redundant mirror and

the other two are basically just

chilling there ready to be put into

action in the event of a failure two

warm spares out of a four Drive array

would sound like crazy Paran oia on

Earth but I assure you that in space

where bit flips from random radiation

are much more common it's perfectly

reasonable I mean other than the

overkill Drive config and the 28v power

conversion that they need to run it on

the ISS there's not much to say about

this thing it's pretty much a bog

standard server there is one cool demo

that they said we could run though oh

yeah they offered to let us pull one of

the drives out of this dummy machine and

live swap it in into this running

machine to show that no data loss will

occur you want to do the honors sure all

they asked is that you put it in Bay 8

see you can see the drive is actually

operating use Bay s that works look at

that I mean that's good rm6 that would

be bad if it wasn't

kokia and then let's check the size

beautiful 3.8 tab exactly what we want

they're not the exact drives that are on

the space station but we wanted a

different capacity to show you that it's

working and those are expensive

look at that status rebuilding I mean

this seems like a lot of extra steps we

could have just looked up the light yeah

the light yeah it's going

success and whether you're looking for a

SAS drive an nvme drive high capacity or

high performance we're going to have a

bunch of Kyo has great Enterprise grade

drives Linked In the description down

below I think I'll let you take this one

apart it's uh appropriately line this

size okay let's take a look at the

second server that's packed into each

Locker the el4 ,000 is a blade chassis

so the servers are basically these

little slide in cards that yeah I know

AR these cute or what wait pull it out

and they go in on the side look at that

instead of from the front wow and they

managed to sneak four kokia xg6 nvme

drives into each of these blades well

when I say each of these blades I should

say they had the power budget for four

drives but they didn't have the power

budget for four blades in the flight

configuration of this system they ship

with just one of the four blades

installed though it should be noted they

do fly up a spare blade per system in

the event of a failure there's just no

way that that poor Locker can support

both these blades and the other server

running concurrently let's put you away

and shift our Focus to the locker now

obviously there's no real up or down on

the

as it whizzes around the earth at around

28,000 km an hour but to improve comfort

for the astronauts they tend to mount

directional items like plants in a fixed

orientation which will put our lockers

the drawers that hold our servers in the

ceiling there are two of these lockers

each containing an identical system load

out for workload sharing and redundancy

and these lockers present some serious

design challenges starting with the fact

that they use a standard that quite lit

Lally doesn't exist on Earth Express

rack to pack the servers in then HP

Enterprise had to get kind of creative

they found the shortest servers they

could and then they stuffed them in

sideways and they're using a combination

of air Cooling and water cooling the air

cooling uses a system on the ISS called

AAA so at the back of these lockers

there's two cold air supplies and then

two hot air returns that handles about

20% of the cooling for the servers op

viously 20% not 100% they're going to

need some more and that's where this

water cooling comes in this isn't a one:

one for how it would be deployed on the

ISS for one thing these fittings 3D

printed mockups real fitting $800 a pop

if you were even allowed to buy them

this tubing cheap vinyl from Home Depot

real tubing must be made of stainless

steel in fact any wetted surface so

anything that comes in contact with

water is supposed to be made out of

stainless steel but we can still

illustrate how it's supposed to work so

on this side these go into a heat

exchanger much like this one this is

actually from the first generation

spaceborn but functionally it's the same

it pumps cold water into here chills the

air inside the system and then takes the

warm water out to be dissipated to the

photovoltaic heat exchangers that are

plumbed up with liquid ammonia coolant

and mounted to the exterior of the

station to sink that heat into space

you need these kinds of special heat

exchangers because while we think of

space as cold and we see people you know

oh blasted out of airlocks and they

freeze over or whatever in movies the

truth is that for traditional methods of

heat dissipation you need air and in the

near vacuum of space well it ain't

there air get

it cheesy jokes aside the two cooling

systems together are good for removing

about 400 watts of heat from each locker

but that's a combined budget so if this

GPU server kicks into high gear well

these CPUs better just chillax for a

little bit now let's talk about one of

my favorite subjects networking there's

four standard RJ45 ports on the front of

the okay I'm going to show you on the

real one ah as I was saying four ports

on each Locker two of them connect both

of the iss's internal gigabit networks

to a separate redundant switch inside

the locker and then the other two links

are going to go between the two lockers

at 10 GB why 10 gig well because for

either backups or for multinode

workloads that is a heck of a lot better

than gigabit and the power budget didn't

allow for anything faster cool I guess

I'm starting to notice a pattern here

anyway that's all pretty standard but

things become less so when you look at

the station side of these Networks

cables this is a 37 pin military spec

locking connector these are designed for

power and data but in this application

just eight of the pins would be used and

it is

$220 for just this part now on the space

station NASA provides these cables for

you but for testing sake really here on

Earth HP Enterprise had to make their

own

fantastic what's really going to blow

your mind though is for all of their

expensive networking these machines do

not have a normal internet connection

just a private link back to Earth that

NASA not only limits to a mighty 1

megabit per second but that they also

encourage folks not to make full use of

also even now in 2024 it doesn't have

247 connectivity pretty much every hour

or two there's a period of downtime that

can be anywhere as short as a few

minutes or as long long as 45 minutes

and that's because they have to

prioritize generating enough power for

the station and when the giant solar

arrays Point toward the sun they can

block line of sight with the satellites

that provide connectivity which oh

that's a fun fact even though the ISS

orbits less than 500 km from the surface

of the Earth our ping times to the ISS

and yes we got to Ping the ISS which was

pretty cool but our ping times were a

atrocious reaching nearly a second as we

uploaded some of the dankest memes that

Earth had to offer LT store.com now we

asked why that is and the answer was

twofold one it's really old okay fair

enough but also too the station's

Internet relay is in geosynchronous

orbit over

35,000 kilm from the Earth's surface and

uh well what I said was H well there's

your problem right there it's just

really far and you might be wondering

well why don't they just use

starlink that's a good question um

someday they might but for now they

don't and HP and the team on the ISS

have to work around the constraints of

the current setup I mean for crying out

loud it took them four years to validate

that you know we can even just run a

normal computer up here and actually

expect this thing to be reliable they

can't just switch to something and go I

don't know I hope it works um oh by the

way here's another fun one there's no

API to determine if if their connection

is up or down so instead what they do is

buffer all their Communications in basic

terms that means that they ping every

second and if the ping succeeds they

send data and then hope that the

connection stayed up during that time

it's a pretty good system okay not a

perfect one but certainly enough for us

to learn a lot from the spaceborn

project even though spaceborn 2 has been

in action for 3 years there's still so

much to learn two of the servers have

hard Hardware raid cards for their

drives for example costing both mass and

power consumption while two of them use

software raid which obviously doesn't

consume any Mass but could impact power

consumption even more depending on the

loads and one of them could be more or

less reliable than the other we won't

know until we try which is kind of a

recurring theme here so if you want to

learn more about the spaceborn computer

project we're going to have some

resources linked for you down below and

we're also going to have a L to some

great Enterprise storage options from

our sponsor kokia we're truly grateful

for this unique opportunity to get

realistically as close as I ever will to

the real ISS a prop in a sound stage in

LA but hey thanks kogia for the

opportunity and for your long-term

partnership