Understanding the Linux Backdoor: Implications for Open Source [When Penguins Cry]

Dave's Garage
4 Apr 202410:06

Summary

TLDRThe video discusses a recent SSH vulnerability back door found in Linux, inserted by a developer in XZ utils versions 5.60 and 5.61. This back door allows anyone with the private key to gain unfettered root access to any Linux system running the exploit. The video highlights the importance of security in open source and closed source systems, and shares a personal anecdote from the speaker's time at Microsoft, emphasizing the significance of thorough code review and build processes in preventing such security breaches.

Takeaways

  • 🔑 A new SSH vulnerability back door has been discovered in Linux, which if exploited, allows unauthorized access to any Linux system without a password.
  • 💡 The back door was inserted by checking in a code that searches for a private key, providing the attacker with root access if they possess the corresponding public key.
  • 🛠️ The vulnerability was introduced in the XZ utils, specifically versions 5.60 and 5.61, which are considered bleeding edge and not widely adopted yet.
  • 📈 A Microsoft employee who works on benchmarking discovered the SSH login delays and further investigation led to the detection of the back door.
  • 🤔 The incident raises questions about the security of open-source versus closed-source systems and whether one is more protected against such vulnerabilities.
  • 🔍 The back door was not easily detected because it was inserted as encrypted and compressed binary data, not as a change to the source code itself.
  • 🚧 The complexity and obtuseness of make files made it difficult to spot the malicious changes, which were added via the build process.
  • 🛑 The vulnerability would have been catastrophic if not detected early, as it could have been exploited by nation-states or other malicious actors.
  • 📚 The speaker shares a personal anecdote from 1993 about an intern who inserted a prank command line switch into MS-DOS, highlighting the importance of code review.
  • 🏛️ The speaker suggests that structured processes and checks and balances, as seen in commercial software development, can help catch and prevent such vulnerabilities from making it into production.

Q & A

  • What is the main topic of discussion in the video?

    -The main topic of discussion is a new SSH vulnerability back door that was inserted into Linux, its implications, how it works, and a comparison between open source and closed source systems in terms of security.

  • What is the potential impact of the Linux SSH back door if it had not been detected early?

    -If the Linux SSH back door had not been detected early, it would have been catastrophic for Linux systems as it would allow an attacker with the private key to gain unfettered root access to any Linux system running the exploit without needing a password.

  • How was the back door inserted into the Linux system?

    -The back door was inserted by one of the main developers of XZ utils who checked in encrypted and compressed binary data that was decrypted and inserted into the build process via the make file, without modifying the source code directly.

  • Which versions of XZ utils were affected by the back door?

    -The affected versions of XZ utils are 5.60 and 5.61.

  • How was the back door in Linux discovered?

    -The back door was discovered by a Microsoft employee who was running benchmark tests on the PostGIS database and noticed that SSH logins were taking longer than usual. His investigation led to the discovery and publication of the back door.

  • What is the speaker's opinion on whether open source or closed source systems are better protected against back doors?

    -The speaker believes that the source code itself is not as relevant as the processes and checks in place. While open source code is accessible for review, the actual protection against back doors depends on the scrutiny and verification processes, which may be more rigorous in closed source systems like Windows.

  • What was the speaker's experience with a back door in Microsoft DOS 6.2?

    -During an internship at Microsoft, the speaker was unaware of a fellow intern who inserted a special command line switch in the copy command to print 'I love sex' repeatedly. This was caught in a manual source code review and the intern was not invited back.

  • How does the speaker suggest that such back doors can be caught in closed source systems?

    -The speaker suggests that in closed source systems like Windows, there are multiple layers of review and verification tests that any change to the product must go through. This includes build lab testing, basic verification tests, and performance benchmarks, which can catch significant changes that could indicate the presence of a back door.

  • What is the speaker's conclusion on the effectiveness of processes in preventing back doors?

    -The speaker concludes that while processes and checks and balances cannot guarantee catching every back door, they significantly increase the chances of detection due to mandatory reviews and tests, making people more honest and reducing the potential for harm.

  • What is the speaker's personal experience with code reviews in the past?

    -The speaker shares that during his time at Microsoft, he was involved in code reviews where changes were carefully scrutinized, especially for critical components. He suggests that the level of scrutiny might vary depending on the popularity and关注度 of the part of the system being modified.

  • What is the speaker's final message to the viewers?

    -The speaker thanks the viewers for joining him and encourages them to leave a subscription and a like if they found the episode informative or entertaining. He also promotes his book on autism for those who are on the spectrum or know someone who is.

Outlines

00:00

🔒 The Hidden Dangers of Backdoors in Linux and MSDOS

Dave Plummer, a retired operating systems engineer from Microsoft, shares alarming incidents of backdoors inserted into Linux and MSDOS. He details a recent discovery of a backdoor in Linux's SSH, allowing attackers unfettered root access to systems without needing a password. This vulnerability stems from malicious code inserted into XZ utils, which, if undetected, could have had catastrophic implications for Linux users worldwide. Dave contrasts this with his early days at Microsoft, recounting a colleague's attempt to insert a non-malicious 'easter egg' into MSDOS. Both stories underline the perpetual risk and vigilance required to safeguard operating systems from internal and external threats.

05:01

🔎 Open vs. Closed Source Security: A Comparative Analysis

Exploring the implications of the Linux SSH backdoor further, Dave Plummer debates the effectiveness of open versus closed source models in preventing such vulnerabilities. He argues that the visibility of source code, as in open source projects, does not necessarily guarantee security, citing the complexity of reviewing changes in make files and the overall scrutiny process. Drawing on his Microsoft experience, he outlines the rigorous review and testing processes in closed source development that might better catch such vulnerabilities. Dave shares a personal anecdote from his internship at Microsoft, illustrating the potential for mischief within development teams and highlighting the importance of oversight and review processes in software development.

10:02

🛡️ Lessons Learned and Closing Thoughts

In the final segment, Dave Plummer reflects on the broader implications of software vulnerabilities and the importance of vigilance in development processes, regardless of the open or closed source model. He emphasizes that while processes and checks are crucial for preventing unauthorized changes, a mix of luck and diligent oversight has historically kept major backdoors out of systems like Windows. Dave's insights offer valuable lessons on the complexities of software security and the ongoing need for comprehensive review mechanisms to protect against vulnerabilities.

Mindmap

Keywords

💡SSH vulnerability

SSH, or Secure Shell, is a protocol used for secure communication over an unsecured network. In the context of the video, an SSH vulnerability refers to a security flaw or weakness that has been intentionally inserted into the Linux system, allowing unauthorized access with a specific private key. This vulnerability is critical as it can provide an attacker with unfettered root access to any Linux system running the exploit, bypassing standard authentication methods and posing a severe threat to system security.

💡Back door

A back door in software refers to a hidden method or feature that allows access to the system without following the normal authentication process. In the video, the term is used to describe a malicious code inserted into the Linux system that enables unauthorized access via SSH. The back door is particularly dangerous because it grants the attacker root access, giving them complete control over the compromised system.

💡Private key

In cryptography, a private key is a piece of information that is kept secret and is used to decrypt messages encrypted with the corresponding public key. In the context of the video, the private key is built into SSH and is used to access the back door in the Linux system. Only those with the private key can exploit the vulnerability and gain unauthorized access to the system.

💡Public key

A public key is a cryptographic key that is made available to others and used to encrypt messages that only the corresponding private key can decrypt. In the video, the public key is incorporated into the SSH of the Linux system, enabling the private key holder to authenticate and gain access. The public key, in this case, is a part of the malicious code that facilitates the back door.

💡XZ utils

XZ utils is a software library and command-line tools for data compression. In the video, it is mentioned that the back door in Linux was introduced through a code change in the XZ utils, specifically versions 5.60 and 5.61. This indicates that the attack vector was not through the core Linux system but through a peripheral utility that is used for compression.

💡Make file

A make file is a file that contains instructions on how to compile and build software. It is used with the 'make' utility to automate the process of building applications. In the context of the video, the make file was manipulated to include encrypted and compressed binary data that inserted the public key into the SSH build process. This change was not obvious and went undetected in the source code review.

💡Open source

Open source refers to a type of software licensing where the source code is made publicly available for anyone to view, modify, and distribute. The video discusses the relative security of open source systems compared to closed source systems in the context of back door vulnerabilities. While open source allows for community scrutiny, the video suggests that this does not guarantee the absence of vulnerabilities, as seen with the Linux back door.

💡Closed source

Closed source, also known as proprietary software, refers to software whose source code is not made publicly available. The video contrasts closed source with open source, discussing the protective measures that might be in place in a closed source environment, such as rigorous testing and review processes, which could potentially catch vulnerabilities like the one found in Linux.

💡Benchmarking

Benchmarking is the process of evaluating the performance, reliability, or other attributes of a system or component by running standard tests and comparing the results to a benchmark. In the video, benchmarking is what led to the discovery of the Linux back door, as a Microsoft employee noticed that SSH logins were taking longer than usual, prompting further investigation.

💡Regression testing

Regression testing is a type of software testing that ensures that newly added code has not adversely affected the existing functionality of the software. In the video, regression testing is mentioned as a process that would likely have caught the Linux back door vulnerability if it had been part of a closed source development process, as it would have detected the significant impact on the login process.

💡Code review

Code review is the process of examining source code changes to detect errors, improve code quality, and ensure that the changes align with project requirements. In the video, the lack of thorough code review is highlighted as a factor that allowed the back door to be inserted into the Linux system, as the malicious code was not detected during the review process.

Highlights

A new SSH vulnerability back door has been discovered in Linux systems.

The back door allows an attacker with the private key to log into any affected Linux system without a password.

The vulnerability provides root access, giving the attacker unfettered access to the system.

The back door was inserted by checking code into XZ utils, which is used in the build process of Linux.

The malicious code was inserted in versions 5.60 and 5.61 of XZ utils, affecting bleeding-edge builds.

The back door was not immediately obvious due to its clever insertion via encrypted and compressed binary data.

The vulnerability was detected by a Microsoft employee running benchmarks on the PostGIS database.

Open source versus closed source systems were discussed in terms of their susceptibility to such back doors.

The importance of rigorous code review and build processes in commercial releases was highlighted.

The potential protective nature of having multiple layers of review and testing in closed source systems was mentioned.

The speaker shares a personal anecdote from 1993 about an intern who inserted a prank command into MS-DOS.

The prank could have been caught by thorough manual source code review, which is more common in commercial software development.

The speaker asserts that throughout their career at Microsoft, no back doors were inserted into Windows.

The importance of processes, checks, and balances in software development to prevent malicious code insertion was emphasized.

The speaker concludes by appreciating the audience for their attention and participation.

Transcripts

00:00

so I don't want to scare you but did you

00:01

know that somebody checked a back door

00:02

into Linux that allows the attacker to

00:04

log into any Linux system I'll tell you

00:07

about that and I'll also tell you about

00:09

the time somebody tried to check a back

00:10

door into Ms Doss and did it from my

00:13

office that's a cool

00:14

[Music]

00:21

story hey I'm Dave welcome to my shop

00:24

I'm Dave plumber retired operating

00:26

systems engineer from Microsoft going

00:27

back to the MSDS and Windows 95 days and

00:30

today we're going to talk about the new

00:32

SSH vulnerability back door that's been

00:34

inserted into Linux what it means how it

00:36

works and what the implications are and

00:38

whether open source is any better off

00:39

than closed source and these types of

00:41

scenarios first off obviously I'm not a

00:43

Linux developer I mean I write code for

00:45

Linux but I've never contributed at

00:46

least not since about 1993 when I

00:49

checked in a bugfix to was it Minix I

00:51

think it was early Linux um or email the

00:54

I should say but suffice to say I'm not

00:56

a Linux contributor or developer so

00:57

while I may not be a Linux expert that's

00:59

probably good for you right now cuz I

01:01

have to explain things in terms that I

01:02

understand which means you will likely

01:04

understand them as well so before we

01:06

look at what happened first what are the

01:08

implications what does this back door do

01:10

somebody checked in some code and it

01:12

looks for a private key and it has a

01:14

public key built into SSH and what are

01:16

the implications well to put it simply

01:22

catastrophic had this not been detected

01:24

as early as it was it would have been

01:26

truly catastrophic for Linux because

01:28

let's say that this this is a nation

01:30

state what they've done is they've built

01:32

a backdoor private key into SSH that

01:35

anybody with the public key or actually

01:37

vice versa they put the public key in

01:38

the product and then they hold the

01:39

private key anybody who holds that

01:41

private key and only that person nobody

01:43

else can use this exploit I should be

01:45

clear about that only if you hold the

01:47

private key which the original person

01:48

who inserted the back door does then you

01:51

can connect to any Linux server with

01:53

this back door inserted in fact you

01:55

don't need a password you just need to

01:56

connect by SSH and the SSH server will

01:58

say hey you've got the right private key

02:00

in your metadata or whatever extra data

02:02

is sent along with login request it

02:04

matches that logs you right in basically

02:07

not only a completely open door it's

02:09

even better than that because it gives

02:10

you root access

02:15

I that side of it's a little more

02:16

technical but suffice to say the result

02:19

is that you get unfettered access to any

02:21

Linux system in the world that you want

02:23

that's running this exploit and you get

02:25

rud access so how' they do it well what

02:27

they did is they checked code into XZ

02:29

utils and when I'm saying they I'm

02:32

talking about one of the main developers

02:34

of the XZ utils the good news is this is

02:36

in 5.60 and 5.61 versions of the XD

02:39

utils and that's pretty much bleeding

02:41

edge it did make it into a few very

02:43

early drops of dros but not it's not

02:46

going to be like in 20.04 won't be

02:47

picked up immediately or anything into

02:49

there so if you're not running on the

02:51

bleeding edge long story short you're

02:53

probably

02:54

safe but surely you can't put a price on

02:57

your login safety I wouldn't have

02:59

thought so either but here we are so

03:01

they checked code into xzu tills that

03:03

long story short bakes the public key

03:06

into SSH and then accepts connections

03:07

from anybody with the private key but

03:10

wouldn't that be obvious it seems like

03:11

this should be obvious and easy to

03:13

detect just by reviewing the source code

03:15

change logs but in fact it's not because

03:17

they were very clever about how they did

03:19

it they didn't modify the source code at

03:21

all instead they injected encrypted and

03:24

compressed binary data that is decrypted

03:26

and then inserted into the build process

03:28

via the make file for the XZ tills if I

03:31

understand it correctly but this means

03:33

there's no changes to the code so

03:35

anybody reviewing the source code

03:36

changes doesn't see any change and who

03:39

reads make file changes I mean they're

03:40

pretty obtuse and opaque to begin with

03:42

and if you have any idea what a make

03:44

file is doing it's either a simple make

03:45

file or you're having a really good day

03:47

because a lot of make files are pretty

03:49

complex and hard to follow once you get

03:51

to a certain complexity of a project so

03:53

at this point we've got a back door in

03:54

the Linux source code and all we have to

03:56

do or all they have to do is wait for it

03:58

to be propagated through out all the

04:00

builds and dros and trickle down

04:02

everybody and if it doesn't get caught

04:03

before that boom hey by the way one

04:07

thing I've noticed lately on YouTube is

04:08

that people are like bang ending their

04:10

videos immediately and I guess this must

04:12

appease the algorithm in some way so I'm

04:14

going to do that today which means if

04:16

I'm going to tell you the stuff that I

04:17

tell you at the end of every episode I

04:18

have to tell you now like halfway

04:20

through so here it is I'm mostly in this

04:22

for the subs and likes so if you're

04:23

finding today's episode to be any

04:25

combination of informative or

04:26

entertaining please leave me one of each

04:28

before you go today and if you or

04:30

anybody you know is on the Spectrum then

04:32

please check out the free sample of my

04:34

book on Amazon where I tell you

04:35

everything I know about autism that I

04:37

wish I'd know in way back then so who

04:39

caught it

04:41

Microsoft well not really Microsoft guy

04:43

that works for Microsoft also happens to

04:46

do benchmarking and so on on the postgis

04:49

database and he was running his tests

04:51

and he noticed that the SSH logins were

04:53

taking a lot longer than they used to

04:54

and this happened all of a sudden so he

04:57

dug into it and he's the hero that found

04:59

the information and the back door and

05:00

published it and let us all know about

05:02

it so had it not been for that this

05:04

would have likely been uncaught now of

05:07

course this promps the bigger question

05:08

of which is more protective against this

05:10

type of back door an open source system

05:12

or a closed Source system and I'd say in

05:15

this case the source code actually isn't

05:17

all that relevant of course it's open

05:19

but so is the make file was anybody

05:21

really scrutinizing the make file and if

05:22

they had would they have caught it if

05:24

they looked at it carefully well nobody

05:26

even at Microsoft is probably hand

05:28

reviewing and scrutinizing the make

05:29

build code changes unless they cause a

05:31

problem if they slow things down they're

05:34

going to take a look at it but if you

05:35

make changes to the make file and the

05:37

binary pops out then everything is cool

05:39

it's probably not going to get that much

05:40

scrutiny unlike a change to the actual

05:42

source code that's going to go through

05:43

several layers of review and approval

05:46

and again I've been gone for Microsoft

05:47

20 years so I'm talking about how we did

05:49

it back 20 years ago which was probably

05:50

pretty sloppy and lazy and horrible

05:52

compared to what they're doing now but

05:54

here's the beauty of a regimental

05:56

commercial release with something like

05:57

Windows NT you don't just check in code

06:00

and then wait for it to show up in the

06:01

build it goes through a lot of processes

06:03

in between first there's a build lab

06:05

that picks up just your change build

06:06

your change independently builds a build

06:08

with your change kicks that out and it

06:11

goes through basic build and when all

06:13

the changes are collected for the day

06:14

that build will go through bvts or basic

06:17

verification tests and that's going to

06:18

check to make sure that all the normal

06:20

stuff works the smoke tests pass the

06:22

build boots does everything correctly

06:24

and that the performance is not

06:25

seriously degraded from previous builds

06:27

and that's how it would have been caught

06:28

early at Microsoft because the

06:29

significant impact to the login chain

06:32

would have caused the regression in when

06:33

log on if it were you know the

06:35

equivalent component in Windows and that

06:37

would have been detected immediately now

06:40

that's not to say that every such change

06:42

would be so it's not a golden guarantee

06:44

or anything but the fact that any change

06:47

to the product goes through both human

06:48

manual review of the textual changes to

06:50

the source code and then through a bunch

06:52

of verification tests and benchmarks and

06:54

several other verification tests that

06:56

make sure the functionality and the

06:58

performance hasn't regressed that

07:00

catches a lot of things it still won't

07:02

catch everything be clear on that so

07:04

there is no Panacea now the other thing

07:06

is when you're checking into code say

07:07

I'm checking into the heat manager well

07:09

the guy who write wrote The Heat manager

07:11

and the person who maintains the heat

07:12

manager is going to very carefully check

07:15

my changes when you get something like

07:16

exz utils I think there was only two

07:18

people working on it so did the other

07:19

person really vet these changes very

07:21

carefully or did lonus you know get out

07:23

of his computer and check every line I

07:25

doubt that so who is actually vetting

07:27

these changes as they go into the build

07:28

and the answer is I don't know because

07:30

I'm not involved in that process so I

07:32

can't really speak to what goes on

07:34

behind the scenes in Linux I can only

07:36

tell you what used to happen in Windows

07:38

how that you extrapolate that forward

07:39

and assume they're more careful now and

07:41

the protective aspects of that now the

07:43

difference is if it's in a part of Linux

07:44

that a lot of people care about a lot of

07:46

people are going to look at it but if

07:47

it's in a part of Linux that a lot of

07:48

people don't care about like xzu tills

07:51

then it's probably not going to get the

07:52

scrutiny that it deserves especially

07:54

when you see the implications of how

07:55

this change manifests so now a bit of my

07:58

own back door stories

08:00

when I was an intern in 1993 I was in MS

08:02

DOS working on MS DOS 6.2 where I worked

08:05

on Smart drive and highe and some of the

08:07

compression engine relocation disc copy

08:11

setup whole bunch of stuff and there was

08:13

a guy in my office that was also an

08:14

intern so we shared an office because we

08:16

were just interns and he decided that he

08:19

was working I believe on the copy

08:20

command and he took the opportunity to

08:22

check not a back door per se but a

08:24

special command line switch so instead

08:26

of you know how with x copy SLS we'll do

08:29

folder

08:30

well with copy he added it so that slart

08:33

which is like asky 254 I forget the

08:35

actual OEM code page at this moment but

08:38

whatever it was he used a special

08:40

graphic character and he made that a

08:41

command line switch and it would just

08:42

print I love sex over and over and over

08:44

and over and over and it would print

08:45

that out I didn't know anything about

08:47

this and then after I'm done my

08:49

internship I get a call from my manager

08:51

at the time who calls me at home and

08:52

says yeah so what did you know about

08:54

this and the answer was I knew nothing

08:57

and uh naturally they did not invite get

08:59

the guy back it was caught in the manual

09:01

source code Change review that I spoke

09:02

of earlier or however they caught it and

09:05

it did not look good for him I don't

09:06

know what the repercussions were I don't

09:08

know what they did I wasn't involved all

09:10

I know is that they took me at my word

09:12

that I wasn't involved and thankfully

09:13

for that because I want up working on

09:15

Microsoft for a long time and I don't

09:17

know if I would have hired me after that

09:18

little incident in my office did I

09:20

really not know about it I didn't so now

09:23

in all my time at Microsoft and in the

09:24

two decades since over this course of 30

09:27

years of Windows and Ms Doss to my

09:29

knowledge there has never been a back

09:30

door inserted into Windows now it's

09:33

possible that one has and it's just

09:34

never been found but I would assume that

09:36

by now there would be some exploit that

09:38

would take advantage of it is that

09:39

diligence process Superior coding or

09:41

pure luck well I'm going to guess a lot

09:43

of it's luck but a lot of it is process

09:46

doesn't mean the developers are any

09:47

smarter it does mean that people have

09:49

day jobs where they are assigned to look

09:50

at code changes in other people's

09:52

products whether they want to or not

09:54

then they are responsible for anything

09:55

that happens as a result of those

09:56

changes and that makes people pretty

09:58

honest and as for the people that aren't

10:00

honest there are processes and checks

10:02

and balances to make sure they don't do

10:04

a lot of harm thanks for joining me see

10:06

you

Rate This

5.0 / 5 (0 votes)

Связанные теги
Linux SecuritySSH VulnerabilityBackdoor ExploitsOpen Source RisksClosed Source SafetyCryptographySoftware EngineeringSecurity BenchmarkingLinux DevelopmentMicrosoft Insider
Вам нужно реферат на русском языке?