Shai Rosenfeld - Such Blocking, Very Concurrency, Wow

all right uh so such blocking very

concurrency

wow that's uh what i'm going to talk

about um so the

the slides are going to be pretty dense

today i'm going to talk about a lot of

material so if you guys

um yesterday when i was sitting here i

had internet so if you guys want to pull

this up on your

laptop or ipad or whatever you can pull

this up it's

this link it's basically my name and

then

sb vc such blocking very concurrency so

you can

pull that up and follow along so like

jonathan says i work at engine yard it's

a company that basically

provisions stuff like windows azure and

amazon and provides you with a

ready stack to use your application with

so you just

it's basically similar to heroku and

other platforms as the services

that you just push your application

there similar to google app engine as

well

you just push your application and then

it kind of works but i'm not going to

talk about

engineer today i'm going to talk about

cats

and because every tech talk needs a cat

and actually the other day when we were

at that beer

tap place jonathan said that we should

put

every tech talking is a cat in it so i

put some images here for you to

enjoy there's a cat there's another cat

all right well i really want to talk

about scaling because this is scale conf

so it's actually not going to be related

to engine yard

what i want to talk about is uh scaling

and concurrency

so really what what is scaling right we

have a certain set amount of time to do

something

we have these things that we need to do

and then we need to do more of them

and that's what scaling is right so

scaling is really doing more things at

once

so when i was invited to come here i

decided okay i'm going to talk about all

the ways you can do more things

okay i'm going to talk about all the

talk about all the things

that you can do and to accomplish this

goal to do more things

so i'm going to map them all out right

and i decided to talk about concurrency

and parallelism because apparently

they're different

and i kept on kind of looking into it

and i realized well there's like a ton

of

models it wasn't really going to put me

off i was going to talk about all the

things

and there's just a ton of calculus and

all these you know mathematical models

and it just keeps on going and going and

going so i decided you know maybe i'm

not going to talk about everything in

depth

but i do i do want to to talk about

concurrency models

so what i ended up deciding on is i'm

going to kind of do a tour

of the different models that exist i'm

going to talk about

some general concepts that all these

models kind of share

and then i'm going to go through some of

the models and talk about them and kind

of

iterate some advantages and

disadvantages that each one of these

things have

and then i'm going to let you sit with

it in your own head and figure it out

because really

this is a huge subject it's i'm

surprised myself that i put all this

information in one in one presentation

because really even one specific model

you can spend a whole lifetime on

right this is it's a really big subject

and so i'm

going to leave it for you guys to to dig

in further

so talk about uh start with the the

general concepts which are the common

ingredients that all these models share

and what i've kind of mapped it out to

is the following things that i'm going

to start

just diving into so concurrency versus

parallelism are apparently different

right so what what is the difference

i found this on the internet says on the

haskell blog and it says not all

programmers agree on the meaning of the

terms parallelism and concurrency

they may define them in different ways

or do not distinguish them

at all and so i decided okay well let's

ask the authority figures

what what is going on and so rob pike

created go was a pretty good authority

figure for concurrency

says that concurrency is about dealing

with a lot of things at once and

parallelism is about doing a lot of

things at once

and they're they're similar but they're

a little different and he basically

talks

in this um this talk that he gave in san

francisco it's a really good talk and

you're going to have resources and links

that you guys can check out on the

slides but he basically says that

concurrency

is is structuring your program your

program to do more things

whereas parallelism is is the fact that

it happens a lot at once

and so the other authority figure i

decided to ask was monty python

figures and actually it was joe

armstrong

so erling is another highly concurrent

language and he

says when concurrency is two queues one

coffee machine

and parallelism is two queues two coffee

machines that's how he decides to

explain it to a five-year-old

it's a similar it's a similar concept

you can see parallelism is is happening

at the same time whereas concurrency is

just the way you structure it to have

two lines

happen at the same time but you still

only have this one

one coffee machine and so the way that

that it made sense to me

was that parallelism is really a

condition that arises when two things

are happening literally at the same

second so for example if you run

two processes or two threads on a

multi-core computer and they're

executing at the same second that's

parallel whereas concurrency is kind of

it's almost like a super set

of parallelism it doesn't have to be

parallel but it can be

and that's it's not entirely true

because some i've i've also read that

some people say that there's you know

there are certain things that are

parallel and not concurrent but i think

it's more of like a

philosophical programming kind of debate

thing so

so operating system mechanisms are we

have things that we need to execute to

do work right so how do we execute these

things

so really it's just threads and

processes right there's not there's not

much else to it

you have a regular operating system

process a linux task

and we have threads native operating

system threads

again same thing but they they just

share memory instead of

being contained self-contained and then

there are green threads

and virtual machine processes these are

basically kind of user level threads

that are done within the virtual machine

that you're running

or the interpreter and they're they're

similar to regular threads except

they're not on the operating system

level and for the purpose of the talk

i'm just going to really uh split it

into

processes and threads and the reason i

put leonardo dicaprio there is because

it's a virtual machine process it's a

process within a process so it's kind of

conception

so scheduling so we have these these

executing things right process and

threads is what i'm going to refer to

now as an executing thing

and these executing things need to be

scheduled so how how do they run like

when do they run

so there's really only two ways you can

that this is usually done right there's

the preemptive way

and then there's the cooperative way and

the preemptive way

is basically saying we reserve the right

to attack anyone at any given time so

basically anything can happen

right you have one thing that's running

and then who knows what happens who

knows how long has passed something else

gets invoked at the same

you know immediately and so that's you

know interrupts our

preemptive and scheduling threads and

operating system are preemptive

and that's preemption whereas the

cooperative uh

way of scheduling is basically it's kind

of handing off the baton to something

it's like

fred is running fred is doing all this

stuff and then he stops he pauses and

says okay

i'm done it's now your turn to do

something and then joe picks up the

baton and starts

you know making hot potatoes or

something stops when he's done he passes

on to harry and then the circle kind of

goes around but they're all

they're all working together and this is

the cooperative way of scheduling

so really what what this is is that it

relies on the program or the thing

that's running the executing thing

that's running to say i'm done

it's now your turn

so this is probably the most important

um common

common ingredient that all these things

share because really we have all these

things that are executing they're

running either preemptively or

cooperative

cooperatively or whatever but they they

need to coordinate between them what's

happening right because they're all

doing work with the same stuff so how do

they how do we make sure

that these things don't trip over each

other this kind of leads me into the

concept of

atomicity and atomicity is basically

the the fundamental reason of why

concurrency is hard

right this is the reason that that you

know people do phds on this stuff

is because things are not atomic and the

reason that it can create havoc is

because if you you look at that example

right there

you'll see you know the first thread is

trying to increment a variable right in

this case we want two threads to

increment a variable the first thread is

going to read the variable from ram

it's going to push it into the cpu into

some cpu register make the calculation

to

you know up it by one and save its local

stack and then

the other thread gets preemptively

scheduled it reads the variable from ram

and if you can see the first thread

hasn't written it back to memory yet so

it reads it and it's zero

and then uh the first thread writes it

back to ram the second thread uh picks

up from where it's left it increments it

it's one

and it pushes it down to ram and it's

it's one whereas in reality this should

have been two

right that's a serious problem because

you can't trust your programs doing what

you're doing it's it's bad

right and um yeah it's really bad

people can die so the way that you

commute these things communicate with

each other

is there really two kind of main ways to

do it one is is shared memory

and shared memory is basically writing

some state or some value to

uh you know variable in in ram basically

and then the other executing thing looks

at that and decides what in it what it

needs

to do according to that value right now

shared memory and it's a pretty

efficient means

of passing data between between these

executing things

the other way is through message passing

and this is a

we'll get to it later but message

passing is essentially instead of

invoking a certain function

that needs to be done you you pass the

message of what needs to be done to some

kind of proxy and then that thing

decides to you know it could either be

invoked automatically you know at the

same time or it could be done later

or whatever but the the idea of message

passing is that you don't directly

invoke something you just

tell something to happen

and um and channels is kind of a subset

to me it's a subset of message passing

except that your interface is through a

stream so instead of uh you having to

like

send a certain message and and have to

deal with that your api is basically

like a file

and this is pretty pretty cool because

if you think about like the unix

paradigm of everything as a file

you know having a file stream api for

message passing is

is nice so

i'm going to talk about the model so

those were kind of the the general

concepts that we're going to use to

try and categorize all these different

things in a way that

that hopefully will be easier to kind of

grasp and then go deeper

you know on your own time so this is the

link again

um because the slides are going to be

you know pretty dense

if you haven't pulled it up you're

welcome to pull it up now

so uh this is the this table that took

me quite a while to put together

and it's trying to take all these uh

these things that we've talked about and

trying to classify all of the the models

that we have

and i'm going to uh iterate through

every one of these and you'll see

you know the um you'll see this the

table kind of row on the top so you can

refer to that

as to when i continue going along

so threads in mutex is is is the pretty

you know uh pretty fundamental kind of

concurrency

thing that you do i'm pretty sure most

of you guys know it yesterday

i noticed that most of you are back in

developers so i'm pretty sure you guys

are familiar with

most of these things um but threads in

mutexism i'm sure is probably

pretty pretty something you're pretty

familiar with so

so again how do we deal with like the

atomicity problem right how do we make

sure that data doesn't corrupt it well

use a mutex right use a lock

and um i'm just going to the table on

the top right there you can see so

this uses threads and mutex is kind of

the name of this this pattern

um it uses threads right and it's it's

preemptively scheduled because operating

system threads are promptly

scheduled and it uses shared memory as a

means to communicate

and it does it through locks but with

locks but shared memory is kind of

the method of communication and it's

concurrent and parallel that's what

those that cp thing

means means you can run it literally at

the same time on a multi-core computer

and all of these models are concurrent

so it's really question if it's parallel

uh parallel or not

and then mutex is an example for this

pattern so how do we deal with

atomicity well we just use a mutex right

we lock around we have that

incrementing thing that we want to make

sure equals two at the end

so thread one tries to increment that

counter

and then thread two tries to increment

it but it can't access that shared data

because there's a lock around it

and so when it reads that variable from

ram it will equals one and then

in the end when it's done it'll be two

two and that'll be good right that's

what we want

so some uh you know pros and cons it's

the biggest advantage is that it's it's

really i mean if you're on any platform

you have threads and mutexes

it's everywhere it's pretty common um

as a programmer you know you you get

familiarized with it pretty much in the

beginning it's

a pretty common pattern and that's

that's an advantage because there's a

lot of resources around it

but uh you know disadvantages is that

because you have to deal with locking

you run into all these issues

of you know live lock and and deadlock

and you know if you have one thread

waiting on another thread then they just

it stalls and your computer crashes and

you see the blue screen

right that's that's basically uh

deadlock

so the next model is threads and

transactions and it's similar to

uh threads and mutexes but it works a

little differently it's kind of actually

like a database transaction so

essentially this uses threads

it's preemptive and it uses shared

memory but instead of using locks it

uses a commit abort

semantic and stm's shared transactional

memory and that's kind of the

the paradigm that's used and so like i

said it's

it's basically like a data database

transaction and

essentially dealing with atomicity is

done by

by explicitly saying this is an atomic

block and then when it needs to commit

when it needs to write to memory it

checks to see if the data hasn't changed

and if it can it'll commit that that

data and if it doesn't

then it won't commit that data and it'll

roll back and all the computation that

it's done will be reversed

well nothing will be written basically

so it can import at any time it can roll

back this is very similar to database

transactions

and so this is kind of the example of if

we're iterating a variable so we would

read it

and then we kind of surround that's not

it

uh we surround the the block of code

that we would need to be atomic in this

like atomic block

and then either it will get written or

it won't get written and what's what's

nice about this is that

you know instead of locking if you lock

around a big data structure

the other thread might need to wait

whereas in reality if the let's say he

was only changing one specific

member of that data structure you don't

need to lock right you've just you've

wasted resources on trying to lock

and when reality you if we would have

used stm two of the things could have

done it at the same time and they both

would have committed

so it's kind of like it's an optimistic

approach because it goes into it takes

into account that

you know you can do all this stuff that

you might be wasting you might be

waiting on for for no reason

so the the that's kind of you know one

advantage is that it's it's a little bit

increased in currency compared to

threads and mutexes

and um you know i don't have much

experience with this but um

they they you know they say that there's

basically stm kind of composes a little

better so if you have two abstractions

that use

uh threads and mutexes and you try to

join them into another abstractions

you still might need to to use locking

to kind of not step over each other

whereas

stm is supposed to compose a little

better you can look at that link if you

want to get into that a little more um

but the the main disadvantage for stm is

that because you have rollbacks

you can't actually guarantee that a

chunk of code is going to complete

because it could roll back

so that's that's a disadvantage you know

if you have some operation that needs to

complete no matter what you know you

you have to make sure that there's a

possibility that you that might not

happen

okay so futures and promises these also

uh use threads and uh this is kind of a

somewhat of a cooperative

scheduling and it's it's a little bit of

a funky

funky pattern in terms of the

categorizations that i've tried to use

it doesn't fit in perfectly but it's a

it's a cooperative model it uses message

passing as in

what you're calling the future and the

promise is is the message

and um and it's paralyzable because it

uses threads

and some examples are data flow and oz

variables

or as programming language and dataflow

variables and this is kind of an example

you can you can see that um what we do

is we say you know pinger dot so this is

you know a pinger and a ponger right we

want to make sure that these two

executing things can communicate back

and forth

and so we'll have a pinger you

know.future and that future is a

reference to something that is executing

in the background and then we can do

whatever we want and then when we call

value we we block until we either get

that value or if it has already

completed we're just going to continue

continue through it right and that's a

future and a future is the reference to

something that's going to be evaluated

and then the promise

oh sorry that's the promise and then the

future is the that value

right and so it's cooperative because

this thing is

is basically pausing until that other

thing is done right

and it uses message passing because that

that reference to that variable is

is that is that message right

so what's nice about this is it kind of

abstracts the the notions of locks away

so you as a programmer don't really have

to deal with locks all you do is

use futures and promises and that

abstraction of oh am i going to step

over myself is

is hidden because of the framework of

promises and futures

and um the disadvantage is similar to

locking which i didn't actually put

there but

essentially you know if you're not ready

to to get that data back like you're

going to block and

you're basically wasting time blocking

on that futures value if it hasn't

completed yet

all right so uh processes and

inter-process communication is actually

i was when i was putting these slides

together i was thinking oh maybe i

should put this before threads and

mutexes because this is this is

really the like canonical oh i want to

do more things at once i'm just going to

run another process of this thing

right and inter-process communication

uses processes it's pre-emptive because

just like

threads the operating system can run any

process at any given time

and i put shared memory because you

could use shared memory with processes

but really it's about message passing

and um so how do we how do we handle the

problem of atomicity with processes well

we just we don't share memory right and

then we don't really have to think about

uh what happens you know how are we

going to make sure that that the

the thing that we're trying to access

that shared thing

is going to be okay well we just don't

share anything if we don't share

anything

then we're all good right we don't have

to we don't have to worry about that

but then we run into issue of oh how do

we communicate well let's pass message

let's pass messages right and so i

take the opportunity to say well message

passing the ipc

is is heavily used with with channels