Lecture 2: RPC and Threads

today I'd like to talk about NGO which is interesting especially interesting for us in this course because course NGO is the language at the labs you're all going to do the labs in and so I want to focus today particularly on some of the machinery that sort of most useful in the labs and in most particular to distributed programming um first of all you know it's worth asking why we use go in this class in fact we could have used any one of a number of other system style languages plenty languages like Java or C sharp or even Python that provide the kind of facilities we need and indeed we used to use C++ in this class and it worked out fine it'll go indeed like many other languages provides a bunch of features which are particularly convenient that's good support for threads and locking and synchronization between threads which we use a lot it is a convenient remote procedure call package which doesn't sound like much but it actually turns out to be a significant constraint from in languages like C++ for example it's actually a bit hard to find a convenient easy to use remote procedure call package and of course we use it all the time in this course or programs and different machines to talk to each other unlike C++ go is type safe and memory safe that is it's pretty hard to write a program that due to a bug scribbles over some random piece of memory and then causes the program to do mysterious things and that just eliminates a big class of bugs similarly it's garbage collected which means you never in danger of priam the same memory twice or free memory that's still in use or something the garbage vector just frees things when they stop being used and one thing it's maybe not obvious until you played around with just this kind of programming before but the combination of threads and garbage collection is particularly important one of the things that goes wrong in a non garbage collected language like C++ if you use threads is that it's always a bit of a puzzle and requires a bunch of bookkeeping to figure out when the last thread that's using a shared object has finished using that object because only then can you free the object as you end up writing quite a bit of coat it's like manually the programmer it's about a bunch of code to manually you know do reference counting or something in order to figure out you know when the last thread stopped using an object and that's just a pain and that problem completely goes away if you use garbage collection like we haven't go and finally the language is simple much simpler than C++ one of the problems with using C++ is that often if you made an error you know maybe even just a typo the the error message you get back from the compiler is so complicated that in C++ it's usually not worth trying to figure out what the error message meant and I find it's always just much quicker to go look at the line number and try to guess what the error must have been because the language is far too complicated whereas go is you know probably doesn't have a lot of people's favorite features but it's relatively straightforward language okay so at this point you're both on the tutorial if you're looking for sort of you know what to look at next to learn about the language a good place to look is the document titled effective go which you know you can find by searching the web all right the first thing I want to talk about is threads the reason why we care a lot about threads in this course is that threads are the sort of main tool we're going to be using to manage concurrency in programs and concurrency is a particular interest in distributed programming because it's often the case that one program actually needs to talk to a bunch of other computers you know client may talk to many servers or a server may be serving requests at the same time on behalf of many different clients and so we need a way to say oh you know I'm my program really has seven different things going on because it's talking to seven different clients and I want a simple way to allow it to do these seven different things you know without too much complex programming I mean sort of thrust threads are the answer so these are the things that the go documentation calls go routines which I call threads they're go routines are really this same as what everybody else calls Red's so the way to think of threads is that you have a program of one program and one address space I'm gonna draw a box to sort of denote an address space and within that address space in a serial program without threads you just have one thread of execution executing code in that address space one program counter one set of registers one stack that are sort of describing the current state of the execution in a threaded program like a go program you could have multiple threads and you know I got raw it as multiple squiggly lines and when each line represents really is a separate if the especially if the threads are executing at the same time but a separate program counter a separate set of registers and a separate stack for each of the threads so that they can have a sort of their own thread of control and be executing each thread in a different part of the program and so hidden here is that for every stack now there's a syrupy thread there's a stack that it's executing on the stacks are actually in in the one address space of the program so even though each stack each thread has its own stack technically the they're all in the same address space and different threads could refer to each other stacks if they knew the right addresses although you typically don't do that and then go when you even the main program you know when you first start up the program and it runs in main that's also it's just a go routine and can do all the things that go teens can do all right so as I mentioned one of the big reasons is to allow different parts of the program to sort of be in its own point in in a different activity so I usually refer to that as IO concurrency for historical reasons and the reason I call it IO concurrency is that in the old days where this first came up is that oh you might have one thread that's waiting to read from the disk and while it's waiting to reach from the disk you'd like to have a second thread that maybe can compute or read somewhere else in the disk or send a message in the network and wait for reply so and so I open currencies one of the things that threads by you for us it would usually mean I can I open currency we usually mean I can have one program that has launched or removed procedure calls requests to different servers on the network and is waiting for many replies at the same time that's how it'll come up for us and you know the way you would do that with threads is that you would create one thread for each of the remote procedure calls that you wanted to launch that thread would have code that you know sent the remote procedure call request message and sort of waited at this point in the thread and then finally when the reply came back the thread would continue executing and using threads allows us to have multiple threads that all launch requests into the network at the same time they all wait or they don't have to do it at the same time they can you know execute the different parts of this whenever they feel like it so that's i/o concurrency sort of overlapping of the progress of different activities and allowing one activity is waiting other activities can proceed another big reason to use threads is multi-core parallelism which I'll just call parallelism and here the thing where we'd be trying to achieve with threads is if you have a multi-core machine like I'm sure all of you do in your laptops if you have a sort of compute heavy job that needs a lot of CPU cycles wouldn't it be nice if you could have one program that could use CPU cycles on all of the cores of the machine and indeed if you write a multi-threaded go if you launch multiple go routines and go and they do something computer intensive like sit there in a loop and you know compute digits of pi or something then up to the limit of the number of cores in the physical machine your threads will run truly in parallel and if you launch you know two threads instead of one you'll get twice as many you'll be able to use twice as many CPU cycles per second so this is very important to some people it's not a big deal on this course be it's rare that we'll sort of think specifically about this kind of parallelism in the real world though of building things like servers to form parts of the distributed systems it can sometimes be extremely important to be able to have the server be able to run threads and harness the CPU power of a lot of cores just because the load from clients can often be pretty high okay so parallelism is a second reason why threads are quite a bit interested in distributed systems and a third reason which is maybe a little bit less important is there's some there's times when you really just want to be able to do something in the background or you know there's just something you need to do periodically and you don't want to have to sort of in the main part of your program sort of insert checks to say well should I be doing this things that should happen every second or so you just like to be able to fire something up that every second does whatever the periodic thing is so there's some convenience reasons and an example which will come up for you is it's often the case that some you know a master server may want to check periodically whether its workers are still alive because one of them is died you know you want to launch that work on another machine like MapReduce might do that and one way to arrange sort of oh do this check every second every minute you know send a message to the worker are you alive is to fire off a go routine that just sits in a loop that sleeps for a second and then does the periodic thing and then sleeps for a second again and so in the labs you'll end up firing off these kind of threads quite a bit yes is the overhead worth it yes the overhead is really pretty small for this stuff I mean you know it depends on how many you create a million threads that he sit in a loop waiting for a millisecond and then send a network message that's probably a huge load on your machine but if you create you know ten threads that sleep for a second and do a little bit of work it's probably not a big deal at all and it's I guarantee you the programmer time you say by not having to sort of mush together they're different different activities into one line of code it's it's worth the small amount of CPU cost almost always still you know you will if you're unlucky you'll discover in the labs that some loop of yours is not sleeping long enough or are you fired off a bunch of these and never made them exit for example and they just accumulate so you can push it too far okay so these are the reasons that the main reasons that people like threads a lot and that will use threads in this class any other questions about threads in general by asynchronous program you mean like a single thread of control that keeps state about many different activities yeah so this is a good question actually there is you know what would happen if we didn't have threads or we'd for some reason we didn't want to use threats like how would we be able to write a program that could you know a server that could talk to many different clients at the same time or a client that could talk to him any servers right what what tools could be used and it turns out there is sort of another line of another kind of another major style of how do you structure these programs called you call the asynchronous program I might call it a vent driven programming so sort of or you could use a vent prevent programming and the the general structure of an event-driven program is usually that it has a single thread and a single loop and what that loop does is sits there and waits for any input or sort of any event that might trigger processing so an event might be the arrival of a request from a client or a timer going off or if you're building a Window System protect many Windows systems on your laptops I've driven written an event-driven style where what they're waiting for is like key clicks or Mouse move or something so you might have a single in an event-driven program it of a single threat of control sits an aloof waits for input and whenever it gets an input like a packet it figures out oh you know which client did this packet come from and then it'll have a table of sort of what the state is of whatever activity its managing for that client and it'll say oh gosh I was in the middle of reading such-and-such a file you know now it's asked me to read the next block I'll go and be the next block and return it and threats are generally more convenient because they allow you to really you know it's much easier to write sequential just like straight lines of control code that does you know computes sends a message waits for response whatever it's much easier to write that kind of code in a thread than it is to chop up whatever the activity is into a bunch of little pieces that can sort of be activated one at a time by one of these event-driven loops that said the well and so one problem with the scheme is that it's it's a little bit of a pain to program another potential defect is that while you get io concurrency from this approach you don't get CPU parallelism so if you're writing a busy server that would really like to keep you know 32 cores busy on a big server machine you know a single loop is you know it's it's not a very natural way to harness more than one core on the other hand the overheads of adventure and programming are generally quite a bit less than threads you know Ed's are pretty cheap but each one of these threads is sitting on a stack you know stack is a kilobyte or a kilobytes or something you know if you have 20 of these threads who cares if you have a million of these threads then it's starting to be a huge amount of memory and you know maybe the scheduling bookkeeping for deciding what the thread to run next might also start you know you now have list scheduling lists with a thousand threads in them the threads can start to get quite expensive so if you are in a position where you need to have a single server that sir you know a million clients and has to sort of keep a little bit of state for each of a million clients this could be expensive and it's easier to write a very you know at some expense in programmer time it's easier to write a really stripped-down efficient low overhead service in a venture than programming just a lot more work are you asking my JavaScript I don't know the question is whether JavaScript has multiple cores executing your does anybody know depends on the implementation yeah so I don't know I mean it's a natural thought though even in you know even an NGO you might well want to have if you knew your machine had eight cores if you wanted to write the world's most efficient whatever server you could fire up eight threads and on each of the threads run sort of stripped-down event-driven loop just you know sort of one event loop Recor and that you know that would be a way to get both parallelism and to the bio concurrency yes okay so the question is what's the difference between threads and processes so usually on a like a UNIX machine a process is a single program that you're running and a sort of single address space a single bunch of memory for the process and inside a process you might have multiple threads and when you ready to go program and you run the go program running the go program creates one unix process and one sort of memory area and then when your go process creates go routines those are so sitting inside that one process so I'm not sure that's really an answer but just historically the operating systems have provided like this big box is the process that's implemented by the operating system and the individual and some of the operating system does not care what happens inside your process what language you use none of the operating systems business but inside that process you can run lots of threads now you know if you run more than one process in your machine you know you run more than one program I can edit or compiler the operating system keep quite separate right you're your editor and your compiler each have memory but it's not the same memory that are not allowed to look at each other memory there's not much interaction between different processes so you redditor may have threads and your compiler may have threads but they're just in different worlds so within any one program the threads can share memory and can synchronize with channels and use mutexes and stuff but between processes there's just no no interaction that's just a traditional structure of these this kind of software yeah so the question is when a context switch happens does it happened for all threads okay so let's let's imagine you have a single core machine that's really only running and as just doing one thing at a time maybe the right way to think about it is that you're going to be you're running multiple processes on your machine the operating system will give the CPU sort of time slicing back and forth between these two programs so when the hardware timer ticks and the operating systems decides it's time to take away the CPU from the currently running process and give it to another process that's done at a process level it's complicated all right let me let me let me restart this these the threads that we use are based on threads that are provided by the operating system in the end and when the outer needs to some context switches its switching between the threads that it knows about so in a situation like this the operating system might know that there are two threads here in this process and three threads in this process and when the timer ticks the operating system will based on some scheduling algorithm pick a different thread to run it might be a different thread in this process or one of the threads in this process in addition go cleverly multiplex as many go routines on top of single operating system threads to reduce overhead so it's really probably a two stages of scheduling the operating system picks which big thread to run and then within that process go may have a choice of go routines to run all right okay so threads are convenient because a lot of times they allow you to write the code for each thread just as if it were a pretty ordinary sequential program however there are in fact some challenges with writing threaded code one is what to do about shared data one of the really cool things about the threading model is that these threads share the same address space they share memory if one thread creates an object in memory you can let other threads use it right you can have a array or something that all the different threads are reading and writing and that sometimes critical right if you you know if you're keeping some interesting state you know maybe you have a cache of things that your server your cache and memory when a thread is handling a client request it's gonna first look in that cache but the shared cache and each thread reads it and the threads may write the cache to update it when they have new information to stick in the cache so it's really cool you can share that memory but it turns out that it's very very easy to get bugs if you're not careful and you're sharing memory between threads so a totally classic example is you know supposing your thread so you have a global variable N and that's shared among the different threads and a thread just wants to increment n right but itself this is likely to be an invitation to bugs right if you don't do anything special around this code and the reason is that you know whenever you write code in a thread that you you know is accessing reading or writing data that's shared with other threads you know there's always the possibility and you got to keep in mind that some other thread may be looking at the data or modifying the data at the same time so the obvious problem with this is that maybe thread 1 is executing this code and thread 2 is actually in the same function in a different thread executing the very same code right and remember I'm imagining the N is a global variable so they're talking about the same n so what this boils down to you know you're not actually running this code you're running machine code the compiler produced and what that machine code does is it you know it loads X into a register you know adds one to the register and then stores that register into X with where X is a address of some location and ran so you know you can count on both of the threads they're both executing this line of code you know they both load the variable X into a register effect starts out at 0 that means they both load at 0 they both increment that register so they get one and they both store one back to memory and now two threads of incremented n and the resulting value is 1 which well who knows what the programmer intended maybe that's what the programmer wanted but chances are not right chances are the programmer wanted to not 1 some some instructions are atomic so the question is a very good question which it's whether individual instructions are atomic so the answer is some are and some aren't so a store a 32-bit store is likely the extremely likely to be atomic in the sense that if 2 processors store at the same time to the same memory address 32-bit values well you'll end up with is either the 32 bits from one processor or the 32 bits from the other processor but not a mixture other sizes it's not so clear like one byte stores it depends on the CPU you using because a one byte store is really almost certainly a 32 byte load and then a modification of 8 bits and a 32 byte store but it depends on the processor and more complicated instructions like increment your microprocessor may well have an increment instruction that can directly increment some memory location like pretty unlikely to be atomic although there's atomic versions of some of these instructions so there's no way all right so this is this is a just classic danger and it's usually called a race I'm gonna come up a lot is you're gonna do a lot of threaded programming with shared state race I think refers to as some ancient class of bugs involving electronic circuits but for us that you know the reason why it's called a race is because if one of the CPUs have started executing this code and the other one the others thread is sort of getting close to this code it's sort of a race as to whether the first processor can finish and get to the store before the second processor start status execute the load if the first processor actually manages it to do the store before the second processor gets to the load then the second processor will see the stored value and the second processor will load one and add one to it in store two that's how you can justify this terminology okay and so the way you solve this certainly something this simple is you insert locks you know you as a programmer you have some strategy in mind for locking the data you can say well you know this piece of shared data can only be used when such-and-such a lock is held and you'll see this and you may have used this in the tutorial the go calls locks mutexes so what you'll see is a mule Ock before a sequence of code that uses shared data and you unlock afterwards and then whichever two threads execute this when it to everyone is lucky enough to get the lock first gets to do all this stuff and finish before the other one is allowed to proceed and so you can think of wrapping a some code in a lock as making a bunch of you know remember this even though it's one line it's really three distinct operations you can think of a lock as causing this sort of multi-step code sequence to be atomic with respect to other people who have to lock yes should you can you repeat the question oh that's a great question the question was how does go know which variable we're walking right here of course is only one variable but maybe we're saying an equals x plus y really threes few different variables and the answer is that go has no idea it's not there's no Association at all anywhere between this lock so this new thing is a variable which is a tight mutex there's just there's no association in the language between the lock and any variables the associations in the programmers head so as a programmer you need to say oh here's a bunch of shared data and any time you modify any of it you know here's a complex data structure say a tree or an expandable hash table or something anytime you're going to modify it and of course a tree is composed many many objects anytime you got to modify anything that's associated with this data structure you have to hold such and such a lock right and of course is many objects and instead of objects changes because you might allocate new tree nodes but it's really the programmer who sort of works out a strategy for ensuring that the data structure is used by only one core at a time and so it creates the one or maybe more locks and there's many many locking strategies you could apply to a tree you can imagine a tree with a lock for every tree node the programmer works out the strategy allocates the locks and keeps in the programmers head the relationship to the data but go for go it's this is this lock it's just like a very simple thing there's a lock object the first thread that calls lock gets the lock other threads have to wait until none locks and that's all go knows yeah does it not lock all variables that are part of the object go doesn't know anything about the relationship between variables and locks so when you acquire that lock when you have code that calls lock exactly what it is doing it is acquiring this lock and that's all this does and anybody else who tries to lock objects so somewhere else who would have declared you know mutex knew all right and this mu refers to some particular lock object no and there me many many locks right all this does is acquires this lock and anybody else who wants to acquire it has to wait until we unlock this lock that's totally up to us as programmers what we were protecting with that lock so the question is is it better to have the lock be a private the private business of the data structure like supposing it a zoning map yeah and you know you would hope although it's not true that map internally would have a lock protecting it and that's a reasonable strategy would be to have I mean what would be to have it if you define a data structure that needs to be locked to have the lock be sort of interior that have each of the data structures methods be responsible for acquiring that lock and the user the data structure may never know that that's pretty reasonable and the only point at which that breaks down is that um well it's a couple things one is if the programmer knew that the data was never shared they might be bummed that they were paying the lock overhead for something they knew didn't need to be locked so that's one potential problem the other is that if you if there's any inter data structure of dependencies so we have two data structures each with locks and and they maybe use each other then there's a risk of cycles and deadlocks right and the deadlocks can be solved but the usual solutions to deadlocks requires lifting the locks out of out of the implementations up into the calling code I will talk about that some point but it's not a it's a good idea to hide the locks but it's not always a good idea all right okay so one problem you run into with threads is these races and generally you solve them with locks okay or actually there's two big strategies one is you figure out some locking strategy for making access to the data single thread one thread at a time or yury you fix your code to not share data if you can do that it's that's probably better because it's less complex all right so another issue that shows up with leads threads is called coordination when we're doing locking the different threads involved probably have no idea that the other ones exist they just want to like be able to get out the data without anybody else interfering but there are also cases where you need where you do intentionally want different threads to interact I want to wait for you maybe you're producing some data you know you're a different thread than me you're you're producing data I'm gonna wait until you've generated the data before I read it right or you launch a bunch of threads to say you crawl the web and you want to wait for all those fits to finish so there's times when we intentionally want different to us to interact with each other to wait for each other and that's usually called coordination and there's a bunch of as you probably know from having done the tutorial there's a bunch of techniques in go for doing this like channels which are really about sending data from one threat to another and breeding but they did to be sent there's also other stuff that more special purpose things like there's a idea called condition variables which is great if there's some thread out there and you want to kick it period you're not sure if the other threads even waiting for you but if it is waiting for you you just like to give it a kick so it can well know that it should continue whatever it's doing and then there's wait group which is particularly good for launching a a known number of go routines and then waiting for them Dolph to finish and a final piece of damage that comes up with threads deadlock the deadlock refers to the general problem that you sometimes run into where one thread you know thread this thread is waiting for thread two to produce something so you know it's draw an arrow to say thread one is waiting for thread two you know for example thread one may be waiting for thread two to release a lock or to send something on the channel or to you know decrement something in a wait group however unfortunately maybe T two is waiting for thread thread one to do something and this is particularly common in the case of locks its thread one acquires lock a and thread to acquire lock be so thread one is acquired lock a throw two is required lot B and then next thread one needs to lock B also that is hold two locks which sometimes shows up and it just so happens that thread two needs to hold block hey that's a deadlock all right at least grab their first lock and then proceed down to where they need their second lock and now they're waiting for each other forever right neither can proceed neither then can release the lock and usually just nothing happens so if your program just kind of grinds to a halt and doesn't seem to be doing anything but didn't crash deadlock is it's one thing to check okay all right let's look at the web crawler from the tutorial as an example of some of this threading stuff I have a couple of two solutions and different styles are really three solutions in different styles to allow us to talk a bit about the details of some of this thread programming so first of all you all probably know web crawler its job is you give it the URL of a page that it starts at and you know many web pages have links to other pages so what a web crawler is trying to do is if that's the first page extract all the URLs that were mentioned that pages links you know fetch the pages they point to look at all those pages for the ules are all those but all urls that they refer to and keep on going until it's fetched all the pages in the web let's just say and then it should stop in addition the the graph of pages and URLs is cyclic that is if you're not careful um you may end up following if you don't remember oh I've already fetched this web page already you may end up following cycles forever and you know your crawler will never finish so one of the jobs of the crawler is to remember the set of pages that is already crawled or already even started a fetch for and to not start a second fetch for any page that it's already started fetching on and you can think of that as sort of imposing a tree structure finding a sort of tree shaped subset of the cyclic graph of actual web pages okay so we want to avoid cycles we want to be able to not fetch a page twice it also it turns out that it just takes a long time to fetch a web page but it's good servers are slow and because the network has a long speed of light latency and so you definitely don't want to fetch pages one at a time unless you want to crawl to take many years so it pays enormous lead to fetch many pages that same I'm up to some limit right you want to keep on increasing the number of pages you fetch in parallel until the throughput you're getting in pages per second stops increasing that is running increase the concurrency until you run out of network capacity so we want to be able to launch multiple fetches in parallel and a final challenge which is sometimes the hardest thing to solve is to know when the crawl is finished and once we've crawled all the pages we want to stop and say we're done but we actually need to write the code to realize aha we've crawled every single page and for some solutions I've tried figuring out when you're done has turned out to be the hardest part all right so my first crawler is this serial crawler here and by the way this code is available on the website under crawler go on the schedule you won't look at it this wrist calls a serial crawler it effectively performs a depth-first search into the web graph and there is sort of one moderately interesting thing about it it keeps this map called fetched which is basically using as a set in order to remember which pages it's crawled and that's like the only interesting part of this you give it a URL that at line 18 if it's already fetched the URL it just returns if it doesn't fetch the URL it first remembers that it is now fetched it actually gets fetches that page and extracts the URLs that are in the page with the fetcher and then iterates over the URLs in that page and calls itself for every one of those pages and it passes to itself the way it it really has just a one table there's only one fetched map of course because you know when I call recursive crawl and it fetches a bunch of pages after it returns I want to be where you know the outer crawl instance needs to be aware that certain pages are already fetched so we depend very much on the fetched map being passed between the functions by reference instead of by copying so it so under the hood what must really be going on here is that go is passing a pointer to the map object to each of the calls of crawl so they all share the pointer to the same object and memory rather than copying rather than copying than that any questions so this code definitely does not solve the problem that was posed right because it doesn't launch parallel parallel fetches now so clue we need to insert goroutines somewhere in this code right to get parallel fetches so let's suppose just for chuckles dad we just start with the most lazy thing because why so I'm gonna just modify the code to run the subsidiary crawls each in its own go routine actually before I do that why don't I run the code just to show you what correct output looks like so hoping this other window Emad run the crawler it actually runs all three copies of the crawler and they all find exactly the same set of webpages so this is the output that we're hoping to see five lines five different web pages are are fetched prints a line for each one so let me now run the subsidiary crawls in their own go routines and run that code so what am I going to see the hope is to fetch these webpages in parallel for higher performance so okay so you're voting for only seeing one URL and why so why is that yeah yes that's exactly right you know after the after it's not gonna wait in this loop at line 26 it's gonna zip right through that loop I was gonna fetch 1p when the ferry first webpage at line 22 and then a loop it's gonna fly off the girl routines and immediately the scroll function is gonna return and if it was called from main main what was exit almost certainly before any of the routines was able to do any work at all so we'll probably just see the first web page and I'm gonna do when I run it you'll see here under serial that only the one web page was found now in fact since this program doesn't exit after the serial crawler those Guru T's are still running and they actually print their output down here interleaved with the next crawler example but nevertheless the codes just adding a go here absolutely doesn't work so let's get rid of that okay so now I want to show you a one style of concurrent crawler and I'm presenting to one of them written with shared data shared objects and locks it's the first one and another one written without shared data but with passing information along channels in order to coordinate the different threads so this is the shared data one or this is just one of many ways of building a web crawler using shared data so this code significantly more complicated than a serial crawler it creates a thread for each fetch it does alright but the huge difference is that it does with two things one it does the bookkeeping required to notice when all of the crawls have finished and it handles the shared table of which URLs have been crawled correctly so this code still has this table of URLs and that's this F dot fetched this F dot fetch map at line 43 but this this table is actually shared by all of the all of the crawler threads and all the collar threads are making or executing inside concurrent mutex and so we still have this sort of tree up in current mutexes that's exploring different parts of the web graph but each one of them was launched as a as his own go routine instead of as a function call but they're all sharing this table of state this table of test URLs because if one go routine fetches a URL we don't want another girl routine to accidentally fetch the same URL and as you can see here line 42 and 45 I've surrounded them by the new taxes that are required to to prevent a race that would occur if I didn't add them new Texas so the danger here is that at line 43 a thread is checking of URLs already been fetched so two threads happen to be following the same URL now two calls to concurrent mutex end up looking at the same URL maybe because that URL was mentioned in two different web pages if we didn't have the lock they'd both access the math table to see if the threaded and then already if the URL had been already fetched and they both get false at line 43 they both set the URLs entering the table to true at line 44 and at 47 they will both see that I already was false and then they both go on to patch the web page so we need the lock there and the way to think about it I think is that we want lines 43 and 44 to be atomic that is we don't want some other thread to to get in and be using the table between 43 and 44 we we want to read the current content each thread wants to read the current table contents and update it without any other thread interfering and so that's what the locks are doing for us okay so so actually any questions about the about the locking strategy here all right once we check the URLs entry in the table alliant 51 it just crawls it just fetches that page in the usual way and then the other thing interesting thing that's going on is the launching of the threads yes so the question is what's with the F dot no no the MU it is okay so there's a structure to find out line 36 that sort of collects together all the different stuff that all the different state that we need to run this crawl and here it's only two objects but you know it could be a lot more and they're only grouped together for convenience there's no other significance to the fact there's no deep significance the fact that mu and fetch store it inside the same structure and that F dot is just sort of the syntax are getting out one of the elements in the structure so I just happened to put them you in the structure because it allows me to group together all the stuff related to a crawl but that absolutely does not mean that go associates the MU with that structure or with the fetch map or anything it's just a lock objects and just has a lock function you can call and that's all that's going on so the question is how come in order to pass something by reference I had to use star here where it is when a in the previous example when we were passing a map we didn't have to use star that is didn't have to pass a pointer I mean that star notation you're seeing there in mine 41 basically and he's saying that we're passing a pointer to this fetch state object and we want it to be a pointer because we want there to be one object in memory and all the different go routines I want to use that same object so they all need a pointer to that same object so so we need to find your own structure that's sort of the syntax you use for passing a pointer the reason why we didn't have to do it with map is because although it's not clear from the syntax a map is a pointer it's just because it's built into the language they don't make you put a star there but what a map is is if you declare a variable type map what that is is a pointer to some data in the heap so it was a pointer anyway and it's always passed by reference do they you just don't have to put the star and it does it for you so there's they're definitely map is special you cannot define map in the language it's it has to be built in because there's some curious things about it okay good okay so we fetch the page now we want to fire off a crawl go routine for each URL mentioned in the page we just fetch so that's done in line 56 on line 50 sisters loops over the URLs that the fetch function returned and for each one fires off a go routine at line 58 and that lines that func syntax in line 58 is a closure or a sort of immediate function but that func thing keyword is doing is to clearing a function right there that we then call so the way to read it maybe is that if you can declare a function as a piece of data as just func you know and then you give the arguments and then you give the body and that's a clears and so this is an object now this is like it's like when you type one when you have a one or 23 or something you're declaring a sort of constant object and this is the way to define a constant function and we do it here because we want to launch a go routine that's gonna run this function that we declared right here and so we in order to make the go routine we have to add a go in front to say we want to go routine and then we have to call the function because the go syntax says the syntax of the go keywords as you follow it by a function name and arguments you want to pass that function and so we're gonna pass some arguments here and there's two reasons we're doing this well really this one reason we you know in some other circumstance we could have just said go concurrent mutex oh I concur mutex is the name of the function we actually want to call with this URL but we want to do a few other things as well so we define this little helper function that first calls concurrent mutex for us with the URL and then after them current mutex is finished we do something special in order to help us wait for all the crawls to be done before the outer function returns so that brings us to the the weight group the weight group at line 55 it's a just a data structure to find by go to help with coordination and the game with weight group is that internally it has a counter and you call weight group dot add like a line 57 to increment the counter and we group done to decrement it and then this weight what this weight method called line 63 waits for the counter to get down to zero so a weight group is a way to wait for a specific number of things to finish and it's useful in a bunch of different situations here we're using it to wait for the last go routine to finish because we add one to the weight group for every go routine we create line 60 at the end of this function we've declared decrement the counter in the weight group and then line three weights until all the decrements have finished and so the reason why we declared this little function was basically to be able to both call concurrently text and call dot that's really why we needed that function so the question is what if one of the subroutines fails and doesn't reach the done line that's a darn good question there is you know if I forget the exact range of errors that will cause the go routine to fail without causing the program to feel maybe divides by zero I don't know where dereference is a nil pointer not sure but there are certainly ways for a function to fail and I have the go routine die without having the program die and that would be a problem for us and so really the white right way to I'm sure you had this in mind and asking the question the right way to write this to be sure that the done call is made no matter why this guru team is finishing would be to put a defer here which means call done before the surrounding function finishes and always call it no matter why the surrounding function is finished yes and yes yeah so the question is how come two users have done in different threads aren't a race yeah so the answer must be that internally dot a weight group has a mutex or something like it that each of Dunn's methods acquires before doing anything else so that simultaneously calls to a done to await groups methods are trees we could to did a low class yeah for certain leaf C++ and in C you want to look at something called P threads for C threads come in a library they're not really part of the language called P threads which they have these are extremely traditional and ancient primitives that all languages yeah say it again you know not in this code but you know you could imagine a use of weight groups I mean weight groups just count stuff and yeah yeah yeah weight group doesn't really care what you're pounding or why I mean you know this is the most common way to see it use you're wondering why you is passed as a parameter to the function at 58 okay yeah this is alright so the question is okay so actually backing up a little bit the rules for these for a function like the one I'm defining on 58 is that if the function body mentions a variable that's declared in the outer function but not shadowed then the the inner functions use of that is the same variable in the inner function as in the outer function and so that's what's happening with Fechter for example like what is this variable here refer to what does the Fechter variable refer to in the inner function well it refers it's the same variable as as the fetcher in the outer function says just is that variable and so when the inner function refers to fetcher it just means it's just referring the same variable as this one here and the same with F f is it's used here it's just is this variable so you might think that we could get rid of the this u argument that we're passing and just have the inner function take no arguments at all but just use the U that was defined up on line 56 in the loop and it'll be nice if we could do that because save us some typing it turns out not to work and the reason is that the semantics of go of the for loop at line 56 is that the for the updates the variable you so in the first iteration of the for loop that variable u contains some URL and when you enter the second iteration before the that variable this contents are changed to be the second URL and that means that the first go routine that we launched that's just looking at the outer if it we're looking at the outer functions u variable the that first go team we launched would see a different value in the u variable after the outer function it updated it and sometimes that's actually what you want so for example for for F and then particular F dot fetched we interaction absolutely wants to see changes to that map but for you we don't want to see changes the first go routine we spawn should read the first URL not the second URL so we want that go routine to have a copy you have its own private copy of the URL and you know is we could have done it in other ways we could have but the way this code happens to do it to produce the copy private to that inner function is by passing the URLs in argument yes yeah if we have passed the address of you yeah then it uh it's actually I don't know how strings work but it is absolutely giving you your own private copy of the variable you get your own copy of the variable and it yeah are you saying we don't need to play this trick in the code we definitely need to play this trick in the code and what's going on is this it's so the question is Oh strings are immutable strings are immutable right yeah so how kind of strings are immutable how can the outer function change the string there should be no problem the problem is not that the string is changed the problem is that the variable U is changed so the when the inner function mentions a variable that's defined in the outer function it's referring to that variable and the variables current value so when you if you have a string variable that has has a in it and then you assign B to that string variable you're not over writing the string you're changing the variable to point to a different string and and because the for loop changes the U variable to point to a different string you know that change to you would be visible inside the inner function and therefore the inner function needs its own copy of the variable essentially make a copy of that so that okay but that is what we're doing in this code and that's that is why this code works okay the proposal or the broken code that we're not using here I will show you the broken code this is just like a horrible detail but it is unfortunately one that you'll run into while doing the labs so you should be at least where that there's a problem and when you run into it maybe you can try to figure out the details okay that's a great question so so the question is you know if you have an inner function just a repeated if you have an inner function that refers to a variable in the surrounding function but the surrounding function returns what is the inner functions variable referring to anymore since the outer function is as returned and the answer is that go notices go analyzes your inner functions or these are called closures go analyzes them the compiler analyze them says aha oh this disclosure this inner function is using a variable in the outer function we're actually gonna and the compiler will allocate heat memory to hold the variable the you know the current value of the variable and both functions will refer to that that little area heap that has the barrel so it won't be allocated the variable won't be on the stack as you might expect it's moved to the heap if if the compiler sees that it's using a closure and then when the outer function returns the object is still there in the heap the inner function can still get at it and then the garbage collector is responsible for noticing that the last function to refer to this little piece of heat that's exited returned and to free it only then okay okay okay so wait group wait group is maybe the more important thing here that the technique that this code uses to wait for all the all this level of crawls to finished all its direct chill and the finish is the wait group of course there's many of these wait groups one per call two concurrent mutex each call that concurrent mutex just waits for its own children to finish and then returns okay so back to the lock actually there's one more thing I want to talk about with a lock and that is to explore what would happen if we hadn't locked right I'm claiming oh you know you don't lock you're gonna get these races you're gonna get incorrect execution whatever let's give it a shot I'm gonna I'm gonna comment out the locks and the question is what happens if I run the code with no locks what am I gonna see so we may see a ru or I'll call twice or I fetch twice yeah that's yeah that would be the error you might expect alright so I'll run it without locks and we're looking at the concurrent map the one in the middle this time it doesn't seem to have fetched anything twice it's only five run again gosh so far genius so maybe we're wasting our time with those locks yeah never seems to go wrong I've actually never seem to go wrong so the code is nevertheless wrong and someday it will fail okay the problem is that you know this is only a couple of instructions here and so the chances of these two threads which are maybe hundreds of instructions happening to stumble on this you know the same couple of instructions at the same time is quite low and indeed and and this is a real bummer about buggy code with races is that it usually works just fine but it probably won't work when the customer runs it on their computer so it's actually bad news for us right what do we you know it it can be in complex programs quite difficult to figure out if you have a race right and you might you may have code that just looks completely reasonable that is in fact sort of unknown to you using shared variables and the answer is you really the only way to find races in practice to be is you automated tools and luckily go actually gives us this pretty good race detector built-in to go and you should use it so if you pass the - race flag when you have to get your go program and run this race detector which well I'll run the race detector and we'll see so it emits an error message from us it's found a race and it actually tells us exactly where the race happened so there's a lot of junk in this output but the really critical thing is that the race detector realize that we had read a variable that's what this read is that was previously written and there was no intervening release and acquire of a lock that's what that's what this means furthermore it tells us the line number so it's told us that the read was a line 43 and the write the previous write was at line 44 and indeed we look at the code and the read isn't line 43 and the right is at lying 44 so that means that one thread did a write at line 44 and then without any intervening lock and another thread came along and read that written data at line 43 that's basically what the race detector is looking for the way it works internally is it allocates sort of shadow memory now lucky some you know it uses a huge amount of memory and basically for every one of your memory locations the race detector is allocated a little bit of memory itself in which it keeps track of which threads recently read or wrote every single memory location and then when and it also to keep tracking keeping track of when threads acquiring release locks and do other synchronization activities that it knows forces but force threads to not run and if the race detector driver sees a ha there was a memory location that was written and then read with no intervening market it'll raise an error yes I believe it is not perfect yeah I have to think about it what one certainly one way it is not perfect is that if you if you don't execute some code the race detector doesn't know anything about it so it's not analyzing it's not doing static analysis the racing sectors not looking at your source and making decisions based on the source it's sort of watching what happened at on this particular run of the program and so if this particular run of the program didn't execute some code that happens to read or write shared data then the race detector will never know and there could be erased there so that's certainly something to watch out for so you know if you're serious about the race detector you need to set up sort of testing apparatus that tries to make sure all all the code is executed but it's it's it's very good and you just have to use it for your 8 to 4 lives okay so this is race here and of course the race didn't actually occur what the race editor did not see was the actual interleaving simultaneous execution of some sensitive code right it didn't see two threads literally execute lines 43 and 44 at the same time and as we know from having run the things by hand that apparently doesn't happen only with low probability all it saw was at one point that was a right and they made me much later there was a read with no intervening walk and so enact in that sense it can sort of detect races that didn't actually happen or didn't really cause bugs okay okay one final question about this this crawler how many threads does it create yeah and how many concurrent threads could there be yeah so a defect in this crawler is that there's no obvious bound on the number of simultaneous threads that might create you know with the test case which only has five URLs big whoopee but if you're crawling a real wheel web with you know I don't know are there billions of URLs out there maybe not we certainly don't want to be in a position where the crawler might accidentally create billions of threads because you know thousands of threads it's just fine billions of threads it's not okay because each one sits on some amount of memory so a you know there's probably many defects in real life for this crawler but one at the level we're talking about is that it does create too many threads and really ought to have a way of saying well you can create 20 threads or 100 threads or a thousand threads but no more so one way to do that would be to pre create a pool a fixed size pool of workers and have the workers just iteratively look for another URL to crawl crawl that URL rather than creating a new thread for each URL okay so next up I want to talk about a another crawler that's implemented and a significantly different way using channels instead of shared memory it's a member on the mutex call or I just said there is this table of URLs that are called that's shared between all the threads and asked me locked this version does not have such a table does not share memory and does not need to use locks okay so this one the instead there's basically a master thread that's his master function on a decent 986 and it has a table but the table is private to the master function and what the master function is doing is instead of sort of basically creating a tree of functions that corresponds to the exploration of the graph which the previous crawler did this one fires off one ute one guru team per URL that it's fetches and that but it's only the master only the one master that's creating these threads so we don't have a tree of functions creating threads we just have the one master okay so it creates its own private map a line 88 this record what it's fetched and then it also creates a channel just a single channel that all of its worker threads are going to talk to and the idea is that it's gonna fire up a worker thread and each worker thread that it fires up when it finished such as fetching the page will send exactly one item back to the master on the channel and that item will be a list of the URLs in the page that that worker thread fetched so the master sits in a loop we're in line eighty nine is reading entries from the channel and so we have to imagine that it's started up some workers in advance and now it's reading the information the URL lists that those workers send back and each time he gets a URL is sitting on land eighty nine it then loops over the URLs in that URL list from a single page fetch align ninety and if the URL hasn't already been fetched it fires off a new worker at line 94 to fetch that URL and if we look at the worker code online starting line 77 basically calls his fetcher and then sends a message on the channel a line 80 or 82 saying here's the URLs in the page they fetched and notice that now that the maybe interesting thing about this is that the worker threads don't share any objects there's no shared object between the workers and the master so we don't have to worry about locking we don't have to worry about rhesus instead this is a example of sort of communicating information instead of getting at it through shared memory yes yeah yeah so the observation is that the code appears but the workers are the observation is the workers are modifying ch while the Masters reading it and that's not the way the go authors would like you to think about this the way they want you to think about this is that CH is a channel and the channel has send and receive operations and the workers are sending on the channel while the master receives on the channel and that's perfectly legal the channel is happy I mean what that really means is that the internal implementation of channel has a mutex in it and the channel operations are careful to take out the mutex when they're messing with the channels internal data to ensure that it doesn't actually have any reasons in it but yeah channels are sort of protected against concurrency and you're allowed to use them concurrently from different threads yes over the channel receive yes we don't need to close the channel I mean okay the the break statement is about when the crawl has completely finished and we fetched every single URL right because hey what's going on is the master is keeping I mean this n value is private value and a master every time it fires off a worker at increments the end though every worker it starts since exactly one item on the channel and so every time the master reads an item off the channel it knows that one of his workers is finished and when the number of outstanding workers goes to zero then we're done and we don't once the number of outstanding workers goes to zero then the only reference to the channel is from the master or from oh really from the code that calls the master and so the garbage collector will very soon see that the channel has no references to it and will free the channel so in this case sometimes you need to close channels but actually I rarely have to close channels he said again so the question is alright so you can see at line 106 before calling master concurrent channel sort of fires up one shoves one URL into the channel and it's to sort of get the whole thing started because the code for master was written you know the master goes right into reading from the channel line 89 so there better be something in the channel otherwise line 89 would block forever so if it weren't for that little code at line 107 the for loop at 89 would block reading from the channel forever and this code wouldn't work well yeah so the observation is gosh you know wouldn't it be nice to be able to write code that would be able to notice if there's nothing waiting on the channel and you can if you look up the Select statement it's much more complicated than this but there is the Select statement which allows you to proceed to not block if something if there's nothing waiting on the channel because the work resin finish okay sorry to the first question is there I think what you're really worried about is whether we're actually able to launch parallel so the very first step won't be in parallel because there's an exit owner the for-loop weights in at line 89 that's not okay that for loop at line 89 is does not just loop over the current contents of the channel and then quit that is the for loop at 89 is going to read it may never exit but it's gonna read it's just going to keep waiting until something shows up in the channel so if you don't hit the break at line 99 the for loop own exit yeah alright I'm afraid we're out of time we'll continue this actually we have a presentation scheduled by the TAS which I'll talk more about go

Channel: MIT 6.824: Distributed Systems

Views: 182,039

Keywords:

Id: gA4YXUJX7t8

Length: 80min 22sec (4822 seconds)

Published: Fri Feb 07 2020