Rust Stream: Ownership, Closures, and Threads - Oh My!

so welcome everybody to a another stream today we're gonna try something slightly different than we've done in the past and I'm gonna try and do something geared towards what I like to call maybe advanced beginners or beginner intermediates depending on how you think about it so this stream is really geared towards people that have done a little bit of rust before they've probably read through the book they've probably done a toy project or something the concepts are familiar if if maybe not quite 100% grasped you're you've at least heard of things like the borrow checker and lifetimes and things like that and today what we're going to be trying to do is go through a project where we kind of put some of those concepts together and really try and see how all these things work together and real quick as I switch over to the to the browser scene here this kind of came from the results of the 2019 rust survey which at the end there's a conclusion at the very end which basically says that we need more learning material for intermediate users and it's kind of hard to know what an intermediate user is some people who are fairly competent and and rust claim that they're beginners some people who or vice versa so not exactly sure what an intermediate user is but what I'm aiming for today is kind of that advanced beginner you've you've read through the book you you've heard of the concepts and today we're gonna put it all together and want to give a quick shout out here to John who's also been doing some streaming as well I mean if you've not checked out John string before or his YouTube channel please do there's a ton of awesome stuff therapy's really super great and he did on Wednesday a stream that kind of aimed at going over specifically over lifetime annotations which today we'll touch on briefly but the truth of the matter is that I'm annotations aren't needed a ton in rust and so this is a really specialized topic I haven't watched a stream on this yet um I kind of wanted to go into this without being biased towards the topics that he touched but knowing his past work for sure it was a great stream and it seems like a lot of people enjoyed it from from what I see I saw online but today we're gonna be doing something a little bit different the most important part for those that are watching live right now is to ask questions we're gonna try and go through this and in a way I'm gonna try to make a bunch of mistakes or try to go down past that won't really work out fully even though I've done I've done this exercise before and I know how this code should work so you know depending on how much I remember of when I've done this kind of coding before I may be biased towards not making mistakes but please make sure that I do is so this is the first stream perhaps in history where people want where I actually want to really make as many mistakes as possible and I'm relying on the chap to make sure that that that happens so what are we going to be doing today well what what we'll be doing is working on a small little library for doing a thread pool and this is not going to be a library that people will actually use at the end of the day there's a ton of things that it won't do it's very naive implementation but the nice thing about thread pool is that we really kind of have to dive into how lifetimes work or how the power of borrowed checker works when things are dropped how long things live for things like that as I said before we won't really be annotating that a ton but we still have to have an understanding of that because the russ compiler will complain when we do something that is not fully correct and at the end of the day what hopefully we should get out of it is as a thread pool library that allows us to execute on a pool of threads where we kind of round-robin around the threads and and we'll write some tests for it and we'll see even the tests will be illustrative of how we should think about the power checker and things like that so hopefully everybody's ready to go again chat let me know if you have any questions so what I'm going to do first of all is create a new project so we're gonna do Cargo new Lib and the hardest part of the day of course is always coming up with a name for your crate I'm going to call it thread pool because I'm feeling very creative today so let's go into thread pool and then I'm gonna open it up and vs code whoops should probably switch over to this so you can see what I'm doing just just for for your knowledge this is what I did before cargo new blip thread pool went into it and then opened up vs code so pretty straightforward and of course done some rest before you know that this is what you kind of get by default from from the library so great so I think the first thing that we're going to want to do is to kind of spec out the shape of what we want to do maybe come up with a with a simple public API that we want to expose and then start filling in the details from there so probably the first thing that we want is a public struct called write pool I would guess and we'll just put nothing in it because we don't know what we need quite yet and then we'll have a thread pool where maybe we do a new function because that seems like it would be good and let's see we'll just do it like this and of course we want that to be pub and then what I imagine we will have is kind of a an execute function of some sort like this and it's gonna take something but we'll talk about that and in just one second what that might be and for now it won't return anything probably today it won't return anything so this is this is what our our our public API is is going to look like right so we have our thread pool we know how to construct it we know how to call execute on it of course that's not gonna do anything right now but just to get our tests kind of bootstrapped let's go ahead and do pool equals thread pool new in order for this to work we need to kind of brink we need to bring our thread pool into scope like this all right what is it complaining about here help resolve thread pool no it's cuz it's a capital P great so now we have our third pool and we want to call execute on it so cool dot execute execute like this and it's gonna complain because we forgot to do a ampersand self up here and the reason I'm doing it percent self here is because we presumably want to be able to call pool execute kind of over and over again like this and so if we were to take it like this this would mean here that we are taking self ownership of self moving it into the execute function and that means that pool here is no longer valid which is the error that we get we're using pool after it's been moved into to execute this is a good time to stop and say if what I just said makes absolutely no sense to you then the stream might be a little bit more advanced then what you're ready for so go back to the book and read that and then come watch this later on or stick around and just enjoy as I type things but if that at least if the words that I said have some meaning to you even if you're not 1000% comfortable with the the ideas that they represent then this is perfect for you and this dream is kind of hopefully kind of solidify those those ideas for you okay so we have execute here and now the next thing that I want to do is I know that for our execute the kind of epi that I want is to be able to pass in a closure or like this and then do some work so like you know let's say print line hello from thread like that and we should be able to kind of do this same here and we should be we should be able to get the thread ID so we can later on look at kind of make sure that we're running these two things on different threads here and stuff but we'll get to that later on okay so the reason it's complaining here is because execute needs that actually take that closure so if you've not done if you you probably use closures before if you've done any kind of rust but you may not have had to make a function generic on closures before and so what we're going to do is do a generic parameter like this and closures are things that implement these kind of closure special close are traits and so we can't just take a concrete typed out of the closure we have to take a generic type that implements one of these closure traits and the most common one is this FN right here and so it's the work or something like that and it's going to be like that okay so what we said here is we have some generic type T it implements this FN closure trait which is a special trait that basically says this is a a regular closure and we'll talk about the other variants of closure traits F in mutton F and once a little bit later on and we know from the signature that that closure takes no arguments returns nothing from it and we're gonna pass that in as work here and what that means is that every time we call execute will be actually calling execute on it will be calling a different execute function that is specialized for the work that is done you know inside of the closures because these closures I know not exactly sure if these particular closures because they do the exact same thing are different from each other but if you have different closures that do different things then they will be different concrete types so is everybody good with that any questions from chat or anything like that great okay so I think now that we've got the kind of the shape of our library up and running then we can start actually doing the implementation work and so I think the first thing that any good thread pool needs is threats probably and so what we're going to do it is spawn off a bunch of threats that will kind of exist in the background and this is not going to be a thread pool that supports kind of lazy instantiation of threats we're just gonna when you create the thread pool you just create a bunch of threads maybe not the most efficient way to do it but but it's what we're going to do for now and so I'm thinking about this and maybe for our thread pool new we want to take like a number of threads that we want to actually create here so we can do something like num threads and I'll just do a u8 here because it feels like you know if you want to create more than 256 threads that's I don't know that's quite a lot so maybe you can make the argument for you 16 but like I use 64 we don't want to be creating I don't know you know millions of billions or trillions of of threats and so what we're going to do here is just do a simple for loop so boom for blank and zero to nome threads we're going to call standard thread spawn like that and we have to have some pass on a closure and so here we've actually spawned our thread and of course we spawn returns back to threat handle this this handle allows us to say okay this right we want to block and wait for it to be done so we can just get that handle back so that starts completely stops complaining about it and here it's saying hey I expected to find a semicolon where's my semicolon at and so we go ahead and put that down there okay so now we're going to have a bunch of threads which is which is really great and you know I don't know if we'll need it but we can go ahead and just hold on to these to these thread handles so I'll just do handles and then we'll do standard thread thread isn't thread handler or join handle sorry join handle and it returns that joint handle returns nothing to us like that and close off our effect here and then we can come down here and say animals like that and so we need to let's go ahead and turn this into a map real quick and we will go ahead and do like this so we're going to iterate over the range map and ignore what we're getting there and we're going to then collect into a vector like like this do we do not need that are here yeah we didn't either okay cool so now we are mapping over our range and for everything responding a thread and then collecting it into our vector of join handles and pollie onto them Gorillaz awesome and then down on our test is complaining because now we don't save the number of threads that we've won so let's just do ten threads for now and that should be good let's see we have some questions here are these rest parts directly mapped to a start or is it something similar to library go routine so threads in rust are OS threads way back in the day before 1.0 Russ did have green threats and they got rid of it the reason for it is that rust is meant to be a systems level language it's meant to be you know it's meant to have zero costs abstractions and if you have green threads you have to have some runtime that keeps track of those green threads and so this kind of went against the Russ philosophy that if the rest philosophy I guess one way you can sum it up is any kind of abstraction if it costs something it should only cost something to you if you actively use it and so if you don't use that abstraction then it shouldn't have any cost to at all but green threads do have a cost whether you use them or not because you need to start up a runtime and and so everybody would have a green thread runtime no matter if they use them or not and so they were removed and Tokyo or acing standard or any of those runtimes now kind of have that that idea built into them and so when you when you use Tokyo for instance you go ahead and start up a Tokyo runtime or what a sinc standard you start up an async standard runtime and then it gives you the ability to spawn tasks which in effect is green threats and let's see a difference between Tokyo threats and standard threats hopefully that explained it and yeah Tokyo thread Tokyo and icing standard I think both use the term task instead of thread to kind of avoid this confusion but yeah you can think of them like like green threads exactly all right awesome okay so we have everything running we have threads that we're going to to spawn to and now what we need to do is figure out that we're how we're going to actually kick off the work so every time we spawn a thread what we want to do is have some way some way basically of one second there so we need to have some way of doing some work checking that if there is some work to do doing that work and then looping back and figuring out if there's more work to do and so I think a great way to do that would be inside of this spawn to literally just loop forever and then like we need to check for work and if there you know once once work do work and then look back kind of thing and so the question is how can we check for work and a great thing that comes with with the standard library come back here is channels which are again going to be familiar for any go prep go programmers in here why is this not working so here we have inside a standard sink MPSC which stands for a multi producer single consumer FIFO queue which allows us basically to send messages across threats which is exactly what we want to do and if you're familiar with go then then go has concepts just like this so I think what we want to do is have a sender and receiver and what the receiver will do is receive what kind of inside of the thread listen for for work do check if there's work block if there's not work and then once there is work it will get it and do the work and then loop back around and so what we want to do here is we're going to have a sender and a receiver and you'll also oftentimes see this as as TX and rx like a transmission and a receiver that I like for today I'm just going to spell it all out and then up here we'll go ahead and we'll do it includes up here so use standard sink em PSC and we need for right now a channel like this so we're going to be doing a channel and what channel channel is generic on some type so it needs to know what type is going to be sending back and forth across across these threats and so what we can do for now is just say that it's going to be unit it's going to be nothing we won't send anything we're just going to basically you know notify this is another way to implement some kind of notification API that just says hey something happened and what we do now is our receiver we're going to move that receiver into this thread here and we're going to check for work so we can do receiver dot receive right here [Music] and receive returns a result here so the reason returns a result is because if the sender is no longer there and the channel is kind of broken then receive will stop blocking and return an error or letting you know that the the channel was kind of was broken up great so we need to I think for now we'll just go ahead and do unwrapped like this and just just crash the program if the sender and receiver is kind of breakup from each other this is not what we should do but we'll fix that a little bit later on and then so we've checked for work and receive again blocks on the current thread which is this thread that we spawned and when it gets a message it then returns and so here we'll have our message and that basically signifies the work that needs to be done and so the question now is you know what work do we need to do and what will be really great as we if we could just give gift this this function down here one receive that and then go ahead and call that work but of course like if we come up here we're sending right now a unit struct or a unit tuple here that doesn't really tell us anything we want to be able to do something like you know like this right like send me a function and then I'll call it but of course if you really read this it's like the size for value of type done standard ops F n cannot be known what it's saying is is that channel requires basically requires a type not not a trait and FN again is a trait so it really needs a concrete type the reason for that is it needs to know the size of the thing that it's receiving for the trait has no size of trade is just a compile-time concept whereas things Struck's I actually have a layout memory they have a defined size so the compiler knows okay I need to reserve this much space on the stack and stuff like that so we can't do we can't do that now I think the best thing to do when you have a trait of some sort and and you just want to be able to work on things that implement that trait but you don't care or even know what concrete types they are um they might be different concrete types which is going to be the case with our with our closures here these closures are going to be different types because every time a closure is created that you know captures its environment differently or whatever it has a different concrete type it's some magical compiler type or whatever but when we don't care about that or don't know about it or can't can't know about it the best thing to do is to take that trait and box it and put it on the heap so that we can call the trait methods in a in a way that's called dynamic dispatch we basically say put it on the heap I don't know what it is but I know that it has these functions and let me call these functions through a virtual table of eat table and so what we want to do here is do this and we'll get a warning here where it says this is a you know this is a trait object this is an object where we call its methods through a V table and it's not yet quite a hard error but in the future you will be required to explicitly notate that in the syntax and say this thing is is dynamic it's a trait object it's dynamically dispatched there is no static that's matched with this Dyn so again the the object that we're sending over our channel is a box it's a heap allocated pointer and it points to a function sorry a closure of some sort and we can call that closure through through that pointer basically so this will work so now what we're getting back here is our work that we want to do and again this is a our boxed you know dynamic a function here dynamic closure and of course well we do just want to call like that so this is great I'm gonna stop real quick I think there was a question and wait if anybody has anything else any other questions about why we needed to box about what dynamic dispatch is anything like that more than happy to to answer those let's see zero dot dot num threads yields the slice is the question no one does not yield a slice this is a let's see if I can get oh you can actually see it right here it's a range it's a range but a few eights that we have there and ranges are simply yeah start in and two numbers together all right okay so another question we got was what is box two dynamic dispatch okay so let's explain that one more time when you call a function in rust like headline - and we have another one if I call food here foo is really a function inside of my binary and when we call foo it's simply what will happen as we jump to this function and and we call it you know and of course there are you know really at the end of the day there's a whole bunch of things that needs to happen like we need to put the arguments somewhere like in registers on the stack we don't need to worry about that but really all at the end of the day this is just a straight function call nothing fancy going on when we define food generically though like this or we're gonna say we have some type T like this and I think we need where as debug debug is in standard format standard is one that debug here so we're taking some type T it's a concrete type when we call foo with something we'll call it with the number you know one as you ate what we're saying here is we will create a foo function that works specifically for you eighths and if we call it again right here where the item is a string like this then there will be another function in our binary that is specifically meant for Strings so now in our binary that we've created there will be two foo functions one that specialized on you ate some one that specialized on strings and if we call foo with different items it will keep on creating more and more special food functions of course if we call foo here with two u8 then this these two Foos are the same thing and this foo is another food that specialized just for Strings okay this is the technical term for this is mono more efficient so you're basically creating a function for every concrete type of thing that the trait that you've set for and what's awesome about this this is very fast it's simply a function call and you have a specialized function for every single type that you that you've created the thing that's not so great about it is that if you use this function with a ton of different types then you'll have copies of your function for all these different types and that might blow up your your binary size so what you gain in speed you'll lose in binary size now let's say we don't want to do that what we want to do is say give me one foo function and I just I don't care what the concrete type is I don't I don't know what it is I don't want to know all I know is that something that implements the debug trait what that means is we instead of calling some type T that happens to implement debug like this we're going to say we are taking literally the debug trait like this this won't work and the reason that it won't work is deep the debug trait is just some compile time fantasy we don't it doesn't really exist in our program it's simply a trade it's a collection of methods that something might that might somebody might implement but we when you call a function you need to know like a specific type it has to be it has to have a well-known size so that Russ can for instance create enough space on the stack and stuff like that and so one thing that we can do is take this by by reference instead and the reason that this works is how how big it is a reference to something well it's good it's a pointer basically now it's that's not quite the the real answer but we Russ knows now how much space the item argument takes up it knows exactly how much space that is it is simply a reference to a trait and of course we're getting the same complaint here that we got it above that I'll show you that this used to work but we need to be explicit now that this is just the trait so we do ampersand done no and so item here now it's it's some pointer to to debug and that means it will go through a V table to get there that's how you call methods on trade objects you go through a V table so while you maybe didn't understand everything there hopefully you understand now when you have generic argument you'll literally create copies of that function for every single type it'll specialize the compiler will specialize for each type and if you have done here then what that means is that you're taking a pointer to some you know some object that's out there and it will go through a virtual call table v table to call functions on that and what this means is you save it's a little bit slower because you have to go through a V table to call it so there's a little bit more in direction which means it's a little slower but there's only one foo function now only one and it's this one right here hopefully that that works today's not exactly about dynamic dispatch so I think that's the last we'll probably talk about that but hopefully that helped a little bit okay so why do we need the next question is why do we need these these right here so unfortunately fortunately or unfortunately the closure traits are not ordinary traits they're kind of special exceptions and the reason that there are special exceptions is because closures are very complicated and what this syntax right here means is that this closure it's a normal closure an FN closure and it takes no arguments we could say it takes one argument which is eu8 and we could say that it returns another argument which is the 32 and now we have a different trait that is a regular closure that takes you ada as an argument and returns at you 32 as an argument but we don't need that we simply just want a we simply want a closure that takes no arguments and returns nothing now you could argue a the designers could have done this and make that work but they decided when it has no arguments you need the you you need the the parentheses there hopefully that helps and the last question is are using our LSPs code or a rust analyzer I'm using rust analyzer I've been very happy with rust analyzer it's not perfect but it's very reliable much more reliable than arla RLS was for me and sometimes it gives you kind of wonky answers to things if you want to like go to code every once in a while it will get confused and say oh I think you mean this thing and take you to the wrong file but other than that it works really fast gives you 99% of the time the correct answer and so everything is good so I would definitely recommend using rust analyzer for your projects and if you haven't heard there is a proposal now to deprecate RLS and move to rust analyzer as the official language server implementation for rust so we'll see if that happens okay great so we've got everything working except we have one error message and let's see what that is here okay so here's the error message let me let me go ahead and open up the terminal and and do cargo check here instead because the the our master pens out a little bit prettier here and it does inside of a BS code here and so what we have here is it says this receiver of you know are boxed function cannot be shared between threads safely okay so we know it thinks correctly might add that this receiver type here is not thread safe that's what this says cannot be shared safely between threads and the reason that it knows that is because the trade sink it's not implemented and let's pop over here to sink and read about what sink is so types for which it is safe to share references of that type between threats so this is where you can you can synchronize between between threats so again it's a big error message but really when you when you read something like this the important thing to remember is when it says cannot be shared between threads safely like it says right here then what you should understand by that is that it is not thread safe this thing cannot be synchronized between threads and it says this because the requirements of send for receiver requires that it's type also influenced and let's look at that inside of the document the docs here to see what that means so this is our receiver type the type that we get from channel as you can see here the receiving end of our channel and if we go down here you can see that it's not sync so receiver is not sync receiver is just plainly not thread safe okay that stinks but let's think about that while why would receiver not be thread safe like what's it's the reason that it's not set right safe and at the end of the day you can think about this as this is a a multiple producer single consumer or another way to say it a multiple sender single receiver channel that's just the way that it was implemented rust doesn't come with another channel type doesn't come with a single producer multi consumer channel it just comes with this multi producer single consumer and so there is only ever going to be one consumer of this thing at a time and the reason for this is that the implementation of this channel assumes that it can safely you know just put the message into the receiver sighs because there's only gonna be one of them so it doesn't have to do any synchronization and make sure okay you know everything stopped the world and put it into this special box or whatever it can just say I know for a fact that there's only going to be ever one consumer at a time and so that's not true for us right we are we are mapping or we're looping over a number of threats were spawning threads this receiver is going to be on on ten different threads right that just doesn't work and so Russ is stopping us and saying hey you're trying to take this thing that assumes that it's running on in one place on one thread and you're putting it on ten threats and if this were another language we would be able to do that and then the whole thing would crash and we wouldn't know why and you know we would be pulling out our hair and stuff like that but luckily we we have rust and it tells us here that that is not possible this thing is not thread safe you need to make it thread safe somehow and so how do we do that what is the best way to take something that is does not allow for multiple things to have access to it at one time and for us to say we want to have exclusive access we want whatever when we when we get at the receiver we want to make sure that we were the only thread that is running this code at the time so think if you're if you've done programming before you might know of some things that allow exclusive access to two resources think about it I can think of one thing what about a mutex let's go over here and look at mutex mutex is a mutual exclusive exclusive primitive it allows you to lock something and when you get when you when you open up that the the mutex when you unlock it basically what it allows you to do is have exclusive access to the thing that's inside of it this is exactly what we need we the mutex will allow us to have exclusive access to this receiver that requires that it has exclusive access okay this is great let's go ahead and say use standard sync mutex and now what we can do is say let receiver cool you Tex new and then down here it's complaining and saying I don't know what Roose what this set is on mutex what are you talking about and that's because in order to use the thing that's inside of the mutex we have to block and now we get another error saying I don't know what receivers on result and that's because locking can returns a result here and let's look at the documentation here for why that is so the air is if another user of this mutex panicked while holding the mutex then this will return an error that's called poisoning we're going to assume that won't happen draw our implementation we're just gonna unwrap this puppy which is actually usually the right thing to do in this case I don't really think it's a wrong place for that I don't think it's really the right thing to do what this means that one of our workers panics here the whole thing will come crashing down basically we're going to panic on on on every thread because the lock will be poisoned and when it's poison it returns an error here and we just call it wrap so basically all the locks all these calls to locks that are happening on on all the separate threads will go ahead and panic and so the whole thing will come crashing down not what we really want in an implementation like this but hey you know for illustrative purposes I think this is this is good enough okay this is wonderful we have a receiver that requires basically mutual exclusion because it's not sync it requires that we only have one thread looking at this thing at a time how do we get mutual exclusion we wrapped in a mutex wonderful it's awesome but we're still getting an error and now I'm saying Sindh is not implemented for this thing okay let's look at the error here this is very so so before it didn't implement sync now it's it's not implementing send and let's take a look at its end here Sindh means that we can transfer this thing across thread boundaries we can we can take it from one thread and put it on to another what sync was you have two things on different threads and they can stay kind of in sync with each other send is you can send this to another threat and it turns out that when we look at mutex here mutex is only send and it's only think if the thing that it implements is also send here and so basically if we go back sorry we go back to send here we can see that it's going it's even giving us a hint an example of a non sent type is the reference count RC so when we when we use our see when we reference counting reference count here is a a single-threaded reference count but it's cousin arc is sent because it uses atomic reference counting it when you increment the reference count in an arc it does that atomically which means that it can handle being on on multiple threads there and so really what we need to do is make sure that our mutex here can be used from multiple threads needs to be able to to implement sync I'm going to stop here real quick for for any questions any questions from anybody so I don't see any questions so the problem that we have here is a little bit so we're trying to to send this receiver across threads the the issue that we're facing here is made a little bit more difficult by the fact that we have closures here and closures are a little bit clever and let me illustrate a point where this makes it difficult what type is receiver right here well it says it's a mutex receiver box Stefan you know FN blow BOTS all this thing but that's it's not really true when we look at lock here we can see it takes an ampersand self and so we don't need ownership of receiver here to call a lock we simply need a reference to it and Russ closures are so clever that they can basically they look at your code inside here and say ok this I'm capturing receiver here from its environment what is the minimum amount that I need from this thing to get this code to compile and that's what I will actually capture in here so receiver this this this closure right here is not capturing it's not taking ownership of receiver receiver is not being moved in here receiver won't be destroyed at the end of here receiver continues to exist up here instead right here what we're doing is we're getting receive a reference to receiver because that's all we need we all we only need a reference receiver but that's actually not what we really want we've we wind a way to tell to tell Russ no is seriously like I want you to move this thing on to this other thread I know you only technically need a reference to get this thing to work you know when you're kind of look inside here but trust me here I want to take this thing and I want to move ownership on to this other thread and the way that we do that is by calling by doing a move here and now what we're telling Russ to say don't be so clever with with these closures don't look at that lock here and say well it only needs am person itself so I'll only capture a reference to this receiver I don't care what it means I really want you to move this receiver onto it so now receiver the this this loop in here receiver here is an owned value and when this thing goes out of scope and it here it will be destroyed and so it doesn't matter if that lock only needs self we actually have ownership of of the receiver here instead okay cool because closures are special like this and the fact that they kind of understand what they're capturing and try and do the right thing of like using the minimum amount that they they need to get things to to work ironically that helpfulness ends up sometimes meaning that the code won't work because what is needed right here is not what we actually need overall in our program okay and because of that what I find when I get into really hairy situations with closures as sometimes try and eliminate using them which I don't normally like to say because most of the time closures are fantastic and you should use them but sometimes it's quite nice to not have to like deal with the magic of closures of how are they capturing their environment and stuff like that sometimes it's nice to just say you know what we're not going to use a closure here and we're going to go and and use a regular for loop instead now I'm just saying this is kind of a dee dee view debug tip and well you know we'll take a look at that in a second how it helps us it is totally possible to get this code working relative easily with closures but I just want to show you how it can can help us debug a little bit so we're going to switch back to a for loop real quick and now what we need to do is collect our handles into a vector of some sort and so it's we get a handle here and now we need to call handles stop push like this and then of course push our handle on so effectively what we have here and they don't need these anymore effectively what we have here is the same exact thing that we had before except we're using for loop instead of a closure with our map function so these these are kind of equivalent cloade except we don't use a closure now okay okay so now we have this we have this error that we have where it's still saying this thing is not sent okay and it's saying that because it's the implementation of send is does it does not work for for what we're doing here so really what we need to be able to do is when we move our ownership into here we need to be able to we need to be able to see when receiver will actually will will actually be destroyed we don't know that right now think about this when will receiver be destroyed right now well we loop over these number of threats we spawn a thread receiver has been moved on two into this thread here that means this thread here has ownership reverse receiver and that means that at the end right here after we've called spun right here then our receiver will be destroyed and of course we're looping over again until this doesn't make sense we've moved receiver onto onto the thread here but every time we create a new thread receiver and the last thread was moved on to that thread and will be destroyed there so we can't have we can't have multiple owners here that's what we're trying to do we're trying to put receiver onto multiple threads here and and each thread receiver will be will be destroyed will have double freeze zero Frank we're moving receiver onto thread number one it gets destroyed at the end thread number two also has ownership of receiver and we'll also call it destroy that that doesn't work the the the golden rule of rust is that there's only ever one owner of a piece of data here and unfortunately we're trying to create many owners of this receiver mutex that we have right here does that make sense to everybody just think when you get into a situation like this you have to think this thing that I'm sitting on to this thread or this thing that I'm moving somewhere else am i doing it multiple times here when I'm looping over and creating a new thread spawning a new thread I've moved this receiver because of this move keyword here onto onto this thread and then when we loop again here and our number of threads I'm going to try to move the receiver on to another thread and I it so I'll have multiple others that can't happen and so you've got a point you get to a point where you say okay we've got multiple owners we need multiple owners how do I take something and allow it to have multiple owners and we've already seen the answer here the answer is arc it is a thread safe reference counting pointer so we know that it's thread safe which is exactly what we need and it allows basically for mold owners of a thing this art will take control of the other thing and allow you to cheaply clone it and then you can you can send it around and the thing that it's that it contains will only be destroyed when all of the different references have been been dropped and that's exactly what we need here when should receiver be dropped receiver should be dropped when it is no longer needed and any of these threats and when does that happen well we don't know until run time and so we need this run time mechanism to keep keep track of that all right I'm going to look real quick at the questions here's a new question why not just write the spawn statement inside of handle stop push so I guess the question was why do we put this into a handle variable and then call handle stop push right here style I don't know I find it more readable this way but if you don't then you can do it the other way totally a style thing okay oh and then there was a question which I apparently got ahead of me and good job chat chat reached the conclusion of arc before I did which is awesome so indeed yes arc is what we need so great we're gonna go up here and we're going to do a standard sink art yeah like this and we're gonna do our canoe here by the way this is a very common pattern you will see arc mutexes quite often the reason is is because you have new Texas for exclusive access and you have arc to be able to have multiple owners and so whenever you need multiple owners but you only want to use that thing exclusively then you need an arc mutex another way to think that if that is do you need multiple owners but you need to mutate that thing if you need to mutate it then you need a mutex because in order to mutate the thing inside of the arc you need exclusive access to it if you don't need to mutate it then you don't need a mutex in mark okay so we're still getting the error message that the reason for that is we now we move the arc into our thread at the end of here arc will will be dropped of course when we drop an arc the reference count will go down to zero and the thing will be destroyed so again we're kind of doing the same thing now but instead we're adding one to an atomic counter and then immediately decrementing it from that counter but that's not what we want what we want to do is instead of moving the same the same arc receiver onto onto our thread we want to create that arc multiple times and move those those different arcs onto each of their threats and so what we do here is we can say something like clone and then receiver this is how you create another another arc and all that happens here in this call to clone that I am right here on this arc is that it increments the the reference count and so now we have another kind of smart pointer arc it has a reference count that's just one higher than it was previously and what we need to do then is move this clone onto onto the thread just like that don't make sense all right let's see where we're at so so we finally reach the point where we've done everything right but it's still not working how do we do this what's going on and the thing when you get to this point that I find most it's just read the error message and see what it's telling you and the error message here is very clear we just haven't been reading it close enough we've been talking about multiple owners we've been talking about exclusive access which is all correct everything we've done so far is correct we've thought through everything we need to have multiple owners because we have multiple threads we don't know when our thing needs to be dropped it just needs to be dropped when all those threads are done with the thing that's a run time you know a run time thing we don't know when that will happen we need exclusive access because we saw that our receiver here is not sync which means in effect it needs exclusive access it's not able to be used in multiple threads at the same time cannot synchronize between threads and so we need that mutex it's very important but what we've failed to do is look at the function that we are wrapping itself the closure we're wrapping itself and that's exactly what our message is saying here done FN cannot be sent between threads safely this there's nothing about FM that requires that it needs to be thread safe nothing about it so how do we fix this what can we do to make this it's quite easy we just go up here and say hey this thing that we're boxing over here not only do we want it to be a closure we want it to be a thread safe closure we want it to be FN and send easy as that we've just added the extra constraint on whatever our closure happens to be that not only is it an FN closure it happens to be a thread safe FN closure and now this thing can be sent across threads and if we go ahead a check it works any questions about that so we're move now on to to how to send our work on to our channel so there was a question can we just use reference a reference to arc instead of clothing colognian no we cannot so let's let's try that real quick we're gonna try instead of cloning the arc we're going to take a reference to it instead we'll try that multiple ways I guess we can say receiver like this and what it's saying is receiver doesn't live long enough let's take a look at thee here it's much better so we're borrowing receiver right here and then we take that that borrow and we move it on to another thread so how long will this thread run for we don't know this work may last forever this work might be an infinite loop might go forever and ever and ever we have no idea we have literally no idea at compile time when we reach this line here we have no idea because we're gonna call this work and all bets are off one happens when in this situation when will receiver be dropped well the ownership of receiver is in this variable that means when this variable goes out of scope then we'll be drunk and that happens here at 25 and so we have a thread running off somewhere it has it has reference to this receiver it's just a pointer pointing to the receiver this thing will run forever who knows we have no idea when it will end but at the end of this function call that will see what's going to be dropped what is this thing pointing to then junk to point into memory that's been reclaimed and reclaimed because the thing was dropped here so taking a reference here doesn't work what we need to do instead is say is tell at runtime when this thing needs to be dropped and what we're doing by calling clone here is getting another handle to our arc here and when do when does the underlying the underlying data that's inside of our arc get dropped it's women all of the arc handles that we've cloned get dropped at the end of that so you know maybe nine of our ten threats here will run for I don't know they'll run very quickly and then the tenth one runs for 25 years only at the end of that 25 years does that handle get dropped when that handle gets dropped are through our arc count finally goes to 0 when it goes to 0 then the underlying data will finally be dropped and so really arc is a way for us to say we cannot tell at compile time when this thing needs to be dropped we need some kind of runtime mechanism to do that and by the way our see the other thing that we talked about is the same thing it's just our C happens to only work on one thread and arc works on multiple threads hopefully that makes sense any any more questions before we get to how to do execute anything else people want me to try maybe you think that the code would work if you did something else these are all wonderful questions it's really great to see to try something out I think the best thing for for learning rust is when you get an error message it's very tempting at the beginning when you're when you're trying to learn something you're trying to get something to work to just like you know throw a clone on it do whatever you know stop doing something just kidding you just want to get the code to work but once you start getting more comfortable with your rust what you need to do is take a step back when you get an error message really try to understand why you're getting that error what's really happening here what is the error actually trying to tell you but is only then will you start becoming comfortable enough to not to not write that code in the first place so there's another question here what's the difference between clone and copy in this case so clone a copy are very different and they're very similar and the best way to understand clone and copy is that clone is an explicit way to to to clone something an explicit way to create two copies of it whereas copy what it does is it gets rid of the idea of moving so you don't move when let's write an example here we'll have some function foo let's say I have a a string we'll call it s so we have some string here and then I'm gonna have another function called bar here and it takes us it takes ownership of a string so another way to think of this is bar takes ownership of of string here what does that mean when when we call bar with a string it will move into this bar function and at the end of the bar function string will be dropped because when do things get dropped when the the value that owns it goes out of scope yeah and so if we want to call bar with our string here we can do that that's fine when is s going to get dropped here it will get dropped at the end of when it gets when bar is called so we move s into bar bar gets called string gets dropped at the end of it and if we try and call bar again honest here we won't be able to do it because it says hey you moved s into bar you moved ownership into bar it's gonna be dropped at the end of the bar bar taking care of it now it will be it's taken care of at the end of the bar but you're trying to use it again the thing is the things already been cleaned up the memory that that string was holding on to it's been reclaimed it's gone that the string doesn't exist anymore and that's what the compiler is telling you here you move in right let's look at it in you know move occurs because s has type string which isn't copy which we're going to talk about in a second and we moved s into bar and then you try to use it again you can't do that so how do we get around this well okay let's have multiple versions of this thing alone and now we have we're moving the clone the copy of of this of s into bar and then we move s into bar here and of course the goal bar in we're gonna get this the same error so we keep going what we call colon again and now you have three separate strings that get destroyed with three separate times but let's take a look at something interesting if I do this here we're gonna add these error messages but let's change this to be a you eight we call this end now we gotta change these and I don't know why I changed from s and just because it feels better that totally irrelevant this compiles but wait a second we moved in into bar and then we moved it into bar again we're using it after it's been moved into the other thing why is that why does this code work but the other code didn't that's because one here is copy what that means is instead of these move semantics where we move things into into functions and then they can't be used you've used after they're moved copy means that forget all that these things will be copied automatically for you so instead of move we just automatically copy and and think about it for a number or whatever who cares it's a number like it's actually cheaper in some cases to move to copy the thing into bar than it would be to take a reference or something like that so like we're just it's fine we're just gonna copy this thing but we can't do that with strings strings owned their value on the heap and so if we were to copy them then we'd have multiple pointers to the same thing you do you know that's if you want that you you know use another language that we don't allow that and rust because when that string gets dropped who should clean up that memory on the heap we don't know there's multiple copies you know if we if we didn't know we would wrap it in an arc and there you go then it happens when the arc gets dropped enough times that the reference count goes to zero hopefully that helps with it did that answer your question you said clone and copy in this case hopefully that answered why we have to clone here we're cloning here because we are creating multiple copies we want to be explicit about it we we still want these arcs to have move semantics we just when we call clone we we get another handle but we don't want that to happen automatically for us great we have another question what's your advice for us beginners who don't code rust in their daily jobs it's tough I mean it's it's tough if you don't do it in your daily job and if you don't have free time which is totally fine if you don't have free time to do coding on top of your job don't you don't you don't have to kill yourself but rust is a is a clearly ace a more complicated language than some other languages because it cares very deeply about performance and so it needs you to be at me gives you a lot of tools to be quite productive but at the end of the day if we had a garbage collector we wouldn't have to worry about Ark because the garbage collector takes care was Swift for instance its garbage collector is Ark it wraps all its values in Ark it's not all it has some smarts about when it doesn't need to do that but but but Swift does this for us but in rust we rust is very very careful about imposing things on you and so you have to be very aware of that I mean the only way to really get a handle on that is by doing more exercises so watch my stream watch John Stream do some coding practices try and get involved in an open source project reinvent something doesn't matter if your open source project is something completely you know has been done a billion times before do it again and you'll learn a lot maybe your your way will be better than others but if it's not who cares you've learned you know the thing the biggest the best project I ever did was implement a chip eight emulator which is a very simple old emulator it's been done a million times it's basically the hello world of emulators anybody who has done an emulator has probably done a chip a dummy layer before but it was awesome I loved it did I do it better than other people no but I learned so much and so you know just go ahead and and do do what you want and have fun with it and you'll learn I hope that helps last question real quick how does how then does clone differ from cow real quick we'll talk about cow is not on our list for things that we wanted to talk about today but we'll touch on it real briefly so cow is an abstraction here as Cal stands for copy-on-write what cow allows you to do is say I don't know sometimes I might have a borrowed value a value that's borrowed and sometimes I might have a value that's owned and I want to be able to use this function with either it then you know if I own the value this function should work and if I know if I'm just borrowing the value this this function should also work let's see there example that they have here so here the what they're doing is maybe you have an owned kind of collection of AI 32s or maybe you're just borrowing this collection of AI 32s this this apps all should work for either doesn't matter if you if you own the thing or borrow the thing and so this is really just a way to abstract over that I don't care if I own it or I'm borrowing it it should work either way hope that helped cool we're going to go on real quick so this is great we started it up our thread pool let me get a drink real quick we started up our thread pool we have a sender and receiver where we can send our our closure that we've put on the heap somewhere we don't know what type of closure it is we just know we can call it that's all we know that's what this box does we know that we can call it and we know that it is safe to send across threats that's what this thing does which is great we have our receiver here wrapped in an arc and a mutex that's because we need multiple of these receivers we need to put it on ten different threads right or however many threads is down here ten different threads but when we actually read from the receiver we need exclusive access because this is a MPSC a multiple producer single consumer and so the type system ensures us by the fact that it is not sync that it can there's no synchronization happening with this type alone we need to synchronize it how do we synchronize things we put her in a mutex we're looping over zero to however many number of threats we clone our receiver arc so that we get another arc handle to it and we move that handle onto our thread and so now each thread has a handle to our arc and we're looping over and over again we're diluting loop and looping and what do we do we call lock to get an exclusive access to our receiver and we call receive on it and then we you know we have to unwrap the lock because it might be poisoned we don't care about that we call receive because the sender might not be there anymore we don't care we're just we're doing this quick and dirty we're just when we get an error we're just gonna blow the whole thing up and then we get our boxed closure that we have and we just call it awesome so our work will be done here then we're gonna loop back around and we're gonna get our lock again we're gonna block on our receiver and wait for for our next closure to be given to us from our sender of course this isn't gonna work because we get a sender here right and what happens to senator well we never use it so at the end of new here we're just gonna drop it and the channel basically goes away whenever the sender or receiver are dropped then the channel know is no longer usable right so we need to you know obviously we need to hold on to this thing so let's hold on to it I think the type for cinder is here so this is our cinder and its generic on the the type of thing that we're holding on to and we see what's holding on to right here is this holding onto our boxed closure here so this is great and we need to put that into our that's right cool okay awesome so now we're not dropping it senator will only be dropped when the thread pool itself gets dropped which you know that's its own problem but we'll come to that in a second so now we want to call execute we've got our function here we want to basically send our our work over this channel right and so pretty straightforward we can get our sender like this and then there's a send on it and it takes whatever type T it is if we do like this we'll get an error because it says I am expecting a boxed thing and they're giving me this type T here which happens to be a closure but you know I'm expecting this thing on the heap okay so let's throw in the EP we're now boxing it this is awesome okay good the weight hey type T here can't be sent between thread safely because it doesn't implement send and we said this hey this thing's got to be thread safe because we're sending it across threads down here right that's why we put this here the rest is saying hey the thing that you're calling execute with might not be thread safe and if we were to send an on threat safe closure over this we would you know put that on if we would send it to a thread and horrible things would happen or not you know which is even worse right we don't know what happened and so how do we ensure that this thing is the right safe pretty easy well this thing is not only a closure but it's also sent right so we have one last error that we're getting to I'm gonna pop this up here and you know the Russ compiler is nice enough that it actually gives us the answer to to this question but it's but we should stop instead of just blindly accepting what the the compiler is telling us we need to stop and figure out what it really means so it says the parameter type T may not live long enough it gives us some help you know you know spoiler alert to help us exactly what we want but let's understand what it means by this so the type T here is any type T we don't know what it is you know as long as it implements this closure trait and as long as it's implement send its thread safe or it's sin double across threads is a better way to say that then this then then this should work but what if type T is itself a reference to something so in other words what if we call execute down here not like this but like this you'll notice here this compiles fine this is totally fine from this perspective right because all up here that we've said is that we need a thing that implements FN and cent which by the way references to closures implement FN and this closure is send and references to something that are send or send but it's still not working why why would it not work well what happens after execute here execute immediately returns execute immediately returns and then right here foo will be dropped and we have a pointer here but what has happened to that pointer it's been put on the heap and moved over to another thread and stuff like that and what is that pointer pointing to now what is that reference pointing to it's pointing to jump memory that's been dropped a long time ago right we don't want that we don't want what we want to say is the thing that we're getting in here should it I can I want to be able to hold on to it for as long as I want to I need to hold on to it possibly for the entire rest of the running program forever and ever and ever because potentially like we may hold on to that that closure for like forever we may never let go of it and that's exactly what this is saying here this type T hey it's a it might be reference if I have any kind of lifetime associated with it which might not live long enough the thing that it's pointing to might not live long enough for that thing to always be a valid reference and so we need some way of saying no this thing is capable for living however long rep we want it to live for and how do we do that with our best friend the static lifetime and what does this do the static lifetime says the thing that I'm sitting in here not only is it a closure not only is its end but it also can live for however long you watch you you are in charge of how long it lives for it can live for this the entire static lifetime of this program it can live forever and ever and ever if you want it to another way to think about static is it has no references it's not a reference and it doesn't it doesn't have references inside of it it's not tied to any outstanding other value out there it is its own entity and it can be made to live for however long we want it to and now if we go down here it's saying hey this thing needs to be static but foo doesn't live long enough doesn't look static it lives for however long food lives for which has just tilt the end of this function here and then once that's done we better not have a reference to fill anymore because nothing's gonna be dropped and then all bets are off if you've done programming it and C or C++ then you know that this is what we've protected ourselves against here is used after free we will free the thing at the end of this function and if we were allowed to pass this thing in here we would be able to use it after we've already freed the thing and so we can just go back to not having references to our closures anymore this passive in like this we've moved our foo closure into execute now and so you know execute is allowed to keep on holding on to it for whatever it does and of course it doesn't keep on holding on to it it moves ownership on into this box and then the box gets sent over itself so on and so forth does does that make sense static lifetimes are one of the most I don't wanna say they're super confusing but they are mystifying when you first encounter them in generic rust program so this is very important to understand any questions alright and we're getting a warning here by the way the warning here is saying we're not using the result that we get from send because again send returns a result because if the receiver is dropped early then when we try and send over we get an error and so we are going to just say the receiver is always gonna be there again maybe not the best thing to do but but for now it's fine and you know what we're done right we're done here and let's prove to ourselves that we're done real quick we'll do something fun here we'll just go ahead and say print start here so before we do the work we'll will print start and but then after we finish the work we will print finish and if this truly is a thread pool what we hopefully should see is is that we'll see start start and then because they're running at the same time and I mean in order to really guarantee that we'll probably want to do something more than just quickly print something out so we can do standard thread sleep and we want to sleep for duration from sex one so will sleep for one second on this thing and I think that we should be able to do this is actually I'm not sure yeah I think cool so we're going to execute food twice here and of course we had to clone because this thing is not copy like we've talked about before okay and so hopefully what we see here is interlaced starts and ends so let's go here and run our test go now well we we've run into an error that rust that's not protected with us from and that's the fact that well actually it kind of has protected from us we are unwrapping an error because on our receive error and if we look at the line here go up here receive returns an error when the sender has been dropped and of course this wind is the sender get dropped the sender is owned by threat pool and so it will be dropped when threat pool is dropped and when is threat pool dropped threat pool is dropped right here so like we call execute execute and then we drop to drop the sender we're done except the receiver won't receive anymore just panics because the sinner has gone bye-bye so how do we keep the senator around for now mmm I think the easiest way for us to do this real quick is just we're gonna block for two seconds here so when will the senator be dropped it will be dropped on line 54 but that time two seconds after you know we've slept for two seconds and so hopefully our our threat pool has been able to run by then kind of a hack but good enough for now let's run that and we get exactly what we we hope for we get the start start finish finish and of course we're getting the same the same error that we got before which is that once the senator was dropped our receiver returned an error and we panicked it's because we didn't rat here if we want to take care of that real real quickly it's it's very very easy we just don't unwrap anymore and we just match and say match on this and if it's okay then we return back work and if it's an error then what we'll do is break from our loop and our threat will will just end [Music] and so now as soon as we get an error and we will get an error when our when our sender is dropped then we'll just break in our our worker will have a graceful shutdown then and so let's just run the test one more time start sorry finish finish no panic great awesome any questions any questions we've got a few more things to talk about some interesting tidbits this is all working great which is fantastic but it won't always work great with all the different types of closures that were were wanting to work with so we're going to talk about that but quickly all right there was one question or am I going to push this to github yes I already actually this so this particular exercise was part of my looking for here part of a workshop that I have given before and so you can find it here in github comm / rila slash rust - workshop in part 3 and here's the challenge that we have spoiler alert we're going to get to that in just a second and you can find the full solution in here so if you want to take a look it's not quite exactly the same way that we did it today but you can you can check it out cool so this all this all works really well let us try to test our implementation by adding some numbers together on different threats so I'll have one number and we'll add them together on two different threads because adding is so incredibly expensive that we need to multiplex it onto a thread pool just kidding and then we'll check on the main thread that the number has been correctly added together so how do we do this well okay we're gonna have to have a number of some sort initialized to 0 and just for our sake I'll do 32 so we have a number but of course numbers are copyright if I like put it onto a thread pool we're not gonna like be able to we needed like we need to put it somewhere so that multiple threads can write to it right if we just move it onto a thread then we're just gonna copy it to another thread and you know then we will be able to like share that memory with one another of course then the naive thing to do is to say you know and ref is is a reference to end not Linda let button ref is a reference to end here and let's get rid of this real quick and what we're going to do inside of su is do it's right to in ref add one to it but then you know that that should kind of work right it's like conceptually but there are several problems with it right one problem is the same problem that we've had before like when when will this number be dropped it will be dropped at the end of this function we have a reference to it this you know we've done our best here to sleep for two seconds we assume that that's enough time to add two numbers together but Brust doesn't know that right and maybe we're wrong maybe the thread scheduler like just really is having a bad day and it takes forever to create a thread or something like that who knows right and to what two seconds is not long enough and our thread has a reference to something that's on the stack here that will be dropped at the end of here and we're back to the same situation that we had before we're back to the situation where we have a reference to jump memory which rust just does not allow right okay so we can't have that so let's let's look at some of the problems that we have let me look at that second so the first one is we're not allowed to clone a ampersand do you 32 or rather we're not allowed to clone a ampersand mut % you 32 here so so that's not good we can change that to this and get a better error message than we had before so we're not allowed to to clone this thing you've just not allowed to clone you're not allowed to just magically create two mutable references to something that would violate a whole bunch of different things you only can only have one reference to something so so that won't work what can we do instead this this will eventually error right here let's let's move on to this this is a more interesting error that we have here so we expected a closure that implements the F N Trade oops it wants the FN trait that's what we've always said we execute will take FN but the closure implements FN mut was as FN mut think okay let's look at the docs real quick going back here so the version of the cooperator that takes a mutable receiver I don't like this since this one's much better instances of FN mut can be called repeatedly and may mute state so how does that different from our plain old FN trait that we had before FM traits maybe call them repeatedly may be called as many times as we want to call them but they may not need States and this is again some magic with with rust closures here it can tell we're trying to mutate something some state that we've captured and so what trait what closure trait does this implement for us well it just decides I need to mutate state I must be an FN mut and you know we are we are expecting just an effing right so this should be a fairly easy fix right luckily we can go up here we can say this is an FN but instead of an FN we want to be able to we it another way another thing to think about is that FN mutts are are basically celibate traits of FN and so anything that is an FN mutt is also an FN so we are strictly gaining capability here we're gaining the capability of being able to mutate state inside of our captured state inside of this disclosure we need to make that change up here and we need to make the change down here and this is now saying work needs to be declared mutable because we're doing mutate it and that's fine so can't can't borrow and it's mutable let's easy to fix this this air right here what's the air that we were get we were talking about before in does not live long enough we are passing this this ampersand but somewhere and it needs to be you know are execute here needs to be static this thing needs to be able to live for as long as possible but we're passing it a reference here like we can't do that this thing this closure is only allowed to live for as long as in is alive in will be long gone before execute allowed to return so this this just doesn't work we're just that's just not going to happen let's let's come to this out real quick so we're good now if we do that but now we need to figure out how do we get a a number that we are allowed to have on multiple threads we don't know when this number will will be well we don't know when we should drop the number but we need to have a pointer to a number or a number that we can mutate but we just don't know when to to drop that thing so one thing you might expect to be able to do is okay we need multiple owners here you know can we do this we need more formers then let's just wrap it in an arc and we should be good to go like this and the answer to that is okay it's the answer it's no the unfortunate part is that the error message that we get is not 1000% clear need to call our growth test the first thing that's saying is just there's no plus equals on on this thing and you're like okay no plus equals what about equals in ref plus and then it's like mmm no can't add an integer and ampersand much of you 32 and you're like what's going on well the reason is when we do when we dereference and ref we're dereferencing the arc so what is what are we getting right here we're getting the ampersand to you 32 so we need the reference again and we need to dereference again here and now we're gonna get the real the real answer to our question the real answer to our question is you cannot assign to data in an arc arc just does not does not implement dere effluent you can't have an arc and then reach in and modify the thing inside of an arc arc is for multiple having multiple owners where but it does not give you exclusive access so you're not allowed to you're not allowed to assume my exclusive access and what does ampersand mother at the end of the day ampersand but at the end of the day is saying I have exclusive access if you have ampersand Meadows something you can be sure that you have exclusive access to that thing but arc doesn't give you that guarantee what gives you that guarantee a mutex but we don't want to go down the mutex route with with a number come on that's ridiculous so we need a way here of modifying the thing inside of of our arc we need a way of having an exclusive access to the thing inside of the arc or shouldn't say we need exclusive access we need a way to modify the number that we're looking at without having exclusive access because arc just doesn't give that to us luckily for numbers there's a way to do that actually that's a talks with these right here are types that provide primitive shared memory communication between threads so you can primitive with primitives you can have if we look down here at atomic you 32 for instance we can look down at fetch add which is what we're gonna end up using to add and you can notice here that fetch add takes ampersand self it does not require its elusive access to add to that number why because this thing is a topic the thing is effectively exclusive access even if you have multiple pointers to it because when you actually do the addition the CPU ensures that you are the there's only one thing doing on it so in a way these atomic adds are like mutexes but guaranteed by the CPU and we won't be talking about what ordering is because it's a whole nother topic that's that's very long and very interesting and I'm not the most qualified person to answer it but now we see here what atomic you 32 is is it gives us an integer type that we can share between threads it basically allows us to have an integer that we can share between threads and modify between between threads and so we're not going to do this anymore what we're going to do is a still use standard what was it standard atomic atomic you 32 sick sorry standard sink atomic you 32 and then here an atomic you 32 then we do our zero in here so now we've now live we have this this atomic you 32 effectively we have kind of no way a mutex around our are you 32 a CPU helped mutex basically and so we can wrap that in the arc just like that and what we can do now is say in ref maybe not the best name anymore but that's fine and what was it called was fetch it had fetch and we want to add one here and we have to give an ordering which we said we weren't going to talk about today in detail with this basically is just the Garrett the level of the level of atomic nature that it is basically having to do with with can the compiler safely reorder and stuff like that we don't have to get into it but we're going to just use the most a sec cast here up what is it called no a cast without me so this is basically just if you've done any kind of multi-threaded programming with these kinds of primitives this is not Russ specific C++ says the same thing Java has the same thing so on and so forth this is just kind of the most hardcore guarantee that we can have so we don't have to think about it okay so this is great we are atomically adding one to our number and multiple clones and what do we need to do here fetch add return something fetch and returns the previous thing right so we need to just ignore that we don't care what it returns all right we don't do this and we're back to to an old friend that we've seen before we have this ere the closure may outlive the current function it borrows in ref this is going back to the same thing of where closures are trying to help well what the closure is doing is looking and saying this fetch adhere now we've looked at fetch add takes an ampersand self and so what gets captured inside of this closure just a reference just a reference here that's all that gets captured and so if we only have a reference in here then we're back to the old problem that we've had before where this end will be dropped but we have a reference to we're back to that problem again and so what we want to say is internal let's move this arc you know atomic u-32 into this closure and how do we do that with move and then you know we should call clone on that because it's not a company and so this works let's run the test and see oh we should probably instead of this they need to be lets in ref dot fetch visit we want to load the number so this will go ahead and get us the number atomically we need to pass in an ordering of sick cast here and we want to assert equal that this is and now it's complaining here that we are trying to use in ref which has been moved and of course if it has been and here we've moved it into here without cloning it so if we clone this instead and now I know it still doesn't work then we'll have to do this instead we'll have to explicitly clone it and so now we can do that and hopefully everything to work fine yep and it passes all right we're reaching the end of the time I wanted to to end just before the end of the hour I want to ask if anybody has any questions that I'm going to end with some potential exercise for you to do on your own and and then well we'll end it for the day so are there any remaining questions for this was there anything we went a little bit faster on this part we're there any questions about why this works why we need an ark why do we need an atomic what was the reason for those things anything like that all right it doesn't seem like there are any questions just to end for the day we moved from it it seems there was a previous question can you repeat the question I might chat refreshed so I don't have access to it sorry about that yes I will post this code and the gist it's also also in here on on github github calm right left rest workshop but I'll go ahead and copy that into it just now oops do it down here and and can send that into the chat as well so going back to our to our exercise real quick we we change from FN death and month because we wanted the ability to to mutate inside of our inside of the the closures that we that we accept there's one thing that we're not allowed to do right now which is effectively take full ownership over over values and allow them to be destroyed the reason for that is because FN mut and FN can be called multiple times and so if you were to move something into a closure and then drop it basically if the closure were to be able to call be called multiple times it would that value would be dropped multiple times and you're not allowed to drop a value more than once and so there's an additional the last closure trade is FN once what I would try to do is come up with an example that not compiled because you are you are dropping something inside of that closure that you're not allowed to you're capturing something in from your environment and then dropping it into closure that will change the closure to implement FN once instead of FN mutter FN and then make sure that you get your code to compile to take FN once instead so a little bit of an exercise for you so some last questions here why are you using in ref at the end and not and the reason down here that we're using in ref and nut n is can't use it why can't we use in let's do in has been moved why has it been moved it's been moved into the arc so at the end of the day mark needs to own the data that wraps because arc is going to be responsible for dropping it when it's reference count goes to zero if that one of the case and arc dropped it's it's data when when it's reference count went to zero but we could still use in here that means that we could potentially use this thing after it's been freed already after it's been dropped already and that's just not allowed that would be you know could lead to things like used after free and and double free and stuff like that so a lot of a lot of problems would be caused by that so that's why we're using in ref here instead I mean if I were writing this code for real I probably would write it like this instead of just inline it because there's no need to have n in its own variable there's you don't really gain anything from that so how do you feel about unwrapped in panics in general especially the library author I assume you should yeah so delight the this library you've this wonderful library or uses some unwraps I think it's fine to use unwraps when you're you know we're we're trying to get something to work we are we can't be bothered to handle all of the different error cases that we might run into so using unwraps is not necessarily a horrible thing but yeah if you're you know once you get to the point where you want to publish this thing to crates that I oh it's not necessarily a bad idea to try to go through your codebase and see if you can get rid of the unwraps in the pics the panics and expects and all the things that will cause your your library to crash libraries in my opinion should almost never have any panics in them unless they are really just to ensure like safety properties or something has gone very very wrong because again you can't can't really recover from panics in your control flow you but if you want the user of your library to be able to handle an error at all you should return a result instead so let's see here and as question how would one go about removing unwraps I think it's important so we removed one I'm wrapping this program right we removed the unwrap right here and the way that we did that was we went to the received documentation so if we go to receiver and then we go to receive here and we tried to figure out like why will this thing return in error and we can read in the documentation that if the corresponding sender to this receiver has disconnected or it disconnects while this call is blocking then this call will wake up and return an error so basically if we get an error if the sender is going if the sender has been you know lost somewhere it's been dropped or whatever and so what we did is say well what should we do when the sender is no longer there well it makes sense if the center is no longer there we're not going to get any more work so what we should do instead is we should just end our our handle here and how do we do that we just break out of our loop and so you what you want to do is go ahead and look at your code and understand what the error cases are and handle them in the way that makes sense for your for your domain whatever it might be all right I want to thank everybody for a great stream there's been a lot of fun I hope you enjoyed it I hope you found it educational and entertaining and hopefully we can do this again please real quick a shameless plug for you to follow me on the Twitter's I mean if I can get to it Twitter comm / Ryan MC Levesque that's where you can find me tweeting about mostly rustling stuff give me a follow there let me know what you thought of the the stream what you would improve and and hopefully we'll see you again another stream in the future thanks a bunch everybody see you around

Channel: Ryan Levick

Views: 25,643

Rating: 4.9957852 out of 5

Keywords: rust, programming

Id: 2mwwYbBRJSo

Length: 117min 38sec (7058 seconds)

Published: Fri Apr 24 2020

Reddit Comments

[u/sroelants]

Great to see so many rustaceans taking note of the survey results and creating new content for the intermediate learners!
Subscribed!

[u/GaianNeuron]

As a dev experienced in C# but new to Rust, this was exactly what I needed. Well worth the watch.