C# Async Programming - Part 1: Conceptual Background

[Music] [Applause] [Music] [Applause] [Music] okay today's topic i changed the course calendar a little bit because last time i saw that you are pretty good in the basics of c sharp so i decided that the topic of pattern matching is something that we don't need to discuss together in school i think this is too simple for you you are good enough you are well equipped your base knowledge is is really good and i will give you in the next few weeks uh one of the homeworks will be to watch one of my existing videos about pattern matching so you will learn about pattern matching but probably you just take a look at the video you understand it and that's all so i discussed two i i decided to change the course calendar a little bit and focus today's lesson on asynchronous programming let me uh describe what the whole goal of this stuff is for me we are working towards entity framework accessing databases using object-relational mappers and when it comes to entity framework it's really important to understand two base technologies one of them is link we will do that next week or we'll start that next week it will be a longer journey because link is a pretty complex and powerful matter and the second thing is asynchronous programming those base technologies you have to master in order to get started with entity framework and my experience is that both base technologies are not a given in a fourth class here in the school or in hotels in general so therefore i would like to focus today's lesson on asynchronous programming okay good we will do some theory i will do a little bit of drawing on the blackboard conceptual drawing so that you understand what asynchronous programming is and why it's important and then we will spend approximately two hours just writing code writing simple code with which i can demonstrate how you work in c-sharp with asynchronous programming especially when accessing the network and when accessing files and databases good fire let's do that let's first do a little bit of conceptual drawing imagine that you have a computer with a cpu has let's say two processors this is my cpu one this is my cpu tube now if you start a program in windows and linux whatever you are starting in reality you're starting a process i think you know that you know what a process is i don't need to go into details if i use any term that you don't know please interrupt me inside of the process a so-called thread is started first question to you what is the difference between a process and a thread what are characteristic differences between these two things yes okay okay i'm fine with that if you start two of your net programs in two different processes one program cannot access the memory of the second process so we have a kind of process isolation boundary with threads it's different there is something which is called threat local storage yes that's true but keeping that aside if you have multiple threads inside of a process each thread can access all the memory of the entire process so you can create a list of something if you have three threads these threads can all access the same list of something for instance that's a big difference good i agree i like that next one how many processes and threads can run on your computer are you limited with the number of cpus well kind of obvious no you are not what's the magic if you have 20 processes running on a computer with two cpus who is responsible for running all these 20 processes yes exactly the operating system scheduler you are perfectly right the processes are distributed across all the cpus and if they they compete for all the cpus okay so imagine the following situation you are writing a c-sharp application that does some heavy calculation maybe you are doing some rendering which is cpu bound maybe you have a game which doesn't use the gpu the graphical processing unit but the cpu or maybe you do some mathematical calculation think of an insurance company who needs to do some some simulation work for insurance programs for instance something like this if you do that you typically split up your work into multiple independent units you write the c-sharp program and you say okay in one cpu i will calculate the upper half of the image and in the other cpu i'll calculate the lower part of the image so you distribute the calculation problem into multiple parts and you run them in parallel that's the point so what we have we have for a certain amount of time here in the cpu one running a certain thread let's call it the thread one and at the same time you have on the cpu two running another thread which is called the thread two in this case for instance and at the end you are maybe getting the resulting data you are transferring it here and here you do the final processing maybe putting these images together or whatever calculation you do understand this is called parallel programming parallel programming means that you have some cpu bound stuff which you would like to do really in parallel at the same point in time multiple calculations are going on in your cpu and modern cpus even your phones they have a lot of these cpus and therefore a lot of things can happen at the same time but the point is if you do parallel programming in c sharp you have multiple threads and these threads are executed on your cpu understand this is not our core focus today you have to have a clear picture in your mind that there is a huge distinction between parallel programming and asynchronous programming for parallel programming you need multiple cpu cores for async programming you do not need multiple cpu cores let's discuss asynchronous programming parallel programming is difficult it's a complex matter and we will not touch it in this course or at least touch it only a little bit we'll see good now let's simulate a computer which only has a single core do you think that in practice this is relevant do you think that as a developer as a professional developer nowadays you have to deal with process with computers that i only have a single core okay microcontrollers you mean iot stuff things like that okay i um i can tell you all of you you will have to deal a lot with computers who have a single core even if you do really hard business stuff if you do web development in large cloud data centers many if not most of the computers have a single core you know why because they use virtual machines and they only dedicate a single core to a single virtual machine and from your point of view you have a single core but of course the machine in the background has many cores doesn't of course in for instance microsoft azure the biggest machines have hundreds of cores so they can run hundreds of virtual machines at the same time on the same box but from the point of view of a single virtual machine you only have a single core that's not the exception that's in many cases the rule of course you have many many many of those small machines so you do you do parallel programming but you do it in a cluster in a server form but from the point of view of a single computer of a single program you often have to deal with a single core not in these virtual machines they are really single core you only have a single core you can only do a single calculation at one point in time yeah it could happen that they have hyper threading but it's uh below your virtual machine you don't see that okay question i also think uh cpus usually don't have that operating anyways it depends yes okay thank you thank you for this comment yeah yeah thank you for this comment yeah good so this is our cpu now let's discuss async programming imagine the following we are implementing a ui application maybe in wpf or winforms or something like this okay so if we start a certain program here let's say this is our ui we are not on the server side yet we will discuss the server side in a second now we are on the desktop a simple windows machine with with simulated one core okay this is maybe the startup of our wpf app it draws the main window on the screen maybe it starts i don't know rendering a beautiful cat image or whatever you want to do it's just the ui and at one point in time here this application is idle it's waiting for the user to do something okay maybe at one point in time we have here we have a a mouse the user moves the mouse and when the user moves the mouse the app becomes active again handles the mouse move or the keyboard input or whatever the user does and then it's getting idle again so at one point in time maybe here your application decides that it would like to go to the disk maybe to a database could also be a network application so you want to call a network service something like this and you discuss that you do exactly that so what you do here is you are running a little bit of code and this code is doing a database query or reading a file or accessing the network i don't care let's stick with the database programming you did i think oracle right last year i think so imagine that you're running complex query against an oracle database at that point in time okay now this this query here will take a certain amount of time okay let's say this query will be processed in the database in that amount of time and now we are talking async programming if you do not do async programming your thread here will actively wait it will block the cpu it will not be busy it will not do a calculation but it will simply sit there and if you compare it with with a with an alarm clock it sits and stares at the clock at the clock and does nothing until the time has elapsed it sits here doing nothing until the database will return the result of the query and now in that area here we have a big problem in that area we are blocking the cpu but we have the user interface which is also a thread and therefore the application does not react on mouse moves for instance the application is frozen for the entire time so that is not a good idea we shouldn't do that if we do if we don't do async programming that will result in very bad user experience same is true for the server we will discuss the server in a second okay so whoa so this is not a good idea what we did [Music] here else can we do well we can run a little bit of code here only a tiny bit of code which issues the database query and now something interesting happens now we are doing async programming async processing that means that we let the database do its job in the background something else is busy not the cpu the disk subsystem is busy the oracle database is busy the network is busy but from the point of view of the cpu we are free it's no longer necessary to block the cpu core it's like an alarm clock we do not watch the uh the clock until time has elapsed we just set an alarm timer and then we can do something else and when the time has elapsed we are going to be informed actively so at one point in time the oracle database will tell us will tell us that it is finished and that will trigger another piece of asynchronous code where we can do the processing of the results of the database maybe we render them on the ui maybe we display a nice chart of the data on the ui and now the important thing is that here our cpu our ui thread is free understand what i mean we only have a single core we cannot do parallel calculations but if we do tasks which are not cpu bound we can't do them in parallel because of asynchronous programming they will not limit the use of the cpu understand what i mean that's super important of course in practice you sometimes use parallel programming sometimes use asynchronous programming you can just do both in c sharp c sharp is really really powerful when it comes to parallel programming can you remember a programming language that you have learned about which does if you don't apply dirty tricks which does not support parallel programming at all but is great in asynchronous programming exactly javascript is single core single grade if you don't apply dirty tricks javascript will always run on a single thread full stop but javascript is one of the best languages when languages when it comes to async programming so c sharp is a little bit different c sharp supply supports both parallel and asynchronous programming questions so far or is that clear so that was a ui an interactive ui now let's get rid of this stuff and think about how it works on the server let's talk about the server so now we have a server let's say with two cpus cpu one cpu two here we have a client computer my case is a laptop but it can also be a phone whatever anything that has a connection wire http anything that has a connection via http to our web server so we are running a web server here uh on our computer the web server currently doesn't have any requests so it's completely idle maybe at the beginning the web server uses a little bit of time for startup starting up the web server setting up any kind of network listeners whatever it does and then it's idle nothing happens my question is what happens if the user sends us an http request http honestly let's say http get i would like to see the awesome cat video so it doesn't http get for any kind of video okay what happens what happens in the web server do we have any idea yeah who is the server the server is a process does the server run multiple threads or is the server a single process a single thread single process maybe it can run multiple processes but let's let's make it simple let's make a simple thought experiment let's say you are writing a c-sharp application which has the web server built in which is typical nowadays so therefore you are running a single process and what about threads how many how many no that's that's not true what happens in modern web servers at least in c-sharp that's the case these web servers they have a pool of threads that's the so-called thread pool and by default i hope that's still the case at least in the past it was the case and i'm pretty sure it's still the case by default the number of threads in the thread pool are equal to the number of cpu cores so what it does here at the beginning when it starts the application it starts a number of threads in the thread pool why doesn't it start a thousand threads what do you think it could start a thousand threads why only two yes it takes a lot of memory and it takes time to start threads threads are pretty heavy weight every thread uses um a pretty decent amount of memory we are talking in the megabytes here not just a few kilobytes or bytes we are talking about megabytes so creating thousands or tens of thousands of threads it's really a problem if you start on a typical.net web server five thousand ten thousand threads it really depends whether you can start them because this thread pool has a limit this thread pull is configurable but there is a limit to what a machine can handle so the thread pull should not and cannot grow indefinitely typically you try to keep the thread pool small not too small it can outgrow the number of cpus over time but typically you try to keep the number of threads small in order to keep your web server efficient otherwise you have to put in a lot of memory okay good so in this case we have two threads each http request is assigned an an idle thread in the thread pool so when an http request comes in a thread maybe this one is currently free thread one is assigned let's write it here with the http request okay so this is our thread one it's currently running on cpu2 and it was taken from the thread pool you see it is no longer in the pool it's no longer it's no longer free and it's now dedicated to this http get request now this http get request does a little bit of processing it's parsing the incoming request and now it finds out that it needs to get the funny cat video from disk and now i think you get the idea because if we draw the disk here and let's say reading the cat video will take a little while if we do if we don't do async programming then our thread will be blocked for quite a long time and at the end we are sending a result back to our client the problem is that we do not just have one client we have multiple clients so if we draw here um maybe here another client this time it will be a phone and this phone will send another http request okay we have a second thread so we can do this one we can do this one something like this it will also access our beautiful database it will take a while and it can send back an http response to our client on the left hand side but now we have a problem what if we have a third client and a fourth and a fifth what if we have hundreds of http requests coming in because our cat video is so hilarious that everybody wants to see it then we have a problem because what the system will do in the background it will create once the threads are gone it will create additional threads t3 t4 t5 it will create additional threads and try to schedule those threads on the cpus but we only have limited cpus you get the idea so these threads they keep waiting and waiting and waiting the cpus get overloaded they are idle they're not doing anything useful in that time these threads they are just blocking the cpu they are not giving it away that the operating system will recognize that these threads are doing nothing therefore it will change its scheduler algorithm and it will create as i said more and more and more and more or more threads but that's not good because every thread that the operating system creates uses a lot of resources understand what i mean that's not a good idea classically dot net was known to be a very an inefficient web server you know why because of us.net developers because we always programmed in.net classically synchronous not asynchronous we programmed like that the javascript the node.js guys they only had a single thread they had to do asynchronous processing so node.js was way more efficient when it comes to web servers understand what i mean that's a problem so we have to change the way of programming we have to change our approach to asynchronous programming even on the server so this here is our first cpu this is our second cpu and http get comes in this is our thread one this is our thread two so here we have a thread processing the incoming http get here we have our nice little database and now comes the important part we are issuing the io operation on the database or on the file system or whatever we do and then we hand back the thread one so this is thread one it's only locked for a very small time and once we are done here the thread gets back to the thread pool so when the second here this one is issuing the request here it can go again to thread one if it wants so again we are doing a little bit of processing from the incoming request and we are accessing the database get the idea so this is something like this then the result of the first database query of this one this is one this is the second database query now a result comes back here which belongs to this http request so we are processing this is number three we are processing the result of the first http request and we can send back the result to this one and then the second i operation comes back and we can process here the second io operation and send back the result to the second client and if you take a look now we only need a single thread and we can process multiple http requests at the same time we are not doing parallel programming here we are doing asynchronous programming here and the important thing is that asynchronous programming is always used when you do any kind of non-cpu bound work so if you access a database a file server a network whatever and if you program like that your web servers will really be efficient and they will scale very well they can handle much more requests than if you do locked programming not asynchronous programming locking threads things like that you have a question the parallel programming comes in automatically when it comes to web server programming because you don't care you as a developer you don't do parallel programming the web server is distributing all the load coming in automatically by using the threads and assigning the threads dynamically to cpus so from the point of view of a web server developer you do not need to program in parallel explicitly because every http request is scheduled automatically parallel programming is really typically used if you have to artificially break a larger problem into multiple pieces and let the cpus work on these pieces independently so whenever you do some heavy processing calculation stuff then you do parallel programming but in web server development and this is primarily what we are focusing here in this course you don't need to do parallel programming on your own it's done by the web server not with node.js node.js is different we told i told you javascript single threaded not multiplied this is the reason why in the cloud so many web servers have a single thread because if you need more threads you should spin up additional servers do you already know docker yeah you know docker yes no maybe not really do you have an idea what docker is kind of lightweight virtual machine the beauty of docker is that you can very quickly spin up new darker containers like a virtual machine so therefore if you have a web server that is handling a certain amount of of load in the cloud for instance and you get more clients coming in watching your cat video you can like that spin up additional machines docker containers more and more and more and more so you are not just using parallel programming on a single machine but you are spinning up machines just like that you are creating another one two three ten 20 100 web servers and they will handle the load and then when the load goes down you throw away the machines that's a question of milliseconds that's the beauty of cloud computing you scale automatically from the point of view of a single server there is no multi-threading but from an overall view there is multi-threading because you have multiple servers okay got it got the difference between parallel programming and async programming very nice i just now hear the clock so we nailed it from a point of view of time management that's very nice that was the theory part okay in the next lesson we are switching gears going to visual studio and i will show you how to use asynchronous programming in c sharp okay thank you five minutes break

Channel: Rainer Stropek

Views: 15,009

Rating: 4.9341021 out of 5

Keywords: C#, CSharp, .NET, .NET Core, Parallel Programming, Async, Async Programming, async await, Tasks, HTL Leonding

Id: FIZVKteEFyk

Length: 29min 48sec (1788 seconds)

Published: Thu Oct 08 2020