Python Threads - MultiThreading in Python and Python GIL - Python MultiProcessing

probably in no other programming language multithreading is so much talked about often confusing and even a controversial topic as it is there in python i believe you must have heard about terms like there are no multithreading in python there is something called gil which is global interpreter lock-in python and because of that multi-threading is not possible well let's talk about each and everything related to multi-threading in python in this video and after this video you will clear all your confusions about multi-threading in python programming language so let's go ahead and start [Music] so welcome everyone to my channel and in this video i am gonna cover these five topics first i will prove you that python threads are real threads what is the meaning of real thread real thread means that for every python thread you create a kernel label thread each created we will see that we will also see how we can create threads in python we will talk about the famous or infamous gil global interpreter lock we will talk about multi-processing in python in brief because this video is not about that but i will cover multiprocessing to tell you what you can do with that and then at the end we will talk about what is the object-oriented way of creating python threads okay so to use python thread you need to import threading and from threading you need to import thread okay this time i am importing so that i can put some sleep in the functions this is not required for using the threads but just to show you something and i am importing threading because i am using some functions of threading which we are going to see in a moment now we can create a thread by using this building thread class how let's see but before we go ahead and create a thread you all know that each thread starts its execution with a function so we must create a function which can be a entry point function for our thread so let's go ahead and create a function called thread function i named it as thread func i am printing a statement saying that starting the thread function i am sleeping for five seconds and i am printing thread function completed okay now if you recall my first topic where i said that i am gonna prove you that python threads are real threads okay so before going ahead and creating a thread we must check how much threads are already there in this particular jupiter notebook handled by python we can use built-in function from trading called active count and if we just see the output of this it says that at this moment of time there are five threads which are handled by this particular jupiter notebook okay we can see the similar information in the top command also let's go ahead and see that so what i'm gonna do is that i will first take the pid of uh jupiter notebook so at this moment of time there is only one kernel run time and you can see this is the kernel runtime and the pid is 5079. well i will say top minus p five zero seven nine okay i will change the interval to zero point five and you know that if i press shift h it will tell me number of threads you can see that number of threads are total in nine over here but here it is showing five why there is a difference though it doesn't matter but the difference is because this talks about the complete jupiter notebook and the kernel this talks about only the python part of it so python part five threads overall nine threads okay so at this moment of time there are nine threads so let's go ahead and create a thread okay to create a thread we will pass the thread function as a constructor parameter so i am calling the class thread and i am passing target as thread function there are other parameters but let's not worry about that because in this video we want to understand the concept of threads okay not the different parameters i will create different video for that so i am creating a thread okay now creation of thread doesn't mean thread is started you need to explicitly start the thread and once you start the thread and go ahead and see the active count it will show you six what was it earlier five now it is showing six when the thread function is completed the active count will be five again now let's see the same thing in the top command so i'm creating a thread okay i'm not starting now in the top command i have total number of threads as nine okay let me start this and you can see that total number of threads are now 10 and as soon as this exit happens total number of thread goes back to nine so this thing is reflected in the operating system so this means python threads are real threads which means you can create threads in python okay confusion clear anyone talks about threads in python yes you can create threads in python no question about that and each python thread corresponds to a kernel level thread okay as we have seen in top command you can check anywhere every python thread will create a kernel thread okay now let's talk about some of the logistics similar to other threads in other programming language you can join you can wait for thread to complete by calling a join so in here i create a thread i can say start and i can say join this cell will not be completed until then this thread function is completed okay we can do that again we can see that you know here star will be there which means this will keep on running till thread function is completed so join will wait for that particular thread to complete and this is the way you prevent your main program from exiting okay [Music] now let's talk about the famous or infamous global interpreter log now before i talk about gil let me clarify that i am talking about the official python version called c python okay gil might not be there in other python implementation okay so let's talk about gil so just wanted to read something from the c python documentation [Music] in due to global interpreted law only one thread can execute a python code at once only one thread can execute the python thread at once so gil is not saying that you can create only one thread gil is saying that you can execute only one thread there is a difference you can create multiple threads but you can execute only one thread let's see the same thing in action now to visualize this concept i had to create cpu intensive jobs okay not the network jobs or file reading jobs where multiple things can happen simultaneously i have created cpu intensive jobs which means that i created a variable called kill thread which is false and see my function i have created a working thread in the while true loop if kill thread is true then return otherwise you keep on looping in this particular thread function so this thread function will run as fast as possible okay so let's create multiple threads two threads and see what happens now before doing that let's uh see that whether indeed my kill thread function is working now active count is five let me start okay now if i go ahead and see top command you can see 10 threads and if i go ahead and say kill said equal to true then you will get back to 9th that means this logic is working if i change it in main thread the another thread will exit now what i have shown you over here i have changed kill thread the global variable in main thread not in another thread which means two threads are working okay things will be more interesting now let's create two threads okay instead of one let's reset the value to false create thread one thread two start both the threads okay now in the top right now number of threads are nine if i create one more thread in top it will be 10 and if i do active count it will be 6 over here i will start another one now in top it will be 11 and active count will be 7 okay so things are working we have created multiple threads now this is the place where gil comes into picture and this is the place where it is different from other programming languages okay let's see how it is now you can see that the cpu usage of these two threads to 100 okay just see i have uh press one to show you all the cores but if i can show you the cpu it's 50 50 so at any point of time cpu usage don't go beyond hundred percent in my system right now i have 12 cores which means the cpu usage can go till 1200 100 means one core okay but you can see over here that the cpu usage remains 100 it doesn't go beyond 100 okay now let me stop the both threads by doing kill threads and now nine threads are there now we have seen that in the case of python i created two threads i run a while true loop in that both the threads the cpu usage is still hundred percent but what happens if i create two threads in a programming language where there is no concept of global interpreter law well so let's go ahead and talk about c plus plus programming language and in here i have created a file called thread2.cpp so in here i have created two thread same thing exactly same no difference the code which i have written in python the same code i have written over here just that i am not using any variable okay so let me go ahead and compile this code and run it okay i am running a dot out okay now in here let me go back and see the pid of c plus plus program the pid is 26722 let's see this top minus p 2 6 7 2 change the duration to 0.5 and if i do shift edge you know there are two threads right now working and you can see that cpu usually is 200 percent approximately you can see that two cores are occupied cp their cores are changing but at any point of time two goals are occupied which means both posts are working simultaneously in case of python with multiple threads half of the time first thread was running half of the time second thread was running but overall combined strength of both the threads never crossed hundred percent so in python if i end up getting 12 threads cpu usage will remain 100 in c plus plus if i create 12 threads cpu usage will go 1200 percent i hope and believe that you understand what global interpreter log touch okay so global interpreter log let you run multiple threads but at any point of time only one thread will be working so in case of python if you are creating multiple threads cpu intensive jobs will not have any impact but if there are other jobs like you know network reading file reading i mean user input micro services connecting to a web server and getting some data back in those cases cpu usage is not 100 by each of the thread because it has to wait for network input disk input and those things are slower as compared to cpu which means cpu will be free and which means if you create multiple threads it will be helpful for you to run your code faster in python now if you really want to understand think of this as you know before multi-core cpus came into picture how your multi-threaded applications was working it will work in a same way in python multi-threading okay so even if you create multiple threads in python it will treat your cpu as a single call cpu that's all okay and that's why if you really want to use multiple cores you need to use multi-processing not multi-threading and remember this is helpful only if you are using cpu intensive jobs like you know our calculation um secret key calculation hash calculation and so on and so forth just a very simple example of multiprocessing you import multiprocessing and you import process from multi-processing in case of thread we imported threading thread from threading in case of multiprocessing we will import process from multi-processing the apis are similar we can create a process function similar to that of our threading functions now i am going to show you one demo but i remember this warning never write any multiprocessing code like this okay so i'm creating two process proc and proc 2 i'm starting first process okay i'm starting second process and now if i go ahead and see the top command sorry let me open for the jupyter notebook itself um it will not come in thread thread will be nine but if you see the cpu there will be two cpu two cores will be utilized hundred percent which was not there which was not possible with single python application or multi-threaded python application okay now if i go ahead and terminate both process we will close all the block handles so that you know handle memory is reduced and things are closed properly now we will talk about multiprocessing later on but let's come to the last topic of this video is all about how to create object oriented threads or object oriented way of creating threads right so what you can do is that you can create your class derived from thread and you can create a run function okay so this is my thread and currently again active count is 5 i can create the thread my thread and if i call start so remember whether you are calling a thread directly or you know deriving the thread to create your own class to start a thread you need to call the start function what start function will do is that it will call the run function which you have created in your class so in here i will start the thread at active count is 6 wait for 5 seconds the thread should exit and active count will be five okay just like raw threads we can actually join also you can see a star over here it will wait for five seconds and then this star will go off and active count will remain five so this was the basic introduction about multi-threading in python we will talk more about multi-threading in upcoming videos thank you everyone thanks for watching we will meet again until the next time we meet good day goodbye you take care

Channel: Code Sports

Views: 1,725

Rating: 4.8367348 out of 5

Keywords: Multithreading In Python, Python Multithreading, Python Tutorial For Beginners, python infamous GIL, Global interpreter Lock, GIL in Python, Multi Threading in Python, python gil, python tutorial, multiprocessing in python, python multiprocessing, python threads are real threads, python multithreading and multiprocessing, using threads in python, CodesBay, Software Development, Programming, Machine Learning, Cloud Computing, Database, codesports.ai, code sports

Id: BhnB45Rf3dg

Length: 16min 51sec (1011 seconds)

Published: Thu Apr 15 2021