Intermediate Python Programming Course

Hey guys, welcome to this advanced Python course. My name is Patrick and I create free tutorials about Python and machine learning. In this course, I teach you all the advanced topics that bring your Python skills to the next level. So who is this course for this course is aimed at an intermediate skill level, you should already have some basic Python knowledge. For example, if you just completed a beginner course and are looking for the next step, then this tutorial is perfect for you. And even if you are already in the intermediate level, you can benefit from this because I share some helpful tips along the way. And we really go into detail of all the different topics. So here's an overview of what we will cover today, the course is splitted into 21 sections. And in my opinion, every experienced Python programmer should know about these topics. Alright, let's start. list is a collection data type that is ordered mutable, and that allows duplicate elements. So let's have a closer look at lists and what you can do with them. First of all, a list is created with square brackets. And within these brackets, you put each element that you want separated by a comma. For example, let's put some strings in here, banana, cherry, and an apple. And if we print this, then we see that all elements are printed, we can also create a new empty list with the list function. So my list two equals list. And if we print this, then we see that this creates an empty list. And later on, you can append items. List allows for different data types. So for example, we can say that our list can contain an integer, a Boolean, and a string. That's all possible. And the list allows duplicate elements. So if we put in another apple here, that we then we see that we have two apples now inside our list. Now if you want to access an element, you do that by referring to the index. So let's say item equals my lists, and then inside brackets, you specify the index, and note that the indices start at zero. So index zero is the first the very first item in this case the banana. And if we print the item, then we see this is the banana. And index number one is the cherry. index number two is the apple. And now if we put in an index that is too large, what will happen, then we get an index error list index out of range. So be very careful with that. Now you can also specify a negative index, so minus one, this refers to the last item, in this case, the apple minus two is the second last item, and so on. Now if you want to iterate over your list, you can do that simply with a for loop for i in my list colon and then do something this crazy, just want to print it. So then we see that for each element inside our list, we print it and note that you don't have to call this I you can call this also x or whatever you want. Now if you want to check if an item is inside our your list, you can do it with an IF and then your item that you want to check, say banana in my lists, colon and say let's print. Yes. Else. Print. No. Now if you run this then we see that the banana is inside. All this let's check if the lemon is no the apple yes So that's a very simple syntax to check if your index is inside your list. Now let's talk about some other useful methods that you can do with the list. First of all, if you want to check how many elements Do you have inside your list, you can do that with the Lang method. And now if we print this, and we see that we have three elements inside our list. Now if we want to append items, we can do that by my list dot append. And now let's append a lemon. Now, print that, we see that a new item the lemon, got inserted at the very last at the end of the list. Now if we want to insert an item at a specific position, we can do that with that insert methods. And now First, we have to specify the index let's say at index number one, and then the item, say a blueberry, and then print it. And we see that at index number one, now we have the blueberry. If you want to remove items, we can do that with the pop method. And this returns the last item and also removes it. So if we assign this to a variable, and print it. And we see that now we got our apple back. And if we print our list, then we see that the app is no longer in our list. Now we can also remove a specific element with the dot remove method. For example, let's remove that. Sherry then we see that the cherry got removed. Now what happens if we specify an item that is not inside our lists, for example, if we have a typo here, then we get a value error. It's not in list. So be careful here. We can also remove all elements with the clear method. So now we have an empty list. Some more useful things that you can do is for example, you can reverse the list with the reverse method. So now the list is in reversed order. And you can also sort your list with the sort method. Maybe this, for example gets clearer for us numbers here also, let's say 4231 minus one, minus 510. And if we start this, then we see that it's now in ascending order. So note that this sort methods, sorts your list in place. So this changes your original list. And if you don't want to have this change, but rather create a new list, then you can do this with the built in sorted method, new list equals sartet, and then your original list and note that if you print your original list again, then you see this is still the same. And if you print a new list, then you see that this is now the new sorted list. Now, some useful trick if you want to create a new list with the same elements multiple times. So for example, I want a new list with five zeros in it, then you can do it like this. So let's say it's put in a zero, and then times five. So if we print this, then we see that we have a new list with five zeros. And we can easily conquer two lists with the plus operator. So let's say I have Another list equals to 12345. And then we want a new list. So a new list equals my list plus my list too. And if we print this, then you your fun new lifts with both elements with both lists inside it. So, yeah, let's talk about slicing, slicing is a very nice way to access sub parts of your list with the colon. So for example, let's create a new list with some numbers 12345689. And let's create a new list and simply call it a and then inside brackets, the A equals and then my list. And inside the brackets, you specify this start index and the stop index. So let's say for example, start index one and stop index five. And if we print this, and we see we have a new list that goes from index one, to index five, and the last item, the last index is excluded. So it said index 123, and index four. So our list has all the numbers from two to five. Now, if we don't specify a start index, then it starts all the way from the beginning. And if we don't specify a stop index, then it goes all the way to the end. And you also have an optional step index. So and then I put in another colon, and then the step index, and by default, it's one. So let's say, this goes all the way from the beginning to the end with a step one. And if I put in a step two, then it takes every second item. And I can also specify a negative index. So this is a SIM, a nice trick to reverse your list. Now let's talk about copying a list. So let's say let's call this list original, and put in some fruits in here, banana, Sherry, and an apple. Now, if I want to create a copy, and I simply do it by assigning it to the original one, then you have to be very careful. So if I print the copy, the nice see that it, it's the same as our original list. But now if I if I modify the copy, what will happen is that it will also modify the original list. So for example, if I append a lemon, and if I print the copy, and I also print the original, then we see that the original list now also has a lemon in it. And this is because with this assignment, both lists refer to the same list in inside the memory. So yeah, be very careful here. And if you want to make an actual copy of your list, you can do it with the dot copy method. So now if we print them, we see that the original method, the original list is still the same. We can also do it with the list function, and as argument we use the original list. So this also makes an actual copy. And as third option we can use slicing, if I just use a colon here, so this means slicing all the way from beginning to the end. And this also makes an actual copy. Now, as a last nice trick, I want to show you an advanced technique that is called list comprehension. So that's an elegant and fast way to create a new list from an existing list with one line. For example, if we have a list with numbers 123456, and we want to create a list with squared numbers, and we can do it like this, inside brackets, we say, i times i, for i in a, or maybe let's call this my list. Now if we print my list and print the second list, then we see that a new list got created where each element is squared. And the syntax is you have your expression, and then a four in loop over your list. So note like the same with iterating, you don't have to call this AI, you can also call this x. And then also use the x here. So that's a very simple and elegant way to create a new list with another with some expression in one line. tuple is a collection data type that is ordered and immutable. It is similar to a list with a main difference to the tuple cannot be changed after its creation. a tuple is often used for objects that belong together. And let's have a closer look at tuples. And what you can do with them. First of all, a tuple is created with parentheses. And within these parentheses, you put each element that you want separated by a comma. So for example, let's put in Max, 2008, and Boston. And if we print this, we see each all the elements inside our tuple. Now the parentheses are optional. So we can leave them away. And it's still a tuple. One special thing is if you just want to have one element inside your tuple. And even if you put it in parentheses, and you write it like this, then this is not recognized as a tuple. So if we have a look at the type of this, then this is recognized as a string. So what you have to do, then you have to put a comma at the end, even if it may look strange. That's the right syntax. So now it's recognized as a tuple. You can also use the built in tuple function to create a tuple from an iterable. For example from a list, so say Max 28. Awesome. If you print this, then we also have our tuple created. Now if we want to access elements, we just do that by referring to the index. So if we say item equals my tuple, and then inside brackets, we specify the index that we want, and the indices start at zero. So index zero gives us the very first item, we print this, then we see we have Mac's index one, we get 28 index two, we get Boston, and if you use an index that is too large, we get an index eiroa index out of range. So be careful here. We can also specify a negative index minus one refers to the very last item. So that's Boston, in this case, minus two is the second last item, and so on. Now, what happens if we want to change the elements inside our tuple like with lists week, if we write my tuple and then get the first index and assign it to a new value, like Tim, and if we run this, then we get a type error object does not support item assignment. So this is not possible because a tuple is immutable. Now we can easily iterate Over a tuple with the for in loop. So for i in my tuple colon and then do something in this case, I just want to print the element. So then for each element, we print that, and we don't have to call this I can also call it for example x or whatever we want. We can also easily check if an element is inside our tuple with an if in statement, so if max in my tuple, and then we say just print, yes. And otherwise, we print No. And if we run this, then we get a yes, so Max is in our tuple. If we check for Boston, it's also inside our tuple if you check for Tim, and we get a no. So very easy syntax to check if something is inside our tuple. So let's talk about some other useful methods that you can do with a tuple. For example, create a tuple with some letters in it. And first of all, if we want to get the number of elements inside our tuple, we can just use the Lang method, Lang, of my tuple. And this returns five, so we have five elements. If we want to count some elements inside our tuples, so we can use my tuple dot count, and then we count the letter P. So then we see we have two letter piece inside our tuple. If we check for the L, we get a one, if we check for all, which is not inside our tuple and we get zero. And we can also find the first index of some specific elements. So for example, my tuple dot index of P. and if we run this, then it returns the first occurrence of this element. So this is an index P. For example, if we say a, and we get index zero, we get if we want to get the index of L, then we get index three. And if we check for an element that is not inside our tuple, then we again get a value error. So be careful here. Um, we can also easily convert a tuple to a list and vice versa with the list and the tuple function. So if I say my list equals and then I use the list function, and put the tuple here, then I get a list out of it. And I can convert it back, I say my tuple to equals, and then the tuple function, my list if I print this, so then I have a tuple again. Now let's talk about slicing with tuples. So slicing is a very nice way to access sub parts of your tuple with the use of the colon. So for example, let's create a tuple with some numbers in it. And let's create a tuple. And then the syntax is we use the tuple, the original tuple. And then inside brackets we specify a start and a stop index. So for example 225. And if we print this, then we have number 345. So this goes from index number two to index number five, and the other last index is not included. So it only has index two, three, and four in it. So if we don't specify a start index, then it starts all the way from the beginning. And if we If we don't specify a stop index, then it goes all the way to the end. Now, we can also use an optional step argument. So by default, this is one. So in this case, it goes all the way from beginning to the end with a step of one. And if we put in a two here, for example, then it takes every second element. And we can also use a negative step, this is a nice little trick to reverse your tuple. Now we can, let's talk about unpacking. So if we create a new tuple, like at the beginning, let's put, let's use Max, 28, and Boston. And we can unpack it, if we write to the, at the left side, we write our variables, so name, H, and city, and then just say equals to my tuple, then we get each separate element and the city. But the number of elements that you put in here must match the elements inside our tuple. So if we just use two elements here, then we get a value error, to many, many values to unpack. But what we can do is we can unpack multiple elements with a star. So for example, if we use some numbers, so 1001234. And if we want to unpack this, so let's say I won, and then a star, and say, I, two, and I three equals to my tuple. And then if we print I one, this is the very first item, if you print by three, this is the very last item. And if we print by two, then these are all the elements in between, and now converted to a list. Yeah, so one more thing that I wanted to show you is to compare a tuple and a list. And because a tuple is immutable, Python can make some internal optimizations. And thus, working with a tuple can be more efficient sometimes, especially when working with large data. So let me copy this in here. In this example, we create a list and a tuple with the same elements. And then we use the SIS dot get size of method to return the number of bytes. And both of them. And if we compare them, then we see that a list is larger, even though it has the same elements as the tuple. And also can be more efficient to iterate over a tuple. And also to create a tuple. So if we compare if we use the time it method, so there's a very nice method in the time, module time at that time it and then you can use a statement and repeat this specific number of times. So in this case, it's 1 million times 1 million times we want to create a a list and 1 million million times we want to create a tuple and then measure the time. If we run this, then we see that it took much longer to create the list than to create the tuple. So yeah, keep that in mind that working with tuples can be more efficient than working with lists. dictionary is a collection data type that is unordered and mutable. It consists of a collection of key value pairs. So each key value pair maps the key to its associated value. And let's have a closer look at dictionaries and what we can do with them. First of all, a dictionary is created with braces. And inside these braces, you put each key value pair separated by a colon. So let's say key name, and then colon, and then the value, Max. And then you separate each item with a comma. So comma, and then let's put in another key value, pair, age, colon 28, comma, and another one, city, Cole on New York. And if we print this, and we see all the key value pairs here, then we can also use the dict function to create a dictionary. And there we put all our keys as arguments. So name equals Mary, comma, age equals 27, comma city equals Boston. And if you print this, we see that we have a second dictionary here. And note that with this function, you don't have to use quotes for your keys. Now, when you want to access the values, you do that by saying my dict and then inside brackets, you give it the key. So you print the associated value for the name, then we get max. And if we want to look up the age, we get 28. And what will happen if we use a key that is not inside our dictionary, for example, check the last name, then this will raise an exception, a key error. So be careful here. A dictionary is mutable, so you can add or change items after its creation. So when we want to add an key value pair, we simply do it like sell, we say my dict and then inside brackets give it the new key. Let's give it an email. And then the associated value max at x y z.com. And if we print it, then we see that our dictionary now has the email key value pair here. Now if we do the same thing again, and the key already exists, then it got gets overwritten. So let's say emails, coolmax, then we see that it still has the key email. And now with our new value. If we want to delete items, we have several options, we can say, we can use the Dell statement, so say Dell, my dict and then off the key name, and then we print it then we see that the name key value pair is no longer inside our dictionary. Or we can use the pop method. So we can say my dick dot pop, and then give it the key, let's pop the age. So now we see that the age is no longer inside our dictionary. Or we can use the pop item method. So prior to Python 3.7. This removes an arbitrary pair and since Python 3.7 This removes the last inserted item. So in this case, it's Python 3.7 and then it removes the city. So we see the city is no longer inside our dictionary. When you want to check if a key is inside our dictionary then there are two common ways to do that. The first one is to use an if in statements so we say if name in my dict and then we can use this key we can say print my dict name. So then max gets printed. If we say last name and want to print it then the if statement is wrong. So this doesn't get executed, so nothing is printed here. Or you can use a try accepts accept statements. So try and then access a key. So let's say my texts, name. And except print, let's just print error. So if we run this, then this statement is successful, so it can print the name. And if we want to access the last name, then this statement will raise an exception, a key error. So an exception is thrown, which is caught here. So this statement then will be executed. So if we run this, then we see that we have the error here. When you want to loop through a dictionary, you have several different ways within for in loops. So you can say for key in my dict, and then print the key, then you can see that this loops through the dictionary, and loop through all the keys. You can also say dot keys. So this will do the same thing. The keys method returns a list with all the keys inside it. You can also loop over the values. So you can say for value in my dict dot values and then print the value. And then it prints the values. Or if you want to have both in one loop, you can say for key comma value in my dict dot items, and then you can print the key and the value. Now when you want to copy a dictionary, you have to be careful. So the most common way to do it is like so let's say my dict copy equals to my dict and just assign it to the original mighty dictionary. And now if you print this, then we see that is the same as the original one. But now if we modified the copy, this will also modify the original one, let's say my direct copy and at the email [e-mail]. Now if we print the copy and the original one, then we see that both dictionaries now have the new key value pair. This is because with this simple assignment statement, both dictionaries now point to the same dictionary inside our memory. So be very careful if use this expression. If you want to make an actual copy, you can use the built in copy function. So if we use this one and print it and we see that the original one didn't change, or you can use the dict function. And as an argument, you pass the dictionary that you want to copy. So if you use this, then we see that the original dictionary also didn't get affected. Now there's a useful method to merge two dictionaries, that is called the update method. Let's create two dictionaries. And the first dictionary has a name, an age and an email. And the second dictionary also has a name and an age but no email, but then it has a city. And if we want to merge these, we can do it like this. They might take a dot update and then with the second dictionary, and now if we print this then what happened. All the existing keys or key value pairs got overwritten. So the name is now Mary, the age is now 27. The email didn't change and the non existing keys the city got edit So, yeah, that's a nice way to update to dictionaries. Let's talk about possible key types. So in all the examples before I used a string as a key, but you can use any immutable type, for example, you can also use numbers as a key, or even a tuple. If it only contains immutable elements. For example, we can say, my dict, and then key three, and so value, give it the squared value, then a key six and 36, and the key nine, and 81. Now let's print this. So this is also possible, but then you have to be very careful, because when you want to access a value, like so, and you want to do it like with lists, and you refer to the index number, for example, say index zero, then this will raise an exception because you have the key error. Zero is not in our list, what do you rather want to do is you want to use the actual key, so the key three is not nine, and then if you print the value, then we see we get the nine. So yeah, we also can use a tuple as a key. So let's say my tuple equals eight, seven, and then create a dictionary and S key, we use the tuple and s value, we use the sum. Now if we print this, then we see we have our new tuple here, and are our new dictionary here. And yeah, so tuples are also possible. But what is not possible, for example is a list. If we use a list here and run this, then this will throw an exception type error on hashable type. That is because a list is mutable and can be changed after its creation. And therefore it's also not hashable and cannot be used as a key. So be careful here. Set aside collection data type that is unordered and mutable. But unlike lists, or tuples, it does not allow duplicate elements. A set is created with braces, just like a dictionary, but we don't put key value pairs in it. But instead just single elements separated by a comma. For example, let's put some numbers in here and print this. Then we will see our set here. And if we put for example, another one and another two here, and print this again, then we see that only one of each element is kept. Because a set does not allow duplicates, we can also use the set function and use an iterable. Here, for example, let's use a list here. This will also create a set, or we can use a string here are example Hello. And if we print this, first of all, we see that the order is arbitrary because a set is unordered. And the order is not important. And we also see that there's only one l in our set. So this is a nice little trick to find out how many different characters are in your word. Now, if you want to create an empty set, and you do it like this just with the braces, then you have to be careful because now if you have a look at the type of this, then we see that this is recognized as a dictionary. So if you want to have an empty set, you have to do it with the set method. Set is mutable so you can change it later on. So now we can add elements and we do this with the dot add method. So let's put in some numbers here and print this. And we can also remove elements again with the Remove method. So let's remove the three and if we want to remove an element that does not inside Our set, then this will raise a key error. So be careful here. So there's another method, let's call that this cart method that does the same thing. So it also removes the element. And if it does not find the element, then nothing will happen. So no error here. We can also use the clear method, of course, this will empty our set, or we can use the pop method. So this will return an arbitrary value of our set and also removes it. So we print this. Then we see in this case, it returned the one and also remove the one from our set. Now, we can iterate over our set very easily with an for in loops. So for i in my set, and then do something, in this case, just print this. So this will iterate over each element and print it. And we don't have to call this I we can also call this for example x or whatever we want. Now we want to check if an element is inside our set, we can do this with an if statement. So if one in my set, and then we print, yes. So the one is in our set, the two is in our our set. And if we check for example, for the four, then nothing gets printed. Now, let's talk about union and intersection. And for this case, first of all, let's create three different sets one with odd numbers, one with even numbers, and one with prime numbers. And now we can calculate the union. So the union combines elements from both from two sets without duplication. So let's calculate the union of odds. And we do this with dots union, and then as an argument, the second set so events and print this. Then we see that now we have all the numbers from zero to nine. So the union will combine elements from both from two sets without duplication. We can also calculate the intersection of two sets. So the intersection will only take elements that are found in both sets. So if we say the intersection equals arts dot intersection events, and if we print this, then we will get an empty set, because arts and events don't have the same elements. Now if we calculate calculate the intersection of arts and primes, we will get all the prime numbers that are also odd. So 357 if you calculate the intersection of events and primes, then we will get back only the even prime numbers. So in this case, only the two now we can also calculate the difference of two sets. So let's create two different sets again, set a with numbers from one to nine and set B with one with 123 10 1112. Now the difference will return a set with all elements from the first set that are not in the second set. So let's call that call a diff equals set a dot difference set B and print this. Then we will see that we will get back the numbers from four to nine because it takes the elements from our first set, but not the ones that are also in the second set so only from four to nine. So if we do it the other way around set be the difference set a then it will take 10 1112 but not these three numbers because they are also here. So then there's a second different method. second difference method, that's called the symmetric difference method. So, the symmetric difference method will return a set with all the elements from set A and set B, but not the elements that are in both sets. So again, so it takes 456789 from set A, and 10 1112 from set B, but not one, two, and three, because they are in both sets. So if I use set a symmetric difference set B, then this is the same thing. Now, union and intersection and the difference method that I just showed you, they will not modify the original sets, they always will return a, a new set, but we can also modify our sets in place. So for example, we can say set a dot update, set B. And now print our set A, then we will see that this updates the set by adding the elements that are found in another set. So without duplication again, so it does not add one, two and three again, but it adds 1011 and 12. There's also a intersection up update method. So set a dot intersection update, set B. And what this does, it updates the set by keeping only the elements from found in both sets. So only one two and three are found in both sets. So only these numbers remain in our set. Then there's also the difference update methods. So set a difference update set B. And if we print this, we will see the numbers from four to nine because difference update, it updates the set by removing elements found in another set. So it also it found it finds one, two and three in the set B so it removes these numbers from our set A. And then there's the symmetric difference update. So this updates the set by only keeping the elements found in set A and in set B, but not the elements that are found in both. So one, two, and three are found in both sets. So these are not taken, but then it takes all the remaining elements from both sets. Yeah, you'd have probably have to play around with them yourself a little bit to make it clearer. And yeah, let's also talk about soap sets superset and disjoint methods. So for example, let's make them a little bit smaller. You can calculate the if set A is a subset of set B. And we print this then this will return false because subset means that all the elements of our first set are also in our second set. So if we use it the other way around set B is a subset of set A, then this will return true because one two and three are also in the second set. And the opposite is called the super set method. So is super set. And in this case, it returns false because a super set returns true if the first set contains all the numbers or all the elements from the second set. So, set B does not contain 546. So it's not a super set. But set a is a superset of set B, because it contains one, two and three. And we can calculate if two sets are disjoint. So this join returns through if both sets have a null intersection, so no same elements. So set a is this joint set B. And if we print this, this will return false because they have same elements. And if we create, for example, a set C and put seven, eight in here, and check set a is this joint with set B, then this will give us true. Now let's briefly talk about copying two sets. If you have watched the previous episodes about lists, for example, then you already know this, you have to be careful, and you want to copied two sets and only do this with a simple assignment. So let's say let set B equals to set one, set a. First of all, if we print this, then we see that we have a copy. But now if we modify the copy, let's say set B at set seven. And if we print the copy and also print the original one, then we see that also the original one changed, because with this simple assignment both points to the same set. So be careful here, you only copy the reference. Now if you want to make an actual copy, you have to use the dot copy method. So if you run this again, and we see that the original set didn't change, or you can also use the set method and use the first set as a argument. This will also make an actual copy. Now as the last thing, I want to show you the frozen set, the frozen set is also a collection data type. And this is just an immutable version of a normal set. So you create this with the frozen set method. And there's an argument you can also put an iterable here, for example, a list and let's print this, then we see our frozen set here. So with a frozen set, you cannot change it after its creation. So if I try to do a.at, two to this will give us an error. Or I can also say try a remove one. This will also give us an error or any of the updates, update methods I showed you, they also don't work. But for example, Union intersection and difference method, they will work a string as an ordered and immutable collection data type that is used for text representation. And it is one of the most use data types in Python. So I hope that at the end of the session, you'll feel comfortable working with them. So let's start. First of all, a string is created with either single or double quotes. So you can use double quotes and then put your letters in here. So let's say hello world, and now we can print this and then we see our string here or you can use single quotes This is probably more common. The only thing you have to be careful is if you have another single quote inside this. So if you have for example, I am a programmer. Now if you try to run this, then this will get a syntax error. So what you can do is you can either use an escaping backslash here, so this is valid. Or you can put your single quote in Side double quotes. So this is again, a valid string. You may also sometimes see triple quotes. So this is typically used for multi line strings. So now I can go in another line. And this is also used for documentation inside your code. So now if we run this, we see that our string goes over multiple lines. Now, you may also sometimes see an escaping backslash, like so. And this just says that the string should continue in another line. But it should not create a new line here. So now if we run this, then we see we have our one line hello world string. Now, if you want to access characters, or sub strings, it's the same like with lists, you access it with inside brackets, so let's say char equal, or let's create a string first. Alright, my string equals hello world. And then you can say char equals and then my string. And then in brackets, you put the index you want. So if you want the very first character, you have to use index zero. So we can print this. So this is the H and F, we use index one, we get the E, and so on, we can also use a negative index, so minus one is the very last character, minus to the second last, and so on. But what we cannot do, for example is we cannot access a character and change it. So if I want to change the first character to a lower age, if I tried to run this, now, this will get a type error, a string object does not support item assignment. And this is because strings are immutable, so they cannot be changed. So be aware of that, we can also access a whole substring with slicing. So then I will say my string and in brackets, I put this start index, so let's say one, and then a colon and then a stop index. And then if I print this, then I will see I get the string, e Ll O. So what this does, it starts at index one, and goes until index five, but index five is excluded. So be careful here. So it has 123, and four, so our string is E Ll O. Now if I don't use a start index, then it starts all the way from the beginning. And if I don't use a stop index, then it goes all the way to the end. So this goes all the way from beginning to end. And then there's another optional step index. So if I put another colon here, and by default, this is one. So this takes every character. And now if I put a two here, it takes every second character. And I can also put a minus one here, and then what it does, it will reverse our string. So that's a nice little trick to reverse the string with this slicing operator. Now, we can concatenate two or more strings simply with a plus. So if I create another string, Tom and I will just call this hello and say this is now a greeting. And then I can say my sentence, I will create a new string that is greeting plus, and then I want a space between them. And then plus again, plus the name. And now let's print this and then we see we have our concatenated string. So very easy with this plus here. Now we can iterate over our string with a four in loop so for i in greeting, and then do something so just print this print every letter So, this goes over our whole string and prints each character. And we don't have to call this I, we can also call this for example x or whatever we want. Now, if you want to check if a character or substring is inside our string, we can do this with an if in statement. So I say if and then I want to check for the letter E. So if e in greeting, and then I will print Yes. And otherwise else I will print. No. So he is inside my word. So it prints Yes. So if I check, for example, for p, then it will print No. And I can also check for substring. So I can check for E Ll, this will also print Yes. Now let's talk about some more useful methods that you can do with strings. So let's say we have a string with some whitespace here, and then we have our hello world, and then some more whitespace at the end. So if we print this, we will see that our printed string also has the wide string. Now if I want to get rid of this, I can do my string equals my string dot strip. So this method removes our whitespace. So now if I print it, we see that the whitespace is gone. And be aware that we this method does not change our string in place, because as I said, a string is immutable. So if I just write it like this, then this will not change my original string. So if I run it, our original string still has the whitespace. So what we have to do is, we have to assign it again to our original one, and then we have the new string with without whitespace. Now, what we can do also, with strings is we can say, we can convert every character to an upper case, so let's say my string dot upper. And then we have all an upper cases, we can also say my string dot lower, then we have all in lower cases, we can check if my string starts with specific character or substring. So if we can say starts with and then we can say h. so this will give us true or we can also check for Hello, also true and check with world. And we will get a false. But we can also check if it ends with something. So if it ends with world, so now we have true. And yeah, if it ends with Hello, then we get a false. Now we can find the index have a character or a substring. So let's say my string dot find and then we want to find O. So this will return the first index that it finds with an O. So index 01234. So it returns a four, we can also check for substrings. So this is the at index three, our l o substring starts and if it does not find a string, then it will return a minus one. We can also count the number of characters or substring it finds. So let's check for how many O's we have in hello world. So this will return to and how many peas do we have? We have zero. We can also replace characters or substrings inside our string. So we can say my string dot replace. And then we want to replace world with a new word. So we want to replace it with the universe. And now, if we print this, then we see our string is now Hello universe. And also be aware here that this will return a new string and does not change this one. So if it does not find this strings, let's say for example, we have a typo here, then it does nothing. So it will still print the original string hello world. Now, let's talk about lists and strings. So let's say you have a string with some words. So let's say we have here, how are you doing. And you want to convert this to a list and put at each word of my string as an element in my list. Now, what you can do then is you can say, my list equals my string, dot split. And if we print our list, then we see that we have each word now as an element in our list. And by default, the delimiter it is looking for is a space. So here, the default argument is a space. So it looks for each space, and then splits our string here. Now, for example, if you have commerce, here, and then cannot find a space, so it, we only have one element here. So now what you would then have to use, you would have to use as a delimiter, a comma. And then again, we have four elements. Now if you have the list, and you want to convert it back into a string, what you can do is you can say, let's say new string equals and then we say, my whip, you know, we say and empty string, and then dot join, and then the list as an argument. And then we print the new string. And then we will see that this will concatenate all of our words, all of our elements in our list. So this will put all of our elements together as a string. And between each element, it will put this string that we put here. So now if you put a space here, then it will put a space between each element. And now we have our original string again. So the dot join method method is a very useful method to quickly join the elements of a list back into a string. And I would highly recommend to remember this one because this is very useful. And let's In fact, let's talk about this a little bit more. So let's say we have a list with some elements. So let's say for example, only A's and then times six. So maybe you know this syntax. So this will create a list with six elements. And now if you have the task to join this into a string, a lot of times what you will see is that you will create an empty string and then you will use a for in loop. So for i in my list, and then you will say my string plus equals i. So let's check this. So it worked. We have our string here, but this is bad Python code. Because what happens here since a string is immutable. This will create a new string here and then assign it back to our original string. So this operation is very expensive. What you should better use it The dot chain method. So as I just showed you, we can say, my string equals, and then we will say an empty string dot, join my list. And then we will print this. So this will also give us the same thing, but it's much cleaner, and also much faster. So let's look at the time of both of these ways. So let's say we have saved from the time at module, we import the default timer as timer. And then we will say here, start equals timer. And at the end, we can say, Stop equals timer. And then we will print, stop minus start. So this will give us the time it takes from here to here. And we will do the same thing here. So if we run this, we will, let's remove this, we see that both both was very, very fast. But now let's say we have, for example, a very large list with let's say, 1 million elements. And now I don't want to print my list. So if we run this now, what we will see that this The second way with a dot join method is much faster. So the first way took more than half a second and the second only point 01 seconds. So forget this way of doing it. And remember that duck shine method. Now as the last thing, I want to talk about formatting strings. So there are two ways to format a string, the old styles are with a percent operator, or with a thought format method. And since Python 3.6, there's also the new f strings. And let's talk about all of these methods. So let's say we have a variable and call it it's, let's say variable equals a string, and we have the name Tom. And then we will create a string and say it's the variable is and then we use the first method, so we use a percent s. And then after our string, again, we use the percent and then the variable. So this tells the interpreter that we have a placeholder with a string here. And then afterwards, we fill this placeholder with our variable. So now if you print our variable, then we will see that our string is the variable is Tom. Now, if we have a number here, we shouldn't use percent s here, we should use percent D. So this stands for integer decimal value. So now we have the variable is three. And let's say we have a floating point. So like this, and if we run this, then we see that we still have three here because we told the Python that we have a decimal value here. So now what we want now is we want a floating point, so we say percent F. And then we have our floating point value here. And by default, it has six digits after the decimal points. So if we want to specify how many digits we want to have, we can say, dot percent dot and then how many digits and then let's say two digits and then.to F. So this will give us two digits after the decimal point. So this is the very old formatting style. The new formatting style is with the dot format methods. So now what He wants to do is as a placeholder, we use braces. And then after our string, we call the dot format method. And then here we put all our elements as arguments. So now if we print this, then we see that we have the placeholder got replaced with our variable. And we can also specify how many digits so we can say, colon, dot two, F. So then we have two digits after the decimal point. And for example, if we have more variables, we simply would place another placeholder here, and then another argument here. So let's say we have var two equals six, and then we would put var two here. And then we will see that we have all our variables inside our string now. So these are the old formatting styles. And the newest way to do it is with the F strings. So this is, since Python 3.6, or newer, you can use the F strings. And with an F string, you would simply put a f between the string and then the string. And then you will also use braces. And inside the braces, you can use your variables directly. So you can use var here, and var two here, and then you don't need this anymore. So if we run this, then we see it worked. And yeah, I think this is much more readable, it's more concise. And it's even faster, especially if you have a lot of variables here. So I would highly recommend using this f string f strings now, since python 3.6. And yeah, what this does is it evaluates the this at run time. So we can also put some operation here. So let's say it's a mathematical operation, like var times two. And then this will, will be evaluated at runtime. So now, we see we have our two times our variable here. And, yeah, so that's it about f strings. And that's all I wanted to show you about the strings. The collections module implements special container data types, and provides alternatives with some additional functionality compared to the general Bert and containers, like dictionaries, lists, or tuples. So we will be talking about five different types from the collections module, the counter the named tuple, the artists dict, the default dict, and the deck. So let's start with the counter. And first of all, we have to import it from collections import, counter. And the counter is a container that stores the elements as dictionary keys and their counts as dictionary values. So let's say we have a string called a with some different characters, a BBB, CCC. And then we can create our counter, we say my counter equals counter, and then we give it our string. And if we print it, then we see we have a dictionary with all the different characters as keys and their count as values. So we have five times a four times B, and three times C. And, like with a normal dictionary, we can have a look at only the items. So this will give us all the key value pairs, we can have a look at the keys. So this will give us an iterable over the keys. And we can also only have a look at the values. So this will give us all the different values. And what's also very helpful is to have a look at the most common element in our counter dictionaries. So we say if we first print our counter again, and then we can see Today we want to print my counter dot most common. And then here how many different items, so I want to see only the very first so the most common elements. So if I print this, then I will get the A with the count five is the most common element. So if I say two here, that will give me the two most common types, so it will also put the B in here. And this will return a list with tuples in it. So, for example, if I want to have a look at only the, I want to see what is the most common element, then I will x have to access the index zero, so, this will give us the tuple at index zero. And then if I only want to see the element, then I will again have to access the first element of this tuple. So against zero, and then I will get the A is the element that is most common in our string. So, we can also use a list here or any other iterable. Yeah, we can also have a list with all all the different elements. So, if we say, print my counter dot elements, and this will give us an iterable over elements repeating each as many times as it counts. So, I have to convert this to a list in order to print it nicely. So now if I print it, and I will see, I will get all the different elements here as a list. And I can, for example, iterate over this. So that's the counter next talk about the named tuple. And of course, first of all, we have to import it. So we say from collections import named tuple. And the named tuples is an easy to create and lightweight object type, similar to a struct. So what I can do is I can define my named tuple. I say for example, let's create a 2d point and call it point equals and then I will say named tuple. And then as first argument, I give it the class name. So typically, this is the same name that I use here. And then as a second argument, I use another string and here I use all the different fields I want separated by either a comma or a space. So I can say x comma y. So this will create a class called point with the fields x and y. So now I can create this point. So I can say p t equals point and then I will give it values for x and y. So for example, I will give it one and minus four. And now if I print my point, then I will see I have a point with x equals one and y equals minus four. And I can also access the fields. So I can say p t dot x and p t dot y. So then this will print the values for x and y. Next is the ordered dictionaries. So from collections import, ordered dict. And the ordered dict is just like a regular dictionary, but they remember the order that the items were inserted. So they have become less important now since the built in dictionary class has also the ability to remember the order since python 3.7. This is guaranteed. But for example, if you use an older Python version, this may be a way to use a dictionary that remembers the order. So for example, let's create a dictionary like so. And then we can append key value pairs like with a normal dictionary. So we say here and brackets give it a key a and a value, one. And let's do this with some more key values. So let's say we have B, C, and D, and 234. And now if we print this, then we see it's the same order as we inserted it. So for example, if we inserted the A, at the very end, then it will also get printed at the end of our audit dictionary. Yeah, since here, I'm using three python 3.7. So in this, I can also just simply use a normal dictionary now and it still remembers the order. Next, we have a look at the default dict. So from collections import default dict. And the default dict is also similar to the usual dictionary container, with the only difference that it will have a default value if the key has not been set yet. So what we will do, we have to create a default dict. And as an argument, we will give it an a default type. So let's say we want to have an int, an integer here as default type. And then we can fill our dictionary, again, let's say D, with the key a is one and D with the key b equals two, and let's print our dictionary. So we will see it here and then we can access the keys. So for example, let's access the key a, and then it will give one and the key p will return to and now if I put in a key that does not exist, so for example C, then what will happen, it will return the default value of an integer. And this is by default a zero. So I can also for example, say I want a float default value. So then this will return 0.0 if it does not exist, or for example, I will have an empty list if it does not exist. So yeah, with a normal dictionary, this would raise a key error. So now this would raise a key error, but with a default dict it would return the default value of the type that we specify. So, as a last collections type, we will talk about the deck. So the deck is a double ended queue. And it can be used to add or remove elements from both ends. And both are implemented in a way that this will be very efficiently. And yeah, let's create a deck so let's say d equals deck and then we can append items like with a list, let's say the append one and the append two and then print it. Now, now, we see our deck here and also we can say we can say d dot panned left. So this will add elements at the left side. So now we can see our three got added here. And we can also again remove elements from both sides. So we can say d dot pop. And now if we print our deck then we will see that the with pop, this will return and remove the last element. So now the two got removed or we can say d dot pop left so this will return and remove the other From the left side, so now, the three got removed. can also of course, say d dot clear. So this will remove all elements, we can extend our deck with multiple elements at a time. So we can see d dot extend, and then give it a list, let's say 456. So this will add all the elements at the right side, or we can say d dot extend left, this will extend all the elements at the left side. And note that now, it will add First, the four from the left side and the five, and then the six. So now six is the most left elements in our deck. We can also rotate our deck so we can say d dot rotate one. And now if we print it, we will see that this will rotate all elements one place to the right, I can also say for example, do not rotate to and then this will rotate all elements to places to the right. Or if I want to rotate to the left side, and I will give a negative number here. So if I say d dot rotate minus one, then all our elements will rotate one place to the left. The inner tools module is a collection of tools for handling iterators. Simply put iterators are data types that can be used in a for loop. So for example, the most common iterator is the list. And the error tools offer some advanced tools. And we will be talking about the product, the permutations combinations, the accumulate function, the group by function, and some infinite iterators. So let's start with the product. So first of all, we have to import it. So we say from it or tools, import product. And let's say we have two lists a equals one, and two, and B equals a list with three and four. And then we say we have a product of a and b, and the product will compute the Cartesian product of the input iterables. So let's print this. So print the product. And then we will see that we have a editor tools object. So this is an iterator. And to see the elements we can come convert it to a list, and then we will see the product. So the product will combine one and three, and one and four, and then two, and three, and two and four. So this is the product, we can also define a number of repetitions. So if we say repeat equals two, then it can repeat. And let's run this and then we see that this is a very large list. So let's make our second iterables smaller, and print this. And then maybe the repetition gets clearer. So we have one and three. And since we can repeat again we do one and three, and we have one and three and two and three, two, and three and one and three, and again with repetition, two and three, and two, and three. So that's the product. Then we also have something called permutations. So permutations will return all possible orderings of an input. So let's say we have one, two and three as a input, and then we calculate the permutations of a and print this again, as list and then we see all the different orderings So we have 123132213231312, and 321. So that's permutations. And we can also specify the length of the permutations as a second argument. So if we want to have shorter permutate, permutations with only length two, we skip the argument two. And then we see different orderings with the length of 22121321, and so on. That's permutations, then we have combinations. So from either tools, import combinations, and the combinations function will make all possible combinations with a specified length. So let's also make an example here. Let's make a list 123, and four, and then say comm equals combinations of a. And the second argument with the length here is mandatory. So in this example, I only want the length two, and then print this again as a list. And then we will see all possible combinations with length two, so 12131423, to four, and three, four. And, and note that we don't have combinations of the same arguments or no repetitions here. And if we want that, we can also use the combinations with replacement function. So then we import it, so import combinations with replacement. And then let's make another combination iterable and say combinations with replacement of a and also of length two, and Prentiss comm with replacements, and then we see that it will make combination of one and itself. So one and one, one and two, one and three, and so on. So this is combinations and combinations with replay replacement. Now, when we have the accumulate function, so the accumulate function makes an iterator that returns accumulated sums, or any other binary function that I will give as input. So let's make an example. First of all, import the accumulate function. And then we can say, we leave the list a equals 123, and four, and then we say, accumulate equals accumulate of a, and print this. First of all, let's print our list and then print the accumulated list. So we see that our list is one, two, and three, and they accumulated sums is 136, and 10. So the first elements stays the same. And then we have one plus two is three, three plus three is six, and six plus four is 10. So that's the accumulate function. And by default, it will compute the sums. But we can also for example, multiply the elements so let's import operator. And then we can give as a second argument, we can say func equals operator.ml. So this will multiply each element so one stays the same. One times two is two, two times three is six, and six times four is 24. And as a third example, let's just use the max. So this will return the max for each comparison. So for example, if we have a five here in between, and have a look at our list than one is the same two now two is the max then compared with five and five is Till the max compared with three and five, still the maximum and compared with four, five is still the maximum. So that's accumulate. Now let's talk about the group by function. The group by function makes an iterator that returns keys and groups from an inner rebel. So let's make an example. To make this clearer, let's say we have our list a equals 123, and four. And then we say we make a group object, and that is group by, and then we want to group A, and we have to give it a key, which map has to be applied. So as key, we can define a function, so let's say smaller than three, and give it an input, and then return x smaller than three. So this will return true or false. And as a key, we will give it this function. And then let's print this. So we will see that this is a group by object and we can iterate over this. So we can say, for key and value in our group object, and then we want to print the key and the value. And then we will see it prints the key and an inner tools, object, group or object. So we can convert this to a list to see the values. And then it gets clearer. So we have our input array. And we group this into other lists. With the comparison if it's smaller than three, so for one and two are grouped together, because they are smaller than three, and the key is true, and three and four are grouped together, and the key is false. Now, we can also use a lambda function here, so I will talk about this in the next video. But as a very short explanation, lambdas are small one line function that can have an input and will do some expression and then will return an output. So I can write this same function in one line with a lambda expression. So I can write lambda, x and then colon, and then simply x smaller three. So this will do the same thing. If I run this, then it will print the same thing. Now let's make an another second example. For this, maybe this is not clear at the first side. So let's define a object persons. And this is a list. And inside this list, we have different dictionaries. And the dictionaries contains a name and an age. And let's say we want to group our persons by the same age, so let and then we say lambda x and simply x and s key the age. And then if we run this and print this, then we will see as keys, we have the different values for age. So we have 2527 and 28. And then we also see that it grouped Tim and then together because they both are 25 years old. And then we have Lisa and Claire. So that's the group by function. Then we also have some infinite iterators. There's the count function, then this cycle function and the repeat function. And the count function is very simple. So if we just say for i in count and then Give it a start values. So let's start at 10, and then print this. So this will make an infinite loop that starts at 10. And then adds one for every repetition. So one, so 10 1112, and so on. And this is still going now. So then for example, if I say, if I is 15, then we will we break, so then it will stop at 15. That's the count function, then there's the cycle function. So this will cycle infinitely through an iterable. So let's say we have a list that has one, two, and three. And we want to cycle through a and print this. So this will print one, two, and three, and then cycle again, one, two, and three, and again, infinitely, until I make some stock condition. So that's the cycle method. And now as a last thing, the repeat methods, so repeat, for i in repeat, and then I want to repeat, for example, just the one, then this will simply make an infinite loop. And we'll print one. And I can also as a second argument, give it the stop repetitions. So how many times do I want to repeat for example, if I say four, here, then it will repeat the one four times. Lambda function is a small one line anonymous function that is defined without a name, and it looks like this. First it has the lambda keyword, then it can take some arguments, then a colon, and then an expression. And what this will do, this will create a function with some arguments, and it evaluates the expression and returns the result. So let's look at an example. To make this clearer. Let's call a function and we call this add 10. And this is equal a lambda with an input. And let's call our input x. And then it should evaluate x plus 10. So this will create a function with one argument, and it adds 10 to the argument and returns the result. And we assign this function to our var variable at 10. So now this is a function that we can call with an argument. So let's call it with with five. And now if we print this, then it will print 15. So this is practically the same as a normal function like this, let's call this at 10. func, and this will take an argument x and return x plus 10. So these two things do the same thing. But the lambda function is much shorter and only in one line. So lambda functions can also have multiple arguments. So let's say let's create a another lambda function and call this mouth and and this is equal lambda. And now we give it x and y. And it should evaluate x times y. So this will create a function with two arguments, and it will multiply these two arguments and returns the result. So now if we print for example, mod two, and seven, then it will print 14. So that's the lambda syntax. Lambda functions are typically used when you need a simple function that is used only once in your code. Or it is used as an argument to higher order functions, meaning functions that take in other functions as arguments. For example, they are used along with the built in functions, sorted map, filter, and reduce, and we will have a look at all of them to make the usage of landac euro. So let's start with the sergeant methods. So you probably already know this, and I also showed this in my video about lists. So let's say we have a list. And we call our list points to the, and the list has tuples with two elements in it. So you can think of this as the x and the y well use of our points. And now if we want to start this, so let's create a points to the sorted list. And then we can call sorted, this is built in, so we don't need to import anything. And now we can start our list. So we want to sort points 2d, and now print, first, print our points and then print our points to the sorted. So by default, this will start our, our list by the first argument, so by the x argument, so one, 510 and 15. But we can also give it a specific rule how to sort it. So we can say we can give it a key argument and the key equals, and this should be a function. And as we now know, we can write a function with a lambda in one line. So we can say, lambda with an argument x. And now let's say we want to sort it by the Y Well, you so by the second index, so then we say, x of the index one. So now if we run this, then we will see that our list got sorted according to the Y index. So what this does is, you can also, for example, give it a or define a function, and let's say sort by y and then give it a index. And in this, give it an argument, and in this case, two argument is a tuple. And then it returns the first index. So now we can also use this function here, so sort by y, and if we run this, then this will return the same result. But now we see with a lambda, we don't need this. And then we can simply use our lambda here, so we can use our lambda here, and get rid of this function. And yeah, that's one use case of a lambda. For example, let's make another example of sorting. Let's sort this according to the sum of each. So therefore, we would say lambda x and then evaluate x of index zero plus x have index one. Now if we run this, then we see that it got sorted according to the sums of each tuple. So that's the sergeant's method with a lambda is key argument. Now let's talk about the map function. So the map function transforms each element with a function. So it looks like this. It has a func, a function as an argument, and then a sequence. So this is for example, a list. So let's create a list with some numbers in it. So 123, and four, and five. And now let's create a another list and call this B equals and now we will want to multiply each element by two. So let's say map. And then as a function, we define a lambda with an argument and evaluate x times to and then as a second argument, we use our list. And then we print this and if we want to print this, then if we simply print it like this, then it will print a map object. So we have to convert it to a list first. And then we can see that each element got multiplied by two. So that's the map function. However, you can achieve the same thing with list comprehension. So you probably already know the list comprehension syntax. It's a little bit easier. So you can write it like this. C equals And then let's say x times two, for x in a. Now, if you print this, then this will do the same thing. So, personally, I would prefer this syntax, it's a little bit easier. But you should have heard about the map function. Now the second function is the filter function. So the filter function also gets a function and a sequence. And it will, this function must return true or false. And the filter function will return all elements for which the function evaluates to true. So let's say let's also give it a six. And let's say we want to filter this. And we say, we equals filter. And let's say in this example, we only want to have the even numbers. So then we create a lambda with x, and we evaluate x modulo two equals equals zero. And then if we run this, we should get only the even numbers. So again, here, we can achieve the same thing with list comprehensions. So we can also write C equals a list and then inside our list, we write x for x in a, and then we can give it a condition, we can say, if x modulo two equals equals zero. So we print C, then we see that this will do the same thing. And as a last function, I want to show you the reduce function. So the reduce function also takes a function and a sequence. And it repeatedly applies the function to the elements and returns a single value. So let's say I have a list here. And I want to compute the product of all the elements. So let's call this product, product A equals and then I can say, read us. And in Python three, I have to import this now. So I have to save from func tools, import reduce. And then I can come call the reduce function. And as a first argument, I give it a function. So I define the function here, again, in one line with a lambda i say, lambda, x, and now it has two arguments here. So function, the function for the reduced function always has two functions has two arguments. So let's say x and y. And then it should evaluate x times y. And as a sequence, I gave it a, so let's print the product. So this will print. Let's make this example smaller, then we can see it has one times two equals to two times three equals six, and six times four equals 24. So yeah, that's the reduce function. And that's all I wanted to show you about lambdas. A Python program terminates as soon as it encounters an error, and an error can be either a syntax error or an exception. So in this tutorial, we will have a look at what's the difference between a syntax error and an exception? What are the most common built in exceptions? How can we raise and handle exceptions? And how can we define our own exceptions. So let's start with a syntax error. a syntax error occurs when the parser detects a syntactically incorrect statement. So for example, if I write a equals five, and then in the same line, I want to print this, this will raise a syntax error because I have no I have to use a new line here. So this will be fine. Or a syntax error can be for example, missing or too many parentheses. So if I try to run this now, this will also raise a syntax error. And now exceptions. So even if a statement is syntactically correct, it may cause an error when it is executed. And this is called an exception error. There are several different error classes, for example, trying to add a number and a string will raise a type error. So, if I say A equals five plus, and then as a string, I write the 10. And now, if I run this, then this will raise a type error, unsupported operand types for plus int and string. So, this is a type error. And now let's talk about some more common built in exceptions. So, of course, there is the import error. So if I say import, and then some module that does not exist, then this will raise a module not found error, which is a subclass from the import error. This is a common exception, then there's the name arrows. So let's say if I have a variable A equals five, and another one, b equals C, and C is not defined yet. So, if I run this, then it will raise a name error name C is not defined, then there's the file not found error. So, let's say I want to open a file f equals open and then the file is called some file dot txt. So if I try to run this, then I will get a file not found error, no such such file or directory. Then there's the value error. Which happens if the function or operation receives an argument that has the right type, but an inappropriate value. For example, let's say I have a list with some numbers here 123. And now I can remove elements from a list with the dot remove method. So I say a dot remove one, so this works fine. So now, I print A, and the one got removed. And if I try to add the two, remove the four, which is not in the list, and this will raise a value error, so list, remove x x not enlist, then there's the index error. So if I want to access an index of a sequence, or of this list, that is not that is too large. So for example, if I try to access the index for, then this will raise an index error list index out of range. And if I have a dictionary, so let's say I have a dictionary with and name, and the name is Max, and it has only the key value pair have the name, and I want to access for example, I want to access the age then this will raise a key error because the H key is not inside my dictionary. Now let's talk about raising an exception. So if you want to force an exception to occur when a certain condition is met, then you can do this with the race keyword. So let's say we have a variable x equals and minus five. And then we say if x smaller than zero, then we want to raise an exception and then we say, race and then we raise simply the base exception, and as message we give it x should be positive. So now if we run this, then this will raise this exception x should be positive. And now if we get given a value of larger than zero, then no exception will be raised. As a second way you can use the assert statements so you can say. You don't use an if statement. So you use an assert statement. So you say assert, and then a condition and the search statements will will throw an assertion error if your assertion is true. Not true. So if you write here, you make an assertion that x should be larger or equal to zero. And now if we run this, then this will raise an assertion error, we can also give it a message here. So x is not positive. And now this will print the message here. And if our statement is correct, so x is positive, then your code will be just fine. So if I ran this, then no assertion is here. Now if you want to handle exceptions, so you can catch exceptions with a try except block. So you write for example, you write try and then call on and then you can do some operations. So let's say I want to try a equals five divided by zero, and this will raise an error. So let's simply run this and show you what happens. So this will raise a cirro Division error, because division by zero is not allowed. So what I can do, then I can make a try except blocks. So I will try this statement. And then I can write except, so if an exception is raised, then the code will continue here. And then I can simply print and our are happened. So if I run this, then your program doesn't stop here, it will continue and it will continue in this line. And you can also catch the type of exceptions so you can ride except exception as E. And then you can print your exceptions. So you can if I run this, then it prints the division by serial message from the zero division error class. Now, it's good practice to specify the type of exception you want to catch. And therefore you have to know the possible errors. So for example, if you know that this is a zero division error, you can simply write or you should write, except zero division error. And then you can do something here. You can also for example, use multiple statements here. So you can try multiple operations. So let's say we want to try five divided by one. So this is this is fine. And then we say B equals a plus and then a string. So we we've already seen this, so this will race. So let's this will raise a type error. So let's print here. So let's catch this acceptive zero division error as E and print E. And now we also want to catch a type error. So then we write type error as E, and then we can also print this. So now if we run this, this will catch the Type error and prints this message, unsupported operand types for float and string. And now if this fails, then this will be catched here, and then this match message gets printed. So now we have division by zero. That's how you can handle exceptions. Now you can also with a try except block you can also have a else clause. So an else clause is run if no exception occurred. So here I print. Everything is fine. And now, if I, for example, make divide by one that's fine, and I want to say a plus four, that is fine. And then the code continues in the else clause. And I also can have a finally clause. So the finally clause runs always No matter if there was an exception or not. And this is, for example, use to make some cleanup operations. So here we print Li cleaning up. So now if you run this, then the else clause runs, and the finally clause runs. And if there is an exception, for example, this, then this line is running. And again, the Finally, clause also is running. So yeah, that's how you can handle exceptions. Now, as the last thing, let's talk about how we can define our own exception. so we can simply define own error classes by sub classing from the base exception class. So we can say for example, class value, too high error, and typically, you want to give your class a name with an error at the end. So the class value to high error. And then as a base class, we use the exception class, and then we can simply say pass. So this is already a valid, defined exception error. So now, we can say, let's write a small function, test value with an input. And now we say if x is larger than 100, then we can raise this value to high error. And by default, it can also have an error message. So we say value is too high. And now if we run our method with an argument of, let's say, 200, then we will see that this will raise the value to high error, so value is too high. And now, for example, we can use a try and accept lock. So let's say try test value, and then accept and catch the value to high error. And then we print or let's print the era as E. And then print, let's catch the value to high era E and then print E. So then, we will see that the message gets printed here. And usually what you want to keep this classes small, but you can write it like any other class. So you can, for example, let's make a value to small error. And also, as a subclass. It has a sub base class, it has the exception class. And now you can, for example, define a custom in it method. So it has the self argument, and then we give it the message and value. And then we can store this variables here. So we can say self dot message equals message, and self dot value equals value. And now, inside our test function, we make another if statement. So let's check if x is smaller than five, and then we want to raise a value to small error. And now we have to give it the message. So the message is value is too small. And then as a value, we give it the x. And now if we catch this, then we want to catch the value too small error as E. And now we have the information about the error so we can print e dot message, and we can also print e dot value. So if I test my function with one, then the value to small error will get raised. So I'm sorry, I'm Miss colon. I forgot the column. So then, um, It catches the error here and prints all the information that I defined here in my error class. Python already comes with a powerful built in logging module. So you can quickly add logging to your application by simply saying import logging. And then you can use this. And in this tutorial, we will have a look at the different block levels, the different configuration options, how to lock in different modules, how to use different lock handlers, how to capture stack traces in your log and how to use rotating file handler. So let's start. So after importing the logging module, you can import to five different, you can lock to five different log levels. So let me copy this here. So the levels are debug info, warning, error and critical. And they indicate the severity of the events. So if I run this, then we will see that only warning error and critical are printed. And this is because by default, only levels of only messages with level warning or above are printed. And if we want to change this, we can do that by setting the basic configuration. And usually we want to do this right after importing them logging module. And then we say logging dot basic config. And then we can specify some arguments here. And for this, I would have a look at the documentation. So in the official Python documentation, you find the different arguments for the basic configurations. So for example, you can set the level and the format, and then the date format. And so in this case, I said level two logging dot debug. And then for the format, I give a string, and inside this string, I can specify this lock record attributes. So for example, I can have the name being locked. And I do this by saying or by writing percent, and then in parentheses, name and then S. Or I can say the ASC time, so this locks the time, then the level name and the actual message. And then I can specify how the time should be locked by saying date format equals and then give a string for the date format. And for this, I can also have a look inside the documentation. So here are the different formatting rules for how to lock the time. So for example, percent m will look the month, then the day then the year, then the hour, the minute and the second. And now if I run this, then we see our new format. And also that debug info, warning, error and critical are all locked. And by default, our logger is called the root logger. So that's because the name here is root. Now, if I want to log in different modules, then it's best practice to not use this root logger. But create your own logger in your modules. So let's say we have a helper module here. And what you do then, after importing, logging, you create your own internal logger here by saying logger equals logging dot get logger. And then you give it a name. And it's also good practice to use this double underscore and then name global variable. So this will create a logger with the name of the module. So it's called helper in this case, and then you can use this logger to lock something so as for example, say logger dot info. Hello from helper and then in your main module. After importing, logging, and setting the config, then for example, if I import this helper module, then it will lock the message from the helper module with the name of this logger. So it's good practice to create your own logger in your modules with this get logger function and then give this name with double underscores here as a name. Now, if I create this log on here, then this will create a hierarchy of loggers. It starts at the root logger, and all these new loggers get added to this hierarchy. And they propagate its messages up to the base logger. So now if I don't want to have this propagation, I can say logger dot propagate equals false. So by default, this is true. And now this will not propagate to the base logger. And now for example, if I run this module and import the helper module, then nothing gets locked here because it doesn't propagate to our base logger. Now let's talk about lock handlers. So handler objects are responsible for dispatching the appropriate lock message to the handlers specific destination. So for example, you can use different handlers to send log messages to this standard output stream to files via HTTP or via email. And ba you let me show you how you set different lock handlers. So first, we create a our logger in our module by saying logger equals logging dot get logger, and then the name of this module. And then I want to create my handler. And let's say I want to have a handler that locks to this stream. So a stream handler. And then this equals logging dot stream handler, though this is a built in class. And I also want a file handler that locks to the file. The file handler equals logging dot file handler, and then it needs a name. So let's say our lock file is called file dot lock. And then typically, for each handler, you want to set the level and format. So we say, stream handler dot set level, for example, set this to logging dot warning. And for the file handler, the file handler should only lock method messages of level logging dot error. And now we also specify some format. So we say, a stream format equals logging dot format. And then inside here, we give it a string, just with the, the same like with the basic config. So let's say we want to have this string here. So we want to have the name of the logger and the level name and the message. And all let's also set the file handler to this format, or just call this format. And then First, we set the formatter to our handler. So we say stream handler, dot set format, formatter, and also we say file handler, dot set formatter formatter. And then at the end, we have to add our handler to the logger. So we say logger dot add handler. And first we want to add the stream handler, and then logger.at handler. And now we want to add the file handler. And now if we use this logger and lock something for example, say logger dot warning, and then we want to say this is warning. And now we also want to have logger dot erawan and Block, this is an error. And now if we run this, what will happen is in our stream, we have warning and error, because our stream handler locks, messages of level warning and above. And then also file handler locks to a file. So now if we have a look, in our folder, there is now this file lock. And this only has the error message. So this is how we can define the front lock handler. Now let's talk about other configuration methods. So we've already seen the basic config method. But we can also use the file config or dict config method. And for this, you will create a file in your folder. And you specify it with this syntax. So you call it logging.com, or logging dot ini. And then you define the loggers, the handlers and the formatters. So in this case, we define two loggers with these names, one handlers, and one format. And then you specify each of these further. So you say logger, and then underscore and then the name of the logger. And then you give it its arguments. So let's say for example, we have a logger called simple example. And this should lock to level debug and above. And it should have a console handler. And then we come and we define the console handler, and this is a stream handler with this formatter. And then we define this formatter and give it the format. And now if we want to use this config file, then in our file, we say, import logging, dot config. And then we can call logging dot config dot file, config, and then give it the file name. So we say logging.com. And now what we can do is we can create a logger with for example, with this name. So this will get the simple example logger. So let's say logging dot get logger. Simple example. And now if we lock something with this logger, say logger dot d back, because it also locks debug, this is a D block message. And now if we run this, then we see we have the message here with this format, the time then the name, the level, and the actual message, just like we defined here for our formatter. And we can also use a dict config, but I won't cover this now. So for this, you should also have a look at the documentation. So the config is just a different syntax that you can use. And then you would right here, logging dot config dot dict. config, and set this config from a dictionary. So with this two methods, you don't have to hard code your configuration in your code. But you can use a separate file that you can easily change without changing the code. So yeah, remember that you can also use these tickets and file conflicts. Now let's talk about capturing stack traces in your lock. So this can be very helpful for troubleshooting issues. So let's say you have a, you run a code that raises an exception. So let's say we have a list with some values 123. And we want to access a value, but we use an index that is too large. So this will raise a index error. And we can catch this by saying except index error as E and then we can say Logging dot error. And by default, this will only now lock the error message live index out of range. But if we also want to lock the stack trace, then we can set the argument e xe info equals true. So e xe underscore info equals true. And this will also now if you run this, this will also include the stack trace in our logger. So now we can see that trace back and the line where our exception occurs. And yeah, so this is helpful for troubleshooting issues. And now let's say we don't know what kind of error we raised. So let's say we just say, accept and catch everything. But we still want to have our trace back, then we can import the trace back module. And we can, for example, look a string. So the error is, and then we use string formatting. So we say percent s, this is a placeholder. And then here, we call this trace back dot format, e x c method. So this will now if we run this, then this will do the same thing. Basically, this will also print this measure message to the lock the error is, and then includes the trace back. So let's talk about rotating file handlers. So let's say you have a large application with a lot of log messages, and you want to keep track of the most recent events, then you can use a rotating file handler that keeps the files small. So for this, let's say, we also have to import this. So we say from logging dot handlers import rotating file handler, then, let me quickly copy this here. You create your logger. Here, you set level. And then here, you create your file handler. So your handler is now a rotating file handler. And then you give it the name of the lock file, then the max bytes. So this means that after two kilobytes, it will roll over the lock to another log file. And it will also keep five backup counts. And then we add our handler to the logger. And then for example, we log a lot of messages. So we say, for underscore, this means that we don't care about this. So for underscore in range 10,000, we look HelloWorld. And now if we run this, we see that in our folder, we now have different log files, all with this Hello, world message. And now if we also have a look at the folder, then we see that each of these files is two kilobytes and after two kilobytes, it gets rolled over. So this is how you can use a rotating file handler. And sorry, now, let's say your application will be running for a long time, then you can use a time rotating file handler. So for this you say from logging dot handlers import, timed rotating file handler, and this will create a lock, a rotating lock based on how much time has passed. So what we will do here is we also create a our handler that is now a timed rotating file handler. And then the name of the lock and then we say when should it roll over. And therefore we can give for example, we can give an S for seconds, an M for minutes, an H for hours, then a D for day. We can also say midnight, or we can give it the weekdays. So, W zero means Monday, W, one means Tuesday, and so on. So in this case, let me rotate this every seconds with an interval of five. So every five seconds, a new file gets created. And we keep a backup of five files. So now if we say, for example, for underscore in range, let's say six, and then we want to lock something. And after this, we want to wait a specific amount of time. So let's import time, and then say time dot sleep. And then we want to sleep five seconds. So now if we run this, we see that our lock file got created. And now if some time has passed, so after five seconds, another log file got created with this timestamp. And then after five seconds, again, another, and so on. So this is the time rotating file handler. And as a last thing, I want to mention that also if you have a lot of different modules, and lock many, many different things, so especially if you use a micro service architecture, then I would recommend to use not locked to this simple messages, but use the JSON format for logging. And for this, I would recommend this open source, Python, Chase and logger. So you can find this on GitHub. And you simply install it with pip install Python, Chase and logger. And then you can define this format, and add this formatter to your handler. And then you log in JSON format. Jason is short for JavaScript Object Notation. And it's a lightweight data format that is used for data exchange. It's heavily used in web applications, so you should be comfortable working with it. Luckily, Python already comes with a built in JS module that makes working with JSON data very easy. So in this tutorial, we will have a look at how we can encode and decode JSON data with this module. So let's dive into it. And first of all, let's have a look at how Jason data looks. So here I have this example file called example dot Jason. And here we see that chastened data looks very similar to a dictionary. So it consists of several key value pairs. And as values it can take strings or numbers, or Booleans, are also nested types, like here, a nested array or a nested dictionary. And we can also have a look at the whole conversion table. And by the way, you can find this on my website, Python minus engineer.com. There you can find written tutorials to all my other video tutorials. And this is how Python is translated to chase and vice versa. So a dictionary in Python is an object and chasen list and tuples are an array is string is a string, integer, long and float are a number and chasten. True and False are also true and false, but with a lowercase and none is now in Chase. And so these are all the conversions you have to know. And let's start working with it. So let's say we have a Python dictionary and want to convert it to a JSON format. And this is also called serialization or encoding. So let's say let me copy this here. Let's say we have a dictionary called person. And this has a name, an age, a city, a Boolean, if it has children, and then titles and this is a nested list. And let's say I want to convert this to a chase an object. So first of all, I have to import the chasen module. And then I can say, if I want to have this in chasen format, I can say person chase Then equals and then I use this module and I use chasen dot dump s, and then the person. So this will dump our object to a JSON string. Now, if I print this, print our person and chasen then I will see that this is now in chastened format. And we can see this, for example, because false has a lower case. Now I can also specify an indent here. And I would recommend setting this indent to four. And now this has a nicer format. I can also specify different separators. And this is a tuple with two values. So here I can specify different separators. So instead of a comma here, I use a semi colon in the space. And instead of a colon in the space here, I want to use, let's say, an equal sign and the space. And now if I run this, then we can see that different separators here, but I would not recommend using different separators, but instead use the default ones. But what's also helpful argument here is to use this sort keys argument and set this to true. So by default, this is false. And now if I run this, then we see that our keys are sorted alphabetically. So this is how we can convert from a Python dictionary to a JSON object. And in this case, to a string. Now, I can also convert it or dump it into a file. And for this, I can say, I open a file. So let's say with open and let's call our file person, the chasen and I want to open it in right mode, open it as file. And then I can say chasen dot DOM, not dump s because S stands for our string, I want to dump into a file. So let's dumb and I want to dump the person object into our file. So now if I run this, then we see that this file got created in our folder. And this contains our JSON data here. So for example, I can also specify the indent here, let's say indent equals four, and run this and have a look at our file again, then we see that it has not a much nicer format. So this is how you convert from Python object to JSON data. And let's say we have chastened data and want to convert it back to a Python object. And this is called D serialization or decoding. So let's say I have our person in JSON format here and I want to convert it back into a dictionary and I will say person equals Jason dot load. So in this case, I want to load from a string and then I will give it the person chasing and chasing and now if I print our person again, and don't print this, then we will see that now we have a Python dictionary again, because here we can see that false is written with an uppercase. So this is how you convert from a chase string. And, like before, if you want to convert from a chasen file and you use the chasen dot load method. So for this we have to open our file so we still have our person dot chasen here in our folder. So we say with open and let's open this file person, that chasen and now we want to open it in read mode as file and then I want to I can say person equals chasen dot load from our file. And then I can print this and if we run this, then we see that this does the same thing. So this is how we can decode chastened data. And now, in this case, we work with a dictionary by Let's say we have a custom object. So let's say we have a custom class, let's call, let's create a class called user. And our user has two instance, variables. So let's say it has a name and an age. Let's say, self dot name equals name, and self dot age, equals age. So now let's create a user object user equals user. And let's say the name max and the age 27. And now let's say I want to have this in JSON format. And like before I call chasen dot dump. So dump from a string, dump as a string. And I want to dump the user. Now if I run this, then this will give a very long error here. And at the end, it says type error object of type user is not chased and serializable. So what I have to do, I have to write a custom encoding function, and this is not very long. So let's say let's create a function called encode. user. And this will take a object. And inside our function, we check if our object is of with this is instant method. So this will check whether an object is an instance of a class. So let's check if our object is of class user. And if so, then we will return a dictionary with all the instance variables as key value pairs. So let's say it has the name. And this equals it, this is our object dot name. And then it has the key h with the value object dot h. And then as a little trick, it will get also the class name as a key. So I can say, object dot with double underscores class and then dot double underscores name. So this would give the name of the class as a string. And then as a value, the value doesn't matter. So I simply put in true. And otherwise, I will raise a type error. Let's raise a type error and as a string or message, I will put this same message here. So now this is our custom encoding function. And now in our dump, or dump s method, I give it that as a default argument. And now here, I use this encode user function. So this now we'll use this function for how to encode the object. And now if I run this, then this worked. So now I can print our user dot JSON. And now we see that we have our dictionary with the name, the age and the user class is key with value true. So this is how you encode a custom object with this default argument. And then there's also a second way so you can implement a custom chasen encoder. So let's say we import from Chase. We import the chase and encoder. And then we create a class called this user and coder. And this is derived from this base chazan encoder. And then we override this default method. So let's say this is called default and this takes self and an object here. And then inside we do the same thing. So we check if our object is of the Last user, and then we'll we will return this dictionary with the class name in it. And otherwise we will we let the base chasen encoder handle it. So we say return chasen encode our default, self and object. And now in our dump or dump s method, I can give it a class argument as a class. Now I use the user in encoder and not the bass chasing encoder anymore. So now if I run this, then we see this also worked. And as a last option, now I can use this encoder directly. So I can say user chasen equals user encoder. Now let's create a user encoder. And then I can say dot encode our user. And now if I run this, then we see this also worked. So this is how you encode custom objects. Now let's say I want to decode our object back. Let's say I have here our user in JSON format. And I want to have it in a normal Python object. So I can say user equals Jason dot load s. And then I will give this user chase in here. Now, if I run this, then this worked. So I can print the user. And now we see we have a dictionary here. So we don't have a user object. So let's check the type of this user, then we see that this is a dictionary here. So for example, I cannot call user dot name, because it's not a user object. But what I have to do if I want to decode this into a user object, I also have to write a custom decoding method. So let's call this decode user. And this will get a dictionary. And now, inside this function, we check if our dictionary contains the user key. So now here, in in our encoding function, we added the user class name as a key. So now, in here, we check if this key is in our dictionary. So let's say if user dot double underscore name in dictionary, and then we will create and return a user object. So let's return a user. And as name, it will get the name from our dictionary, so name equals dictionary and then the dictionary with the key name. And as a age, it will get the age of the dictionary. So age equals Dictionary of age, and otherwise, it will simply return the dictionary. So then still, the decoding will work but it will be decoded into a dictionary. And now we have to use this custom decoding method. So we can say in our chasen dot load or chasen dot load s method, we can specify an argument that is called object hook. And now we set this object hook to our decoding message. And now if we run this, we see that we have our user object so we can let's print that type of user. And then we see that this is now a class user. And we can access its instance variables for example, user dot name, and here it prints max. So this is how you decode custom objects. So Python comes with different built in modules to generate random numbers. In this tutorial, we will have a look at the random module for pseudo random numbers, the secrets module for cryptographically strong random numbers and the NumPy random To generate arrays with random numbers. So let's start with the random module. And first of all, we import random. And this is used to generate pseudo random numbers for various distributions. And it's called pseudo random because the numbers seem random, but they are reproducible. And we will see how we can reproduce the data in a second. But first of all, let's have a look at the different functions. So the easiest one is random dot random. So let's say a equals random dot random, this will print a random float in the range from zero to one. So let's print a. So this is a random float in the range from zero to one. Now, if you want to have a specific range, we can use random that dot uniform and give it a start and a stop. So let's say our range is from one to 10. Now this will produce a random float in this range. Now if you want to enter chess, we can use random dot Rand int, and give it the range. And if we run this a couple of times, hmm. Now it's not happening. But this range will actually now we got it, this will include the upper bound, and you might expect a behavior with this is not included. So for this reason, you can use the RAND range method, so this will do the same thing, it will pick a random integer in this range. But here the upper bound is not included. So this will never pick the 10 here. Then there's the random dot normal variate function with a mu and a sigma. So let's give it zero as mu and one sigma. And this might be useful if you're working in statistics. So this will pick a random value from a normal distribution with a mean of zero and the standard deviation of one. So let's have a look at how this normal distribution looks. This is the normal distribution for different means and standard deviations. So in this case, we use zero and one, so we have to have a look at the red line. And this will pick a random value somewhere in this range where our red line is not zero. So this is the random normal variant. Now the random module also comes with different functions to work with sequences on let's say, we have a list and call it my list equals. And let's create a list with different characters. So if we print this, we will see that each character is now a element in our list. And for example, now we can pick a random choice. So let's say a equals random dot choice from our list. And print this so this will pick a random element. Now if you want to pick more elements, we can use random sample and give it the number of different elements we want to pick. And this will pick unique elements. So it will for example, never pick a twice. And if we want to have a behavior where elements can be can can be picked multiple times, we can use the random dot choices method. And here we have to use k equals three. So this will do the same thing, but now we see it can pick elements multiple times. Then there's also the random shuffle methods. So let's say random dot shuffle our list so this will shuffle a list in place. Now if we print this, then we see that the elements are now shuffled. So these are the most common functions to generate random numbers. Now are Yeah, I said that these are pseudo random numbers, because they are reproducible. And you can do this with the random seed method. So I can say random dot seeds and give it a value here. So let's say for example, one, and then I can do different random operations. So let's say I want to have, I want to print random dot random. And I want also want to print some random integer. So let's say random dot Rand int in the range from one to 10. Now, if I run this, this will produce some random numbers. And then I can reseed again with this value with the same value here, one, and then do the same set of operations. And now if I run this, then we see that these are now exactly the same numbers here. Now, I can also, for example, now I can see it with a different value here, let's say two. And then I do this operations. And then again, I will see it with one and do these operations. And at the end, I will seed with two again and do these operations. And then I run this and now let's have a look. And now we see that all our operations are our random numbers, with a seed of one are now the same, and then all, where I use the seeds to all these random picks are now the same. So this is how you can reproduce your data with this random seed functions. And because these numbers are reproducible, they are not recommended to use for security purposes. And for this purpose, you should use the secrets module. So we can use import secrets. And this only has three functions. And they should be used for things like passwords, or security tokens or account authentication things. So for all these purposes, you should use the secrets module. The disadvantage is that it's it takes more times for these algorithms, but but they will generate a true random number. So and it only has three functions. So the first one is secrets dot ran below. So let's say a equals secrets dot ran below, and then it has an exclusive upper bound. So this will produce a random integer in the range from zero to 10. And 10 is not included. Then you have the secrets dot random bits method. So this will return an integer with K random bits. So for example, let's give it four bits. Now, if you you're familiar with bits and bytes, so for example, here for bits means that it can has four different random random binary values here. So the highest possible number here, and this case would be 1111. So this is 15. So this is two to the power of three, which is eight, then this is two to the power of two, which is four. So eight plus four plus two plus one equals 15. So this will generate a random number in the range from one to 15. No from zero to 15, sorry. Then you also have a secrets choice method. So let's say I have a list. My List equals list and with some characters in here, and then I can use A equals secrets dot choice and my list. And this would pick a random choice that is not reproducible. So this is the secrets module. And now if you're working with arrays, then you can use the NumPy module. So if you have not installed it, just use pip install NumPy. And then you can say import NumPy as NP. So usually, you will do it like this. And then you can say, for example, you want a array with random floats, then you say A equals NumPy dot random dot Rand, and then give it the dimensions. So in this case, I will put in three here. So this will produce a 1d array with three elements in a year. So three random floats here. Now, if I can, I can also use more dimensions here. So I can three, this is now a three by three array. Now if I want to have random integers in a range, I can say, Rand end and give it the range from, let's say, zero to 10. And here 10 is excluded. And then I can give it this size. So let's say also size three a 1d array with random integers. Now, if I want to have a array with higher dimensions, I have to use a tuple here, so I can use a tuple and say, three by four, for example. So this will create a three by four array with random integers. Then, this will also have a random shuffle method. So let's say I have a NumPy array with different dimensions. Now print this array. And then I can say NumPy, dot random dot shuffle, and then our array and now print the array. And this will only shuffle the elements along our along the first axis. So this will never switch elements in between, but only switch elements in the first axis. So this is the NumPy random module. And one important thing you have to know is that the ret NumPy random generator uses a different number generator than the one from the Python standard library. And it also has a different seat. Well, seat functions, so I can also say NumPy dot random dot seeds, and then give it a value, let's say one. And then I can do some operations. So let's say NumPy of print this print NumPy dot random dot Rand three by three. And then I can receipt and do the same thing. And this will now reproduce the same, the same array. And the important thing is that you should use the NumPy random seed method instead of the seed method from the random module that we've seen previously. So these are two completely different seed generators. decorators are a very powerful tool in Python and every advanced Python programmer should know it. In this video, I show you the concept behind decorators how you can write your own decorators the difference between function and class decorators and some typical use cases. I promise you that once you have understood the concept, it is not as difficult as it seems in the beginning, and it might improve your Python knowledge a lot. So let's start there are two different decorators, function decorators and class decorators. more common is the function decorator and it looks like this. So you have a function, call it def. And let's say call it do something. And that does nothing in this case. And above your function, you have an add sign, and then some other function name. So some decorator function name, let's say my decorator. So this is how the decorator syntax looks. And what this does a decorator is a function that takes another function as argument and extends the behavior of this function without explicitly modifying it. So in other words, it allows you to add new functionality to an existing function. So in this case, this function would be extended with the functionality of this decorator. And in order to understand this concept, we have to know that functions in Python are first class objects. This means that like any other object, they can be defined inside another function passed as an argument to another function, and even returned from other function. So now, let's have a closer look at the concept. So let's say we want to, we have a function and call it print name. And this will simply print LX. And then we have a decorator function, call it start and decorator. And now as an argument, it takes a function. And inside our decorator function, we have an inner function called and we call it wrapper. So def wrapper, this is a wrapper function, which, which reps our function. So he inside this wrapper function, we execute the function. And then as I said, I can extend the behavior. So I can do something before. And I can do something after it. So before I say in this case, simply print start, and after it, I want to print and, and then after creating this inner wrapper function, I also have to return it. And now, to apply this, let's first of all, simply, let's execute the print name functions. And if I run this, it prints LX. And in order to apply the decorator, I assign this print name function to now to our decorator function. And as argument I take the print name function. So now the print name function has this new functionality. So now if I run this, we will see that it prints Dart then executes the function and prints Alex and then it prints and, and now the decorator function will do the same thing as this line. So now if I write at start, and decorada, then I don't need this anymore. So this now that's the same thing. If I executed now, it will also print start Alex and end. And now this is how we can extend the behavior of a function with a decorator. So let's see what happens if a if our function has some arguments. So let's say we have a function, call it at five and this takes an argument and then it returns X plus five. And now if I tried to run this at five and this argument I give 10. Now, if I run this, I will get a type error because our wrapper takes zero positional arguments, but one was given. So here, I need the same arguments is here. And to fix this, I can use the arcs and quarks. I will talk about this in another video in more detail. But basically with this syntax, I can use as many arguments in keyword arguments as I want. And now inside our wrapper function, I also call this function with the arguments and keyword arguments. So let's write it like this. And now if I execute it, then it works. So this is how you apply arguments. And now what about the return value, so let's store this in a result. So let's say result equals, add five, and then print the result. Now, if I print this, this will print none here. And to fix this, I also have to save the result of the function here, and then return it from my inner wrapper functions return result. And now if I run this, it can print the result. And now I said last thing, what about the function identity. So let's print the help function of f5. The health information with this help function, and also let's print the name of this function with this double underscore method. Now, if I run this, this will print that help function wrapper, and the function name is also wrapper. So Python got confused now about the identity of this function. So in order to fix this, I can import func tools. And here before my wrapper, I apply also a decorator. That's called func tools, dot wraps, funk. So this will now preserve the information of my used function. So now if I run this, I see that it now knows the help on function at five. And also, our function name is now again, at five. So this is all to complete the decorator funk decorator syntax. So this now is a template for a decorator that you can use for all your function decorators. So let's say call it my decorator, then you can do something before the function, then you execute the function. And then you can do something afterwards. And then you return the result and return the wrapper. So this is the template for a nice decorator. And you can also have a look at this on my website, Python minus engineer.com. Yeah. So now, as we've seen here, we see here a decorator that takes a function that takes an argument so decorators can also take arguments. And what this means this is basically now two inner functions, so an inner function within an inner function. And to make this clearer, we'll look at another example. So let's say we have a function, call it greet, and then it takes a name. And then inside it will print. And now we use an F string. And I've shown this before in another video about strings, so we can say hello. And then inside braces, we use the name. And now we use a decorator and call it repeat and give it an argument num times and set it to three. So I want a repeat decorator that executes this function three times. So how does this decorator now look? First of all, we have the outer function repeat, which also takes num times and then inside it takes out decorator function as we've seen it before, so we have a Decker Ray down. So define a decorator. Repeat. And this takes a function. And then inside here we have our wrapper. And this takes arcs, and Clark's, and we decorate this with our func tools dot wraps Decker decorator. And then inside our wrapper, I simply want to repeat this the number of times I've given here. So I say for underscore, because I don't need this for underscore in range num times. And then I say result equals our function with the arcs and the quarks. And then I return the result, then I return the wrapper, and then I return that decorator. So now if I execute greed LX, then this will be executed the number of times I've given here. So now if I say executed four times. So this is how the concept behind decorators with arguments work. And now let's also talk about nested decorators. So you can stack decorators on top of each other. So you can, let's say we have a function and call it let me copy this here. So let's, we have a function, say hello, which gets a name, then it prints a greeting and returns the greeting. And now we can debug this, we can decorate this with our start. And decorator as we've seen it before. Now, let me copy this here inside. This is our start end decorator, which will print start and end after our function. And we also decorate this with a second decorator and call it D block. And now let me copy this debug decorator in here. So, this debug decorator extracts the name and the arguments and the keyword arguments and then it prints the information of this function it executes the function and then it also prints the information about the return value. So, this will basically print some more information about this function. And so now if I apply multiple decorators to add a function, they will be executed in the order they are listed. So this means now if I say for execute say hello LX this will first of all execute the debug function and then inside the debug function, it will execute the start and decorator function. And then inside this function it will execute the say hello function. So now if I run this, we will see that first of all, it prints calling say hello this is from my Dubuc wrapper. Then it prints start from the start and decorator. Then it prints Hello LX then ends and then again I'm here I am prints the function name and the return value. Hello Alex. So this is how you can apply multiple decorators. And now it's the last thing Let's talk about class decorators. So instead of a function decorator, you can also define a class decorator. So let's say we have our function, say hello. And then it simply print. Hello. And I want to decorate this with a class decorator. And I call this count cause. So class decorators do the same thing as function decorators, but they are typically used if we want to maintain and update a state. So in this example, I want to keep track of how many times I have executed this function. So let's create a class call it count calls. And this has a init method. And it takes self Of course, and then it takes the function just like the decorator function. And then inside the in it, I will save the function as class variable, or as member variable. And I said self funk equals funk. And then I will also create a state. And I call this self dot num calls. So and this is zero in the beginning, so I want to keep track of how many times this got executed. And now in order to write a class decorator, I have to implement the call method. So this also takes self, then the arcs and the quarks. And this is the same as the inner function in our function decorator. And now, sorry, this also has trailing double underscores. And now the call methods allows me to execute a object of this class just like a function. So let's, as an example, let's just print Hi, there, here. And now, let's say I create a object of this class called cc equals count calls. And this takes a function here. So this example, I just use none. And now, since I've implemented this call methods, I can say CC and execute this as a function. So now, if I run this, it prints Hi there. So in our example, I don't want to print Hi there. So, what I want to do now, I want to update the state. So I say self dot num calls plus equals one, then I want to print the number of calls. So, I print this is executed self dot num sorry self dot num calls times and then now this is my man, I also have to execute and return the function. So I say return self dot func and now I call the function with all the arguments and the keyword arguments. And now if I say, if I run this and I say Say hello, then Oh, sorry, self num calls. Now, if I run this, then I will see this is executed one times and now if I run this again, then I will see. Now this is executed two times. So here I could keep can keep track of how many times this is executed. So this is how you can implement class decorators. And now let's talk about some typical use cases of decorators. So for example, you can implement a timer decorator To calculate the execution time of a function, you can use a debug decorator like you've seen before. To print out some more information about the called function and its arguments, you can use a check decorator to check if the arguments fulfill some requirements and the depth the behavior accordingly. You can register functions, like plug ins, with decorators, you can cache the return values. Or you can add information or update the state generators or functions that return an object that can be iterated over. And the special thing is that they generate the items inside the object lazily, which means they generate the items only one at a time and only when you ask for it. And because of this, they are much more memory efficient than other sequence objects when you have to deal with large data sets. They are a powerful advanced Python technique. So let's have a look at some examples. To understand how they work. A generator is defined like a normal function, but with the yield keyword instead of the return keyword. So let's define a function call it my generator. And here I can return or I can yield some values. So here I use the yield statement and yield a value. So I want to yield one. And then I can have multiple yield statements inside a generator function. So I can, for example, also yield two, and then yield three. And now I can create a generator object. So I can say ci equals my generator. And now if I print this, and this will only print that this is a generator object. And now what I can do, for example, I can loop over this object. So I can say, for i in ci, and then I print the value. So this will print one, two, and three. And I can also get the values one at a time with the next function. So I can say value equals next ci, and then I can print the value. So this will print one, and this will execute the function and runs until until it reaches the first yield statement. And here, it returns the value and pauses at this line. So the next time if I want to get the next value, again with this next function, so again, I say value equals next ci, then it will continue here and runs until the next yield statement. So it runs until here, and returns to and pauses here. So if I run this, now, it will, it will print one and two. And if I do it again, then it will also return and print three. And now what will happen if I try to run it a fourth time. So now if I run it, this will raise a stop iteration, because a generator object will always raise a stop iteration if it does not reach another youth statement. So yeah, this is how generators work. And you can also for example, use them as inputs to other functions that take iterables. So for example, the built in sum function takes a iterable. So I can give the generator object here and I can print this. So this will calculate one plus two plus three equals six. Or I can, for example, use the built in sorted method and put the generator object here. So this will return. This will create and return a new list with all the objects in a sorted order. So for example, if I have it the other way around three to one, and then with this, I can sort it again. And then it prints one, two, and three. And now let's have a closer look at the execution of a generator function again. So let's say I have another generator, and I call it countdown, and it takes a starting number. And then I say first of all I want to print starting. And then I say while num is larger than zero, I yield, the num. And then I also want to update the numbers. So I say num minus equals one. And then I create my generator object. So I say CD equals countdown. And for example, I want to start at four. And now if I, let's first of all, run this, and notice that this will not print starting here, so nothing will be executed here. And now the first time, I want to get the first value with, let's say, value equals next of this countdown generator object. Now if I run this, then now it will start from the beginning of this function and execute it. So this will print starting, and then it will run until it reaches the first yield statement. And here, it will return the number and stops at this statement. So I can also print the value. And then it prints four. And again, the next time, I want to continue here with again, with this next statement, let's say print, seed print next CD, then it will continue here, it will remember the current state, so the current number is four, then it will update the number now the numbers three, then it will continue in the while loop. And then it stops again, at this line, and now returns three. So now if I run this, this will also print three, and then again, it remembers the state and the next time I continue, it will continue from here, and so on. And again, if I run this a couple of times. Again, if I print next, then it will also print to and now it will also print one, and now it will raise the stop iteration. So this is the execution in detail. And now let's have a look at the big advantage of generators. So as I said, generators are very memory efficient. So they save a lot of memory when you work with large data. So what this means is, let's have a look at an example. Let's say I want a function, call it first n and it takes a number as input. And this will return a sequence with all the numbers starting from zero all the way up to n. So usually what you would do is you create a list call it nums equals an empty list, then you also say num equals zero. So, this is your start number and then you say while num is smaller than n nums dot append num. So you at the current number to a list then you update the current number. So you say num plus equals one and at the end, you will return this list. So you return nums and now I can say for example, I can say my list equals first n and give it for example 10 and then I can simply print this so now this will print all the numbers from zero to nine in a list. And for example I can also calculate now the sum of this. So this will print 45. And now here with this way, all the numbers are stored in this list. So this takes a lot of memory. And now if I use a generator instead, I can say I define another function first n underscore, Jenna rater. And now it also takes as input, and now I don't need the list anymore, I simply say num equals zero and also the while loop while num is smaller than n. And here, I simply yield the current number. So I yield num. And then I also have to update the number. So I say num plus equals one. So this is the whole implementation of this as a generator object. And now I can, for example, also print the sum of this first and Jenner Raider object. And now you see this will give the same result. And this will also print 45. But here, I don't have to save all the numbers inside this array. So I can save a lot of memory here. And for example, if I analyze this, I can import sis. And now I can get the size of this object. So I can see size persists precice dot get size of this object, this will return the size of this object in bytes. And again, here, I also say, print sis dot, get the size of this object. So first, I print the size of my list object. And then I will print the size of the generator object. And here we see that already, the generator object is smaller. And now let's say I don't have 10 numbers in here. But let's say I have 1 million numbers in here. And the same number of elements in here, then this you see, this takes way more memory. So and use cases like this, the generator object is very useful. So remember this. And another advantage of the generator object is that we do not have to wait until all the elements have been generated before we start to use them. Because we can, for example, get the very first item with the first next statement. And we don't have to calculate all the numbers. Yeah, so this is the big advantage of generators. Now let's have a look at another example to practice the generators. A typical example is the Fibonacci sequence. So we say define feeble, not cheap. And this will give this will get a limit as argument. And the Fibonacci sequence works like this. So the first two numbers are zero and one, and then all the following numbers are a sum of the previous two numbers. So now, we have zero plus one is one. Now one plus one is two, one plus two is three, and so on. So then we have five 813, and so on. And to implement this as a generator, first of all, we have to store the first two values. So we say a and b equals zero and one. And then we say while a is smaller than our limit, we yield the current value, so the current value is a and then we update the current value. So now we say A equals B. And also we in the same line, we update the B value, and now the B value is the sum of A plus B, the sum of the previous two numbers. So we say a, b equals B, and so a is B, and B is a plus b. So this is the whole implementation of the Fibonacci sequence. And now we can say for example, fib equals Fibonacci and as a limit for example, like if it 30 and Now I can loop over this object, I can say, for i in fib, and then print I. And now we see this will print the sequence until, until this limit. And now as a last thing, let's have a look at generator expressions. So generator expressions are written the same way, like list comprehensions, but with parentheses instead of square brackets. And this is a very simple syntax and shortcut to generate some generate to implement the generator expression. So I can say, my generator equals and now I use parentheses and here I can use an expression with a for in loop. So I can say I, for I, in range, for example, 10. And I can also use an if statement, I can say, if I model two equals equals zero, so this will put all the elements all the even elements from zero to nine in a in my generator object. And so for example, I can print or I can loop over this object, so I can say for i in my generator, and then print i. So this will print 0246, and eight. And this is similar to the list comprehension. So the list comprehension works the same way, except that they use square brackets here instead of the parentheses. So I can say, my list equals this expression. And then if I print the list, this will on print the same sequence as a list. And by the way, I can also say I can convert a generator object to a list with the list function. So I can say print list, my generator, and this will do the same thing. And again, let's analyze the size of this. So let's say print sis dot, get size of this object. and here also I want CES dot get size off this objects. And now they here they are almost equal. But let's say again, I have a large number 100,000 Then again, my generator object is much much smaller and saves a lot of memory. So with threading and multi processing, you can run code in parallel and speed up your code. And in this tutorial, we will learn what is the difference between a process and a threat, the advantages and disadvantages of both how and why threads are limited by the Gil and how we can easily use the built in threading and multi processing module to create and run multiple threads or processes. So let's start with the difference between a process and a threat. So a process is an instance of a program. So for example, if I'm running one Firefox browser, then this is one process. Or if I'm running one Python interpreter, then this is one process. And a thread on the other hand is an entity within a process. So a process can have multiple threads inside processes take advantage of multiple CPUs and cores. So you can execute your code on multiple CPUs and parallel processes have a separate memory space. So memory is not shared, but between processes and they are great for CPU bound processing. So this means for example, if you have to, if you have a large amount of data and have to do a lot of expensive computations for them, then with multi processing, you can proceed As the data on different CPUs and this way speed up your code and new process is started independently started independently from other processes and processes are easily interruptible and killable. And there's one Gil for each process. So this avoids the Gil limitation. And I will come to the Gil or global interpreter lock in a second. Now, there are some disadvantages. So process is heavyweight. So it takes more, it takes a lot of memory and starting a process is slower than starting a threat. And since processes have a separate memory space that memory sharing is not so easy. So the so called inter process communication is more complicated. And now on the other hand, threats, so as I said, a threat is an entity within a process that can be scheduled for execution. And it's also known as a lightweight process. And a process can spawn multiple threads. So all threads within a process share the same memory. And they are lightweight. So starting a thread is faster than starting a process. And they are great for IO bound tasks. So this means input output tasks. So for example, when your program has to talk to slow devices, like a hard drive or a network connection, then with threading, your program can use the time waiting for these devices and then intelligently switch to other threads and do the processing in the meantime. So this is how you can speed up your code with threading. But on the other hand, threading is limited by the Gil. So the Gil allows only one thread at a time. So there is no actual parallel computation in multi threading. So threading has no effect for CPU bound tasks. And they are not interoperable and kill killable. So be careful with memory leaks here. And since threads share the same memory, you have to be careful with race conditions. And a race condition occurs where when two or more threads want to modify the same variable at the same time. So then this can easily cause bugs or crashes. And yeah, that's the difference between processes and threats. And now I mentioned a couple of times the Gil. So let's talk about the Gil. And this is also known as the global interpreter lock. And this is a lock in Python that allows only one thread at a time to execute. And this is very controversial in the Python community. But why is it needed. And this is needed because in C Python, so C Python is the reference Python implementation that you get when you download and install Python from python.org. So the gala is needed because in C Python, there is a memory management that is not thread safe. So in C Python, there is a technique that is called reference counting for memory that is used for memory management. And this means that objects created in Python have a reference count variable that keeps track of the number of references that point to the object. And when this count reaches zero, the memory occupied by the object can be released. And the problem now in multi threading is that this reference count variable needs needs protection from race conditions where two threads increase or decrease the value simultaneously. So if this happens, it can either leak, it can cause leaked memory that is never released. Or it can incorrectly release the memory while a reference to that object still exists. So this is the reason why they introduced the Gil. And a couple of ways to avoid the Gil if you want to use parallel computing is to use multi processing Or you can use a different free threaded Python implementation and not c Python. So there's, for example, Chai THON or iron Python. Or you can use Python, Python as a wrapper for third party libraries. And this is the way it's it works in NumPy artists Sai pi modules. So they are basically just wrappers in Python, that then call code that is executed in C. So yeah, that's enough theory. And now let's jump right into code. So let's start with multi processing. And for this, you simply say from multi processing, import a, the process I'm sorry. And now I create a list called processes, where I will store all my processes. And now I define a number of processes. And a good number usually is the number of CPUs on your machine. So you can say import o s, and then we say num process processes equals o s dot CPU count. So on my machine, there are four different CPUs. And then I will create the processes, so create processes. So I will say, for i in range, num processes, P equals a new process. And this takes two important arguments. Now the first one is target and the target function. So this is a callable object or a function that is then executed by this program process. So I have to define a function here. So I say, let's define this up here. So let's say in this example, let's say their square numbers. And here, I will say for i in range 100. I will simply say i times i. So this is a dummy example, that's basically not useful, but just for how to show you to show you how to use different processes. So this is the function that my process should execute. So I say target equals square numbers. And if my function here has some arguments, so then I would also need to specify arcs equals and then as a tuple. Give the arguments here. So in this case, I don't need them. So now I created my process. And then I say processes dot append, my process. And now I want to start each process. So I say for p in process, and then I say P dot start. And then I also have to join the processes. So I say for P and process, P dot join. So this means that I want to wait for a process to finish. And while I'm waiting, I am blocking the main thread. So here I am waiting for all processes to finish. And I blocked the main thread until these processes are finished. So now at the end, I can for example, simply print and main. And I will only reach this point when all processes are done. And now if I execute this, let's For example, let's also import time and tear. Just to show you the different processes, I will wait some time and say time dot sleep 0.1 and now I am having a look at the activity manager or the task manager. So here I can filter for processes. So I say I filter for Python. And as you can see that I've already two Python processes running, they all have a different process ID. And they all, it's also shown how many threads are inside my process. So now if I'm executing this Python file, then we will see what will happen. So it takes a couple of seconds. And now we see five Python processes coming up. So this is the main process, and then the four process processes I created here. And now after a couple of times, after this is finished, they will disappear again. So we can see that there are actually different processes now running on my machine. And this is how we can use multi processing. And now let's talk about multi threading. So the threading API is very similar to the previous multi processing IPA API. So here, I say from threading, import threat. And then here, let's call these threats, and num. Now I call this number of threats. And let's simply say I want to have 10 different threats. And then for i in num threats, now I create a threat. And this takes the same arguments. So it also has to define a target. And if my target has some arguments, then I would also have to specify the arcs here. And then I say, threats dot append my threat, then I will start each threat. So I will say for t in threats, T dot start and also join them. So I will say for T and threats, T dot join. And now let's have a look at the activity manager again. Now if I'm running this Python file, then we will see takes a couple of seconds. Now we will see one process coming up with 11 threads inside so the main threads and the 10 child threads that I created here. And now processing is finished and the threads disappear again. So this is how you can use the threading module. In this video, we will go into more detail about the threading module. So we will quickly recap how we can create and start multiple threads, then we will learn how we can share data between threads and how to use locks to prevent race conditions. We will also learn what is a daemon process and how we can use a queue for thread safe data exchanges. So let's start and let's quickly recap from the last video how we create and start threats. So this is the code where we left off. So we say from threading import threat. And down here we define a num so we want 10 threats here. And now we create our threats. And for each threat, we give them a target method. So this is the function that the Stan executed by this threat. And then for each threat, we also have to say thread dot start. And also threat dot join. So join means that we wait and block the main threat until the threat is complete. So yeah, this is how we can can use the threading module. And now let's go into more detail. And let's talk about how we can share data between threats. So since threads live in the same memory space, they have access to the same data. So this makes sharing data very easy. So we can for example, just use a global variable here. So let's define a global variable. And in this case, I will call this database value. So and I will set this to zero in the beginning, so this should simulate a database now And now in our main code, what we will do is, we will first of all we print, print the start value. So we print our database value here. And then we will create two threats. So let's say threat one equals threat. And this will get a target method that we will call increase, and also threat a second threat. So threat to that does the same thing. And then for each threat, we say threat start. And also, threat join. So we wait for the threads to complete. So thread two dot join. And at the end, we print the end value. And then we again, want to print a database value at the end. So and now we have to define this increase methods. So we say define increase in here, we want to get and modify our database values. So in order to modify the global variable, we have to say global database value. And now we can use it here. And now let's make some dummy code. So we want to simulate some database access, we want to get the value from the database and store it in a local copy. So we say local copy, equals and here, we can simply copy it from our database value. And then we want to do some processing. So here, we simply say, local copy plus equals one, so we want to increase it. And then we simulate that this should processing should take some time. So we import time. And then we wait some time here. So we say time dot sleep 0.1. And then when we are done, we want to write our new value back into our database. So we simply copy it back and say database value equals local copy. So this is our increase function. And now let's run this. So now we have two threats. And if we run this, let's clear our console. And let's run this again. So we he sees start value is zero, and end value is one. So now you might be wondering, why is this one because we have two threats. And both threats should increase our database value. So now the end value should actually be two. And now why is it one and this is because we have a race condition here. So a race condition happens when two or more threads try to try to modify the same variable at the same time. And now let's step through this code what is happening here. So when we say thread one dot start, then it will get the database value and store it in a local copy. So in the beginning, this is zero, and then we will modify the local copies. So now our local copy is one. And now since we say time dot sleep, our program can intelligently switch to the other threads and use the waiting time. So now it switches to threat number two. And now thread number two invokes this increase method. So it also copies the database value in the local copy. And the local and the database value is still zero because we didn't write it back here. So now thread two also has a local copy, which is zero and then it increases it's to one and then we again say time that sleep so we can switch back to threat number one. And now threat number one copies it's it's a Copy that is one into our database, and then it's done. And then we are switched back to thread two again. And that now also copies its local copy that is also one here into the database value. So this is why the end value is one. And now how can we prevent this. So for this, we use the lock object. So we say from threading, import, lock, and then we create a lock here. So we say lock, dot lock equals lock. And now we say our increase method gets a lock. So we have to give this year in the arguments. So we say arcs equals lock. And since this is a tuple, with only one element, we also need a comma here. So Python needs this comma here in order to know that this should be a tuple. And also, for our second threat, this will now get the lock as an argument. And now with a lock, we can so a lock prevents another threat to access this to access this code part at the same time. So now we can say lock dot acquire. So it basically has two, only two methods. So we say lock dot acquire. And now we can process and modify the value. And at the end, when we are done, we say lock dot release. So and we should always, every time we acquire a lock, we always have to release it. So Otherwise, this will block and never release. So then we are stuck here. And now what is happening with this lock. So now our let's run this and see if this works. Lock dot acquire. So Oh sorry. This is a lock object. So now let's run this. And now we see that it's correct, our end value is two. So what happened here, so now our first threat got here. And since it locked the state, now it can modify the value. And it will not switch back to our threat number two here, because it's the state is locked. So it can count it continues and runs and copies the local copy that is now one into our database, and then it releases the lock. So now our second threat can enter this code part. So it also gets the database value. And this is now already one and then it modifies it to two and writes it back. So now this is working fine. And as I said, you should never forget to lock to say lock dot release. So there's a recommended way to use locks. And this is to use a lock as a context manager. So you can simply say with lock colon, and then use the part of the code here. And then we don't need to say lock that release. So let's also get rid of this. So if we run this, then we see this also works correctly. So this context manager acquires and releases the lock for you. So yeah, this is the concept of locks. And now let's talk about how we can use queues in Python. So queues are excellent for thread safe and process safe data exchanges and data processing in multi threaded or multi processing environments. And for this, we simply we have to import the queue so we say from queue import queue. And now let's get rid of this. And first let's have a look at how a queue is working. So a queue is a linear data structure that follows the feefo or First In First Out principle. So a good example of a queue is a queue of customers that are waiting in line where the customer that came first is also served first. So, let's create a queue object. So we say q equals Q, and then we can put in some elements. So we say q dot put one, and Q dot put to two. And also, let's put in us a third object, so say q dot put three. And now our code looks like this. So first, the one is, enters our Q, then we put in the two, and then we put in the three. And here, this is our front. So the beginning of the queue. So now if we want to get the first value, we can do this by saying first equals q dot get. So this will get and remove the first item. So if we print first, then this will print one. And now our thread, our queue only has three and two inside. So this is how the queue principle is working. And there are a couple of other important methods. So first, you can check if a queue is empty with Q dot empty, this will return true if the queue is empty. And then in a threat threading environment, whenever you get a object with queue dot get, and then you process this object. When you are done processing, you should always call queue dot task done. So this now tells the program that we are done processing with this object and can't can can't, can continue. And there's also a Q dot join method. So this blocks until all items in the queue have been gotten and processed. And this is similar to the thread dot join methods. So with this, we block the main thread and wait until all the elements in our queue are processed. So these are the the important methods you have to know. And now let's look at an example to how we can use this. So we say Um, also, we want to define a couple of threats. So we say num threats equals 10. And then we say for i in range, num threats. And here we create our threat. So now we say threat equals threat. And as a target, it needs to get a function. So we will define this in a second. And then we say thread dot start. And we will also now use a demon threat. So we say threat dot demon equals true. And I will explain what this will do in a second. So by default, it is not a daemon thread. And then let's define our function. So our function, let's call them worker, and define this worker function up here. So we say def worker. And now we use an infinite loop. So we say while true, and then we say value equals q dot get. So this will get two arguments, this will get the queue and also a lock. And here, we will get the first value inside our queue with Q dot get and then we will do some processing with it. So in this case, we simply want to print the values so we say print and let's import the current so let's say from threading import current threads, so we want to print this here. Let's use an F strings so we can say we are in and then we are in our current threat dot name. And in this threat we got the value. So we simply want to print this here. And then we are done. Remember, we have to say q dot task done. And now what this is doing, this is an infinite loop that is now starting. And since we don't have values inside our queue, this Q get method will block and wait until items are available. So now we wait here, so we have to fill our queue with elements. So we simply say for i in range, let's say one to 21. So we want to fill this with all the numbers from one to 20. And we say q dot put AI. And then at the end, we have to say q dot join. So we block the main thread and want to wait until all the items have been gotten and processed. And yeah, then we print and main. And now let's run this code and see what's happening here. And here, we also have to, now this worker gets no, let's leave this lock, and only gave it a cue. And now as arguments, we have to give it in a tuple. Again, we give it the cue and don't forget the comma here with one item. And now let's run this and see what's happening. And now we see that we have threads with different names from one to 10. So we have 10 different threads. And they get the values from our queue and can process the item here. And the order might not be sequential. But what is important here is that with a queue, we can easily exchange the data in a thread safe environment. So this value, this queue gets call is thread safe. And also the queue put calls are thread safe, so no other thread can write at the same time into this queue position. And now let's run this again. So in this time, we, this time, we got lucky, and we got lucky again. So what might happen here is that multiple threats might try to print at the same time. So there might be print statements that are in the same line. So two statements in the same line and no line break. So to make it work correctly with we should say with lock and use a lock here. So let's also give this a lock. Let's say down here, lock equals lock. And then as argument, it also gets the lock. And now this should work fine and never produce confused lock statements. And yeah, let's have a look at what is happening here again, and why we use a demon threat. So we are our threats, entered this infinite loop. And then it blocks here because we have no items inside our queue. And then as soon as items are available, then it can continue here and process the items. So in this case just prints it and then after all the items are done, we can continue here then this will unblock and then we will continue and print n main and then we will leave the main threat. And now a demon threat is a back ground threat that will die when the main thread dies. So you might be wondering, we have an infinite loop here. Why do we Why does our program correctly stop so we say and main and then it's it's done and I can use my command line here again. So a daemon thread dies when the main thread dies. So if I reach this statement and then exit the main thread, then all the threads die. And so the worker method and the wild true loop no longer gets invoked. And this is why we use a demon threat here. So by default, this is false. And if we don't use a demon threat here, then our program will still continue here in our wild true loop. So, what we should do then is we should use another mechanism, for example, some some signaling mechanism like an event to say that now we are done and we can exit the wild true loop. So then we should have used here, if some condition and then break. So yeah. In this video, we will go into more detail about the multi processing module. So we will quickly recap how we can create and start multiple processes. Then we will learn how we can share data between processes, and we will recap how to use locks to prevent race conditions and how to use queues. And at the end, we will learn how to use a process pool to easily manage multiple processes. So let's start and let's quickly recap from the last video how to create and start processes. So we say from multi processing import process. And then down here we define a number of processes. So a typical good choice for this is the number of processes on your machine. And you get this with Oh s dot CPU count. And then you create your different processes with process equals process. And this takes a target is a function argument and the target is a callable function. This is that this then executed by this process. So we define a function appear, this simply squares, some numbers. And then we give this to our process here. And then for each process, we call process dot start. And also process dot join. So this says says that we want to wait for all processes to finish and block the main program until all these processes are done. So this is all we need to set up multi processing. And now let's go into more detail. And first, let's talk about how we can share data between processes. So in the last video with multi threading, we learned that we can easily share data between threads with a global variable. And now with processes, processes don't live in the same memory space, so they don't have access to the same public data. And because of that, they need special shared memory objects to share data. And there are two shared memory objects that we can use, we can use a value for a single value, or we can use an array. So we say from multi processing, import value and import array. And down here, let's first start with a single shared values. So we say shared number. And this is now a value. And this takes two arguments. First, we have to give it the data type as a string, so we give it an i for integer and a starting value. This is no cirro. And first of all printers so we say number at the beginning is and then we say we access this shared number with or this value with share number dot value. So now if we run this, and we see that this is zero, and now let's create two processes that should modify this number. So we say process one equals process. And as a target, it gets a function that we call at 100. So let's define this up here. So let's define adds 100 and this gets a number and then it should modify this number a couple of times. So we say for i in range 100. So 100 times it should say number though value plus equals one. So it should increase this by one. And we also want to modify a behavior that takes some time. So we say, time dot sleep and 0.01. And so here we give it this at 100 to our process, and it also needs arguments. So we say art equals, and this is a tuple. So here, we give it this shared number. And be careful. Since this is a tuple, with one element, it also needs a comma here so that Python knows that this is a tuple. And then we create a second process that will do the same thing. So process two, that should add 100 to our shared variable. And then we say process, one dot start, and process two dot start. And then we wait for them to complete. So process one dot join, and process to the chain. And now at the end, we again, print our numbers. So we say number at end is and then access it with share number dot value. So let's execute this and see what happens. So the beginning is 100. It's zero. And now we got lucky. Now let's run this a second time. And now it's not 200. So it's only 168. And why is that because here a race condition happened. And I will not explain this in details, please have a look at the previous video. There I explained in detail how race conditions occur. So a race condition occurs when two threads or processes try to access and modify the same shared variable at the same time. So in this case, both processes try to read and write into this object at the same time. So some operations might get lost here. So in this case, it's only 168. And to prevent this, we must use a lock. So we say from multi processing, import lock. And I also talked about this in the last video, so please check this out. So a lock prevents another process from accessing this at the same time. So to use this, we create a lock object, so we say lock equals lock. And then we give this to our function to function. Yes. So this now also takes a lock. And a lock has two important methods. So first, we say lock dot acquire. And then at the end, we say lock dot release. So as soon as we say lock dot acquire, it will set this in a locked state. So this means that while this is running, no other process has access to this code and can execute this part here. And then when we unlocked the state, again, with lock that released and the second process can also execute this. So this is all we need to prevent multiple process to modify this at the same time. So now let's run this. And now we get 200. Let's run this again. And yeah, still working. And a better way to use locks is to use locks as a context manager. So whenever you say lock acquire, you always have to call locked release, then at the end, otherwise, this will block and your program cannot continue. So don't forget this. And you can use a lock as a context manager. So we say with lock, colon, and then your code. So this automatically will acquire and release this for you. So yeah, this also works. And now this is how we can share a single value. And now let's share a array. So we say shared array equals array, and this also needs a data type. So in this case, that's given a D for double And here given a list as initial values, so we say, put in 0.0, here, 100.0, and 200.0. And then we say our array, at the beginning is, and let's say shared array. And then we have to access each element with inside brackets and then with the index. And we can also use slicing here to access all in the indices. So let's just put in a colon here, and then again, print this at the end. So at the end, we want to print our array. So array, at the end, is this. And now we have to change our functions, this now takes multiple elements. So this takes numbers. And then we have to go over each number here and increase it. So first of all, let's also change this parameter here. So let's say shared array. And now, in our, in our function, what we want to do is we want to go over each number and increase it. But be careful here. So we cannot say for number in numbers, and then simply say number plus equals one. So now if we run this, this will print our error at the beginning and at the end, and this is still the same. And this is because this loop here will create a local variable called number that is then increased. So this has nothing to do with our shared value object. So in order to do this, we have to say for, and let's say for i in range, and then the range has the length of our array. And then we access each element with numbers, dot i, and save plus equals one. And now let's run this. And now we see that it got modified. But we also have race conditions here. So don't forget the lock. So we say with lock, and then our modification operation. And now we increased each element in our array by 200. So this is how we can use the shared value and shared array. And we can also use a queue to exchange elements between processes. And I also already showed this in the last video. So a queue can be used for process safe data exchanges. And so in the last video, we set from Q, import Q. And there, we have to use a slightly drif different queue that was formed from the multi processing module. So this has all the same methods except the the tasks done and the tasks and the join method. So a queue is a linear data structure that follows the first in first out principle. So the first element that you put into your queue, that is then also the first element that gets retrieved when you want to get elements. So let's make an example to use a queue and exchange data between multiple processes. So in this case, let's say q equals Q, and then create two processes that should do should access and write to this queue. So we have a process that gets as a targets, it gets a function that we call square. And as arcs it gets some numbers and our queue and then we curate a second process that has In a second different function here, so we call this make negative, and it has the same arguments. So it will write to the same queue. And now let's define our functions here. So we say that f square, and this will get some numbers, and Q. And then we say, so in this case, for i in numbers, we calculate the square and put it into our Q with Q dot put i times i, and then a second function make negative. And there we also it also takes some numbers and a Q. And here, we also go over our numbers for i in numbers, and then we say q dot put minus one times i. And then let's say, let's start our processes. So pros, one dot start, and process two dot start, and then process one dot shine, and process two that shine. And here, we don't have to call q dot join, because there is no methods cue that join. But what we can do is we say while our queue is not empty, so while not Q dot empty, and then we want to print each element. So print q dot get. So this will return and also remove the first element in our queue. And yeah, now let's run this and see what happens. numbers is not defined, oh, sorry, I have to create a numbers variable. And I will say this is a range object from one to five. So or five should be included. So I say from one to six. And now this will print each element. And we see that both processes have access to this queue and can write put elements into it. And then in our main process, we can also access the queue and get the elements back. So this is how we can a can use a queue. And now as a last thing, let's talk about a process pool. So a process pool can be used to manage multiple processes. So a process pool object controls a pool of worker processes to which chops can be submitted. And it can manage the available processes for you and split, for example, data into smaller chunks, which can then be processed in parallel by different processes. So let's have an example how this works. Basically, a pool takes care of a lot of things for you. So you don't have to consider a lot. So we simply say from processing, import pool. And then down here, we create a pool. And we say, so we say pool equals pool, and then it has two or four, let's say it has four important methods that you have to know. And for the rest, I would recommend to have a look at the documentation because there are a lot of more methods but the most important ones are map, apply, join and close. So what we want to do is we want to create multiple process that should access a or execute a function. So we call define a function cube. And this takes a number and returns the cube. So it will will return number times number times number. And now here we can simply say or we create some numbers. So numbers equals a range object from zero to 10 or 10 or only to nine. And then we say, pool dot map. And now we map we have to give it a function, so we give it the cube, and the numbers. And this will return a result that we can then print. So print our result. But first of all we have to, so this will, what this will do this will automatically allocate them the maximum number of available processes for you and create different processes. So typically, this we'll create as many processes as you have cores on your machine. And then it will split this iterable into an equal into equal sized chunks, and submit this to this function. And this function is now executed in parallel by different processes, or by different processes. So this is all you need to write and then the pool will take care of the rest. So this will allocate the pools, it will split the data and then run this method in parallel. And when it is done, it will return the result. And we have to call pool dot close. And then we can call pool the join. So this means that we want to wait for the pool to process all the calculations and return the results. And we have to remember that we should call pool that close before. So now if we run this, we can print our result and we can see that it has the cube here. So yeah, this is how we can easily use the the pool to run different processes with a function. And if we simply want to have one function executed by a pool, then we can say pool dot apply. And then also the cube. And then in this case, it will only has one number year. So we can for example say Apply numbers, the first element. So number zero, so this will execute a process with this function with one argument. And yeah, so this is the most important things about pools. And there are also asynchronous calls to this map and apply functions, but I will not cover them here. First of all, we will learn what is the difference between function arguments and function parameters. Then we will talk about positional and keyword arguments, then about default arguments and variable length arguments. So what are the arcs and quarks arguments for then we will talk about container unpacking, we will also talk about the difference between global and local arguments inside of functions. And finally, we will have a look at how arguments are passed to functions and if they can be modified within a function. So let's start and let's quickly talk about the difference between our arguments and parameters. So parameters are the variables that are defined or used inside parentheses where defining a function. And arguments are the values passed for these parameters while calling a function. So let's make an example. Let's say we have a function called print name, and it gets a name. And then we simply print this name, then this name here is our parameter. And when we call this function, let's call print name with a string LX. Then this is the argument for this function. So there is a difference when we talk about them. Now let's talk about positional and keyword arguments. So we can pass arguments as positional or keyword arguments. And let's make another function as an example. So let's say we have a function foo, that has three parameters A, B, and C. And we simply want to print them Print A, B, C, then we will call this function with positional arguments. So we can say foo, and then just one, two, or three. So this will print 123. Or we can also use keyword arguments. So we say A equals one, b equals two, and C equals three. So this will also work. And note that if we use keyword arguments, then the order is not important. So I can say, for example, C is one, and a is three, and a is the first one that is printed. So let's see what happens. So it prints 321, and not 123. Like I like the orders here. So when using keyword arguments, then only the keywords matter and not the position. We can also use a mix of both. So I can use a positional argument first, that's a one and then I can use keyword arguments. So b equals two, and C equals three. So this will also work. But I cannot use another positional argument after a keyword argument. So if I try to call it like this, then this will raise an error. And also, if I try to assign a a again, so A is the first positional argument. And now if I use a as keyword argument, then this will also raise an error. So yeah, that's the difference between positional and keyword arguments. And yes, sometimes it's better to use keyword arguments, because it makes it more clear what they present. Or we can rearrange the arguments in a way that makes the most readable. So yeah, then we have the possibility to add default arguments. So I can say D and give them give this parameter a default value. So let's say d equals four. And now if I, I can call this fool, and just with three arguments now, one, two, and three. And let's also print D here. So if I ran it like this, then it will take the default value for D. And I don't need it here. But I can also give it a different value. So I can say, seven here. So this will print 1237. Yeah, so default arguments must be at the end of your function parameters. So for example, if I have one here, b equals two, and I try to run this, then this will give an error. And now let's talk about variable length arguments. So probably, you've also already seen functions that looked like this. So they have some parameters A B, and then have at this star, and arcs argument, and sometimes also with with double stars and quarks. And now what these are, so this is a function. If you mark a parameter with one asterisk, or one star, then you can pass any number of positional arguments to your function. And if you mark your parameter with two stars, then you can pass any number of keyword arguments to this function. And typically they are called arcs and quarks, but you can call them whatever you like. So also, for example, C. So inside this function, let's print a and b first, and then this is a tuple inside your function. So we can go over this tuple and say for arc in arcs, and then print arc. And the quarks argument is a dictionary so I can say For key in quarks, and then I want to print the key and the value of this dictionary entry. So I say quarks key. And now I can call this function. For example, at least it needs the two arguments A and B. So I can say, one and two. But then I can also use as many positional arguments as I want. So I say, maybe 345. And then I can use some keyword arguments. So I can, for example, say six equals six, and seven, equals seven. And now let's run this. And let's see what it prints. So first, it prints the two positional arguments one and two, then it goes over our, our arcs. So this is 345. So it prints each number in a different in a new line, and then it goes over the keyword arguments and prints the keywords and the value. So this is how we can use variable length arguments. And for example, I don't need them. So I can also simply use keyword arguments here. Or I can use some more positional arguments and don't use the keyword arguments. Sorry. So this is also possible. Now, let's talk about force keyword arguments. So sometimes you want to have keyword only arguments, and you can force enforce that. So you can, for example, give a write a star here, and then some more arguments after this. So let's say C, and D. And then I want to print them here, print ABCD. And now every parameter after this star must be a keyword argument. So if I write it, like call it like this, then these are positional arguments. And then this will raise an error. So I have to say C equals three and D equals four. And then it works. Or, if you, for example, use the arcs variable here. And then each parameter after that is also a keyword only parameters. So let's say C, and D, and then simply print the last two. So if I write it like this, or for example, to make this more clear, let's call this last. And then for arc, in arcs, print, arc, and then print. Last. So if I caught it like this, then it's missing, then sees this parameter only as your arcs. And then it says that the last keyword parameter is missing. So I need another one. And now I needed as keyword arguments. So I say last equals 100. And then it's working. So this is how you can enforce to have keyword only arguments. Now let's talk about unpacking arguments. So if we have a function, let's say again, fool with three arguments, A, B, C, and we simply print them a, b, and c. Now let's say we have a list, my list equals 1012. Here, then we can easily unpack this list into our function in a function in the function call. Have this list unpacked into the arguments. So I can say, star and then my list. So this will unpack the first item into a second into B and the third into C. And this also works with a tuple here, so I can have a tuple. Here. The only thing is that is important is that the length of your container must match the number of parameters here. So for example, if I have another item here, then this won't work. Now if I have a dictionary, so let's say my dict equals, and then the, it must have the keys with the same names as your parameter names here. So a, and then some well you want, then the second B, and some well you and also see. And some well you then I can unpack a dictionary with two stars here. So I say two stars, and then my dict. And then this will also work. But here, also, the length of this dictionary must match the number of parameters here. And also, the keys must match the name the parameter names here. So for example, if I use e here, then this will also raise a type error. So yeah, this is how we can quickly unpack a dictionary or a list into our function arguments. And now, let's talk about local versus global variables. So let's say we have a function. Again, foo. And this now First, let's say we have a global variable somewhere. So we call this number and say, the number is zero. And then inside, we create a local variable x and access this allow a global variable, so we can say x equals number, and then let's print let's say number inside function, and then print x. So and then we can call this. So let's call foo, then this will print the number inside the function, so we can access this number here. But if we want to modify it, so let's say number equals three, then what will happen, then this will raise a, a arrow here, because then what this will do here, this will create a local variable that is now different than this global variable. So if you want to modify this, then we first have to say, global number, and this is the name of this global variable. And then we can say number equals three. So this will work. So now if we print the number after our function call, then this will print the new value three. And if we now what will happen if we don't write this global here, and we don't have this and simply assign number two three. Then now what will happen if we run this now this x is not available anymore. So let's run this. So this will Print zero, it prints the number that is still zero even after the function call where we set number equals three. And this is because here, we create a new local variable. So this has nothing to do with this global variable. And this is the, this only lifts inside your function. And it will not modify your global variable. So if you want to modify the global one, then you have to write global number here. And now it will print three. Yeah, so this is the difference between local and global variables. Now let's talk about parameter passing. So maybe you've heard already of the term, call by value or call by reference. And in Python, it's a little bit different. So it uses a mechanism, which is known as call by object or call by object reference. And there are two rules that must be considered. So parameters are passed are the parameters passed in? No, sorry, the parameter passed in is actually a reference to an object, but the reference is passed by value. And there is a difference between mutable and immutable data types. So this might be a little bit confusing. But this basically means that mutable objects like lists or dictionaries can be changed within a method. But if you rebind, the reference in the method, then the outer reference will still point to the original object and is not changed. And immutable objects like integers or strings cannot be changed within a method. But immutable objects contained within a mutable object can be reassigned within a method. So let's look at some examples to make this clearer. So let's say we have a function foo. And this takes an argument x, and then it reassigns. This so it says x equals five. And now let's say we have a bar equals 10. So this is an integer, and then we call foo with this bar. And after this, we want to print all of our now this will still print 10, even if we assign X to five here. So because what happens here that var is an integer, and this is an immutable, immutable type, so it cannot be changed. And that this will create a local variable called x here, that has nothing to do with this. So this is the same with the global and local variable difference. But yeah, so immutable objects cannot be changed, but mutable objects can be so let's say this will get a list. And then we can modify this list. So we can say a list dot append an item, so let's append four, and let's create my list. And this is, has three elements one, two, and three. And then we passed this list and then call this function with the list and then afterwards, print the list. So then we see the list got modified. So immutable objects can be modified within a function. And also immutable objects within a mutable object can be changed, so that immutable integers within this list can be changed. So I can for example, also say, list and access the first index, so index zero, and this is now let's say minus one 100. So this will also change the global list here. And but what is what is not possible so if we rebind a mutable reference here. So if I say for example, first if I say, a list equals, and then let's say 200 300 400. And now I call, I create my global list here, I call this function, and then I print it, and it will still print the original list 123. And this is because I rebind, the reference here. So this is now a local variable, a list with this new ways and new values. So this has nothing to do again with the global variable. So yeah, maybe now the four points are more clearer. So again, mutable objects can be changed. immutable objects cannot be changed. But immutable objects contained within a mutable objects can be changed. And like here, if we rebind, the reference in the method, then the altar reference will not be changed. And let's have a last very quick difference that how this can affect your list. So first, if we say, a list, instead of writing append items, we can, for example, say plus equals and then a new list. So if I write it like this, and now if I run this, then my outer list here, my global list, got affected by this. But now if I say a list equals a list, plus this, then if I run this, then this will not change the original list. So this is a slight difference, but it can have a big effect. Because here again, this will create a local variable. So be careful with this slight difference. So plus equals, again, will change the list. So in this tutorial, we will talk about the different use cases of the asterisk or star sign in Python. So it can be used for multiple different cases like multiplication and power operations, the creation of lists or tuples, with repeated elements, for arts quarks, and keyword only parameters for unpacking lists tuples, or dictionaries into function arguments, for unpacking containers, and for merging containers into a list or merging two dictionaries. So we will have a look at all of these use cases. First of all, of course, there is the simple multiplication operation. So let's say result equals five times seven. And then if I print the result, then this will print the multiplication of these two. Or if I use two stars, or two asterisks, let's say two, and then two stars and then a four, this will be a power operation. So this is two to the power of four equals 16. This is one use case, then it can be used to create lists tuples, or strings with repeated elements. So let's say I want to have a list called Ciro's equals, and then I write one element. So I say, one item here, so zero, and then I write times 10. So this will create a list with 10 elements, and each element has a zero. So this is my list. I can also put in multiple initial items here. So if I write it like this, then this will repeat zero and 110 times. I can also use a tuple here. And it also works with strings. So if I say, let's say a B here, then this will create a new string with 10 times A B So next is to use the star or asterisk for the arcs and quarks and keyword only arguments. So if you don't know what this means, please watch my last video about function arguments. So probably you've seen a function that looks like this. So that define a function called foo. And then it has some arguments. And then also some arcs with one star, and with quarks with two stars. So and then let's print A, and now arcs is a tuple. So I can go over this tuple for arc, in arcs, and then print arc. And quarks here is a dictionary. So I can say four key in quarks, and then print the key and also the dictionary value of this key. And now, I can call this function with the A and B arguments. So let's say one and two. And then for this arcs, I can use as many positional arguments as I like, so I can say 345. And then I can also use as many keyword arguments as I want. So I can say, for example, six equals six, and seven, equals seven. So this will print my function here. Um, forgot the beat here. And then if I only use one star here, and then another parameter here, then all parameters after this star are keyword only parameters. So if I want to print C, I cannot call call the function like this. So because here, the last item must be a keyword argument, so I have to write C equals three, and then it will work. So this is another use case of the star operator to enforce keyword only arguments. Then we can also use the asterisk for argument unpacking. So let's say I have a list, my list equals and it has three elements. So 012, then I can call this function and unpack this list here with one star, and then my list. So this will work. And the only thing that is important here is this is that the number of arguments must match the number of parameters here, the number of elements in the list must match the number of RF parameters here. So if I have another one, then this will raise an error. And this will also work with a tuple. And if I have a dictionary, so let's say my dict. And then this must have the parameter names as keys, so a and then a value, one, B, and the value, and C and the value, then I can unpack this dictionary with two stars, and then my dict. So this will work. And also the number of elements must match the number of parameters here, and also the key, the keys, the name of the keys must match the name of the parameters. So if I have a different key here, then this won't work. Then the asterisk can be used for unpacking containers, so it can unpack the elements of a list tuple or steps into single and multiple remaining elements. So let's say I have a list called numbers and This is, let's say 123456, then I can unpack them, let's say I write, star and then beginning and then a last value. And this is equal numbers. So let's print beginning. And let's print last. So this will unpack all the elements except the last one into a list, and then it will unpack the last item into a single number. And, yeah, be careful here, this will always unpack your elements into a list. So if I have a tuple, here, then unpacking works, but it will still be a list here. So if I run this, and it looks like this, I can also unpack the, or put the star sign for the last item. So this will unpack the first number into the first element into one number and all the remaining elements into a list that is now called last. Or I can use this in the middle, so I can say beginning and then star, middle, and then last. So and then I can print the middle here. So now middle is my list with the elements between so if I run this, it will print this. And for example, I can also unpack more numbers into single element. So I can say second, last, and then here print. Second, last. So this is how we can unpack multiple items into a list. And we can also use the star operator to merge iterables into a list. So for example, if I have one tuple with elements, one, two, and three, and then I have another list, so my list equals 456. And then I can say second, or let's say, new list, equals and then I say I, in brackets, I put my first iterable here, so I can say star, and then my tuple. And then I can put in the second iterable here, so my list. So if I print the new list, then this will be a new match list. And I can also use a set here. So if I use a set, here, my set, then this will also work. So this merging, works for lists tuples and sets into a list. Or I can merge two dictionaries. So if I have one dictionary, call it dict, a equals and then some elements here. So let's say a and one, and B and two, and then I have a second dictionary, so let's say dict b, this has the keys C and D with the values three and four. And then I can create another dictionary. So let's say my dict equals and then inside these square brackets, I use two stars and then the first dictionary and then comma and then again two stars to unpacked this Second dictionary. So this will merge multiple dictionaries into one dictionary. Now if I print this, then I can see that I have one dictionary now. And yeah, I think that's all the important use cases of the asterisk sign. This tutorial, we will talk about copying. So we will learn how we can copy mutable elements with a built in copy module, and the difference between shallow and deep copies. And we will also have a look at how to make actual copies of custom objects. So let's start. And first of all, let's have a look at the assignment operator. So let's say we have a variable called orc, and this is now a number. And now if we want to make a copy with an assignment, so we say copy equals original, then this will not make a real copy, it will only create a new variable with the same reference. So now both variables point to the same number. And now for immutable types like this integer, this is not a problem. So let's say if we change the copy, and say copy equals six, then this assignment will again, create a new variable. So they they are now both independent. So if we print the copy, and if we print the original, they are different. But when we deal with mutable types, so for example, a list then we have to be careful. So let's say we have a list here with some elements. So let's say 01234. And now we make a copy with this assignment operator. And then if we change elements of our copies, so let's say we want to change the first item and say this is now minus 10. And now if you print both the copy and the original, we see that also the original has the value minus 10. Here, and this is because this assignment operator doesn't make an actual copy. So to make an actual copy, we can use the built in a copy module, so we can say import, copy. And then we have to make a difference between shallow and deep copying. So a shallow copy is only one level deep. So at the first level, it makes an actual copy. But then it only copies references of the nested child objects. And then there's the deep copy. So this will be an a full independent copy. So let's start with an with a shallow copy. So to make a shallow copy, we can say copy equals, copy that copy, and then the original. And now if we print both, we see that the original didn't get affected. So only the copy here has minus 10. And, for example, with a list, there are several different options to make shallow copies. So we can also say copy equals original dot copy. So this will also work. Or we can use the list function and give it the original as an argument. This is also possible. Or we can use list slicing, so we can say pork, and then the slicing operator. So this will simply be from start to end. So this will copy all elements. And this will also make an actual copy or a shallow copy. So this works fine if our element is only one level deep. And now let's say we have a nest or nested lists. So let's say we have a first lists here, a list inside a list and then a second list here. So with some more elements, three, so this is our original list. And now we make a shallow copy. And now we change an object or an item that is at the second level. So we say copy at index zero, so in this list, and then again at index zero, so this element, or for another example, let's make index one here, so this is this element. And now this we want to set to minus 10. And now let's see what happens. So if we run this, we see that both the copy and the original now have minus 10. Here. And this is because a shallow copy is only one level deep. So to make an actual copy in all the levels, we have to make a deep copy. So we can say copy dot, deep copy. And now if we run this, we see that the original didn't get affected. So this is the difference between shallow and deep copying. And for the built in types, like lists, dictionaries, or tuples, we can use these methods. But we can also use it for custom objects. So let's say we have a custom class and call it person. And now in the in it, it gets self Of course, and then it gets a name and an age. And then we say self dot name equals name, and self dot age, equals age. And now let's create two persons person one equals person. And now as the name it gets LX, and as an age, let's say 27. And now let's make a copy, simply by assigning it, so let's say person two equals person one. And now if we change person two, dot h, equals 28. And now if you print person two dot h, and we also print person, one dot h, then we see again, both got affected because this is not an actual copy. So here, we can use copy, copy. And now if we run this, we see we have a shallow copy here, so the original person didn't get affected. But again, now if we have a deeper structure, so let's say, let more let's first create our person class, and let's say we also have a class company. Um, this gets this has an init method, so in it, self and now this gets two persons, it gets a boss, and an employee. So self dot boss equals boss, and self dot employee equals employee. And now we create two persons. So one boss, so boss, might be older. And now a second person. Cho was a little bit younger. And now let's say we want to have a company so we say, company equals company, with our person one, and our person too. And now if we want to make a clone of this, so if we say, company, clone equals company, or let's right away, make a shallow copy, so we can say copy dot copy. And now if we change some variability here, so let's say one boss turns a year older, so let's say company clone, dot boss dot age, equals 56. Now, and now let's print this print company clone dot boss dot age, and also print the age of the boss of the original company. So let's say company, boss, ah, then again, we see it got affected because this is only a shallow copy, and the age is at the level two. So this will, again only be a copy of the reference here. And in order to make this independent, we have to say copy dot deep copy, and now if we run this, we see that the Original bosses still 55. So this is the difference between shallow and deep copying. So we will learn about the concept of context managers, and what are they used for, we will then have a look at typical examples of context managers and how we can implement our own context manager. So, context managers are a great tool for resource management, they allow you to allocate and release resources precisely when you want to. So a well known example is the width open statement. So, in order to open a file, we can say, with open, and then a file name. So let's call it note stuff. txt. And we open it in write mode, and s, a, and give it a name here. So, inside our width statement, we can use this name now. So we can say file dot write, so we write something into our file, some some to do. And now when we leave this with statement, again, this width statement or just context manager will make sure to correctly close our file again, even if there is an exception somewhere here. So if we would have to write this as a full code, it would look something like this. So we say file equals open, and then note stuck text in write mode. And then we have a try block. So we try to write into our file. So write some to do. And now we have an A Finally, clause. So this will be executed with or without an exception. So no matter what happens, this will, will be executed every time. so here we can say file, dot close our file again. And then our resource is freed up again correctly. So now if we compare this and this, then our with open statement looks much cleaner, and much, much more concise. So this is the recommended way to open a file. And this is a typical example how we can use context managers in order to open an AI file and allocate the resources. And then after leaving, it's also make sure to correctly free up our resources again. So typical examples is like in this case that with open statements, then, for example, to open and close database connections, or another typical example is the lock. So if you've watched my tutorials about multi threading and multi processing, you already know how to use a lock. So if we have a lock, so we say, from threading, import, lock. And now if we create a lock, so lock equals lock. So whenever we acquire a lock, so we say lock dot acquire, and then we can do something here safely. So this is now thread safe. But after that we always have to call lock dot release. And if we forget this, we might run into a deadlock here, and our program won't continue. So never forget to say lock dot release when we had locked out acquire. So a better way to do this. And also much simpler is to say with lock, and then do something here. This will automatically acquire our lock when we enter this with statement and then it will make sure to say locked at release when we leave this with statement again. So this is also a typical example. And now let's say how we can implement a context manager for our own classes. So in order to do that, we have to implement the ENTER and the exit methods. So let's say we have a class and call it managed file. Now of course this has an in it and it will get a file name here. So we simply store the file name, say self dot file name, equals file name. And now we re implement the same functionality as With the with open statement, just in order to show you how this is done. So, now what we have to implement is we have to implement the Enter method. So this will get self. And then we have to implement the exit method, this will also get self, and then it will get an exit exception, type, an exception value, and also an exception trace back. Now, I will talk about this in a second. But first of all, let's implement both of them. So, the Enter methods will be executed as soon as we enter the width statement. So, here we want to allocate our resource. So, in this case, first, let's print enter, to have a look at where this will happen. And now we allocate our resource. So we say self, we create a file and say self dot file equals and now we open it here. So we open it with the file name, and open it in write mode. And then also inside the Enter method, we want to return the allocated resource. So in this case, we return self dot file. And now in our exit methods, we want to make sure that we correctly close the file. So we say if self dot file, so if this is not none, then we say self dot file, dot close. And then print, exit. And here, let's print in it. And now this is all we need to use this class as a context manager. And now we can say we can use a width statement. So we can say, with Managed File, and this will get the file name notes dot txt, s file. And then we can say file dot write, some to do. And now let's say let's see what happens. So this will, let's also make a print statement here. So let's say print, do some stuff. So we see here, that init method gets called when our object gets created. Then as soon as we enter this width statement, the Enter method gets called so enter is printed, then our resource is allocated, then we can do some stuff. And afterwards, our exit method is called as soon as we leave this with context here again. So now let's talk about what will happen if an exception occurs. So we see here that Python Python passes the type the value and the trace back to the exit method. So you can handle the exception here. And if anything other than true is returned by this exit method, then the exception is raised by the width statement. So let's say let's print continuing here. So in order to see if we reach this code, and now, let's also print. The, for example, let's print, we want say, exit exception, and then print the exception type. And the exception value. So now if we run this, we see that our exception here is none. So no exception here, exception type, and the exception value is none. And now if we try something here, this that won't work, so let's say file dot some methods, so this will not exist inside our class here, so it doesn't notice some method method. So this will raise an exception. So now if we run this, then we see inside our exit function, it still can close our file even if there is an exception. So it reaches this code. So then it prints the ACC chat exceptions. In this case, it's an attribute error. And the error is that we don't have this some methods. And then we can exit this function. But then our width statement will raise an exception. So we won't reach this continuing here. And now if we want to handle this exception ourself, we can, for example, say, we check if exception, type is not none, then prints that here's an exception. So let's say exception. Exception has been handled. And now in order to not raise an exception, we have to return true here. So let's say return true. And we don't want to print this anymore. And now let's run this. Now we see, we did prints exception has been handled, then it exits our width statement again, and then no exception here from our width statement, and we can continue. So continuing is printed. So yeah, this is an example of how to write our own class as a cost as a context manager. And we can achieve this with implementing it as a class with the ENTER and the exit functions. But we can also implement it as a function. And to do so we have to say we have to import something. So we say from context lip import context manager, and we have to use this as a decorator. So and then we will create a function that is a generator. So if you don't know, or are not familiar with generators and decorators already, and please have a look at my other tutorials, because I already talked about them. So now let's create a generator here and call this open Managed File. And this also will get a file name. And then here, oh, sorry, I misspelled it. So Managed File, and then inside here, we want first, of course, want to open our file, so we say f equals open file name, in write mode. And here we have to write a try and a finally clause. And inside the try statement, we want to yield the file. So here, we would, we want to write everything that would otherwise end up in our enter function. And then we want to have a finally clause and here we write all the content of the exit method to free up the resource, the resource. So here, we say F dot close. And then we also need to decorate it with our context manager decorator. And now we can use this function in a width statement. So we can say, with open Managed File and call it notes dot txt, and then S f and then we can say F dot write and then write something. So this will also work. And now let's go over this again what will happen here, so is because this is a generator, so this will first make sure to allocate our resource and then it will try to yield our resource. So and by yielding it it will temporarily suspend its own execution. So we can continue here and use this file. So then we can do some operations with this file. And then when we exit the width statement again, then our function here continues running. And then the finally clause will be executed, and our file will be closed again. And also we can handle exceptions here. And yeah, so this is the second way how to use a context manager. And that's all I wanted to show you about context manager. I hope you enjoyed this tutorial and if you liked it, please leave a like and subscribe to the channel. See you

Channel: freeCodeCamp.org

Views: 1,187,607

Rating: 4.9702063 out of 5

Keywords: python course, python tutorial, python, intermediate python, python nprogramming

Id: HGOBQPFzWKo

Length: 355min 46sec (21346 seconds)

Published: Thu Aug 27 2020

Reddit Comments

[u/davehodg]

Is the course any good?