The FULL Guide To Itertools For Python Developers

how's it going guys in this video we're going to be talking about it tools and I'm going to be showing you all the methods that come with this module so that the next time you have to use it you'll know exactly what each one of them does so you won't have to read all of that documentation now first I'm just going to copy and paste in the first section from the documentation which explains to us exactly what iter tools are so ETA tools are functions that create iterators for efficient looping the module standardizes a core set of fast memory efficient tools that are useful by themselves or in combination together they form an iterator algebra making it possible to construct specialized tools succinctly and efficiently in pure python but to keep it short and simple it just provides us a very quick and memory efficient way of creating iterators which are extremely memory efficient now the first function we're going to use is dot count and when you use account it makes an iterator that returns evenly spaced values starting from a starting point so for example if we have a counter and we say itter tools dot count it's going to take a starting value such as 10 so this will start creating values from 10. and optionally you can also insert a step so you can say step two if you want to get only those even numbers back well I mean if you wanted to step two at a time and something interesting about this is that you can also step with floats but for now we're just going to create an iterator using count starting from 10 and here we're going to type in 4i in counter and we're going to print I and it's very important that we insert a break here so if I is equal to 20 we're going to break out of this Loop because this iterator is infinite which means it's going to start from 10 and it's going to create numbers all the way until the end of existence or whatever that limit is on your computer so it's going to make the number incredibly large so you need to add a Breaking Point and if we run that it will generate the numbers starting from 10 all the way to 20. and yeah of course you can add a step such as 2 and it's going to do 10 12 14 16 18 and 20. so that was the first method next we're going to use something that's called a cycle so to do this we need to First create a list of elements and I'm going to make an array of strings so a b and c and I'm very aware that instead of creating a list you can just say ABC but I'm going to stick to this convention of typing it in a list because in reality this can be a list of anything and that's just to demonstrate that it's part of a list but let's create this cycle so I'm going to call it cycler is going to equal itatools.cycle and we're going to insert the list now what a cycler does is essentially take that list and create an infinite loop with it which means we can iterate over it as many times as we want so here we can type in for I and letter in enumerate cycler and what we're going to do is print I the letter and a separator which is going to be this colon and we need to set a breakpoint because this is infinite and as with all generators this will go on forever unless we have a specific breakpoint so if I is equal to 20 we're going to break now if we run this and we make this a bit bigger you'll see that it's going to start looping from a then B then C then it's going to start at a again and this actually creates a copy so once we exhaust the original list it's going to start iterating through the copy and it's going to do it for as many times as you want to do it so that's how you can use cycle next we have something called repeat and this one's actually very straightforward so we're just going to create a string which is called string and we're going to create a repeater which is going to take it tools dot repeat and we just need to pass in what we want to repeat and the amount of times we want to repeat it which in this case will just be 10. now if we try to print this you'll see that we're not going to get those items 10 times because it will return to us this strange object so with a lot of iterables you would have to convert it to a list if you want to see it in a list format so if you run this now you'll get a list of 10 strings that say string but you can still do things such as for I or 4 string in repeater we're going to print the string and we'll get all the strings back up to 10 times because there are 10 strings in this repeater so those were the three infinite iterators that come with it tools now I'm going to leave it to Future me to explain the next section of it tools which are iterators that terminate on the shortest input sequence so now future me is here to continue the video and we're going to continue with the next method which is accumulate so to demonstrate this we're going to create some numbers and it's going to be a list of one two three four and five and we're going to say accumulation is going to equal itatools.accumulate and we're going to pass in the numbers now let's print the list of these numbers and we can make this bigger again so if we run this we're going to get 1 2 3 4 5 because that is the list of the list but what we wanted to do is get the list of the accumulation so if we run that you'll see we'll get 1 3 6 10 15 because it starts with 1 plus 2 which gives us 3 and then three plus three which gives us 6 and then 6 plus 4 which gives us 10 and 10 plus 5 which gives us 15. so it's slowly accumulates these numbers and if you want to get very funky you can even import operator or you can use any function you want but I'm going to be using operator here and as a second optional argument we can pass in operator dot multiply and then you will get the accumulation of the multiplied numbers next we have chaining so to do this we're going to create two different lists a which is going to be a list of one two three and B which is going to be a list of a b and c and we're going to create a variable called combined which is going to equal itata tools dot chain and inside here we can pass in as many iterables as we want and what it's going to do is chain them which means we're going to treat these two iterables as a single chain or as a single sequence so here we can now print the list of combined and if we run that we will get a single sequence back and as I said you can provide as many as you want so we can even duplicate the same lists or provide it multiple times and we'll get a single list back from it now we have a more interesting one which has to do with compressing so first we'll create a list that holds the values of a b c and d and we need to provide some selectors which I will explain in a moment what they are but they have to be true and false values and they have to equal the amount of elements in the first list so for example we can add false true true and false now we can create a variable called compressed which is going to use itatools.compress and what we need to pass in is the list plus the selectors and if we print the list of the compressed we're going to get V and C back and to explain this very bluntly we have the selectors here that choose which elements from the list will be returned so 0 being false it's going to say that okay a is not going to be returned because this is false one is true so it's going to select b c is true and D is false so it's going to select the elements that you picked from that list and it's not true that it has to match precisely because if you do exclude one it's just going to stop at C since that's where this exhausts so this can stop earlier depending on how many selectors you provide moving on we have drop while and we're going to create a list once again which is one two three four five six seven and we're going to type in remaining is going to equal it tools dot drop while and here we need to provide a Lambda expression and the Lambda is going to take a number which is going to check for the condition that n is less than 3. and we're going to pass in the list then we can print the list of this remaining and if we run it we'll get three four five six seven and the reason being is that it's dropping the elements where this evaluates to true and it's going to drop everything that is true and as soon as this evaluates to false it's going to print the rest of the list it's going to stop elaborating it and it's going to just print the rest of the list so for example if we have let's say 1 and 1 and 2 after D7 those will still be printed because this is only dropping the value while this is false until it evaluates to true and then it's going to stop doing that and if you have a billion numbers that are under the value of 3 or less than 3 this can have a large startup time because it's going to continue making that check until this becomes false and then it will print the remaining of the variables for the next example I'm going to create a list of a range of 100 or maybe it's misleading to say this is a list because it is a range but the concept is the same we're going to pretend that's a list of 100 elements and we're going to create a filtered list so filtered is going to equal it tools dot filter false and here we're going to pass in a Lambda which takes n and we're going to add the modulus of 10 to the list or not add the modulus of 10 but check that n modulus 10 is equal to 0. so the way this works is that when this evaluates to false it's going to return it in the list so everything that evaluates to false will be part of this filtered list of this filtered object because once again with every itter tools object you do need to combine it or convert it to a list if you want to see the results in a list format otherwise it's going to be treated as a generator so here we can type in filtered and when we run that we're going to get 0 10 20 30 40 all the way to 90 because 10 module is 10 elaborates to or evaluates to zero which is false it's a false value and so does 30 modulus 10 it still evaluates to zero one being true zero being false it's going to return all the false values and you can also just insert none if you prefer and that's just going to return the false values from this list so if we have let's say 0 1 false true and we run this we're going to get 0 and false back it Returns the false values and that's what filter false does now the next one I'm going to show you is called Group by and that allows us to group consecutive Keys into a new iterable so for example let's or actually I'm not going to write this all out because this is really tedious but the list is going to look like this it's going to be a list of tuples which has a and one a and two B and 3 B and 4. so as you can see they all have something that is repeating so what we can do is type in a new variable or create a new variable called grouped and I might add a space here so it's easier to read and we're going to use it tools as always dot Group by and here we want to group by or first we need to provide the list we want to group by and I find this very annoying that this is the first time that you need to provide the list first and not the predicate but that's how it is and we need to provide how we want to group it by so we're going to do something simple here we want to group The Elements by the key of each one of these tuples so here we have k at the index of zero because we want to group it by these letters so we want all the B's to be together we want all the A's to be together and we want to see of course to be together so now with that being done we can say four key and values because there can be multiple values in grouped we're going to print key and we need to convert that list of values or that iterator of values to a list if you actually want to see the values otherwise these values will be returned in an iterator object and the separator is going to equal a colon and a space now if we run this we're going to get these grouped back so you can see a is with a B is with b and c is with C so that was the basic way to use this but there's another example that I must show you and to do this I'm going to replace all of this with numbers because this might be easier to show you that it's grouping consecutive elements and not just the ones that look the same so for this example we're going to create a variable called example super creative and it's going to return to us a list of G which is the groups for key and group in ITA tools dot Group by and inside here we'll pass in the list now we will print this example and what you'll see is that it's going to group them by whatever is consecutive which means 2 is not exactly going to be grouped with two unless it's in an executive order so 2 and 2 must be together for it to actually be considered a group and that's actually all I wanted to mention with this quick example is that they must be in a consecutive order for it actually to be grouped for the next one we're going to use something called I slice and this is a way of slicing lists into iterables and it's very similar to just using the syntax that we have in Native python with the colon but of course it returns an iterable so here we're going to create a temporary list or a list of letters and we're going to create a variable called sliced which will take itatools.i slice and here we want to slice the list starting from 2 for the rest of the list so we do need to define the end limit and if you put none it's going to be indefinite so by providing that and printing the list of the sliced we should get cdef so it's starting from two if we do not provide none it's only going to go up until 2 and there's not much to say on this one it was that simple I mean you can easily use the slicing syntax but if you want to create an iterable use the itatools.ieslice now we have another one that's quite funky and this is called pairwise so if we have a list or we're not this is going to be called the list but it's just a string so a b c d e so it's a string of characters and if we say ped is equal to itatools Dot pairwise and we pass in that list and print the list of pad you'll see that it's going to pair each one of them as far as it can go so a b b c c d and d e those are the pairs it created from this list if you have two or less or actually if you have less than two it's not going to print anything because it can't pair anything so you need to have at least two elements in your array and this string at least consists of five elements or five different characters so it was able to pair them and that's really all you can do with this so it helps you pair Elements which means if you had a list of numbers this is one two three it's going to give you a pair of each of those numbers so one and two and two and three moving on we have star mapping and to demonstrate this I'm first going to create an example so we're going to once again import operator or actually from operator we're going to import power and here we're going to create a list of tuples once again so we have one two or that's a bad example I need this to be 2 3 I need this to be 2 4. and two five because we want to get the power of 2 for each one of these so next we can create something called star mapped and that's going to equal it tools dot star map and first we're going to pass in the function so here we're going to pass in power followed by the list and we can add a underscore here so we get rid of that green line but what's happening here is that we're passing in the function and this can also be a Lambda if you want but it's going to Loop through the list and it's going to extract all the arguments so 2 and 3 2 and 4 2 and 5 it's going to extract those and loop through this and return the result of these being placed inside power each time so if we print let's say the list of Star mapped you will see that we will get 8 16 and 32. now if you have a function that does not support more than the amount of parameters that you're providing it it's going to throw an error because of course power only takes two arguments but to demonstrate this example a bit more clear I did create a useless function and this function is called example it takes arguments and it creates a temporary list and then for each argument in those arguments it's going to append that argument to the list as a string with X to it so it does some very minor processing and I did not mean to remove my list I need to get that back so we're going to create a list right above X and this list will contain one two three four five six seven and so on so you can see these are arguments of of varying lengths now instead of power we can just pass an example which takes as many arguments as we have so this actually can be set to 4 if we want and when we star map this it's going to place all those arguments into example and that's why they call it star map because you have the star here which takes apart the arguments and when you run this you'll see that we'll get a list of 1x2x3x and 4x for the first element then we'll get 4X 5x for the second and 6X and 7x for the final one so it effectively placed all those arguments into example next we have the opposite of drop while and they call this take while and what it does is create an iterator that returns elements from iterable as long as the predicate is true so let's create another funky example of a list so list is going to equal one two three four five six and one so here we're going to say taken is equal to itatools dot take while and we're going to create a Lambda expression that says we should only take while a is less than 4. and we need to pass in the list then we can print the list I've taken and when we run this we will only get one two three back because as soon as this evaluates two false it ignores the rest it's only taking it while it's true so as soon as it becomes false once again it's going to ignore the rest and if this is false from the beginning it's not going to take anything so it only takes elements while it is true and then it stops for the next one I'm going to change the list up a bit so now we have one two three and ABC and here we're just going to return independent iterators from a single iterable so and to do that we're going to create something called a t so here we'll type in itatools.t and it's going to take the iterable and the amount of times we want to duplicate that so if we type in three we can say for iterable in T and print the list of the iterable and when we run that we'll get three identical iterables which are completely independent and finally the last one from this section is one of the most common ones and this is called zip longest so here we have three different lists that we want to combine or zip together and python does have one that is called zip which is exactly the same except the one from ITA tools provides us with some extra functionality so for example here we can type in zipped is going to equal s dot zip longest and here you pass in as many lists as you want there's as many iterables as you want so a b c and you can also provide a fill value for the values that are just not there because you can't really combine these all if there are some missing so by default it's not going to go any further than the first two if one of the lists has only two values and to demonstrate that we will first start with saying for a B and C in zipped and then we're going to press or press print a b and c with a separator being a colon surrounded by spaces so if we run that or actually I did not know this but by default it's going to pass in none for each one of the elements that you do not provide so as you can see for true and false we only have two so in general if you were to just use zip it wouldn't go much further than that so here's an example we're using zip you'll see that it will only go to two because this is the longest but with zip longest it's going to go as far as you wanted to so even if we only have two values in C it's going to zip all of them until five but what I was trying to mention earlier is that you can provide a fill value so this can also be a string or some other value and instead of none it will replace it with that string so it's a very efficient way to zip lists or iterables now moving on to the third and final part of iter tools we're talking about combinatoric iterators and I had to read this let me just show you what it looks like these are the times they used in the python documentation I can barely read that combinatoric iterators so these are the ones you probably see the most being used with it tools or the most popular ones that you see around YouTube or in tutorials and you probably heard of permutations or combinations or products and that's what this section is going to cover so first we're going to be talking about how you can create a Cartesian product of input iterables so to do this this we're going to create two lists or two iterables one's going to be of one two three and the other one is going to be of ABC now let's create an output which is going to take itatools dot product and we want to pass in A and B here and what we're going to do is say 4 T in list of outputs we're going to print t and if we run this we're going to get the Cartesian product of these two lists so we're going to get one a one B one C two a two B two c and so on and you can also provide a repeat value so repeats and we can say we wanted to repeat two times so it's going to duplicate these lists two times and we're going to get these large tuples such as 1A 1A 1A 1B 1a1c so it effectively duplicated this and created a much bigger Cartesian product next we have permutations which allow us to get all the permutations from a selected iterator so for example we'll create something called permutations and we're going to get itatools.permutations and inside here we'll pass in the list and all we need to do now is say four group in the list of permutations and we're just going to unpack them so we're going to say print unpack the group and the separator is going to be a space and if we run this you'll see we're going to get all the permutations of a b and c so ABC ACB BAC BCA cab and CBA none of these are repeating in sequence as you can as you've noticed so we effectively got all the permutations of a b and c if we did not unpack this and we just excluded the separator it would return to us a tuple of the permutations so just to make it look cleaner I decided to unpack them and add the separator so you could see the permutations of these letters and optionally you can also provide how long you want the permutations to be so if you want the permutations to be of a length of 2 you can insert two as a parameter or as an argument and it will only provide you with permutations of a length of two moving on we have combinations and here we'll create a tuple of 0 1 2 3 and we'll get right into it by creating a variable called combinations which will equal itatools.combinations and here all we have to do is provide a list now when we print the list of combinations or maybe it's better if we do the same thing I did last time so 4G in list of combinations we're going to print G we should get all the combinations of 0 1 2 and 3 back and I will is missing a positional argument so I'm going to insert two and this is the length of subsequences that we want to get back from the input iterable so if we run that we'll get 0 1 0 2 0 3 1 2 1 3 2 and 3. so these are all the combinations you can have with these input variables and if you add a three it's going to show you all of the other combinations so these are all the combinations you can have with these values which means if three was used with two and one that's all you can put inside and it's never going to repeat a sequence in here but what if you do want it to be able to repeat sequences well it's a tools has one method that is very similar to this one and that is combinations with replacement but now when we run this you'll see that it's actually going to repeat the numbers unlike with the previous combinations so you might have something that says one one one but with the previous combinations we would never get that because you cannot have the same element repeating and by same element I mean the same one at the index of one because if we go back to combinations for example you can see we have 0 1 2 but if we provide another one here it will place that one inside there because there are other combinations that you can create with that one this one is completely independent to this one over here and that's why you can get the same value repeating but it's never going to be the exact same value because these two are considered independent so if you actually want to get repeating values you're going to have to type in combinations with replacement because that will give you the repeated values as well when you are generating these combinations and with that being said we've concluded every single method that is included with it tools I know I didn't really explain where you can use these but I did show you how they can be used so possibly in a future video I'll show you exactly where we can use some of these but for now it's just good to know that you understand how these work so when you do need to use them you know immediately what they do or you can kind of remember what they do because that will really contribute to your coding faster and more efficiently when you know that these methods actually do exist but anyways with that being said do let me know in the comment section down below if you found this video you're interesting or there's something interesting that I missed I would love to read about it and with all that being said as always thanks for watching and I'll see you in the next video

Channel: Indently

Views: 14,605

Keywords: pyton, pyhton, pythn

Id: baZpqVmR488

Length: 29min 11sec (1751 seconds)

Published: Thu Mar 09 2023