Speed Up Your Code With Cython

[Music] what is going on guys welcome back in today's video we're going to learn how to massively speed up python code using cython so let us get right into it all right so the first thing we're going to do is we're going to open up a command line and we're going to install psiphon by saying pip install siphon like that it's written like python with a c in the beginning and it is a library and a module but it is also a programming language or you could say an extension of the python programming language so it basically takes python and adds statically typed variables for example to the python language and by using these c like principles we can speed up the python code massively because python makes a lot of things easy for us we can for example just say a equals 10 without specifying any data type and then we can say print print type a for example and we're going to see okay it's an integer but then again i can go ahead and say a equals hello world and then it's a string and in python you can do things like that and because of that you only know what data type a certain variable is during runtime you cannot know in advance now of course we have talked about type hinting and we know that type hinting is possible and tie painting allows developers to get rid of the uncertainty if you use tools like my pie but it doesn't make things faster because it just it just makes the whole thing a little bit more stable and more predictable for the developers but it doesn't actually speed up the code because you're not changing anything in the inner functionings of the python language whereas with syphon you can actually do static typing which is not possible with type hinting and in today's video what we're going to do is we're going to write a simple function that is going to find some prime numbers so we're going to have a function signature like that we're going to have um i don't know find primes and we're going to have amount and what we're going to expect as a return value is a list of the first uh x or in this case the first amount of prime numbers and we're going to do this in vanilla python and then we're also going to do this in psythen and we're going to see the difference in execution time all right so let's get started with the implementation of the vanilla python version we're going to say def and we're going to call this i don't know prime finder vanilla and we're going to pass an amount here and we're going to say the list of the prime numbers that we already found is going to be empty in the beginning we're going to say we found zero prime numbers and the number that we're going to check first is going to be two and then we're going to say while the amount of found numbers is less than the amount that we demand we're going to say 4x in the primes that we already found because the only thing we need to check for is if the new number that we're checking is divisible by the primes because if it's divisible by any other number it's going to also be divisible by the prime so this is just mathematics here we're going to say if the number that we are currently checking is divisible by any of the prime numbers so if the division results in the remainder of zero then we're going to say this is not a prime number because it's divisible by something otherwise if that is not the case you can just append an else here to the four which basically means that if if we don't break out of this for loop we're going to go into this else branch here and we're going to say if that is not the case primes append and we're going to append this number because there was not a single number below it not a single prime number below it that is able to divide it so it is a prime number and we can say found plus equals one and what we're going to do uh every times we're going to increase the number in order to progress to the next number so number plus equals two and in the end we're just going to return to prime so a very simple function this is the vanilla version in python uh now we're going to write the same thing using um using the siphon syntax now we're going to get some probably some errors while coding because it's not going to recognize the syntax so don't be confused by that but we're going to call this prime finder um optimized because syphon actually allows us to or allows itself to optimize code so here we're going to also pass amount but here we start with the first thing we're going to say int amount in order to signal that this is going to be an integer again we're going to get this error here but don't mind it we're just going to continue coding here because this is siphon syntax later on we're going to rename this to main.pyx because that is the file type that we're going to code in but i think if i do it right now we're going to get no syntax highlighting at all so we're actually no actually we get syntax highlighting we just don't get it here okay uh it doesn't really matter so we're going to do it like that then we say cdf which is a sith and keyword for c definition int number so we're going to do the same thing we did up here we're going to define a number and we're also going to define this x control variable and we're also going to already define the found here as an integer because we don't want to define them on the fly and we're also going to define the empty prime number list but here since we're using uh c like programming style we need to already say okay how large is this array going to be we cannot just say okay make it larger by appending elements no we need to say okay this is a fixed size thing so we're going to say int primes and let's just pick an arbitrary limit of 100 000 for example so this is five zeros there you go and we're just going to say okay in case i choose an amount that is above that number we're just going to take the minimum off either the amount or a hundred thousand so if we choose something that's above 100 000 we're just going to default to a hundred thousand so you can change that of course if you want to what we're going to do then is we're going to say found equal 0 again and we're going to say number equals 2 as we did before and after that we're going to see while found is less than amount we are going to say 4x in primes here we need to do it a little bit differently because we're not just saying for x and primes uh or actually we are saying the same thing but we don't want to go through all the primes because primes is uh or has a hundred thousand elements and we don't wanna go through all the zeros because then we're going to get the division by zero so all we need to do is we need to go up until the index of found which basically means that if you found zero you're just going to do nothing uh so you're not even going to take the first number because up until zero means not including zero um and if you have one you're just going to look at that one so this is what we need to do here and if we do the same thing if the number divisible by or if the number is divisible by x then we're going to break because that is obviously a prime number otherwise we're going to say primes on that particular index so on the next uh free space is going to be that number just need to indent this here and found is going to be plus equals one so we're going to increment it let me just see here if i'm not blocking anything no seems good i'm just going to make it a little bit smaller here um there you go so found plus equals one and also number plus equals 1. so one thing we need to do differently here also we need to say return list is going to be p for example for p and primes up until fount so that we only have the elements that we need and we're going to return that list as a return list so as you can see the whole logic of how we find the prime numbers is the exact same we're not changing anything so it's not more efficient here we actually have more code here so we have a fixed size array with unnecessarily many free slots if we don't need them we have type definitions we have all this extra code here like that so actually if anything this is going to be slower if if it's just for the amount of code so it's not a more efficient algorithm but we're going to see that this thing here this function is going to be way faster than this vanilla function up here all right so now we have all this code but we cannot just go ahead and call the functions down here and measure the differences in execution time what we need to do is we need to compile the whole thing because we have these uh c like concepts here that we first need to translate into c code and then we're going to end up with a binary file so what we need to do here is we need to create a setup.py file so we're going to call to open up a new python file setup.py and for this we're going to i think also need the terminal and we need to say pip or actually i think this is part of core python i think it's part of chord python but if not we need to say pip install setup tools but i think it's part of core python so you don't need to do anything probably but what we definitely need to do is we need to import from setup tools we're going to just import setup and we're going to also import from siphon dot built we're going to import sythenize this is the function that we're going to use and the only thing we need to do here is we need to say setup equals and in here we say ext modules equals cytonize main.pyx that is all we need uh why is this oh sorry we don't say equals we just call setup that is how it works and we don't just run this but we have to type in a terminal command so we navigate to the directory where the setup file is and then we say python or python3 py build x or build underline x or ext minus minus in place like that and that's going to build this into a binary file and what we're going to do after that is we're going to have a second script an ordinary python script nothing with syphon uh we're just going to call this test.py and as you can see first of all we have this thing here which is the actual um the actual compiled binary but we also have a c code so we also have a c file here a c script or c program a c source file you could say uh but you can see it's full of preprocessor directives so you don't need you don't really need to understand what's happening here uh also don't think that c is always that complicated this is just a python uh translation so it's a little bit uh yeah more extensive i don't really know what's happening here but this is the optimized c code and then we also get this now we actually don't need the source file anymore we only need that and we can use it by just saying let me just see the syntax i think we only need to say import uh main or depending on what you called your script in the first place you just need to use this name here so import main in my case and we're also going to import time to measure the differences and now we can just go ahead and say main dot and we can call the function so in this case we had prime finder vanilla like that and we can for example say we want 40 primes and then we can say print the result which is the list and let's see if this works so as you can see we got 40 primes now let me just see if this works if i rename this so let's just call this something else like that so it's not no longer called main so we're definitely not uh importing this and if i now run this you can see it still works now the important part is now to figure out if it's faster uh if the siphon optimization is faster than the prime vanilla finder and for this we're going to first of all say start vanilla equals time dot time we're going to copy that into end vanilla and we're going to print and vanilla minus start vanilla like that so first of all we can just run this and see that it works 40 is of course nothing let's change this to 4000 for example and you're going to see that it takes a little bit longer but still not very long even with vanilla um but we're going to increase the number in a second here so first of all let's do the same thing let's just copy that and do the same thing for the optimized version so i'm going to call the start c and nc and we're going to call not vanilla but optimized like that we're going to also copy this here c and c like that so if we now run this you're going to see that this takes half a second and this takes 0.04 seconds now we might say that's not really important but let's change the numbers to let's say i don't know ten thousand ten thousand is not uh not not a small number when it comes to finding all the primes and as you can see it takes quite long to get a result it took like three seconds 3.18 seconds to get a result from the vanilla function and it took like a third of a second not even a third of a second from the optimized function and you know we can increase the numbers here as we go on we can say for example 20 000 and this is going to take quite some time i think this is going to take like 12 seconds for the vanilla version um but we're going to see that it still gives us the result and then we're going to see that this is way faster now it's still going to take like 1.2 seconds or something like usually a 10th of the time that it took this function this is going to be the execution time of this function um but as you can see like this needed 12.89 seconds and here we just needed 1.12 seconds for the exact same result and this is not going to really change if we do for example 30 000 here i'm not going to do it here so i'm going to comment that out but this is not going to take too long for the optimized version it's it took like 2.6 seconds if we did this uh with up here with 30 000 we would probably take like 28 seconds or something and as you can see this is just because we have static typing we didn't do a lot of magic there we didn't change any algorithmic uh complexity things we didn't all of a sudden cache something because caching i did a video on speeding up your code using caching this works sometimes if you have recursive function calls and so on but it only works in some situations and this didn't even use any caches or anything special we just optimized the code with static typing we sacrificed some of the python flexibility for a massive improvement in execution time and speed and performance so that's it for today's video hope you enjoyed it hope you'll learn something if so let me know by hitting the like button and leaving a comment in the comment section down below and of course don't forget to subscribe to this channel and hit the notification bell to not miss a single future video for free other than that thank you so much for watching see you next video and bye [Music] you

Channel: NeuralNine

Views: 104,520

Keywords: speed, speed up python, python faster, make python faster, cython tutorial, cython, python with c, python c, python c tutorial, python speed up tutorial, python faster code, python speed

Id: Ic1oE6SEOBs

Length: 16min 37sec (997 seconds)

Published: Wed Jun 02 2021