Why is Python so Slow?

so is Python too slow now the reality is Python is an extremely slow programming language when we look at something like Java C and C++ it's embarrassing how much faster they can do things than Python in fact specific algorithms and applications can actually do things about a hundred to two hundred times faster than the native Python language so in this video we're going to discuss why that is and the different techniques that we can use as Python programmers to speed up and run more concurrent Python applications now before I get too far it is worth noting that wood python lacks and speed it makes up for in development time and cost typically developing programs in python is much faster simpler and ends up costing less as there's less labor involved with the project now this is a massive advantage in in many cases you can actually write you know a Jabba whip like code in Python and about four or five times faster than you'd be able to do it in that language so this is something to consider and this actually brings me nicely into the sponsor of today's video which is kite now kite wants to help you write your Python code as fast as possible by using their awesome autocomplete engine for Python which happens to be a plug-in for your IDE or text editor kite runs a machine learning model on your computer while you type to show you the best possible completions it ranks all of its completions by relevance so you get shown the best ones first and it has a ton of awesome features like intelligent snippets which save you a ton of time when you're writing code one of the best parts of kite is actually the co-pilot window now the copilot window acts as a companion window while you're coding you can pop it up on the left or right side of your screen and it shows you relevant documentation following your cursor throughout the file kites awesome I personally have been using it for the past about two months I absolutely love it and the best part of it is that it's free and you guys can download it at the link in the description so why is Python so slow now there's a variety of different reasons why this language is slow but I just want to combat a few rumors here the main reason this language is slow is not the global interpreter lock although that definitely is a factor to the speed and the way that we can write you know faster Python programs that is not the reason fundamentally why the language is slow the reason the language is slow is because it's dynamically typed now we're going to talk about this more in detail but I want to comparison to a language like Java which I actually personally write in now in Java everything is statically typed and this language is actually compiled before it's run unlike Python that's compiled at runtime through an interpreter now what happens in Java when you write code is you need to define what type each of your variables are gonna be what type you know your methods and functions are gonna be returning and you pretty much have to define exactly what everything's gonna be throughout your code now although this leads to you know much longer development times and takes a much longer time to write your code what it does is increase efficiency exponentially when you're compiling now the reason this actually works and the reason this works so much faster than Python code is because if you know the type that a specific variable or object is gonna be you can perform a ton of different optimizations and avoid performing a ton of different checks while you're actually running the code because these checks are performed at compile time in Java essentially you can't compile any Java code that hasn't actual or you know even just like typed errors while you're writing that code so you're gonna try to compile it and it's gonna say you know this type isn't accurate you can't do this you can't compile it because it knows that when it comes to runtime that's not gonna work so essentially all of these checks that actually need to be performed in Python when the code is running are performed beforehand and there's just a ton of optimization done because of this statically typed length now you might be asking well why doesn't Python do this well Python is dynamically typed which essentially means that any variable can change its type and can change its value at any point in the program while it's running this essentially means that we can't actually compile the entire program beforehand because we can't do all of these checks at once because we don't know what type these variables are gonna be they're gonna change at runtime different things are gonna happen and because of that you know we can't get all this optimization that we might have in a lower level language like Java C or C++ and that is kind of the fundamental reason the language is slow this typing this dynamic typing and any fast language is gonna have a compiler that's gonna run through its gonna make sure that everything is good it's gonna do all these checks before it actually ends up running the code at runtime where what happens in python is all of your code is actually compiled and checked at runtime so rather than you know compiling it before and taking all that time beforehand while you're running the code a ton of different checks need to be happening to make sure that you know this object is correct these types are proper everything is working the same now that is kind of the fundamental reason and that's what we're gonna keep going back to in this video we talk about some other slow parts of Python then the next thing to talk about is obviously the lack of concurrency in Python this is going to be your major kind of factor on speed if you're writing an application in Java C C sharp you can kind of spread everything out throughout multiple threads and what this allows you to do is utilize all of the cores of your CPU so to kind of break this down in modern day computing most of us have four core CPUs or higher and that allows us to actually run for tasks at the exact same time concurrently now with Python this isn't possible well Python says is well for each interpreter we can have at most one thread running at a time and a thread is just you know you can think of it as some kind of operation that's happening on a CPU core so that means that even if we create a ton of different threads in our Python program we can only be using one CPU core wow you know a Java program or a C program could be using all eight or could be using all four which is obviously going to lead to you know a 4x increase in speed now we can get around this in Python by using something called multiprocessing but I'm gonna discuss some of the issues with that in just a second now a lot of you are probably wondering why the global interpreter lock exists in Python why would they make that a feature of the language well this boils down again to the dynamic type of Python so the way that memory is actually managed in Python and I'm not an expert on this and any means I've just kind of looked up a few things and made sure I kind of had an idea what I was talking about is that it's not thread safe now what that means is that if two threads are to separate you know pieces of code are trying to access one specific object in memory at the same time you're gonna run into issues and essentially we can't allow that to happen so what we do is we say at least what pythons done is said well we're gonna have a global interpreter lock that means that only one thread can run at a time to prevent this from happening because we know one of the main issues with running multiprocessing and multi-threading applications is you have deal with locking and sharing memory and that is kind of one of the things that I'm gonna get into now with the multi processing module in Python so although concurrency is possible in Python by using the module called multi processing which essentially allows you to spawn another Python interpreter that can run its own threads it's very difficult to actually achieve effectively I've tried it personally obviously you know I'm not a pro but I've had difficulty doing this because of the way that memory is managed whenever you do this you need to shut up set up a shared memory object which allows you to actually transfer memory between one Python interpreter to the other this is difficult to do and it's not you know very it's not intuitive to actually get working so that's kind of one of the issues with it and even though you can do this you still run into a lot of the speed issues with Python coming from the way that it's interpreted and the way that it works you can't actually achieve multi processing and concurrency in Python but it is difficult and you have to deal with much more difficulties and kind of things getting in your way than I would say in Java C and other languages like that where you can just you know make threads and they will run automatically on the other cores without having to define shared memory objects and locks and stuff like that so other than multi processing how can we speed up our Python code well one way to do this is to actually use C code as an extension to our Python library or whatever it is that we're creating now python is really built on top of C a lot of people don't know this but there's a lot of you know functions and things that you're actually using that are written natively and C and that Python kind of just has an extension for that allow you to use them and that's why for example the sorting algorithm in Python will run you know much faster than if you write your own just native sort actually in Python we have a lot of optimizations done and a lot of things written lower-level in C now this actually means that you can do this yourself so if you need to create something that's gonna run very quickly in Python and you can't use a different language what you can actually do is write that algorithm and see and import it into your Python code as you know in extension so you can run that code faster than if you had just written it natively in Python so to sum it all up here python is slow mainly because of the way that the language is built there's a lot of other factors but because of the dynamic typing involved in python we're not able to introdu a lot of optimizations in our compiling and interpreting that we have in other languages like Java now this leads us into well how can we actually make Python usable and much faster now Python obviously is usable and in a lot of cases you don't actually care a ton about the runtime of your code or it would be nice if it ran faster but at the end of the day you know a few milliseconds isn't gonna be drastic to whatever application or product you're building and in this case Python is great because it's much faster to develop and to write code in and typically a lot easier and more readable when you're developing a project now to actually increase the speed of your Python code you can introduce some kind of threading and concurrency you can use the multi processing module which will allow you to have multiple interpreters running at once and get past that global interpreter lock but you may run into some issues with shared and locking memory now other than that you can also write C extensions for your Python code obviously this isn't gonna be for everyone as you need to know C to do that but if you're really in a pickle you need to get something fast and you need to use Python for it writing something and C and importing that will speed up your code by a drastic amount of time especially if you have like searching or sorting algorithms or things that are just really intensive computationally if you write those in seed that's gonna speed up the execution of time of those exponentially and that's gonna save you a ton of time and a ton of headache rather than trying to learn you know an entire new language when you can just kind of pick out some pieces of C and write a small extension for your Python code so with that being said that has been why Python is slow how to make it a bit faster and just some information about the language in general I'm sure I might have made some minor mistakes here if I had please don't hesitate to leave a comment down below and let me know just like you guys I'm learning as well and this is my kind of surface level understanding of the Python language so with that being said if you guys enjoyed the video make sure you leave a like and subscribe to the channel down below and let me know what you want to see

Channel: Tech With Tim

Views: 535,652

Keywords: tech with tim, why is python so slow, is python too slow, python speed up, python slow, python language to slow, python programming, why is python slow, why is python slower than c

Id: x2IQP8iug3c

Length: 10min 44sec (644 seconds)

Published: Thu Oct 31 2019