⚡ 5 Horrifying Python Techniques to get you fired - Andrew L

I am here to talk about python uh python has some very good things going for it python it's very quick to iterate and make scripts um it's got lots of Library support so you can do pretty much anything with it the syntax is very easy to understand even for newcomers it's very flexible language and in most cases it's quick enough that's not to say it's quick but it's quick enough for uh most use cases if you're writing performance critical things probably not python but in general it's fine and python as many people are aware is a dynamically typed language that is types are not checked at compile time and types are only checked at one time so you can conceivably run into a situation where you get an error halfway through your program because you're trying to add an integer to a string um however python is also a dynamic language and the definition of this is a bit harder to nail down but uh effectively Dynamic languages uh can support creating new types and new code at runtime things like the exit and eval functions and um defining classes at runtime this Dynamic nature of python allows us some very useful built-in things so for example pdb and sys.centrics are the built-in python debugging functions that are written themselves in Python which allow for powerful viewing of what's going on inside your program while it's running there's the inspect Library which allows you to view stack frames the source code of a given function and so on you can have classes that are defined or not depending on what the version or platform you're running the code on which allows you to do Platform specific optimizations and you have runtime reflection which lets you see the fields of a class at runtime and monkey patching which lets you overwrite methods with customers and so the standard life we use is the stand Dynamic stuff in very interesting ways so you have a built-in thing that lets you do Dynamic dispatch on a function just by adding a decorator and it looks at the type of the first argument and picks the best implementation and then in terms of external libraries there's latexify which given a function uh will return the latex form of that function which is very useful however I'm not here to talk about good python I am here to talk about cursed python so if we take a look at the title slide this is a program that took two days to write because it is a beautiful piece of artwork And yet when you run it it prints the Zen of python beautiful is better than ugly and my favorite one of these is readability accounts sure python sure um so the Xena python uh it's they aren't rules but they are ideas on how to write nice clean pythonic code these are the ones we are going to explicitly violate you can make Arguments for the others but these ones are fairly common dry so let's get right into the first of our uh interesting cases variable binding most people think of variable binding as just the arguments to a function and assignments and that's broadly speaking true um however there's a bit of nuance and where there is Nuance in Python there is Jank so let's have a look at some of that so multiple assignments it's a very useful feature of python that lets you assign to multiple things at the same time um and so for example here the V1 and V2 point to the same list this is useful for defining constants and for getting multiple views on the same object which does have its uses however it also lets you write code like this and the question is after the first line what does a contain it contains the list containing itself which is a bit of a suffice when you run into it the first time um and similarly after the second line a contains the list containing itself and the empty list which this feed it's a it's a feature of python and it has its good uses but it also has its Shenanigans the match statement was introduced in Python 310 even if you've not come across it before the syntax is fairly clear you have something and you are checking is it similar enough to the first case if not second case and so on so here when you run this it will take the other Bond because even though it is structurally the same as the first Bunch it fails the condition so if we run this which is similar note that the underscore here means match any pattern don't bind any names I just want it to match everything and if we run this we of course get a syntax a name hour I'm just kidding no uh a print six um because The Binding of the names happens on the structure matching and only then after the names have been bound does it check if the conditions are fulfilled and so um what that means is that this match statement is semantically equivalent to a normal assignment um which is not at all confusing variable binding has its other side variable unbinding uh which in most cases is just the delete statement so let's have a look at these where here is the variable a valid it's fairly intuitive that it goes like this on the left a is valid from where it's defined or it's deleted and on the right a is valid within its own function and for future Reasons I'm also going to annotate the other variables that already exist in scope um so then the question becomes for a try except what is the lifetime or the scope of the variable e your first answer would be oh well it's it's just that it's valid from the beginning of the exception block to the end and it's invalid after that and the answer is broadly correct we'll come back to that you see uh C python uses the reference counted memory model and so to prevent reference Cycles um which would cause your program to be unable to free any memory uh from this point forwards um see python turns the try accept block on the left into pretty much the one on the right there's a little bit of nuance but it's effectively equivalent and what that means is that this is in fact incorrect the correct layout would be this if you have a variable called e in the outer scope it gets deleted if an exception is raised which means that a function like this when it gets called has a chance to delete the global variable e you won't necessarily know about it because for example the outer memory hour can be raised at any point in the program and then your variable is gone so be careful with naming your variables moving on we have annotations which most people will know as type ins because that is the generally accepted use of annotations and annotations are interesting they are gradual so you can add them as and when you want and they aren't necessary annotations are stored in two places either on the object itself or in a magically created uh double underscore annotations object that python inserts whenever you uh add a necessary annotation these are what the variables contain note that it does not have a attribute called annotations and this is fine this allows for type checking which is always a good thing and it's also worth noting that python doesn't actually treat these as they are python converts them into other forms under the hood so it looks like that and it's I should say that the function annotations are the added to the function at the time that the function is compiled so you can't separate the function annotations but you can separate the variable annotations even though it never typically is however if we if we think back as to what the annotations were it wasn't it wasn't this and it wasn't this it was this which makes you think because that is the representation of a dict containing the class end not the text that we put in but the actual Class end though I should say that this is planned to be changed in the future so what this means is because there is no type level logic in Python everything is an object that it must be evaluating things so if you have a tight hint of five that's fine python can deal with that and lists that's all fine but it's just it's just an expression so you can evaluate the expression and you get your uh corresponding typing but wherever it becomes truly cursed is that in Python functions are also Expressions which lets you have these annotations and also printed the output I know indeed and so if in theory we write a program that opens the file that we are currently running reads the file closes the file because that's just good practice and then prints the contents and we put it through the magical one minifying machine um we have this program which if you ignore type hints like many editors do you have this is not quite a note but it's it does nothing effectively however when you run it it prints itself um so yes and on the subject of turning things into one line we are going to turn things into one line because mostly because we can and I'm sure everyone's felt oh there are too many lines in this program let's make it fewer so the question now becomes what can we turn into one line well we start with these three main things of python syntax we have statements of the type simple and compound and expressions expression is already one line so we're sorted there and so what we want to do is we want to turn all the statements into expressions and can we do that the answer is yesh there are some cases in which we can't but broadly yes we can turn everything into an expression so what we want to do is we want to take python source code we want to read it into an abstract syntax tree which the standard Library does for us we want to turn it into a syntax tree representing an expression and then we want to use the standard library to turn it back into source code and I mean we just need to transform this in tax nodes right how hard can it be well to begin with all the nodes that are already representing Expressions they're already done for us so that is 53 out of the 111 AST notes um are done for us that's 48 and I will take that so now just just the other ones right um so the trivial ones are ones that are either already ignored by python or have a corresponding expression format I should note that the X4 note here is the statement containing an expression as opposed to an expression but it's still easy to pull out the expression here's where we get into the things where you actually have to think of it the ones that require care they aren't difficult and you can solve them with some very simple python tools however in the spec and the docs and so on uh there are edge cases in their design which you just have to account for so it's uh you have to be systematic in how you approach it but it's not that difficult to take on then there come the cast ones here are four and your rays can't really be done however it can be mostly done delete you can only do with implementation specific tricks try can be implemented with with and with can be implemented with try however the docs do not say that there is a way to do it uh there is however a way to do it and then unfortunately there are some nodes that we can't transform into one line which are the async ones the scope modifying ones and the generators it might be theoretically possible to do so but you'd have to re-implement pretty much the entirety of the Python interpreter in Python which kind of defeats the point and then the final class of them are the match uh nodes for the match statement as seen earlier for this one it's fairly simple we have this spec for python structural pattern matching and you can just re-implement re-implement it in Python which is about 200 lines long and allows you to turn normal readable human match uh match statements from this into this Abomination which does exactly the same thing as the default match even including the variable binding Shenanigans and it also lets you take perfectly normal and innocent Pi game programs and turn them into this which was 283 lines long originally and uh has increased its uh character count by 50 it also runs 16 times slot so yes you can one lineify pretty much everything in Python I should also to go back to an old slide point out that the title of the slide is a program and is in fact an expression so if we hop over to here and we copy that expression and we just hop into python all expressions in Python evaluate to something so then the question is what does this program evaluate to and it could be none it could be a string or it could be something else uh something entirely unexpected now we come to our final and possibly most cursed item uh to demonstrate importing the import Machinery in Python is very complicated normally you just write uh import module and you're done but the stuff that goes on behind the scenes is quite interesting I should note that when you do an import from a module that is equivalent to importing the module extracting the item and deleting the module as long as we can attack the import we can also attack the from import so what actually happens when you import a module in python python finds the module you're talking about some most modules are found on disk some modules are built into the python interpreter some modules can be loaded from zip files or network resources but it needs to be found first uh then python creates an empty module type effectively a namespace with the built-in spec information about where the module came from uh then python runs the code from the module and captures all the variables and functions and classes you define and then it stores that defined stuff in the namespace it actually does these last two steps the other way around um presumably to detect cyclical Imports and other things like that the equivalent code for the running is that and finally it binds the originally requested name to the module however this leads to Shenanigans here we're going to import a file module.py if that module.py well contains an hour then you'll get an error and it looks all very sensible this is entirely expected however if we have this mystery module um then when we run it we get an error in the standard import Library which you shouldn't do so what what's causing this what are these mystery contents these mystery contents deleted the module from these system modules and so back here where we add the module to the imported lookup that is system modules and because you are executing the module that you are importing you can modify system modules during the import process we can change what objective module will be now I looked long and hard to find an existing library that used this technique and I did eventually find one which is a python error handling Library you can import the library and use the library as a function you can use the library as a decorator and you can use the library as a context manager and it simply removes ours and this is amazing um because it uses a a little talked about feature of python the ability to modify the import system and uses it in a useful fashion there is one more import related thing which is mathematics so here we have the file called one so that's import one oh if I can spell so one it's it's just a module there is nothing funny going on here however well what is one plus one what one plus one is two and we now have a file called two which we can import so let's do some more maths 2 to the power of two plus one is the module five and there's now a module called five and you can keep going and on and on and on and uh negative numbers and everything is supported and this is uh defining additional functions on the module itself um which I'm sure has a use but you know uh we're not here to provide productive uses of the Python techniques and there is just one final thing one final truly truly cursed aspect of python you see in the background we have had uh the Zen of python represented by our little logo here if we run it again and we just add the output to the file n dot pi um this is not a valid python file it is syntactically invalid uh it's just a piece of art with some comments at the top so of course it can't do anything um thank you for listening

Channel: UWCS - University of Warwick Computing Society

Views: 230,647

Keywords: computing, uwcs, university, society, coding, programming, python, cursed, breaking python, cursed python, cursed programming, code exploits, python exploits, zen of python, destroying python, horrifying python, python oh no, wat talk

Id: t863QfAOmlY

Length: 16min 12sec (972 seconds)

Published: Tue May 02 2023