Python's Class Development Toolkit

since I don't need any description of who I am this first slide just kind of ignored that as the Who I am I've built a lot of tools that you use everyday in Python the new style classes have been out for almost a decade now and we have a bunch of little tools that people read about here and there but they typically don't learn how all the tools fit together and they don't learn it in the context of the problems they solve in the methodology that you use to bring these all together so I hope to show you that today we're going to build a simple class we're going to encounter common problems and show you the Python standard solutions to each of these problems individually these tools are nothing collectively it forms a very good toolkit so I hope to teach you in a very different way than Shiva then you've seen before which is we will do problem solution pairs here's the problem what is the solution that Python offers it to you another challenge with the way that people are typically taught classes is you write a class and you test it and you consider yourself done you don't typically challenge yourself to think about how the users use that class so in this presentation in addition to making a class we're also going to show user code there's a principle that says every new user will stretch your code in a different way than you possibly could have imagined I'm going to hope to through this example start to raise your awareness of how people will abuse your classes and how you can help them out how are you guys enjoying your PyCon so far that organizers have just done a fantastic job this year we really need to thank them I think everything's just run smoothly it's a it's really been great I'm hoping to sneak my child up here and show him off again he's got a cute little up icon t-shirt on today all right let the games begin I'm going to try and make this light-hearted and fun I hope you guys laugh a lot as oh we go along while I'm filling in technical different information by the way how many of you would classify yourselves the coders that's good this is a closed class there's going to be code on the screen none of this high-level stuff that you have to do at keynotes so here's our plan well learn the a development toolkit we'll also see the different ways that users exercise your code and then I'm going to make fun of our friends up in San Francisco practicing their Lean Startup methodology slash religion how many of you've heard of a Lean Startup how many of you've heard of agile is there a relationship between the two absolutely the agile methodology was designed to get rid of waterfall where you go quite far in your development before you realize you develop the wrong thing and the idea was to replace the entire waterfall with a bunch of tight iteration cycles but the idea is you do a little design a little coding a little testing and then iterate and that testing and coding faced informs the next phase of the design it makes you a little bit smarter about what you do so that you don't have to see 10 moves ahead all you have to do is see one or two moves ahead if you're doing the wrong thing you can adapt rapidly and make a change in your code early on before the code does become heavy this is the agile methodology people will describe in many ways but tight iterations and rapid adaptation is its core feature what does the Lean Startup methodology someone got the bright idea hey the way we did things in calm one point Oh was not a good way raise fifty million dollars spend it all go to market and if we had the wrong product sorry goodbye 50 million dollars the new way is to borrow three million dollars and to put out a Minimum Viable Product take it to market and get some feedback right away and do very rapid iteration so that you're constantly pivoting and adjusting so that by the time you've run out of money you've achieved what the magic Nirvana of product market fit people pay you massive amounts of money you become very wealthy buy an island in Hawaii or have an IPO at $38 a share and get married all kinds of good things can happen to you hypothetically speaking all right let's go back to the beginning the graduation to limp you graduate you have two choices you can get a haircut and get a real job are but you could be the CEO of your own startup I say go for it which will be more fun let's go to the startup all right well borrow a moms pension fund to get going I mean how badly could that turn out and so we're going to be CEO of our own startup and we'll use the Lean Startup methodology how do you spend the first three weeks of your time thinking up the name of your company in the elevator pitch but it turned out what we studied in school was we got a PhD in circles we know all about circles in the world in these circle technology where you're going to be a circle company I'm going to save you guys two weeks by coming up with your company name circuitous and your elevator pitch your elevator pitch is advanced circle analytics company how does this apply to your regular programming I put this in a doc string right at the top of a module what goes at the top of a module it's your elevator pitch what the heck it is you're trying to do a lot of people skip the module docstrings so they're skipping their elevator pitch that says what it is they're doing the quick orientation to the code should you skip the elevator pitch no it's absolutely essential but I've just saved you three weeks of development for your company all right so we have some code let's get going start coding let's make a class circle what do you notice that I put in my class did I start with pass now started with a doc string did you know I've only type three lines of code so far the module level doc string the class definition and the module are the doc string for the class a hundred percent of what I've written so far is fully documented how's that they didn't used to do that in the old waterfall okay so start with this why should you start with the documentation well one it's nice to be fully documented at anytime two is particularly nice to be able to run it through a nice tool like Sphinx and get a PDF out of it hand it to a product manager and before you've written your first line of code so can you go sell this our internally to a company you make the documentation you hand it to your users and say can you use this when it's the easiest time to get feedback on the usability of your product before you've written it you will never be more agile than at this point but I'm concerned is this advanced technology we promised an advanced circle analytic toolkit is this an advanced class no it's an old style class unless it's Python 3 in which case it's automatically a new style all sim to 7 here so this is not advanced technology let's fix it ok we inherit from object what does that do for you it enables the toolkit of all the things I'm about to show you in fact if you go out and try any of the things I show you and they don't work the number one causes almost always you didn't inherit from object when you inherit from object you get some extra capabilities things the object knows how to do the old-style classes didn't know how to do this is the advanced technology just by inheriting from object next thing on the list is instance variables everyone here has made a class before let's remind ourselves what instance variables or for instance variables are where you put the the data for each instance that's unique to that instance if you have anything that is not unique to the answers does it belong in an instance variable no do i everybody says no all of you made classes before you all know that do I go from company to company looking at giant code bases full of lots of instance variables I have nothing to do with the instance yes you see it everywhere don't do that the only thing pertinent to this circle at this point is its radius we store that that varies instance by instance by the way is to under an it a constructor no you know that because at the bottom of the screen it says it's not a constructor what is self self is your instance it's already been made by the time and it gets called it's calling the class that makes the instance so an it is are often improperly represented as a constructor it's not it's an initializer it takes an existing instance self and it populates it and it takes the radius and stores it in the dictionary the goal of instance variables is to get the data unique to an instance the goal of dunder init is to populate that instance for you I'd ask you learn something new but I'd really hope the answer is you haven't learned anything new yet but we're framing the conversation for what comes ahead next thing is we need a regular ol method regular ol method you can tell us a regular method because this argument is self whenever I'm teaching Python one of the first questions is why does it have to be self doesn't have to be self no it could be this it could be the letter S why do we use self because it's a cultural norm and you want your Python to look like Python to other Python programmers but you could use any variable you want why did cuido even require us to put self shouldn't it be implicit and some languages it is I think this was a accidentally good design my understanding is that he spent three months building the language with modules and functions and got it working fairly well but it wasn't object-oriented used up all his vacation and then later he's like oh I need to add in objects I believe it was done in one day what he did was he indented all of the deaths he took the functions that he already had and ran them in their own namespace class is a name say if you actually cut the code out of a class and stick it in a separate module and import it it looks remarkably like the class dictionary how many of you knew that what's inside is effectively a module did you know I can put print statements in in the class definition and when you define the class it'll print you can put four loops in there you can actually alongside the diffs open and close files unbelievable this is against the rules in Java and C++ it's not Java and C++ just say it so the way the class works is the definitions run is if they were in their own module and then the module dictionary becomes the class dictionary who learned something new now cool and so I believe that the way cuido designed it was he simply indented the functions and stored on the class dictionary he needed a way to get to them and so the only programming that was necessary for that one day was to program the dot the dotted lookup will reach inside other class find the function and then take the first argument and fill in self for you hey what's going on with this 3.14 it's a matching number it's a constant right just tuck it in - are we doing things the right way no that's not advanced circle technology we should use reusable components let's import from math and use math dot pie in what ways is that better one it make sure that we're going to use the same pie throughout our code it's maximizing a code reuse but it's a PI a constant how about it's a variable that never changes there's actually a difference if you run this code on an old version of Python you'll get a 32-bit version of Pi if you run it on a modern Python you'll get a 64-bit version of Pi and if you take the same program and upload it to a Cray it'll run out to 128-bit version of our PI your program will self adapt when you do things like this all right so modules for code reuse as this just started out so darn basic you can't believe it you've only learned one new thing which was that what runs inside is effectively its own module we'll go to new and exciting places by the way is this could work yeah it's a simple enough class what are we going to do next we've done an area for a circle what should we put in a perimeter sounds nice how about a color how about a secant how about an inscribed Pentagon you're going to need all of those things right what's yangmi stand for you ain't going to need it so yagna is a something people used to toss around is just a random acronym but now it's part of the religion the religion of Valene start-up says don't put load down your product initially with a whole bunch of features all you're doing is investor money moms pension plan without knowing with the clients I whether I actually need it so part of the discipline is to stop throwing mountains of stuff into your classes start with a minimal possible the more you throw in the less agile you get should we add a whole bunch of things right now and give up our agility no what should you do commit push and that's the agile part of it the Lean Startup part of it is I have MVP Minimum Viable Product you ship the product it is not enough to check it in you have to stick it into a customers hands early on to get some feedback perhaps they don't need no stinking circles it's better to find out now before you spend all of moms pension fund just say it okay so we get ready to ship I thought I had a shipping on the next one I guess not all right Oh share data we didn't put a version number on all the instances share the same version number we use class variables for that notice the instance variable is a stored in an it remember the job of the initializer is to populate the empty instance but a class variable shares the data so version is shared across all instances should you put shared data at the class level or the instance level class level my version number I put it in quotes that makes it a string why didn't I do a floating point number is version point eight plus point one equal to 0.9 yeah if you haven't had any bad experience out with binary floating-point numbers someday you will in the meantime strings work really well they're easy to parse another common way to do the version numbers is with tuples but it allows some of the exact comparisons a little learning point here don't use floating-point numbers for your version numbers just say like well I printed out the clients of the documentation for this thing and they say they're version 0.1 oooo 1 what does that tell you about the quality of the code use strings all right so we have Minimum Viable Product what do we do ship it so I'll we print up a tutorial how to use our class we've announced we're secure this we can look up our class variable we can make an instance by calling the class and we can look up an attribute radius and we have a method marry area what more could a person want right shift out the door and we have to put it in the hands of some people who can use it because remember your customers inform you in a way that you can't be informed otherwise if you don't have customers stick in the hand of some other user every new user will stretch it in a different way user number one was nobody would pay us for this but our friends back in academia they want our software and they use an interesting way their goal is to make a research grant proposal to compute the average area of random circles they're starting in their proposal with in equal 10 but it is their hope if they get a DARPA grant they will never have to graduate and could stay in school forever not getting either a real job or doing a start-up making them very happy the surprise for us is that they used an in equal 10 you're like wow I when we made this we thought if only one or two circles these guys may make lots of them so one thing to anticipate for how people use your classes is possibly they're going to make many many instances and these guys are fully exercising the API taking a look at the radius and making the area call and using our random circle generation plus it gives me a chance to show off list comprehensions why did we use x range here memory efficiency it scales and it's fast for the most part anytime you expect in to be bigger than a very small number always use a range why did these guys see the random number generator reproducible results oh that's a new feature in science reproducible results how often you get that ok so new technology ok the next customer oh is this rubber sheet company our friends hood academia refer to sod new customers this is great because mom was getting worried because customers number one didn't pay us customer number two can write us a check the rubber sheet company but they have a special request they want a perimeter method is now a good time to add it yes there is a known use case there's actually lots of things I've considered putting in the collections module how many of you would like a bye marry tree how many of you would like a skip list how many of you would like a bloom filter until you ask me for it until you demonstrate the use cases where you're showing restraint and putting it in we're shipping the product we know you need instead of how about I add five hundred things to the collections module and you can't find the three or four that are useful to you restraint is important for your customers as well as for yourself too many features is a bad thing in addition to losing your agility so now we put in the perimeter and these guys go to use it and here's their code they actually cut several holes in our one rubber sheet so one rubber sheet has three cuts in it they make three circles Luke over it print them out and they said is they called area and said it had a cold area what is this all about well it turns out when you take rubber and heat it up it stretches so they did something we didn't expect they changed the radius after they changed the radius to allow for the expansion of the rubber sheet they recompute the yaw while warm area was this an anticipated use case now I think it's hard to see this one coming basically if you expose an attribute expect users to do all kinds of interesting things with it if it's variable it will change sooner or later it is not a constant in this case the radius of a circle was not constant after creation how do you guys feel about exposing the radius at the time so that the customers can both look at it and modify it who feels icky about that ah all of my Java and C++ programmers feel icky about it why because there's an ironclad rule in the Java and C++ rule and most object-oriented language and that are in Clyde rule that you must follow is do not expose your attributes what should you do instead getter and setter methods people don't get to access methods but what would that do to our poor client look at this line here what would it turn into they would have to do a see get radius x 1.1 take the result of that and call see set radius isn't it awful looking code it made our customers suffer for it so let's just do things the Python one way and the Python way so consenting adults language will leave all the doors locked and hey we'll just risk whatever happens to us remember there's no private and protected in Python how do you feel about exposing your attributes in Python is common and normal when you go back to the other languages it's not common in normal and you absolutely must not expose your attributes because if you need to change the underlying implementation at some point you are absolutely toast you are locked into your client code and can't change in Python you expose your attributes yes in other language to you absolutely not different rules don't let people in impos java rules when they are coding in Python and vice versa it has a common fault that people who get good at one language and check the rules of that language into another these are completely different animals okay the next customer is a national tire company we were referred but true or false this is my first rodeo false will I tell you yes at the appropriate time I set it up we wait the tension builds to a climax and knock it down later I promise to tell you but not now how about that I'm having a good time with you guys all right by the way after we get our rubber sheet company we got a check in the mail wasn't a big check mom was happy Oh pension funds starting to come back it looks like she may be able to retire after all but now she hears we might get a national tire chain that's awesome these people will write big checks maybe we can get out of moms of a back bedroom and get an office down in the soma district you know how that goes no air conditioned no he but you're hanging out with all the cool people at Twitter and Craigslist and Wikipedia and Yelp and all that sort of thing so trade the air condition you're still being paid in stock options don't get me wrong you know not actual spendable green things so the third customer is the national tire chain they're the ones that going to get our office for us and they need a customization they track their tires by their inner radius they are very interested in the inner area of a tire but when you put a rubber on top of it its circumference change and they need to know that so they can do accurate statistics for your domitor and a speedometer so they need the perimeter to do something different how do customers commonly change your functionality they do it through sub classing most people who write classes don't spend time thinking about how subclasses will use their code so we're going to see a sub class or do that they are over a sub classing tire overriding perimeter when they call 1022 what method gets called dunder init do you see a dunder init here no it goes up and borrows the one from the apparent method when they call area it goes up to the it looks first entire and then into the circle class as a parent but when we look at perimeter it looks first entire before it looks at the parent and it finds the perimeter so this is the parameter that gets called now does the parent get called it all yes we actually refer to it inside our perimeter so we've intercepted the call and call our parent if the parent eventually gets called this is the section is called extending the parent doesn't get called at all the section is called overriding so we extend the parents method we let them the parent compute the perimeter and then we scale it up by 25% for the rubber this is common in normal you should expect every class you write somebody's going to subclass it and do something horrifying to one of your methods just saying expected is this common in the software field absolutely how many of you know the lay of the language cover awk aww Kay the idea behind op was the people who wrote it were saying people who wrote C and said and they said well said is really good at regular expression of checking and doing some substitutions C is program but well how about we combine the two a C like syntax with regular expressions and their intention was that awk would be a programmable version of said they said their jaws dropped the floor the first time they saw an ox script more than ten lines long they never imagined this I got a chance to interview with one of them at Google and I showed him a flight simulator I wrote in a long time ago it outputted ansiedad sis I didn't have curses on the system that I was building on and actually I built all of the instrument displays and it worked perfectly well did they anticipate that use case you have no idea what people are going to do with your code when you publish it just saying okay then next company is a national graphics like company by the way when we got this uh tired company we got written up in Wired magazine is that cool a little of free advertising yeah mom's a little more happy now and so we get a call from the national graphics company yeah we're dealing with these big people they they change the rules of the game a little bit they are not going to take our product right out of the box they're going to have custom needs in fact they have many suppliers they have a circle supplier a square supplier a triangle supplier a Pentagon supplier and across supply a supplier and they don't want a different interface to each of them so what they need is a converter method because all they're interested is as a bounding box of the shape that's how the graphic company manipulates shapes the properties like radius are something that applies to a particular shape so they need something that will take their bounding box AG and I'll convert it to a radius is this a fun way to interact with our class no it's tough for the client they have to use the converter function everywhere do I see this sort of thing all the time yeah I go from client to client and they provide published converter and adapter functions it's not the right way the API is awkward what the customer really wants is a different signature they don't want to have to construct a circle from a radius they want to construct it from a bounding box diagonal this is called a constructor war as constructor Wars are typically fought necessarily with customers though they sometimes happen right when you're first creating a class and when you define have a constructor war several people want different czar signatures for the constructor the usual way is decided is who has the most power well I'm the lead we will use foam bounding box diagonal or in this case the National graphics company is the biggest customer we'll just throw all of our other customers code out the window and destroy them in favor of our new client you've never seen this happen in real life right okay so there's a constructor war Brian what's the right answer a name Kramer is actually a fairly a reasonable approach I say the right answer is whenever you have a constructor war everyone should get their wish you should have more than one constructor if there's a constructor war people are digging in their positions for a reason they need that constructor they shall not be denied so is this common and normal sure we do it in Python itself the one true way to make a date time of course is a year month and date but perhaps you work with time stamps at which I'll point that constructor is really awkward so we give you a second constructor from time stamp that will convert this time stamp into a date some of you just like to count the days from the beginning of the Gregorian apocalyptic calendar and you know who you are and that is actually the name of the calendar we use Gregorian apocalyptic apocalyptic calendar which I had never heard of before I encountered the day time module so you can use the number of days as a beginning of a time to compute the ordinal are you can just ask for now these are four different constructors from day time now kind of speaks it for itself but the the preposition used here is typical in alternative constructors you use the word from so the day time gives you several different ways to make a day time object how many of you knew that already okay how many of you knew that cuido and Tim Peters designed this module how many of you know that they're really good at Python they provided alternative constructor should you do it - yes you should copy all of their behavior size or at least on when it comes to design not all the other behaviors tim peters one of the smartest people I know he also date trades with Python probably making a fortune and on Facebook all the time I get to a little farmville notices from him don't copy everything they do I'm not kidding it's true and then here's one that I put in dict from keys that constructs a dictionary from a list of keys how many of you knew that was there he goes to back to all the way to I believe Python 2.1 it is not as well-known as it should be it is an alternate way to construct our dictionaries so the solution is whenever people want multiple constructors you should give it to them and don't decide on one over the other the technique for doing that would be to provide an alternative constructor via class method so in this case the class method we can call circle from BBD the bounding box diagonal and that will convert to a radius and then build a circle that way a person doesn't need the converter function they just call the appropriate constructor just like you would call date/time from time stamp you can call circle from BBD and it will make a circle instance for you do you dig it yes that is what class methods are for yes sir is this my first rodeo why actually five bucks of that question I used to say five bucks all the time and then Sam started sending me a PayPal invoices so that's bringing that to so no more verbal uh five bucks it was actually an excellent question what the cola from BB D is to make an instance of a circle so shouldn't we call circle here absolutely but there's a problem the it works fine for the graphics company they can call from BB D 25 and make themselves a circle but it should also work for some classes what about these tire people shouldn't they be able to call from BB D they should but what our code will currently do is call from BB D which is hardwired to make a circle and not a tire it is very common for people to devise classes and never think about their poor subclass ease so we failed to think about the subclass ease what do I need in order to parameterize a call to know whether I was called from a tire if I'm from BB D how do I know whether I was called from a tire or whether I was called from a circle yeah we pass it in as an argument so a class method says change the first argument from clause self to class we need two classes of parameter so we know what to construct I just changed circle to class excellent anticipation of the next slides calling me out just like this morning and the key knows that code doesn't run that wasn't you though all right all right so we've got another so the summary here is when their constructor wars everyone should win provide the alternative constructors the technique for doing that is class method the parameter to it is class be sure you use that parameter because it will support the sub classing thumbs up if you got it sweet let's drive on new customer request add a function the angle to grade is a function that is a trigonometric function vaguely related to a circle so it converts the angle to radians computes the tangent and says that a 5 degree slope is equal to a 8% grade you know to help the trucker's I convert between their tables between what they see in their in klonoa and what is on their charts says how good the brakes are going to work do you guys like to watch Ice Road Truckers it's great you ought to see what truck loaded with dynamite does going up an icy Hill which is a 10% grade and it doesn't quite make it to the top and slides back in other words this is an important problem so I put this in my Alfa release and I hand it out to them and disasters ensue one thing that happens is some people who were working with fear classes grabbed our function because it's just sitting out there unrelated to circles does this work for spheres it does not spherical trigonometry is different from plain trigonometry that's why they call it plain trigonometry so people were using it inappropriately the hyperbolic people well that's right out there their trig functions are entirely different they don't even resemble triangles anymore and Moberly can people even find this code they're like these circle people gave us a circle class I did adder of it I did a help and I couldn't find angle degrade anywhere why not because it wasn't in the class what's the solution put it in the class and dentate so now I slide it inside we've got a regular method and now when people call it they put take an instance of circle angle to grade that way the sphere people never call it the hyperbolic people never call it and the circle people know where to find it a lot of improvements one thing sucks though the thing that sucks is really you have to create an instance just to call a function a hint that you're doing something wrong as we passed in a variable self do you see self used inside there is nothing in this function that needs to know about an instance of the circle this is an awkward API to call and you're wasting a variable as a solution static method for the purpose of static method is attached functions to classes probably you guys knew that but you couldn't articulate exactly why you would do such a thing you do it to improve the findability of the function and to make sure that people are using the function in the appropriate context circle angled to grade it is not used that often which is why most people don't know what it's for so if you learn something new you learn how to articulate its purpose why in the heck would you put a function in the class in part because that's where people look for it when I was doing some consulting for a company with a weather modeling they had a cloud class and lots of information related to the cloud people who work with the clouds occasionally need to convert inches of mercury to millibars and back they're looking for the conversion function does that have anything to do with clouds now that's a straightforward conversion that doesn't have any to do with clouds but people look using the cloud kit we're looking for it and they needed it then you put it all the related tools in the toolbox okay the next one out so here's the trucking company making its call circle angle to grade it's a nice clean call we didn't have to instantiate first the correct context is present people would not use this with spheres and the method is findable who learned something too all right fantastic oh we got a problem big government client they send in an auditor and they're going to tell you not what your products should do they should tell you how it does it theirs is OS for this this is called micromanaging when you tell somebody how to do something and they said we're going to get the big government contract which would make mom really happy we get to have our IPO I'll get real secretary paid vacations office manager those stock options might look like something all we have to do is a piece of government to get the big contract and so ISO one one one zero I made that one up says that you are not allowed to use the radius here in the area calculation instead you must call the perimeter and then back into the radius the reasoning for this is that at one time they divided a perimeter into an area and they had an inconsistency between the two and it caused an airplane crash a lot of government rules and regulations that seem arcane almost always have a catastrophe behind them so there's actually legitimate reasons for other silly rules so I change this to call perimeter and then I back in to what the radius is and compute off of that was a simple change I think so all problems have simple easy to understand logical wrong answers and this is one of them what my problems with the variable name self as it implies that it's you but it's not you it is you are your children whose code to this mess up entire company they subclass stop perimeter we are not calling our perimeter we're calling its perimeter its perimeter scales it up so we've now broken their area method they should be free to override any a method they want without breaking one of the other methods and now we just broke their code I know what you're thinking there must be a better way how about we keep a private copy of it so here's the problem with a terror company their perimeter code and our better way is we're going to make an underscore perimeter equal to perimeter we will have a second copy of it we'll say go ahead and override it all you want but I'm going to have this self called the underscore perimeter our original copy that way the subclass R can override and you know what this fixes their code doesn't seem like a great technique you keep a spare copy of something that waives someone overrides it you've still got the original it seems like a reasonably good idea but if it's a good idea for you it's a good idea for them too so they adopt this idea of well we're going to protect our reference here by making another parameter hello hello you're copying me you're copying me you're so annoying you're sorry and you're starting to try and win this game it's the copying game on the playground in first grade they're copying you like you're ugly they go you're ugly that didn't work I'm ugly no I just insulted myself how do you win the copying game the way you win the copying game on the playground is you use somebody's specific name Heather is ugly now if Heather copies you Heather is saying Heather is ugly you won this is the way remember we got a copying game here any naming convention they use we need to do the same thing so the answer is going to be the name should be underscore tire under perimeter and this one and so if you include your own name this will refer to underscore circle perimeter and then they will all copy us and do underscore tire perimeter and we won't have a naming conflict anymore clever first game first grade skills coming back to save the day in a programming environment who learned something new this practice really works it works so well we've automated it for you and it is the double underscores in Python where my de-evolutionator saw my double perimeter and my dunder perimeter got chopped off on the bottom of my slide apologize for that but the dunder perimeter will automatically name mango for you and rename this variable to underscore circle under inter perimeter then if the tire people use the double underscores to they'll get under tire under perimeter who learned something new the purpose of the double underscore is what we call a class local reference we are making sure that self actually refers to you most of the time self means you or your children but occasionally you need it to be you people will mistakenly represent this is the way to make things private in Python spy thought about privacy now it's a consenting adult language we don't leave the locks on the door the intention of the double underscores was not about privacy the intention was to use it exactly like this it's actually the opposite of privacy is all about freedom it makes your subclasses free to override any one method without breaking the others okay we've half way to a government contract but we're also in violation of ISO 22 22 this is a huge problem their new rule is we have one little change we looked at your instances and you're no longer allowed to store the radius you must take the radius out and replace it with diameter and you can't have both what code does this break it breaks our entire class we'll have to go in and now we get to keep the API the same it's how we stored internally the changes so we can accept the radius in the constructor are the initializer but we actually have to store the diameter and then we'll have to change all the functions they use it and all of our client code including that which was published in Wired magazine in the Wall Street Journal the books are already out the door we are going to destroy all of our clients code by this change isn't horrific what is it that we wish we had done in order to access the instance variable for radius what a Java person would do getters and setters if we had just done this from the beginning we would be able to make this change easily now and store the diameter and lookup the diameter do you guys understand why now you never expose your instance variables there must be a better way we're sending it right here we've written these getters and setters I've just wish that I didn't have to change any code I just wish that whenever somebody did a dotted look-up on an hour radius that it wouldn't match changed to a get radius and every time they said it would magically change to a set radius and if you wish hard enough the fairy godmother appears and as property and what property does for you is after the fact without changing any of the rest of our code or any of our code or any client code we can add this in and magically it will every time we set radius including in our own code this will no longer store in the instance variable this will call this setter and actually store the diameter this is why in Python you don't have to exposing your attributes even if you never use property it's a big win for you why because python api's are clean and beautiful they don't have getters and setters everywhere in fact if you find yourself divided assigning a getter and setter you're probably doing it wrong all you're doing is making it awkward for your users to access dotted access is a lot easier if you actually need to make a change later after the fact it's really easy to do by putting in a property this is a big big win in our language so when I get people coming from C++ and Java it's the hardest thing for them to give up they just feel naked exposing their attributes like I get over it there's a problem later we can just fix it by adding on a property and it's effortless in the meantime your code looks very beautiful you learn something new now you know what's properties for you get a benefit from it even if you never use it the benefit is every time you design design a class you don't have to put getters and setters in it and that will make your classes shorter I'll make them raw run faster and it'll make the API it's more beautiful I think it's a big win is something you can do in a dynamic language you can't do in a compiled language fair enough Oh was I clapping because I'm done or because it's awesome awesome and it's this question yes and I clapped back to you because it was actually an excellent question all right so now we're rich and famous we've got a lot of money we paid back munching mommy's pitching plan she's wealthy we've bought an island in Hawaii and gotten married on the day of our IPO the problem is when you're billionaire you no longer know who your friends are so your only real friends are the ones that you had before you were a billionaire and it was those poor academic guys like the research grant and they're going to go out and make 10 million circles to improve mankind's knowledge of average circles and they call you up there madder than heck hey as we start to scale up you are eating up an enormous amount of memory did you know that every instance has a dictionary and that but between the instance and the reference to the dictionary our circles are over 300 bytes each which is kind of big for all they do is store the radius there must be a better way how can you drink down the size of your yes it's Lots if you pick up a design patterns book this is called the flyweight design pattern the flyweight design pattern says mate when you have many many many instances make them lightweight and slots is a way to do it what slots we'll do is say let's allocate just one pointer for the diameter and nothing else no dictionary you lose the ability to inspect the dictionary you lose the ability to add additional attributes so this is just an optimization you always save it for last only when you're scaling up to a bajillion of instances how many minutes tomato I'm exactly on time this is good so you implement the flyweight design pattern saving it for Less by putting in Dender slots it suppresses the instance dictionary and it saves an enormous amount of memory and now your friend scanner are free to run two million circles without eating up all the memory on their machine who learned something new do you do us lots first or last last because if you do it first you lose the ability to inspect the dictionary you lose the ability to add new variables all along the way but if you stick it in right at the end you get memory efficiency and memory efficiency on a modern machine is particularly important because as we discussed in earlier talks the cost of a cache misses is expensive as a floating-point divide it'll slow down your code right a bit I'll take questions about one minute I would like to summarize unless I truly need to stop right now what's up I did not I did not break tyre because we have our getter and setter for the radius it turns out the from the users point of view there is no change at all the only people who would be hurt would be a subclass at storing an additional attribute like color right wrong the interesting thing about slots if some a subclass is you it the slots doesn't inherit you are the in the dictionary suppression capability of it disappears and so subclasses going to add their own attribute so that's a fine thing it means subclasses can go about doing the thing and I have not broken their code I have an excellent point I knew that's the one that you intended me to make I appreciate that and I actually believe that to be the case this man knows this stuff all right the summary this is our toolkit what do you need to inherit from in order to get all these other things to work object has it all these other things it turns out those double underscore so did only the Dumble underscores off the list that's because I was out of space I will add double underscores to the slides before you get them but the double underscores will work even on old-style classes but instance variables works for the information unique to an instance class variable is for anything you share a regular method is a method that has self and that how you use a regular method whenever it actually needs self to walk out on the instance data class methods the primary purpose is alternative constructors whatever people fight over constructors always implement a class methods what I do is what cuido does is what tim does is what you should do - they need class why do they need it so that you can create subclasses like tire as well make sure you use the CLS whenever you make a class method static methods you can tell a static method because it doesn't need either self our class its primary purpose is to attach functions inside a class so that they become findable and so they get used in a correct context property transforms dotted access into method access and this allows you to freely expose your instance variables it's a very big deal slots implements the flyweight design pattern makes your classes lightweight and a free I'll add the double underscores to this for class local references and you will have nine tools that professional Python programmers use every day who learn something new sweet no time for questions I will take questions outside yeah okay

Channel: Next Day Video

Views: 236,319

Keywords: psf, pycon2013, talk, RaymondHettinger

Id: HTLu2DFOdTg

Length: 45min 56sec (2756 seconds)

Published: Wed Mar 20 2013