Protocol Or ABC In Python - When to use which one?

today i'm going to talk about should we talk about that first yeah let's let's talk about that first the letterboard is gone i mean it's not gone i still have it but it's no longer in the videos and that's for two reasons actually one is it was kind of screwing up my recordings you know i forgot to replace the text with something else and i had to redo the recording again and another thing is that i felt very stressed about having to come up with these awesome one-liners all the time and i was afraid i was going to disappoint you so instead of putting this leather board there and giving me all that stress what i'm just going to do is to include a few more stupid jokes in the video itself and then hopefully compensate by that anyway this video is not about letterboards it's about protocols and abstract based classes they're quite similar in python but when you consider them as part of the software design i think they have actually pretty different uses so that's what i'd like to show you today an example where i apply both protocols and abstract base classes and have at the same time very different results in terms of the software design my channel focuses on explaining intermediate to advanced programming concepts mostly in python but i also like to talk about software design principles and design patterns and basically anything that helps you create a better software design i've made a lot of mistakes myself in designing software applications in the past and one of the most important things that has helped me is to basically write down those things i also want to share what i wrote down with you for free you can get this at ariumcodes.com design guide it's a seven step guide that helps you think about your software design and you can read through it pretty quickly apply directly to what you're working on so get it for free at ariumcodes.com design guide now let's take a look at today's example the example i use today to talk about abstract base classes and protocols is an internet of things service there is an iot folder here that contains a couple of files containing some useful classes and functions and there's a main file that i use to test a few of these things so let's first look at the files in here so we have a device file that contains an abstract base class called device it has four methods connecting disconnecting sending a message to the device and retrieving status update and you see i'm going to use these methods later on in the rest of the program it also imports a message type from the message file so this is basically an enum that contains different types of messages switch on switch off change color play song open close and so on you can add more to this if you wanted to and there's also a message class that contains a device id so that's the device that the message is intended for it contains a message type and some data which is in this case a string probably you want to do something a bit more complicated if you're really creating a system like this then we have a number of devices that inherit from the device abstract base class and therefore implements these methods that it defines connect disconnect send message and status update there's a hue light there's a smart speaker and there's a piece of curtains smart curtains of course these are not real objects i just print out a few things so that we can see how everything fits together but there are three different types of devices that we can use in this system then we have a diagnostics function that takes a device as a parameter connects to a diagnostic server retrieves a status update from the device and then sending that to the server again it's not actually doing that it's just to show how things work and then we have a service file that contains the actual internet of things service this is basically going to be an object that's responsible for keeping track of which devices are part of our local network and that we can send messages to so there's a register method and an unregistered method see it maintains a dictionary of devices via strings which are the ids of the device and when you register device it connects the device it generates an id it stores it and then it returns that id as well we have a method for getting a device or looking for device given an id and finally we have a method called run program and the program is in this case simply a list of messages and it's just going to send the messages to the devices that these messages are for and in the main file i create an internet of things service you see i have all these imports here i create and register a few devices or the hue light speaker and the curtains i create a few programs so i have a wake up program that is a switching on the hue light the speaker playing a song and opening the curtains and when i go to sleep well i want everything to be off and i want the curtains to close so those are the two programs and then using the surface i can run these programs and then collect some diagnostics in this case i collect some diagnostics of the hue light when you run this program you see that it connects first to the devices then it runs these two programs the wake up program and the sleep program and then finally it connects some diagnostics and sends a status update to the server so this example specifically uses abc's as you can see here we're not using protocols so there's a few things that are important to know when you use abstract base classes so one thing is that if you want to inherit from an abstract base class obviously you need to write that explicitly you need to import the class and you see that happening here in the devices so each of these classes is a subclass of device devices imported from the file iot iot dot device and that's the case for all three of these what you also see is that device is used in other places as well for example in the diagnostics module the function that collects the diagnostics needs to know that it gets a device so it also needs this device class and for the service this is actually the same thing so we're importing the device here because well we need to register devices and we need to call the send message method in order to run the program so that's what this means in terms of dependencies everything that relies in some way on the abstract base class either by implementing it or by using objects that are a subclass of that need to know about this class so you need to import it and if you're using a type checking tool something like an abstract base cloud can help you clarify what you need it's going to point out errors for example if i go to the abstract base class and i would comment out this connect method just like this then well here you're not really going to see problem because it just each of these classes just defines a connect method but if you look at the surface now you see that we're going to get a type error here that connects is unknown it doesn't exist so you can't call it right so let's make this abstract method again part of the device and then obviously you see in the surface that the problem is gone now if you go to one of the device subclasses let's say we are going to comment out this connect method here so now hue light no longer implements all of the abstract method that it needs to and well here obviously we don't get a problem because it's not dependent on the hue light if you look at the surface it's also not directly creating the hue light so there's also no problem here if you look at the main file you see that there is a problem here can't instantiate abstract class hue light so having type checking switched on helps you find these problems before you actually figure them out when you run the program so when you run this program it's going to stop at the point where it's trying to create an instance of hue light so that's how abstract base classes work at the moment you create the instance then it's going to do the type checking and verify that the the methods are all there and that there are no abstract methods because you're not allowed to create an instance of class that has abstract methods protocols were introduced in python 3.8 as an alternative to abstract based classes and they work differently from the typing point of view abstract base classes rely on so-called nominal typing and that means if you want a typing relationship to be there a is of type b then you need to explicitly write that down in your code for example using inheritance so if you have an abstract base class you create a subclass of that abstract base class you inherit from it and that establishes the relationship the python interpreter uses that relationship to determine whether or not the types match protocols are quite different they rely on structural typing and that means that instead of having to explicitly say that something is of a certain type of a protocol python looks at how the structure is of these objects do they have the same methods do they have the same properties if so then it is assumed that the types match so that means that the usage of protocols is actually also quite different you don't establish generally inheritance relationships with them you don't inherit from a protocol class but a protocol defines the interface that is expected in the part of the program that refers to it so if you have a function or a method in a class that gets an argument of a particular protocol type then anything that implements those methods that has those properties can be passed as an argument to that function or method and structural typing that will actually do the comparison of the structure of the objects is going to make sure that your program works as you expect it to this fits very well with python's runtime type checking system that treats two objects the same if they have the same methods and properties that's also called duct typing so now let's change the example to use protocols instead of abstract base classes and see what happens so i'm going here to the device class and i'm no longer using an abstract base class but i'm going to use a protocol there we go it has also automatically imported this and because we're using protocol class we no longer need these abstract methods here so i'll remove them and normally what you also do with protocols is that if you don't provide the implementation which normally you don't you write these three dots let's do that here and here as well so that's what we have now i can remove the import here our devices because it's a protocol class no longer need to directly inherit from the protocol class the normal way you would do this is use doctyping instead so i'm just going to remove this inheritance here there we go and let's also do that for the smart speaker class and for the curtains class so that means we can now delete this dependency here it's no longer needed in the diagnostics bit we still need to know that it's going to expect a device that's so that it can call a status update function similarly in the surface we also still need to import the device class because we're expecting it to have a connect method and a disconnect method and a send message method so on the main file nothing changes here we're still creating exactly the same objects i can run this now we're going to get exactly the same result so the main difference now is when we change this to a protocol is that we no longer have the import dependency here in the classes that implement that particular protocol and now the typing system also works differently than what we had with abstract base classes if i go back to the hue light and i comment out this method again let's see what happens so comment out the connect method and then go back to the main file you see there is no problem when we create an instance of the class because well there is no inheritance relationship it's simply a class that we can create an instance of so that's not a problem what you do see is that you get a type error at the point where we're trying to register the device because register device which is part of the internet of things service expects something that follows the device protocol that implements the device protocol and that's of course now where the system breaks down because huelite no longer has a connect method and when i try to run this you'll see that i won't get an error when i try to create the instance but i'm going to get an error when i try to call the connect method because that simply doesn't exist in the hue light class because it didn't follow the protocol correctly so this is a key difference between how abstract base classes and protocol classes work it's it's the point at which you see the problem occurring with abstract base classes that's when you try to create the instance with protocols it's when you try to use the instance at a place where it's expecting to implement that particular protocol a big difference between protocols and abcs is that protocols are mainly there to define the interface to other parts of the program that use the protocol in some way so i have a method that has an argument of a type protocol and that helps define what the method expects this reduces coupling for example the collect diagnostics function doesn't need to know anything about devices that can connect disconnect or send messages it only needs to know that the object has a status update method that it needs to call similarly the internet of things service doesn't need to know that devices have a status update method but it only needs to know that they have a connect disconnect and send message method because that's what it's using so that means that protocols are also very natural way of limiting the interface and the definition of the interface that you're going to need what i've done until now is define a device class that contains everything that is part of a device but with protocols you don't necessarily have to do that you can actually split the interface in two and define the part that's useful for diagnostics and define the parts that's useful for the internet of things service with abstract based classes you can in principle also do this but then you're going to need multiple inheritance and to me it feels like a much more natural thing to do with protocols than with abstract base classes so let's look one more time at the example and see how this works and what the effect is so let's now split this protocol class into two parts so that each part of our program the service and the diagnostics part can use only that which it needs so what i'm going to do is i'm going to copy this class and i'm going to put it into the surface file there we go so it's a protocol class still so i have to import it here and we also need the message type and the internet of things service only uses connect disconnect and send message so status update is actually not needed here and it doesn't need to know anything about it so the only thing that the internet of things service needs to know is that it has a class device which has connect disconnect and send message methods there's an error here i think i've imported it here so now of course i don't need to do this import anymore so i'll just remove this and it just has its own device class that it's using in the methods now in the main file this doesn't change anything because that's actually the only thing that a protocol does it just defines the interface of what you're going to need and i can do something similar for the diagnostics part so let's again i'm going to copy this class and now i'm going to diagnostics and i'm going to put it here and let's also import a protocol class there we go now the interesting thing is that collecting diagnostics doesn't need to know anything about connecting disconnecting or sending messages so let me just remove these things here and perhaps we can also name this differently instead of device we could name this diagnostics source because it's a source of diagnostics and let's also change the name here so now it's a diagnostic source i could change this name just for completeness so now here we're just dealing with diagnostic sources and this import is also gone if you go back to the main file still this will work just fine because we're really specific about defining splitting our interface and defining exactly what the different parts of the program needs and that's i think a nice way to think about using protocols so protocols belong with the thing that defines how it needs the protocol so this part belongs with diagnostics because that just defines the interface of what it expects as an argument and similar for the service this defines what it expects a device to have if you want to use it with the internet of things surface and now we can delete this file because well we no longer need it because it's it's not used anymore right so we have our devices so it simply relies on a message type we have diagnostics that only uses the protocol m import we have the the message classes that are used in different places and then we have the surface that also only relies on the protocol and on these message classes so the coupling due to the protocols the coupling between servers and diagnostics has been reduced a lot because we no longer need these imports the only thing we need to make sure is that our devices implement the right methods and what's nice is that because diagnostics and devices are in principle different things we could have a class that only implements let's say the diagnostic source protocol so it only retrieves status data so it's not something that you connect or disconnect to it doesn't adhere to the device protocol and then use that just in the collecting diagnostics part without it being part of the internet of things service and you can kind of achieve the same thing with abstract base classes but you'd have to use multiple inheritance which again i'm not really that big of a fairness for interfaces like this it's fine but still it introduces a couple of extra imports that are not typically needed i feel this is a very natural way of thinking about protocols you simply define the interface of what another part of your system expects and you pass it an object that adheres to that interface that implements that protocol i hope this example shows you that although abcs and protocols are closely related there is a kind of conceptual difference that you can exploit the way i look at it is that abstract based classes conceptually belong to their subclasses so that's part of a class hierarchy that you're done using in other places of your program whereas protocols belong more closely to the places where they're used i also wouldn't say that abstract based classes are obsolete because of protocols i think for example abstract base class can be a great solution if you have control over who creates the subclasses and you have all that put together in a separate module or something and you want to keep that nice and tidy together because conceptually protocols belong more at the place where they're used they might make more sense if you have a huge project with lots of different classes that implement that particular protocol and that you want to use in your application and maybe in some cases you're not even allowed to change those classes for example if they're part of a third party library that you can't change well you could use the protocol mechanism to still decouple your application from that particular third party library without having to change the library and have it implement some kind of abstract class that you have to define then so in that sense i think protocols are really useful when you want to connect different libraries together and still be able to reduce the coupling a cool thing about protocols is actually that you can define the same protocol multiple times for example if you have several modules that you want to be completely independent from each other but there's still overlap in the kinds of objects that they expect you could define the same protocol the same class in multiple files and then those files you can use everywhere without having to rely on the other file on the other hand if you're using protocols that way if you're not relying on inheritance to define the relationship then you also lose some of the advantages that inheritance gives you like being able to create some useful methods already in the super class that you can then use in all the subclasses another thing that inheritance helps you with is if you define the subclass and you explicitly inherit from that abstract base class then at that point your ide and the type checking system in your ide will help you identify any mistakes that you make while you're typing and with protocols this isn't possible because there is no nominal relationship between the protocol and the class that implements the methods that the protocol defines so overall by using protocols you do lose some of the advantages that inheritance offers but it also feels like a very pythonic way to define the relationships between the different parts of your program and what i particularly like about it is that it offers a really natural way of splitting up an interface and only define the parts that you're using in that specific area of your code doing this has a name by the way it's called interface segregation it's part of the solid design principles proposed by robert martin i talked about that in an earlier video i've put a link in the top right and that's also why this is called dog typing not because we're docking something but because if something walks like a duck and quacks quacks quacks if something walks like no this is stupid okay i hope you enjoyed this video if you did give it a like so the youtube algorithm spreads the word consider subscribing if you want to watch more of my content thanks for watching take care and see you next time

Channel: ArjanCodes

Views: 26,688

Keywords: protocol or abc, protocol python, python abstract classes, python tutorial 2021, software development, software design, learn python, python 3, software development course, python abc, abstract base class, python protocol class, python duck typing, python duck typing vs abstract base class, python duck typing explained, python tutorial, python classes, learn python the hard way, python 3 programming tutorial

Id: xvb5hGLoK0A

Length: 23min 45sec (1425 seconds)

Published: Fri Oct 29 2021