Network Automation using Python on MikroTik (REST API Part 1)

hey everyone and welcome so in our previous video i told you about rest api feature of the new microtech router os v7 and also i introduce you to postman api testing suite but our objective is to automate our network testing through postman is not going to automate our network that is just a tool so that we can test the apis we can see what parameters to send and what parameters we are going to receive from the router that's it but how do we automate our network with that so the answer for that is python so what we are going to do today is we are going to use the same rest apis and we are going to build a code in python remember our objective in the last video we want to configure the devices for our customers at the least we don't want to automate the whole network in one go we want to build it up slowly and steadily so to do that the first thing that we are going to do today is we are going to configure our device through rest apis using python we are going to understand how do we use python and how do we use rest apis and how do we configure the device using python and the rest apis available on microtech router os v7 so let's get into it so on a computer uh let's start by downloading pycharm so pycharm is a ide for python and it's a great software for newcomers you can download the community edition and if you want to buy you can buy the full edition also but community edition will do once you have downloaded it uh just install pycharm in your systems and then we'll go from there so once you have installed pyrocharm and you started you see a screen something like this and why are we using pycharm and a not a notepad or some other textured editor ideally you can write your python scripts in a notepad or a text editor also but pycharm we are using because it's an integrated development environment that means that you can debug the code in the software itself and you also have terminal here so in case you want to install certain packages of python you can install everything can be done from here itself so this is the best way to go about i usually use sublime text and that's my go to uh editing software for python or any other type of scripts and all but pycharm is something that will be helpful to you when you're beginning and later on you can shift your focus from pie charm to any other text editor that you feel comfortable with so here let's make a new project let's name our project automation underscore microtech and we'll select the virtual environment we'll be using a python 3 for this in case you have more than one python installed like python 2.7 or something then you can select that but it is recommended that you use the latest python and in case you do not have python free then you can go to python.org and install the new python free so let's create this and so great your new project is ready and this is the initial thing and if you want to run this code that has just been given to you you can do run here and high pi charm is the output that you are getting is because this is printing high by charm so i'll not go into details what this program is doing so let's delete everything here we don't need this we'll write a blank program for ourselves so if you're new to python don't get overwhelmed with it python is very easy to learn i'll be giving you some syntaxes which are not difficult to grasp while we are making the code and i'll be giving you ways that you can learn python very easily by going to google and searching certain stuff so don't get overwhelmed by it the easiest way to learn a language to is to have something to develop for if you're learning language just learning the language it will be boring and difficult to do so this is my personal experience i have learned python just by making scripts from it over the past two years it takes a bit of time i agree but it's never too late to learn or never too late to start so you can go to google and you can just search web requests on python so if you remember that we were sending web requests https requests while using courseman so that's a web request so how do we send a web request through python and i have already clicked on a link here which is a blog and this is a great blog if you're starting out and it has everything that you need it tells you how you can write code in python it will explains you about rest apis also so let's look at this code and we are going to be using the same methods that we discussed in the last video that is get put post deletes and we are going to be doing this so let's just get straight into it and the first thing that we need to do is we need to install a module which is a request by default this doesn't come so let's go back to our pycharm and in pycharm what you can do is come to terminal here and just write the simple command pip 3 install request so currently uh for me uh i have already installed it so there was no installation done but uh if in case you have not installed a request you will get an installation progress bar and if in case you are returned with this error that your bip is out of date don't worry there's a command given so you can run python minus m pip install upgrade once installed the new pip will be installed paper is basically your package manager which manages the packages for python pythons have lots and lots of modules lots and lots of packages you can use tons of them and pip is something that can help you out in identifying those packages you can install very easily with it so you can use pip and the best way to find about a package is just go on google and search about the package and you will get to know so great and now we have install pip and if you go back to the blog we see that it says import a request so what we are doing is now we are importing that module in our program so the first thing that we want to see is that if we can get if we can execute a get request that we did last time around so to do that we can use response equity requests method is get here and you see that now there is a suggestion because we are using a function get inside request which we have just imported and it says that we need to enter url parameters data headers cookies files so what we need to do is we need to enter first our url so if you remember our url is https our router ib then what we need to do is we need to send uh authentication so how do we send authentication so if you remember postman we were using basic auth and basic auth package is available inside request package auth so you need to do is from requests dot auth import http basic auth so our complete url is our ip address of our router rest interface let's just get the list of interfaces that are there right now and let's use that package of authentication so we want to use http basic auth and the back what we need to do now is there's a suggestion that we need to enter the username and the password so great let's enter the username and the password for it so the username is api and the password that i'm using is my secret sauce one two three so great now we have sent our response and now what we are looking for is we are looking for a reply from our router so if you remember that reply from router was in json format so let's just do one thing right now let's just print the response function here of json so we remember our responses in json format so let's see what happens so don't worry if you get errors something that i'm gonna get don't worry about it so let's just see what happens so we just ran and what just happened so there was a bunch of errors that have come let's review them once by one by one and let's see how we can rectify them so the first problem is that there is an ssl verification error and this error if you remember when we were entering chrome also this error was coming so what is happening is the ssl is getting verified here also by python and it is not able to understand what certificate authority has issued this ssl so to overcome this what you need to enter in your request get is one more parameter that is verify is equals to false we don't want it to verify our ssl so let's just run once again and see what happens awesome so we have got actually a response back with all our interfaces but this is not a readable format to make it more readable let's show one more thing so we need to import another module which is json json is an inbuilt module so you don't have to worry about installing it through pycharm and this time around when we print what we need to do is use this module and the method that comes in it which is dumps and what we need to print is the object so if you see the hint that is given is object and then what we need to do is we need to indent it also so we'll use this and then comma we indent it for perfect so now our output is in a readable format that we can read and this is something that we can work on again if you do not understand how to do it just go to google just write a json response python beautification so how to beautify json in python i mean there is tons and tons of resources that you will find here uh there are many ways that this can be done and this is just one of the ways that we use we did a json dumps of the data this is our data so our data was passing in response.json variable and we just did an indent is equal to 4 awesome so now we can actually use this data and we can build upon it so if you want to search say how many ethernet ports are available or how many bridge ports are available so we can actually search this thing but first if you're worried about this error so it's the same error that this is an unverified https request adding a separate certificate verification is strongly advised you can visit this to ignore this so to ignore this what you can do is you can enter these two commands which is import url lib3 and in this what you can do is a url lib3 dot disable warnings so once you disable the warnings what is going to happen is anytime you send a request this warning will not show again so the warning has gone so don't worry don't be like how do i know this thing if you were to search if if you were to delete this and run it again all you need to do is insecure warning unverified https request just copy this thing go to google and just search for it and you will be given tons and tons of answers on it and you can just copy paste those answers there are certain things that you do not require in regular need and are available readily on the internet somebody else has also asked for it so you can copy that answer and you don't have to stress over it a lot great so let's now do something more on this so we have hardcoded our url our username and the password so let's just make a variable now and from that variable what we are going to do is we are going to fetch that variable every time we want to hit an api because every time the endpoint is going to be different sometimes we want to fetch interface sometime we want to fetch bridge sometimes we want to fetch ip addresses so let's make a string so https and single colon uh 103 our router ip and rest so this is our standard endpoint variable that we can use every time so we don't have to type the whole https router ipe slash rest because this is going to be common for everywhere and the username is also going to be a common thing so let's not use it everywhere and the third thing is going to be the password which is going to be common so let's not use that also everywhere perfect so now what do we do here so since this is variable we don't have to put it in inverted commas we can just write url but we are trying to hit the interface so we can concatenate this string with another string using plus sign and again then use single inverted commas and concatenate it with the rest of the variable that we want to insert and in this case is slash interface and in basic auth we'll change we'll remove this and we'll put user name and we'll remove the password from here and we'll enter the password variable here and let's run the script and see what happens now so great everything is still running everything is still awesome so great we are reaching somewhere but we do not want to print this so now we know that the response that we are getting is in json format but there is another thing to note here in python anything between the square brackets is known as a list and anything between this curly brackets is known as a dictionary so dictionary has a key and a value associated with that key so like rx packet here is the key and the value for that rx packet key is three zero nine zero two one five and what we really want to do here is we want to search all the keys which have say type as a bridge or which have type say ethernet so let's get into this list and dictionaries how can we fetch these values so to fetch these values to iterate through a list or what we need to use is a for loop so for loop is just a loop that will go on until a condition is met and once the condition fails it comes out of that loop so what we basically want to do is we want to iterate through this loop and what we want to fetch is the interface name so for interface in a response dot json what we want to do is print the interface so let's do one more thing let's use the print command here and in double inverted commas this is an interface and let's print one more thing here after the interface is done we just print a bunch of dashes so do note here that in for loop there is a gap left in front which is four spaces and in python this is probably the only syntax that you have to remember indentation you need to indent after loops after defining a function after defining a class and after if then else statements beyond this you do not need any special syntaxes so let's see what happens this time around when we print great so this is an interface and the first interface gets print printed and then there's a bunch of dashes then this is an interface punch of interface gets printed and then there is dashes so now we know that interface basically contains each of that list elements separately so now what we need to do is from that interface we need to fetch a key which is type right so let's print this interface with the key type and let's see what happens this time around so this time around what we got back was just these four things that is ether bridge bridge vlan that is our router has four interfaces one is ethernet two are bridges and one is a vlan awesome so what we now want to do is we want to print if the interface is a bridge we want to print its id and we want to print its name so let's put an if statement here so if interface the name what we are searching for is type is equal to a bridge then we what we want to do is print the interface dot id sorry we need to use the square brackets here and we need to put inverted commas also okay so id and what we need to print is the interface name great so let's see what happens this time around okay so we got two bridge interfaces and both of them got printed but we cannot decipher between the id and the interface so no worries let's do one more thing let's concatenate this one more and at least to make this more readable so let's say that this the id of the bridge interface is this id let's concatenate this with a plus sign and then after that let's leave a gap and then let's say the name of the interface is this again let's indent the name perfect now let's see what happens perfect so now we have the id and the bridge name printed on our screens perfect so what we are doing right now is the url is fixed the username is fixed and the password is fixed so let's do one thing this time around so let's get the ip address the username and the password from the user and let's fetch the data then so to do this uh what we need to do is let's create a variable ip address and let's say it's equal to input so with input what we're gonna get is a prompt and let's inform the user what we want them to enter so please enter the ip of the router from which you want the list of bridge interfaces perfect then we're gonna prompt them for username so let's say input and let's say please enter the username of this router great and in the last we're going to use the password so input please enter the password of this router great so now let's do one thing let's bring all of this one code here and let's cut it here and paste it here and what we're going to do is the url now becomes minus the ip address so url now becomes this this will become concatenated with ib address and the username we already have and the password we already have so we don't need this anymore if we have a url we have a response and we have our instance so let's run this and see what happens this time around so here let's enter our ip address let's enter our username and the password and let's see what happens awesome we got the ids of our bridge interfaces and the names so now we have made a small script with which we can get the ids and the names of all the bridge interfaces in any of our routers all we need to do is enter an ip address enter the username and the password and fetch the data so we don't have to login to each and every router now we can do it just by the script so this is pretty cool right so this is the way that you can build your own scripts and these scripts can do a specific task this is just the beginning of it so don't worry if you are wondering how am i going to do it it is simple you can come back to me you can send me a message in the comment section below and ask me anything if you want a specific script to be built we can all build it together we can use our github i'm posting a small script that i've made on the github the link is below you can just review it once if you don't understand anything please do come back to me do let me know in the comment section below what do you think about today's video and do let me know if you would like such videos where we do live scripting and we do specific scripting that you would want to understand and want to build together with me please do press like button if you liked this video and please do subscribe to my channel to get the latest updates until we see the next time goodbye

Channel: MKSIT

Views: 1,541

Keywords: mikrotik, network automation, rest api, postman, postman introduction, postman basics, mikrotik basics, networking technologies, ips networks, api development, api testing, mikrotik routeros v7, routeros, mikrotik rest api, mikrotik api, mikrotik automation, python, python rest api, python network automation, rest api automation testing, python project

Id: IvCeDAsO3hs

Length: 26min 14sec (1574 seconds)

Published: Thu Oct 07 2021

Reddit Comments

[u/amaneuensis]

This is pretty cool! Thanks!

I wish MikroTik had opted to adopt YANG configuration modeling instead of a proprietary REST API. Granted, the REST API is easier to use; on the other hand, it could have saved them, and me, some work by opting for the de-facto standard.

I guess I could write an adapter.

[u/Alx_xl]

Is there a way to create a policy to let the api just manage a specific section like DNS or Interfaces?

Because imo even with "no sensitive" the whole router is compromised if i give the api write access or am i seeing that to critical?
Anything changed with v7?