Automate Juniper Devices with Nornir and PyEZ | nornir_pyez | Junos DevOps

i was talking to john recently you know him as ipv0 and he was telling me knox you need to learn nor near you need to see where it fits into the picture of things where it fits into the big picture and how it actually works and the power that it can bring to the table and here's the thing i did a couple weeks ago i started getting into nornier and learning all the bits and pieces and now i've drank the kool-aid i'm all aboard with the power of nor near and how it greatly simplifies scripting when it comes to interacting with more devices now throughout this video we're going to talk more about where nor near fits in what does it really do and how does it really work i'm going to give you some resources where you can get started learning nor near too but one of the things that's cool about nor near is that it's very agnostic it doesn't care which platform you're working with or which transport you're working with as long as you have the ability to create your own transports or your own inventory or all of these other things you can just plug it into nor near and let nor near do the rest of the work for you so one of the things that i did recently was i took the juniper pie ez sdk the python library for working with juniper devices and i fit it to work within nor near so what we're going to cover in this video is how you can get started working with the juniper nor near pi easy library that i created let's go [Music] so if i'm describing nor near to someone basically what i would tell them is when it comes to learning nor near it's a little tough at first because there are three major moving parts to understand how it all fits together but once you've gotten through that initial learning curve then you're off to the races and it's smooth sailing what we should talk about first is what nor near really does at the end of the day let's go a couple weeks ago i launched my blog and as recently as last night i took to start talking about nor near if you scroll down just a hair there's a blog post right here called understanding nor near for network automation now here's the gist of nor near nor near is really broken down into three components not unlike the nvc design pattern and i kind of make the argument they call nor near a network automation framework and i i guess that's technically correct to me it feels more like a network automation design pattern which i think is really cool nor near does some things that you may be familiar with already the first one is it separates inventory management from everything else that you do this is exactly like ansible think about how ansible handles all of your inventory and the host variables in the group variables there's even default settings and environment variables and that's exactly how nor near handles inventory too the code the playbooks that you write for ansible are separate from the inventory there is a part where you say run it against these devices but that's kind of the point so the first thing that you should know about is inventory and i kind of write all of these right here on the blog you may want to read this too because it's going to go into a little bit more detail and set up comparisons that you really care about the second thing is you choose the transport mechanism yourself that plugs into nor near so you can use something like netmeco as your transport library you can use something like scraply or scraply netcomp you can use things like napalm or in my case what i did is i choose to use the pi ez which is the juniper sdk as a transport really what pi ez is over the hood is it just simplifies how we use nc client to interact with juniper network devices but it's very very powerful and very mature so the last piece that we should talk about is the actual code itself you write one function one time then the way it all works the way it all comes together is nor near will take your one function it'll apply it to all of the devices in your inventory or a subset of them if you want and it uses the transport mechanism that you specify so if i scroll down here here's an example of how nor near works within pi ez all we have to do is we initialize nor near with a config file the config file points to all of the inventory pieces that we have within our library or within our directory notice nowhere in here are we seeing anything about hosts or we're seeing ip addresses or usernames or passwords that's because this config file is initializing nor near and it says all of the inventory is located over there worth pointing out you don't have to use like yaml files you could also hook it in to other things like netbox to get all the inventory and secrets if you wanted to then beyond that we say we want to run this specific function in my case it's pi e z interface terse this is the exact same thing as show interfaces terse within a juniper environment or it'd be like show ip interface brief within a cisco environment and then it spits out the results so what really took place under the hood is this command right here the run method the run method says perform this block of code or this code that you've written against every device that's in your inventory and in this case it's using the pi easy library in order to do that that's how nor near works so the cool thing about it is it does all of this stuff concurrently it executes this code we only wrote the code one time and it executes the code across all the devices concurrently it gathers all the results and then spits the results for each device out or you can interact with those results programmatically it's very very cool again i highly encourage you to read through this blog or more importantly if you're ready to get started really working with nor near and understanding all of the different components and pieces that makes it work john mcgovern aka ipv0 has content on cbt nuggets dedicated to advanced network automation using nor near it will take you from zero to hero within cbt nuggets using the nor near framework so i highly encourage you to check that out so once you understand the point of nor near and where it fits in then we can talk about how to actually use it within the network automation context of a juniper device i have an srx300 here a physical srx300 firewall within my environment here it is let me just bring it up right here and you can see there it is i'm on the command line of this juniper firewall the cool thing about the pi easy library is that we can now interact with this device programmatically in fact that's kind of the entire point of juniper and junos the operating system i mean look if you're new to juniper look at this configuration tell me what does that look like to you huh looks like structured data it actually looks like json and that's because under the hood juniper devices are all performing programmatic interactions to each other every command that we type like show ospf interfaces if i pipe this to display xml rpc the command show ospf interface is actually issuing an xml api request the rpc under the hood that it's issuing is this rpc right here get ospf interface information feels a lot like netconf right that's because this is what eventually became netcomp juniper operating systems have been operating like this for a long time the other cool thing is if i just take off the rpc i can see the response in os in xml in the programmatic output now i don't have ospf running on this device just yet but here's the thing this is what's really going on under the hood is this just sending xml rpcs and receiving xml data back we can do the same thing with json we can do it with raw text that's kind of the entire point and now we can use pi ez along with nor near and hit all of these devices and issue all of these commands programmatically all simultaneously and then take programmatic actions against them so here's how i'm going to get started working with the nor near pi easy library and interacting with this srx300 firewall i've got my get bash pulled up here if i give it a docker ps we'll see i'm running netbox right now but i'm not running anything else so let's say let's just start a blank python container where we're going to do some development with nor near pi ez i'll say docker run detached interactive let's give it a name like pi easy tester we're going to run the python container and start off in bash the container starts up and now i can use vs code to attach to it so down here in the bottom left corner i'll say attach we'll choose attach to a running container and there's my pi easy tester so the container starts up and the first thing i'm going to do is fire up a new terminal and we're going to get some of the requirements installed here first thing i'm going to give it is a pip install nor near under so i got to spell it right nor near underscore pi e z press enter and there we go we see it's installing 0.0.5 that's the release as well as all the requirements in order to actually get pi easy up and running that includes nor near xml2 dict the nor near utilities and of course pi ez let's also install rich just because we like pretty output right let's go ahead and get that going now what i'm going to do is i'm going to make a directory real quick we'll call this maker let's call this pi easy tester and cd into pi easy tester and now i can add this folder right here pi easy tester and this is where i can start using this as my staging grounds first things first i'm going to create a new file that's going to be my config dot yaml file i'm going to create a new folder called hosts a new folder called groups and a new folder called defaults within defaults i'm going to create a new file called defaults dot yaml within groups i'm going to create a new file called groups.yaml and within hosts i'm going to create a new file called hosts.yaml last thing is in my root directory i'm going to create a new file and we'll just call this runner.pi or something like that main.pi whatever it doesn't matter we're just going to call this is going to be my main script where i perform my network automation tasks now it detects that this is a python library so let's go ahead and install all the things that it needs and at this point everything should be ready for me now what we have to do is we have to start putting the config in place the hosts in place the groups in place and the defaults and paste i'm just going to paste these in again i can't encourage you enough to go watch the content on cbc nuggets to learn how these yaml files work together to build the inventory pieces but you don't need to be bored to death watching me type in ip addresses and usernames or not i'm just going to paste these in now and then we're going to jump back to the runner.python script where we can see the pi ez library actually in action okay so fast forwarding a little bit we've got my config that's in place that points to my different yaml files i've got my hosts which identify the ip address groups which identify a basic username and password for a lab environment as well as things like the dns server and ntp server is arbitrary data and then i've got a default section that just kind of overrides all of it again i can't stress this enough go to cvg nuggets to understand the inheritance model and how all of these different live these different inventory devices and these different gamma files actually apply to your real script now it's time to actually put together a script where we can use the pi easy library in order to perform some operations the basic operation that we can do is the moment pi ez connects into one of our devices it actually retrieves facts about the device so we created one of the methods that we could use it's just high easy facts so that we can get the facts on our device i'm going to paste in my import statements right here and we'll save it real quick and you'll see exactly how this is going to work we're importing pi easy facts from the library that we just installed we're going to initialize nor near which is going to find all the inventory files based on what we specified in config one of the nor near utilities that we're going to have is it's going to print the result of all of the operations that you did against all of the different devices and then lastly we have the ability to do some pretty printing if we want to do that using rich so if i paste in this block of code here's how it's going to look i'm just dynamically finding what is the path that this file was being run in that way i can point it to exactly where the config file is when it initializes nor near it's going to find that config file and then when it finds that config file it'll know exactly where all of the inventory specific items are i'm just telling it run the specific task called pi easy facts that's where it goes and collects the facts and retrieves all of the results back now in this case i'm printing out the result of that task that's an important thing within nor near that you'll learn a lot about is that it's going to just print the result of that task for the device that we ran it against so here we go i'm just going to run this script real quick we'll say it's a python file and watch it take place also worth pointing out while it's running this see the print right here this is going to be a pretty print because we actually imported it from rich so it's not the traditional print it's going to be a nice print and there it is right there there's the output let's bring it on up and we see the full output of the fax from running the pi easy facts thing now the thing to keep in mind is in my inventory i only have one srx device what would have really happened is if i had a huge inventory it would have run this fax command against all the devices again i can't stress this enough look at this code there's no for loop there's no specifying that there's an inventory and we need to loop over each one of the inventory and run this command we just say use the pi ez fax and execute this command across the entire topology that's the entire point nor near is what's handling connecting into each one of those devices simultaneously using the pi easy library the big benefit here is you write your code one time you write one function and then nor near handles the rest now as a different thing that i'm just going to show off real quick let's do print results so you can see what the nor near result looks like and we'll say response if i run this python file one more time real quick you'll see how the output changes a little bit when we use nor near's built-in print result there we go so in this case we actually get some output that says we're running the fax and then we get this green output that says the fax command was successfully run and then it shows you for each one of the results or each one of the objects in your inventory what the results of that run were so now let's step up our skills a notch and do something a little more complicated like changing the configuration recall earlier that i didn't have ospf running here's how i would go about actually changing the ospf configuration using the pi ez config module up here on import i'm going to say pi easy config and what we'll do is we'll build a payload and i'll hit enter a few times and payload now this payload could come in a bunch of different forms it could come in the form of json it could come in the form of raw text just like this output looks like right here because it's not quite json right we don't see quotes and we don't see colons but we can actually use this exact structure or convention to bring a payload in using pi easy config so we've got json we've got that ascii text version of json we've got xml or we could even use set commands to push in data beyond that with the pi ez config method we could even use jinja templating so we could specify the path to a template and then the path to the data or we could retrieve the data from our host file or group file and it will combine those all together this is all part of the power of pi ez and the pi easy config method you have a bunch of different ways of changing the config so if i use xml here's what i'm going to do i'm going to start typing the xml here and what i'm going to do is i'm going to come back to it when it's done because this config is going to be you know kind of long i'm just going to use an xml subtree so it's going to start with the configuration then it's going to go protocols and ospf an area then the interface from the name and that's how it'll run ospf so this is where it's going to start to get fun you see now on the screen we actually have the payload that we're going to be sending in the protocols ospf area name interface we say which interface we want it to run on and i'm also giving it a priority of 200 hopefully this one becomes the designated router with a priority that high so in the past when we were running this here's what we did we told nor near to run one specific command but one of the cool things that we did with this particular nor near payload and especially when we're working with juniper devices we have a difference between the candidate data store and the actual running or committed data store so what we have to do here is we're going to be issuing this payload in to the candidate data store and i can tell it to either commit that data store right now that way this payload becomes active right away i can also use the pi easy diff function so that way we can see a diff of what's changed within our configuration and then once we've validated that that diff looks good we can tell it to commit so rather than just running the pi easy config just raw right out of nor near what we're going to do is we're actually going to build a function that runs a series of tasks like issuing the payload in checking the diff to make sure it looks good and then committing the candidate data store to the running data store so here's what we're going to do i'm going to say def to define a function we'll call this ospf config and it has to take a parameter of a task remember that the task is really what's being run against each one of these devices so when we define a function we're saying for each one of these devices we're going to be sending you a task to run so all of our functions take a task that way it runs this block of code against each one of our devices whenever nor near starts to run it we're going to be sending in the config so let's grab the response back here and save it in a variable called send response we'll say for our task run the pi ez config operation i'll explicitly say this is going to be the task name that we want to be running next i'll explicitly say my payload is going to be the variable that we specified above payload the pi ezconfig method itself the function itself takes a set of different variables these could be the payload this could be the template path this could be the template variables as well as what is the data serialization that we're actually sending in in my case we're sending in an xml payload rather than a set command payload or a json payload so we have to specify that too the default operation is that it just defaults to the raw text which was this format here this would be the text payload it's not json it's not xml it's just kind of the juniper specific text output so here the parameter that we'll specify now is going to be the data underscore format and it's going to be xml and we'll close my parentheses now this sends this config into the candidate data store it doesn't check for the diff and it doesn't actually perform the commit operation so for that we're going to import a couple more libraries but it's worth pointing out we could use commit now and set it to true and it would automatically perform the commit operation and quit out of the actual ssh or netcom session so i'll hit save just to format this out and we're going to import pi e z diff as well as pi ez commit so we'll say if we do get a response back we'll run another command we're going to run another task and we're going to save this as diff response we'll say task dot run and this time we don't have to specify anything special here so we can just say run pi easy diff what pi e z diff does is it checks the candidate data store versus the committed data store and returns the differential about what's changed then once we have a diff response if there is a diff response showing that we were able to successfully apply our configuration to the candidate data store we'll actually run the commit operation saving that in commit response so let's say task dot run and we'll use the pi easy commit operation now it's worth pointing out this performs commit check to validate that this will pass the configuration change before it actually performs the commit operation if commit check fails it will actually perform a roll back by default so we're not altering the data store in a bad way that way somebody logs in later goes what the heck's going on here we want to make sure that we clean up our mess if anything goes particularly wrong so now all i have to do now is in the nor near run command we're going to tell it run the ospf config function so nor near is going to pass in the task itself for each one of the devices in our inventory and then it runs all of these commands against each one of those devices in my case it's just an srx device we're going to use the nor near print result right now that way it prints each one of these tasks out individually and we can see the results of each one of these tasks so here goes nothing let's apply ospf to this srx device we'll start debugging there's the python file let me bring the terminal output up also worth pointing out this does take a while my srx device is really old it's got a crappy little data store like a little usb thumb drive that i think it's running on uh so the commit operation itself sometimes takes several minutes i've increased the commit timeout to be 300 seconds as opposed to the default 30 seconds sometimes you actually do run into an issue where the commit response comes back a little slow because you have a huge running config or something like that so the pi ez config default timeout for the commit operation is actually 300 second seconds as opposed to the 30s and there it is look at that as i was talking we got a successful deploy message first of all here's the configuration that it was sent in we see under pi easy config this was the config operation that was sent in using xml here's the pi easy diff look at that it shows we're adding all of these lines to the running config and there's all the pluses and then lastly when it goes to commit we've got a successful commit the moment of truth let's just get out of this let's actually watch us do show edit let's go into edit protocols and show and there it is right there we've now added ospf into the mix if i go to top exit and if i do show ospf interfaces there we go right there so using the pi easy library in conjunction with nor near i could now run this operation against every device in my inventory by only coding it one time that's the power so what i want to encourage you to do is if you really want to get into this and see all of the different functions and operations that are available within pi easy indoor near first of all check out github as well as the read the docs links make sure you have some fun exploring those read the docs links especially if you want to start using jinjo templating we cover how to do that in the docs as well so that's been getting started with the nor near pi ez plug-in connector thanks for stopping by y'all i'll see you in the next one

Channel: Data Knox

Views: 2,375

Rating: 4.9540229 out of 5

Keywords: juniper automation, juniper python, juniper nornir, juniper pyez documentation, juniper pyez install, juniper pyez day one, juniper pyez show commands, juniper pyez rpc, juniper pyez nornir, juniper nornir pyez, juniper jncia-devops, juniper jncie automation, juniper jncia automation, juniper ansible, juniper python on box, juniper python op script, jupyter python ide, juniper automation python, nornir python, nornir network automation, nornir network, nornir netconf

Id: Ql0ZAPY-fHo

Length: 23min 27sec (1407 seconds)

Published: Tue Nov 24 2020