Python for Network Engineers – A Practical Guide for Getting Started – Jeremy Schulman

so my name is Jeremy Shulman I will tell you a little bit about me but this deck is maybe six slides I'm going to spend some time showing you a demo a live demo and then I'll have some more slides after the demo and hopefully there'll be some time for Q&A I like to do interactive kind of stuff so if people do you know want to interrupt and have a question that's like burning in your mind that's cool but you can wait to the end that'd be great about me I am a software engineer that's how I identify as a software engineer but I've been in the networking industry for over 20 years always on the vendor side so I would say I'm a software engineer but I'm competent in in networking I could take any network set of instructions and actually understand and do things with them but I am by no means a network engineer I've never run a global network that's you know kind of its own art and science in my mind you guys are a special breed in that way in 2012 I had a very unique opportunity that opened my eyes to what now people are talking about is DevOps and DevOps for networking but I happen to be a Juniper at the time and now I'm back where I wrote the the puppet agent that ran on a switch right so I had to literally write a ruby stack for net comp because that's how Juno's talks and then I wrote a puppet agent with the help from puppet labs to do this and it really opened my eyes to what is now in everybody's head about how do we automate the network but this is back in 2012 when I when I saw this happen and so I spent a lot of time building libraries so if how many of you are juniper customers and how many of you are also Python programmers and how many of you are using Juno's PI easy yeah that I wrote that and so juniper like has been doing a great job keeping it up and supporting the community behind it so again quick hands again python programmers do you identify as developers or you just you believe you know I write Python code but I'm a network engineer it's okay okay that's cool I mean so two years ago I was in Mountains view where we did a DevOps for networking summit and there was 120 people and only 5 people raised their hands at that point in time so it's actually really nice to see a lot more people getting interested in this space I've had other roles I do a lot of stuff in the open-source I'll share these slides you can google my name you'll find stuff I'm I like to be pretty active my new role at juniper is actually being a mentor to new college graduates so I've come back to Juniper to teach you know these young kids coming out of college how did you network automation or build networking tools in a way that I think is a fun and interactive way with customers so I work in the field I'm not part of the business unit I work in the field so any of you guys that are juniper customers and you want to talk with me more about that I want to take my new kids on field trips that'd be cool all right but this deck is my own deck actually I just joined Juniper like a like literally last Monday so anything I say isn't Juniper stuff this is just just me all right so there's been a lot of talk in the industry about how network engineers need to become Python developers and I think that's crap what I believe is that network engineers need to competently use Python to achieve a singular goal which is to increase the love the reliability in your network operations that's it right if you're if you're saying well why should I automate my network or what am I supposed to be doing what does automation mean to me this is a very simple way to answer that question are you doing something that increases the reliability in your network operations if the answer is is no which it probably isn't you're probably as to what degree that answer is so always think about it in this way we don't do automation for the network just for the sake of it there's a purpose there's a business reason you're going to do it and the reason isn't because you're your management thinks you want to sleep at night or have a personal life or you're going to work less they know they they they want a reliable Network because their business runs on that that makes sense to you guys all right good all right so in this agenda I'm going to really talk about what I believe that network engineers should should go on this journey about how to use Python because if nobody really kind of gives you a path to follow and guardrails along it you're not really sure what to do where to get started what makes sense what makes sense for you so I'm going to share what I think are good insights and you know we can have a lively discussion about it and then the point of the demo is I want to introduce something called Jupiter notebooks has anybody heard of Jupiter notebooks I'm really impressed how many of you are using them to actively do tasks to automate your network suite this is great all right you can leave or you can you can be that guy in the crowd that'd be great all right so I'm going to introduce you to this technology that and if I say Jupiter or juniper networks or Juniper notebooks itself it's a slip of my tongue this has nothing to do with juniper Networks is the technology that came out of the science and engineering disciplines as I think as a replacement to Mathematica anybody knows what Mathematica is and then we're going to do a live demo and this demo is how can we take lldp data that we've collected from the network and dynamically draw a network diagram because nobody likes to draw the network diagram so why don't we just take data off the network and just automatically draw it I'm going to show you how to do this and this is something you can go home and do yourself that is what I believe all right who plays cards against humanity' alright great you're gonna like this game then all right so managing networks awesome in theory kind of a mess in practice right manual via the CLI you know we know this this is terrible we want to get away from it why because it's not reliable because humans are not reliable right SNMP based products is a terrible mess for reasons that that our technology I mean they had a good reason but they're just really hard to work with and so nobody really wants to use them but it's kind of like what most products are today but trying to integrate SNMP based products into your environment I believe you know and anybody who's in the yang you know model camp I I will debate this with them heavily I'm not sure how the yang thing is going to be much different than this thing because enterprise based mids and native based yang models to me are kind of the same thing so maybe they'll get it right this time I don't know vendor products we can always rip on vendor products right because they've been a dumpster fire of reliability I mean you can you can get some piece of software from a vendor but it's always been terrible because they generally care about selling you a box but nowadays the their their motivation to sell software is actually much higher so I'm hoping that vendor products and their quality and the reliability will go up and historically it's been a mess and Sdn I don't I still don't know what what people want to use this term for so we're just going to kind of not use it anymore and then people will talk about orchestration systems are gonna save the world and you know maybe but ok then there's DevOps so let's make DevOps for networking and the people who are in DevOps are already having a problem to define what DevOps means to them because the marketing people took over and it's terrible ok then then there's everybody wants to talk about networking on a mission but nobody knows what it is or maybe and then this is obviously my favorite there are people who will get up on stage and tell you that all network engineers need to be software engineers and this is an epic fail and I will argue with them over you know any amount of scotch until they they submit so this is the part where I want to talk to you where doesn't how does a network engineer begin and where should they actually stop now this is taken from a research paper from the 1980s by some Berkeley professors on the topic of how does anybody learn a new complex skill so I didn't make this up there's a white paper on this and and these are the steps that they define with specificity what does it mean to be a novice what does it mean to be competent what does it mean to be proficient somebody Chris just said in the last thing well if you're proficient Python programmer he used the word proficient but I'm going to use these words very specifically and I'll use it in an analogy because this is the best way I can explain it how many of you cook right great ok so I'm not allowed to cook for reasons that I won't talk about but I'm allowed to chop and prepare right so I'm allowed to dice an onion right or Shifa nod you know dinosaur kale but those things those those actions are what a novice learns like how to dice an onion how to hold a knife that's a novice skill because what you're learning is skills out of context this is somebody says dice an onion you know what to do and you can be pretty good at it when you're competent that means you can follow a recipe so I get plated I love plated plated it's great because I get a recipe I get all the ingredients and I can actually participate by chopping onions and doing those things now I can get better or worse at being competent in following a recipe right and recipes could be easy and recipes could be hard and if I want to make chicken pot pie the recipe for chicken pot pie I could probably do but for Chateaubriand probably not so much right okay so how does this relate to network engineering if you're a network engineer if you can follow a mop a method and procedure a set of instruction then you can do it reliably you can competently follow those instructions right but when you cross the chasm into being proficient this is the person who can create the recipe like I if you gave me a bunch of food items and said hey make chicken pot pie and I had no recipe to follow you would not be eating chicken pot pie all right so there's this giant chasm that you have to cross to become proficient in something so if you're a network engineer and and when you're competent you're a network engineer you've been a network engineer for probably three or four years you're competent but when you cross the chasm into becoming a senior network engineer or an architect that's when you can from a blank sheet of paper create a network design that adheres to some business objective from a blank sheet of paper that's your skill you're proficient in that but it takes a lot of experience to understand the why you do something to create that architecture or to create that recipe or to write that piece of code now what I've learned or my message here to network engineers are just getting started is you should stop at being competent right unless you want to be a software developer because once you cross that chasm into Perficient you're not a network engineer anymore you are a software developer that has domain knowledge and networking so some people are starting to use the word network reliability engineer to mean this thing but if you ask Google what an SRE is a site reliability engineer it is a software engineer who build infrastructure tooling right so if some of you want to go on this journey you know this is again another five to seven-year journey to become proficient right I'm a software engineer of twenty years right it took me seven or eight years to be proficient to write from a blank sheet of paper and create something like you know spy easy that would be useful to a broader set of people it's this hard stuff and then just I won't even talk about becoming an expert or master because it's off topic but it requires a massive amount of time and energy ok how does this relate to networking right so networking to me I see networking as a complex distributed system you have all these boxes written by software from different companies over decades of time that are magically going to work together right and in this in this environment network engineers basically do two things they config stuff and then they do a bunch of show commands to make sure that whatever they configure the network is working they show stuff so I I have this chart where I talk about configuration management and situational awareness and then I break it down by things that are only done on a device in isolation I call that host only so if you're configure if you're configuring something only on the host like changing the SNMP community strings there's a little risk in that kind of action but if you do something that affects the distributed system like changing routes and brought prefixes that can take down the network that that's a high risk thing right same thing with with the situational awareness if I'm doing a command on a singular device for example you know show IP route to sea routes that's that's something you can grok and understand but if you are looking at the entire network as a holistic system to understand is this service operationally correct you know you're going from different boxes to different boxes running different commands and correlating data in your mind in order to understand is it correct this is incredibly complex incredibly complex companies are built around this kind of idea to do all of these things so when I talk to network engineers about well what kind of tasks should they even go after start with these very simple situational awareness tasks really kind of for two reasons one it's if it's a read only option if you're only doing read only activity the risk is low and if you're trying to do these really complex tasks then my advice is to you know buy products that do these complex tasks and learn to automate their API learn to communicate with their API so maybe it's OpenStack or maybe it's you know some product that's on the market there's a lot of really great software products that are coming on the market and they all have api's so but I would never encourage a network engineer to try to build a complex distributed network monitoring system I've seen many companies try and you know it's great they tried but then they failed and it wasn't good so I like to think of this activity as you know your network is a crime scene and as a network engineer you're the detective and the idea is is build yourself these capabilities to get the information off your network in a reliable way that allows you to analyze it yourself if you don't have these complex products you are that complex product like everything is in your head and you're the detective taking all this information and you're trying to synthesize it in a way that you can say it's not the network's fault or it is that make sense to you guys all right so now I'm gonna talk about Jupiter notebooks because everybody says we have an API learn Python it's like saying how to draw an owl first you draw two circles and then you draw the rest of the owl right it's just it doesn't work that way and one of the things that I learned very early on with the the Juno spicy stuff was I would always demonstrate it in what it's called the Python interpreter which is like a COI you know you can type in some stuff and you see some stuff and you type in some stuff and you can see some stuff and and people saw what I was doing and they said hey it kind of feels like a COI this is pretty cool I'm not writing a program Jupiter notebooks takes this to the next level and what Jupiter notebooks are I'll skip this this is like how you install it it takes five minutes it runs on your laptop and you get a web page and you can launch them stuff I'm gonna do this in the demo but what did what it presents is an interface that allows you to type in text or pictures or snippets of code and you can run them piece by piece and if it blows up you can go back and fix it and you can try it again and it has built-in help and it has tab completion just like a CLI so what I learned with networking with network engineers is they like a CLI experience so this gives you a CLI like experience and it allows you to do something that you're doing today which is take any method and procedure document that you already have written down and you can literally transcribe it into Jupiter notebooks and then automate only the little pieces that you want to automate maybe other pieces you don't because you don't have to automate all the things you only want to automate the things that are going to increase reliability you know because you don't want to do repetitive tasks that make sense all right so we're gonna do a demo this is the demo part so there's a tool called Jupiter and you say notebook and I'm gonna run it in the background okay so what this did is I'll make this bigger so what this did is it popped up this this window and I'm going to click on my notebooks and a notebook is simply just a text file you can check it in you conversion control it it's if you look at it in a raw editor it just looks like JSON but when you bring it up this looks pretty good when you bring it up what you'll have you know you know it'll produce a table of contents for you and it will show you essentially the content of your jupiter notebook and so if i double click in a cell this is just markup like so i can just type in any kind of text like you can see here and i can hit shift enter and and that will execute that cell so this picture that you see here this is basically just markup saying you know load this PNG file that i captured okay so if you're looking at python code what you'll see is some of these cells allow you to enter python code and you can execute it okay and when you execute it you'll see a little number here so this is a Jupiter notebook that I have already run and the purpose I'll tell you the purpose of the story and we'll get into it let's say that you're working at a zoo and this is your network I've just made up fictitious animal name boxes with fictitious ports and you've dropped you've drawn this the Visio diagram or whatever now let's say that you've captured in a CSV file you know the the LLDPE relationships and and what you want to do is you want to take that data and you want this picture to be automatically drawn for you right wouldn't it be great I mean the last time I did this presentation they asked me to make this demo so I'm assuming that this would be right and the way you can look at this peak picture and read it and think and this is kind of goes back to the data modeling thing we heard in the last session think about it this way dog port 3 is connected to cat port 2 from this picture so the blue boxes represent the devices the brown boxes represent the interfaces and the little circle represents the link in between that's obviously a piece of meta this is part of a meta model meaning there isn't really a physical link other than like say the cable and let's say that the CSV file looks like this right very simple looking file now the idea here is I could literally click a button and run this notebook and at the end of it I expect to see a picture so while I'm going through this you can see that there's a little bit of code in here and I mean this is not going to be a code walk through I'm just showing you there's real code here and you can mix in you know running some code and seeing some output and having somebody explain the process so if you have a mop if you have a network engineering mop it says ok add a VLAN to this interface and that's what they're supposed to do and then there's a little piece of code that actually remotely logged in maybe it's using yang you know to change the interface tat and you you run it and says ok that was done now you go to the Dement the next thing right and maybe you say ok we'll show the status of interfaces and then there's a little piece of code to do that you can step through this process so there's some text and techniques of Python in here that are very handy like creating a two-way map but as a result of this you know we can see what the code is doing I can see that I created this little table that has my cabling information that says well what is 8 port 1 connected to and says well a port 1 is connected to bear port to you so I'm I'm looking at the data structures of my program as I'm going along to make sure that what I'm writing this it's actually correct and at the end I'm running a thing to say make me a diagram and in this case I mean again I'm using this technology from the science community called meta plot lib which they use to do all other signs pictures and graphs and there's another piece of software called network X which is a graph and analyst and analytics and computation tool and and and what I'm using it for is to do that layout automatically for me I'm saying here's a graph of data and I want you to use a layout called spring layout to make the little things you know spread out automatically cuz I don't want to lay it out and so the rest of this is just loading data into the graph and at the end there says draw and here this is basically saying well I want certain shapes and sizes to look a specific way you know maybe I want the interfaces to be tan and I want the host to be blue and at the end this will execute and produce hopefully this picture now let's do the demo part because I explain what this is doing and so what I can do is I can reset the output I can say restart and clear output so imagine that you know you have a new data file and now you want to rerun this notebook so what you can see here is that these numbers are blank this means that it has not been executed and if I scroll all the way to the bottom we can see that there is no picture okay now if I say sell run all this is going to run all of the cells from top to bottom and you can see it's kind of executing through this at the end you can see I got a graph right and you know if I wanted to play with the colors I could play as the colors if I wanted to change the layout because right now I'm using something called spectral layout and this this network X tool has something called spring layout and if I wanted to see what spring way out would look like I just changed one the line of code Irie execute that cell and now I can see what the spring layout looks like because maybe I was just curious I wanted to see what that looked like and maybe I want to tweak the numbers to see and move things around this is just an example to show you the interactive nature of this tool Jupiter notebooks right so my thesis is that if you're a network engineer and you have mops that you could you could take any mop and convert it into some form of execution to increase reliability into the process do you guys think I have this hold water with you do you guys think that this is something that people could do that you could do even if you're not a Python programmer like if somebody gave you a recipe you could you could run it and if you already have that recipe if somebody taught you how to convert that recipe into a Jupiter notebook would you want to do that show hands you want to do that you think this would make your network more reliable right not just because you think it's fun I mean it's fun but that's me alright anybody who wants to see this in more detail see me afterwards I'll be around but this is the end of the demo unless you guys had any specific questions about Python code I'm happy to like talk Python into your ears fall off but now I'm gonna go back to the rest of the slides yes yeah it was actually developed for this purpose because scientists collaborate on their data and their experience experiments so this this is actually the reason why it was developed now there are million little plugins that are built for Jupiter notebooks that have pop-up dialogues and create dashboards and all these slider visually there's so much work that has been done because data scientists and and and you know that engineering discipline they're not Python programmers but they simply want to harness the language to achieve a result so we don't have to reinvent any of the stuff there they're literally continuously building this technology all right so that's Jupiter notebooks I think it's pretty cool I did the live demo all right where do we go from here so let's say that you're just not you haven't even started playing with Python you know you want to what I would consider novice topics are on this list now I'll talk about how you should learn them in a minute but you can see that this list is you know not small but not terribly large you can become fairly good at being a novice Python you know become novice ly aware of Python within a week and you're all really really smart people I believe you can do that thing I listed the things that I think are important for the use cases that I've built over time with with network engineers these are all the libraries that you're going to need to learn you'll get these slides guys now every single one of these libraries is its own thing you need to learn how to do this has anybody used napalm for example like napalm great I could I could write a Jupiter notebook that uses napalm to like touch a bunch of device all right but there's all these libraries these are the libraries that I use the most when I'm doing you know Network automation tasks and you can see that there's a lot of them here and the more you use them in the context of a recipe the more proficient you will get or the more competent you will be at using them until you can understand why you would want to use a bi-directional you know hash dictionary yak-yak-yak so if you're getting started this is what I would suggest is next steps take a formal Python class like from Global Knowledge or one of these online places you can do it in a week it's a fixed cost you can get tested you can get certified make your company pay for it I mean if they want you to do this thing they should pay for this training do not do do it by yourself you not because you can't but I mean I'm speaking for myself I need I need the discipline and motivation to attend a class and be you know get it done otherwise I get distracted and then I won't do it there are some really good Python news newsletters everything's free the two that I really really like is from a guy named Dan Bader and and then Python weekly comm I read these things every day these news feeds every day I'm a Python developer this is my world download a real IDE editor do not write Python code in notebook or VI just please just just don't do it just please you know get a real editor it's free you can play with you know PyCharm you can get it for free they they have a commercial or a professional version it's it's not expensive but trust me to start there now when you're starting to build competency like once you have your core your core novice skills and you're trying to build competency you have to use Python every day just like you're on the COI every day just start moving your your your rhythm of businesses well can I do this in a Jupiter notebook can I run this command kind of thing I'm used to pa'dar notebooks and then do take industry classes there's a lot of really smart people really focused on this problem space Yvan Pepin Yakut IP space offers a lot of really good classes Network to code offers a lot of good classes and Kirk Byers he runs both a free and a paid version these are all really smart people they're super passionate about this the space they they'll offer you knowledge and various tools and technologies like ansible and salt and whatnot and then a book just came out that is highly recommended mat Oswald and Jason Edelman and Scott whoa really smart guys poured a lot of time and energy in it support the community by their books I'm not I can't say that but I just did I think it's a great book actually and it really helped out so with that one of my good friends who was also just back at Juniper Derek winkworth when I had him review this this is what he told me about network engineers he said network engineers are proficient at quickly becoming competent and I get that like if you're a master routes which person and you need to know a little bit of load balancing or firewall you can pick it up really really fast so I believe that you guys can be competent in Python at a really good clip but you know be patient set realistic goals just remember how long it took you to become you know a senior network engineer I mean it's all about setting realistic goals and and having a community to work with and and get feedback and help from so that's my my shtick this is who I am you're welcome to ask me anything I have no idea how much time I have or if I went over I'm sorry but what I got one minute Wow all right I'm happy to answer any questions or I'll be around for a while any any questions is everybody gonna go home and try Jupiter notebook say yes yes alright good I will help you I will help you alright thank you that's my show

Channel: CHI-NOG

Views: 7,710

Keywords: chinog, python, netdev, jupiter demo

Id: 6INPb9EAgBI

Length: 29min 17sec (1757 seconds)

Published: Sat Jul 07 2018