How do I get started with Network Automation? | Interview with Principal Engineer Eric Chou

[Music] uh me do oh [Music] because i need you here [Music] me [Music] because i need [Music] me [Music] oh [Music] i need you here [Music] what's good youtube welcome to the lab every day youtube channel the podcast today we're talking one of my favorite subjects with one of the people that has inspired me he's an author let me see if we can get that good an author a principal engineer for at networks he's the host of network automation nerds podcast he's a mentor he's worked for some very large companies doing some very large networking and to top it off he also does security so for those of you that are paying for network automation you can stay for security because he can provide that information as well we're here with ed cho what's up eric hey how you doing duwan uh and hello everybody who's watching the streaming i appreciate you being here i think i've mentioned it in a lot of other channels or appearances before i think the the biggest gift that you could give people nowadays is your attention and so i appreciate you know you guys staying here and of course if you're watching the recording afterwards that's just as good so thank you guys and glad to be here hey man i'm glad you're here um you know we've been trying to do this for a while yeah yeah so so for people who's not aware like we've been trying to schedule this since march i look back on my calendar like the first time i said hey let's talk to duan and his community it was like march and just one thing after another but but here we are man we're good yes so man uh i'm gonna start first let me do some shout outs to everybody's tune in let's do that first so we can level set yep for those of you that watch the live stream you know how i love to start these out i like to see where you are coming from to do some shout outs so put your area code in the chat so i could shout you all out let me see where everybody's coming from and well white 404 229 916-819-869-214-809 what's good john what's up cisco 216. ohio in the building 242 bahamas 828 dallas texas that's what's up 225 from the 601. saudi arabia look people here from all over the world eric to get this knowledge you're about to drop on them nice thank you thank you guys so i i've just pulled out the page now you know i'm i'm a noob not like a veteran like zuhan so so i just pulled out just now but thank you guys i'll uh keep an eye out on the comments as well for sure for sure all right eric so you've been in it all of your contact information in the description of the video so if you all don't know who eric is you'll find out from this interview but let's kind of start from how you guys started at nit and then how you kind of got started in automation what led you to where you are today yeah um i got my start in it when i was five my my dad got me a computer and i saw my first game in you know when i was six no just kidding no none of that was true i i i listened to a lot of podcasts and i listened to a lot of you know read a lot of bible one and i'm most jealous of people who could say that right like got it made they just like okay my dad worked for nasa and you know he taught me our first line of praying and i sold for so like i wish i could say that but it's not true i actually did not start at nit until way later in college um in fact i'll tell you a funny story later right but um but it's it kind of illustrate how ignorant i was as far as i t and and hardware and computers are concerned i was scared of it basically but but i got started when i was in college i was a finance major but there was this little thing called the internet was about to take off and uh so i was like oh you know i got to get on that train you know i already spend all these time in um in uh finance i don't want to waste it but you know i still want to get it so i got a minor in uh i think it was uh cis so computer information systems so it's not quite like coding or programming but it's computer related and i think that was um and you could have two tracks one of them is graphic we're using flash and dreamweaver so this is my you know okay boomer moment so you could what's dreamweaver that's okay you're excused um but um it's it's basically there's this tool to do html and uh a little bit of css so anyway so that's how i got started i i was a cis minor and i was doing a lot of internships which i would highly encourage people to do if you haven't tried out or give that idea do some internship to try out things that you like or not um i did a bunch of internship as much as i could and uh toward the last quarter of college um i was interning at a local isp it's a tier 2 isb unfortunately they're not around anymore it was part of the casualties for bomb you know this whole thing that blow up but um but i was an intern over there i was you know the first one in the office i was in the tech support i was there to open the office and but it fit in my schedule right like it's um uh you know you wake up early and you you work your eight hours you go to class and you maybe go to more classes and or you know whatever and um and so i was intern over there they offered me a full-time position after i graduated and i took that position and uh so as an isp you do a lot of system and administration it was also they were also a var so they were a cisco reseller so then i started taking uh certification courses and um you know just kind of went that path so for a lot of people they always wonder whether you know um that i started out with coding or how many classes that i take uh the answer is none right like i took one java class in college and i i i didn't flunk uh but i i didn't do good that was probably one of the only seas i had in college and um i was scared of it from there on and i went the cisco certification route and eventually got my ie back in 2008 and of course right like i cling the hands and go that's it like i'm not done right no not really you know even though you're ccie but i was working for amazon at the time and they were not big in certification or anything like that the only reason i eventually pursued that was because that was a goal of mine but i knew even back in 2006 you know two years before i got my ie that this was not going to be that propeller that advanced me in the career um so yeah that was that was kind of in a nutshell that's how i got started in it was internship and then kind of just went through the cisco routes and then self-study for the most part for coding automation and all of that and so people probably uh known me for um you know kind of working for public cloud providers and so let me go back a little bit so after the isp i went to work for time warner cable which was my first exposure for a large you know service provider environment and that was really eye-opening after time warner i went to work for cisco lenses as a product engineer working for cable modems and so on then went to juniper as attack again so you see this theme where i keep on going to a support role and then um and then i uh our family my wife and i at the time moved to seattle i didn't i didn't really have a job so it wasn't like i deliberately looked out for this this new thing there was no cloud computing amazon was very much a retailer at that time so i just kind of stumbled upon it of course you know 2006 was when um one year before they launched ec2 so s3 was there but ect wasn't so the service really took off after ec2 was uh was launched and of course the rest is history and then so uh so i worked for in during that hockey stick growth kind of period was very exciting was also another time to get exposure to data center networking went to work for microsoft azure also in the early stage and um you know and then uh and then 8-10 uh joint a10 with my good friend we started this new team that's what that's a security bit that duan uh mentioned about was we're very focused on uh ddos mitigation so shout out to my a10 friends as well so my first question is you know i didn't mention that you had a ccie you don't talk about it a lot why is that is it it i'll literally i'll let you guys see it for the most part because it expired [Laughter] so so i got in 2008 and then i was i went for the written exam for ie voice so now it's collaboration and it was you know voice and um call manager that sort of stuff but i was doing a lot of voiceover ip at work but it was mostly sip based so it was really just unrelated and you know after a while you kind of evaluate your return on investment for your time and that just didn't turn out to be a great investment and so as much as i hated um i had to just pick and choose and so eventually i came renew and i did not renew it um put on probation and you know it was just kind of yeah it wasn't i'm not i'm not doubting people's path and you know it's all it's good that they're they're focused on ie focus on certification de for example but for me i'm just saying for me personally it wasn't bringing me a lot of value at the time especially with employers and you know in the multi-vendor environment and they were actually driving a lot of the the changes the white papers and so on with vendors but not necessarily cisco so it just wasn't bringing me enough value for the time investment i need to put in and in case you were wondering coding is pretty hard so i had to spend a lot of time on it you know so you know if i compare the two um and i'm glad i did i chose python over uh you know in python it's everything else associated with it besides besides furthering my cisco certification path okay i think this is a great way to kick off the first question i have for you as a new network engineer or as an existing network engineer you learn networking you build your foundation you get your ccna right which path would you recommend going learn python learn network automation or continue on to a higher level like networking network concert or something yeah i'll be i'll be lying to tell you if i could tell you that answer right only you could tell that answer yourself i would say take a hard look at the big wins or the priorities of the company if the company is a cisco var and you actually get material benefit from getting more certificate certification for example when i was working for the var they actually get x amount of discounts just by having multiple you know certified people on the team and that would automatically get you a lot of resources that you need and a lot of support that you need for continuing your certification path however if you were in environment such as i was that they don't you know i'll tell you the the the benefit the bonus the quote-unquote bonus i got for getting my ie number was a hundred dollar you know certificate and i bought a watch for it so it's like you know it just wasn't valued um in that environment right like we were doing a lot of stuff for sure but it wasn't anything that the iu would cover i mean it was still nice it was kind of the this hat on your feather to to put in but it just you know yeah you gotta you gotta i think for me back to your question you should probably answer that yourself but one way to answer that question is really to uh take a close look at where the company is emphasize right now and if you still want to work for that company you know go go with that flow because it's so hard to swim against the current if let's just say for a while i told you i worked for juniper right even with working for juniper i was you know still studying for ccie and that's twice as hard i mean honestly it's like you're dealing with juno's command or at the time nest screen commands and at night you're going back to cli and you're spending twice amount of time for half as much work for half as much gain so i think the right question is really to to take a look at your current work environment and decide for yourself and if it doesn't really match um what you're going after i'll tell you one of the reason i left microsoft was understandably so because of microsoft where they are their code base is in c sharp they're at the time oh i'm getting some feedback uh do one is my voice still okay yeah it's still good it's like popping in and out could be the connection but you're good okay i'll tell you we got a problem we're good okay um okay do you have do you have your sound on on youtube since you got the chat pulled up turn the sound off there okay let me do this yeah i've closed all of my uh windows but anyways we'll we'll continue as long as you could hear me okay yeah we're good okay i'm i'm having a lot of statics so uh hopefully voice could still make through but but i'll tell you you know going back to what we're talking about so one of the reason i left microsoft was because i i see my path going forward as mostly python related and network automation related and but i was asked to do a lot of c-sharp work at work and therefore you know i as much as i like microsoft as much as i like the team i was in i left the company and kind of pursue that goal of my of furthering my python path and continuing what i feel more passionate about we got a question from sam jones around your ccie what field did you get your cci here i'm sorry so you were you were asking what track i got my ccie in correct correct okay so it was routing and switching so at the time there was no like i think it's now enterprise networking or whatnot uh but for me it was uh routing and switching okay cool cool all right can you hear me are we good no i'm having a hard time so do you mind if i drop off and sign back in yeah that's fine it's cool okay all right let me uh let me end that all right let me see and we're still here stand by let's see yeah what's that here stand by just a moment to air comes back thank you everyone for tuning in if you all got questions i would love to hear them in the chat what certifications are you all working on and if anyone is working on network automation let me know i would love to hear that so we can kind of talk about it if you're working on python if you're using ansible that'd be great to hear from you all deep work oh yeah that's an excellent book if you have not read it you should looks like eric is joining us again it was let's see um vassal genuinely said i'm a ccnp certified but i have a question i got to read in python i'm facing an issue to make logic you know what i have some great book song logic um i got some great books on logic hold on let me see actually i got it here so two books um this all the first one is grocking algorithms this book here it's an excellent book to learn logic you know how to understand how to write algorithms how to sort bubble sort all the different types of sorting that are out there and it really helps you think outside the box the next one is taking your python to the next level with fluent python this is this book right here it opened up my eyes so much so both of these books along with when it comes to network automation you know this is a great place to start with eric's book so both of those or all three of those books are like extremely extremely helpful and have been helpful for me hey john can you still hear me okay yes hey you did it can you hear me all right yeah i can hear you oh my god that was uh that was like somebody constantly playing this buzzing sound in my ear oh good man all good it happens especially when we're live thanks everybody for staying on yeah i appreciate it yeah sorry about that so um so back where we're talking about so um i think you would choose the your own path and sometimes as a heart of a decision to leave a great company like microsoft as it was that's what i chose to do i can't say i never thought about the implication or what would have happened if i did make choices otherwise but but still you know i think i don't think we could spend a lot of time looking back and so those were those were kind of the um i guess uh points i try to make is just you know go for go for whatever you feel is passionate about and you know at the end the thoughts will probably connect that's really good to know because you don't hear too many people talking about letting like their certifications expire and with you being in the cloud in the early infancy stages you had the vision you know of course you were working at juniper and still you know working on your ccie but once you got in microsoft and amazon you saw you know what let me focus on where i'm at and not focus on this other other things that are distractions you know what i mean yeah it was a hard choice really um it was really hard because you know you spent so much time i spent probably five or six years and multiple attempts at ie right but to let that go it was just kind of hard i think if i was able like nowadays if i was able to renew that without like another written test for for example if i could just take some i guess you could renew that with education units nowadays um if i do that then um [Music] let me uh i think my video froze up so let me start again so if i could do that then i probably would have renewed it um but it is what it was uh it was what it was so i i did not now since you were you know with amazon and uh michael azure did you pursue any cloud certs once you got into those spaces yeah you know there were none there were there was no certification offer um at the time um at least the first so i was there from 2006 2010 for amazon and then so to be technically correct i was when i would when i joined microsoft i was part of the infrastructure team so it's called the global networking service and global networking service in itself divided into a different some different teams that aligns with different uh they call properties so my team was aligned with being and uh being at the time had the team that does this is all i think public information that they were doing autopilot so autopilot is really the automation team that drives bing's infrastructure that gets eventually merged into the the azure infrastructure and so on so right now there's only one infrastructure team at microsoft but as you know with all big companies you know there's a lot of different teams different focus everybody's really kind of focused on their own bits and so to be technically correct you know i wasn't uh joining azure per se because there was still you know kind of moving parts around that but the second time around when i left microsoft and rejoined microsoft later on i did join azure proper and in the sre team man i admire that no okay so man i i don't want to leave like your career because there's so much like to start in this foundational area of how you got your start now with aws from my understanding they use a lot of custom tools and even hardware um where you still have any work on routers and switches by cisco why you with aws yeah you know i think that's a um that's an interesting question so i think it's something that they try to keep close to their chess aws especially but we could talk about azure so azure obviously is very open about their sonic project and that was kind of their um it really depends on the property so if you were looking at microsoft it's really just one company but underneath there's eight or seven or eight different i forgot the track but seven or eight different companies that achieve revenue of one billion dollar or more or some some sort of run rate that will get them that so you got xbox windows office 365 azure of course and all of these properties um sort of have their own unique requirements at least to begin with and they all eventually move to azure but um but you know a different rate and a different adaptation progress so i would say if you're talking about azure um it's always been working with vendors but also drive their own they give their requirements to vendors and you know you leveraging their purchase power to kind of um implement those unique features in them and for other properties who are uh maybe who are not adopting azure at the same rate they're still very much traditional traditional networking involved where you're configuring routers you're configuring switches um and doing a lot of operations and that's where you know it's kind of interesting on the the cross section of yeah we know network automation is on the is on the rise and this is something we have to do but at the same time you know we're looking at azure for their adaptation of you know just moving to the cloud and we have all these fuzzy middle of you know what do we do with network automation okay you've been doing network automation for let's say what 14 15 years is that it's safe to say no actually so so i would probably say the network automation journey for me really started um maybe around 2016 2015 range i mean so it's it's kind of a gradual process um so maybe around 2010 is when you know have the exposure and kind of decide to go in but um because of my involvement with the python community a lot of people think you know automation will ask me if automation equates to python of course not right like so automation is so much more than just python it's about programmability you can have gui tools that make you do automation it's about apis about all the ecosystem that surrounds it so i think even in 2010 because there was so little resources on network automation there was so little information floating around unless um unlike today that it was a lot of period of time it was just for me to uh kind of finding out what the resources are kind of finding out my own path trying out different tools and so in my last stream where i i did a talk about like what are the tools for getting started with python and network automation i i kind of call myself the the dummy for um trying out different tools and that was very true it's like if you've seen those um car dummies to try like crash dummies then that that's how that's who i was it's like i was trying different tools and see if they would do the trick and so for example you know when i was trying to um create network apis like a front-end network api i try pyramid i try bottle i look at django i look at flask and all of these stuff were took time and took trial and error a lot of documentation reading and just my own exploration so i you know back to my book what i tried to do was i aggregate all of this experience and just you know tell you uh try to tell you something that i've done so you don't have to repeat my mistakes so for example in chapter nine i talk about using flask i talked about the reasons of using flask and that was from you know previous experience of trying out different stuff and really didn't fit the bill for network engineer you know bottle was too simple django was uh you know have a heavy learning curve for network engineers so flask was kind of right and pyramid uh you know maybe was not as mainstream as uh python or flash and stuff like that so that took a long time and that took a lot of effort to to just get it right so i would say you know i kind of took detours here and there um especially when i was at uh you know at teams where they they may use different tools like or different languages like c sharp or uh or whatnot so those were all all you know helped me in a way but um they're just kind of the detours um detours like i took i mean as you know uh duan so nowadays the career path is not just a straight line right it's like almost like a monkey bar so you know you go here for a little bit then you come here and then you go up a little and then you you you hop to another tree or or what not so um so you know honestly that's how my path was like it was just uh mashing together and trying to find my own way and uh i read a lot i listen to podcasts a lot and try to gain insights such as this great community you know this is probably one if this committee existed i probably would be like tuning in every week and just like be fully engaged you know man this community is awesome you know you're a huge part of that and you know i think right now with where we are everyone's excited because you can see the change and the opportunity to be a part of the change is like greater than ever you know the companies are actively looking for engineers that know python that know automation and understand this stuff you know what are your thoughts on that yeah i would say it's very exciting to to be in this time um uh you know i mean i'm not trying to promote my book too much i'm just gonna reference you know like so in the intro no it's okay i'm not trying to sell books you know but um in the intro i i i quote dickens and say it's the best of times it's the worst of times so i truly believe it's really one of the best times to be in network engineering right now because in any time of change you're on the in the same playing field right you're competing with people who were um maybe double ccies and uh but you're we're all on the same level in playing field trying to learn python trying to learn network automation and so that creates a lot of opportunities and one of the hardest things to predict um in technology in life in general is you can't tell which which you know technology is going to win right um if you could tell you spend you know 100 of your money 100 of your time learning that but the problem is you don't really know but now it's kind of clear that uh network automation is probably i don't know at least in my opinion it's probably one of the technologies gonna stick around it's not a fad um and python for me really worked out well for you know it just became i think it was the month of october that it overtook 20 years of dominance by c and java to become the most popular language so all of these indications you know you know objective indicators indicates that these two technologies are here for the state so you're free to invest a lot of time it's not going to be time wasted and it's going to be effort well well you know well spent in my opinion anyways so it is exciting and it's more exciting that we have a clear path forward so you just have to be able to you know manage your time manage your effort and learn those skill sets now when we talk about a clear path we of course start with aries book because it provides no i the reason why i say that is your book is going to teach someone python is going to introduce someone to ansible is going to introduce you to flash like you said it's going to introduce you to snmp tools it's going to introduce you to interacting with um azure and aws it's going to teach you all these different pieces along with ci cd your book covers a wide range of infrastructure automation you know what i mean so yeah i appreciate that yeah yeah yeah now the question i have let's say someone struggles with learning python what advice do you have for that person that um is looking to get into learning python or just learn automation is python right place for them or should they go somewhere like ansible i think um i think the advice i would give is just try try them out you always want to just try them out and see which one click so for me i struggled for years to learn pro when i joined some of the companies back then the pro was the dominant language for scripting for system management and for network automation or network management back then right there was really no network automation but but i struggle and it doesn't really fit the way i think so if you look back at pro it promotes multiple ways of doing the same thing and so everybody is like hey look look how beautiful this code is i did it in two lines but it's very complex right like you know maybe only the author would understand it i mean i'm biased but um but when you look at python it you know if you look at the zen of python it promotes one best way of doing things right so that's true different philosophy it doesn't mean one is better than the other it just means python works with my brain better so coming back to network automation and your question is ansible the right way to go is python the right way to go or is go laying the like all of these you could just try them out the beautiful thing with nowadays there's so many tools at your disposal and it doesn't really cost you anything to try them out except your time so try them out to give it an honest shot and try it out to a point where you feel comfortable in saying okay i understand the basics of it and and then make your decision on whether to go further with that you don't have to spend you know you don't have to go out and get a degree you don't have to you know uh spend all of your time doing it but give it an honest shot and try it to a point where you feel comfortable in doing one or two tasks and make your decision back then for me it's python um it's it's ansible as well but not too not so much i think i would probably for me probably use ansible to get started but once you get started you get comfortable um it is written in python so you could go ahead and learn python write your own module maybe or you know just fully switch to python it's up to you at that point but try different things and give it a shot see if you like it or not when it comes to excuse me like your book and or not even just your book but learning python and learning network automation how important is linux it's for me it's it's it's crucial because um in production i know it's not true for the enterprise world but for the last i don't know 14 years or 15 years psych every production environment i've been to uses linux um yeah so so that's and especially nowadays you look at these open uh network uh network operating system um like your sonic for example or your nokia's sr linux so all of these operating system is gearing toward i mean already operations of guarantor and some flavors or linux so to me it seems like it has one and um you know so for me i think that's kind of the fundamentals you don't have to be a linux expert i certainly am not but you know i'm comfortable in the environment and it's not something i have to think about when i log on to you know i try to try out some new project and then say okay run this and ubuntu run this in uh you know red hat or not that i i would i need to go and look at the references or whatnot so i think that's probably a good benchmark for yourself is just to be familiar with it you don't have to be able to you know contribute to kernels or you know recompile linux kernel you just have to be comfortable operating in that environment yes totally agree with that i think not just for network automation but for network engineering troubleshooting i've always had better tools and been able to determine the problem faster by utilizing linux it's just so many built-in tools inside of linux that just made my job a lot easier that was my thoughts yeah and though and the best part about it is they're for most part they're free free as in free beer and a lot of more times they're free as in freedom so free and beer are just free to use right um and there may be some licensing around it but free as in freedom is you know you could go and trace all the way back to the source code and if you spend if you really want to you can actually change the way the tool works so i i don't know i don't i don't i can't i can't recall the last time i saw a linux tool that is neither like you actually have to pay a lot of money to use it or um to con to contribute to it um so so i think those are really good things to keep in mind that you're you're there's really practically no ceiling in these tools that you could use at least in my opinion right right i got a question from mark kaye he says eric along the same lines of why you would choose python how do you approach selecting the right tool for each situation that's a great question um i'll give you a a similar thing that the path that i went down for the last two months or so so we were looking for a solution to uh to stream data so um you know and not in the same way where we need to put into a database retrieve it and put in a database that just becomes a lot of overhead so we were really trying to trying to find a better tool than what we had or it's basically a brand new tool um i just went out and searched i just went out and see what other people are doing we're in a great community where we there's very little chance that you're the first person who encountered this problem so i went out and i went to see which tool the best of them use so in uh in for example i will go and see what how how does netflix deal with this problem how does uber deal with this problem i know i'm not uber i know i'm not netflix but uh but i want to see how they deal with it so i know where the ceiling is and um and same thing with network automation right so if you're interested in network automation you go out and see how how does how does facebook do it and they're very open about their their uh architecture uh their um the technology they choose why they create their own os so even though you're not facebook you're not google but it's always good to see what they're doing so that was the first step you go out and see what other people are what other people are doing and what kind of success they have and then second is once you know that you go out and try them if you could chances are these companies are using some sort of open source tools some sort of uh you know tools that are that has a lot of documentation so in my case you know many of them uses this tool called kafka so i went ahead and uh you know tried it out get some uh get all the resources i could whether that's books courses and uh lap it up try it out and then you know kind of do a prototype on what to use so if i overlay that with network automation so that means okay so you know maybe they're maybe these companies are using a particular library that you could try it out maybe these companies or some of these companies are using ansible they publish their documentation that you could try it out and then overlay that with your own situation prototype it um and then you know you probably by this stage you have a good idea of what tools to use so that's kind of this parallel path where you know it doesn't exactly align but i think the steps and the concepts works well in both situations you just go and do your research uh prototype it as soon as possible and then drive up your your uh your conclusion and kind of repeat the rinse and repeat on these three three processes that's great advice but i have a question about that um in network engineering particularly operations right we don't really have time often to research problems like in software development agile you know we have spikes and that's built in where you can take out time to just research our um a problem how does a network engineer that has a heavy workload find the time to research a solution you got any advice on that yeah i think that's a that's a real problem for us i think we're so operational and so interrupt driven that it's always that you're on this uh you're running in this this this wheel that you just keep on running this treadmill it feels like you're running really really hard but you're going nowhere so you gotta i think in my opinion you gotta break out that loop so either a lot of us have chosen to learn things uh in off hours and pick a problem to solve so there's one way to do it that i've seen a lot of people have success to it and i myself did it at various points in time is i spend some time off hours to learn the technology that i think are relevant and go back to the day job and solve one problem for them to illustrate uh that it's a valuable technology that it's it's applicable and it brings real value to your day job and once you demonstrate that then you go ahead and say well you know if i were able to automate this task then i'm if i do more of this then i may be able to gain more ground and you so you break out the cycle of always interrupt driven just by a little bit and you start this new cycle of a positive feedback where you know the more contribution you make the more confidence they have in you the more resource they give you and more the more time you could spend on it so i think you the the point really is to break out the cycle however you want to do it whether it's you you front load and put up a lot some efforts on your part or uh if you're lucky enough to have a manager who could support you in just in just taking friday off like one friday of a month to just you know pursue those goals um then you could break out your cycle that way but but i think it's a big problem and hopefully by providing these resources like what do one is doing like a lot of case studies that you know maybe devnet is putting out or maybe some other people have talked about that you could convince somebody um if you're just you know overwhelmed you convince somebody that this is a good path to go down to and willing to give you a shot um so if you so i think one of the the most successful way people could do that is really finding one problem to solve in your day job um if you just say hey mr manager can i go learn python or network automation chances are you're not going to be successful but if you're saying hey i read this case study that by leveraging python i could automate our backup strategy for our nightly you know configuration backup so we could have a more you know everybody could sleep a little better and uh you know have more confidence in our collaboration drift and so on um can i have one friday out of a month to explore this and even if you don't find the solution right away you can report back your progress and just so that everybody is aware the progress that you're making um like i said you know we experiment so not all experiment worked out but at least you document it so the next guy could say oh you know eric already tried a b and c he haven't tried dnf so i'm gonna give those shots so you're still bringing company value and you're still bringing your team value in fact your teammate will probably treat you better because you just saved them five hours of digging through something that wouldn't work so i think i think that's what i've seen people be successful in doing is just to somehow break up that loop whether you're a uh good talker like the one that you could convince your manager or you you just you know up up front load your some of your work um solve real problems and start another positive loop um as opposed to you know just always be interrupt driven i i hear you i think it's a real problem and it's not a perfect answer but i hope it helps no it really does help um i think like if we take a step back from what you just said one of the things we have to think about is oftentimes especially when you're younger in your career you're in operations and you're working on a job to where you know you may be supporting the edge you may be supporting the data center you may be supporting whatever in your network and then when you go home you're working on the certification you know what i mean that's not that's oftentimes not in line with what you do in your day job so i think one of the things that you're that you're saying that's important is to talk about work on the things that you are actually doing because it's going to make it's going to add value to your organization and you can actually add value to your resume to talk about the things that you've learned and that you understand deeply yeah absolutely i would agree 100 100 and you put it so much better than i did [Laughter] not at all not at all i think you know i learned a lot from you know watching you reading you listen to you speak and then others in the industry as well and then being in my own um environment to where i'm overloaded boiled down and i don't have the type of time and it's like ah okay i really want to get this ccna security but i'm not even working on firewalls so what can i do right now that's going to make my job easier and it's going to add value to my resume you know yeah absolutely so i know back in a lot of the companies i have they actually have these open door meetings where everybody would meet uh mostly leadership and management but the doors open for everybody to actually participate in and listen in on what are the priorities of your company or what are the priorities of your team to participate in and what i've seen is i mean i try to make to these meetings as much as possible but i rarely see junior engineers attend these meetings um it's probably because they're overworked it's probably because they they may not even realize these meetings exist but you know i would go out when i was when i was serving as a mentor for younger engineers i would always drag them along and say even if you don't have any inputs or whether if you just participate you wouldn't know the decision-making process of the leadership why they picked this project over over another and think about the value proposition that they're doing and then how you could fit into that value chain and then break those value chain down into specific items that you contribute to i am positive by doing that your leadership your management would appreciate you um for that for example you know if they're picking one project over another because of cost set because of cost optimization you know they need to cut x amount of cost per quarter then you're looking back and say okay i manage the network um where can i fit in into that value proposition is it okay if i try to prolong the features or or the lifelong life cycles of these hardware maybe i could by doing something such as automation or by doing something such as um you know uh finding out bugs a lot sooner or uh you know maybe i could write a script to um automatically you know uh troll automatically crawl through the weekly publishing you know uh vulnerability for example and match that to our device or i could write a script to actually you know upgrade our device os so that we're uh we're in a safe place then um that will prolong the life cycle of these devices so on so forth right so that's just one example but like you said it's you know you have to bring bring that value back into your day job that would keep you so much happier it will motivate you and it will um keep everybody around you to support you better and that in itself just feeds it back into that positive loop again yeah that's one of the things i miss about being in the office because i'm a sponge and i'm nosy so we're in the office and there's a white board and there's a network designer i've never seen i'm like hey what's this or somebody's talking about something that's going on or there's a meeting that i i i wasn't invited to but i feel like i should know what's going on i may walk in but it you know being remote it's hard to have that you know i would say that fly on the wall where you kind of learn just by being around people what advice do you have for engineers that are working remote and trying to learn and get up to speed with let's say network automation or even improve as a network engineer yeah that's the that's the hardest part for being remote i would um agree as well it's especially for junior engineers or who's somebody who's just started out it's really hard to learn by osmosis at this point in time um so i would probably advise to just be more like you said you know for like a better term be nosy right like you you actively seek out the advice of other people you actively participate in the meetings that you see are happening virtually or you know hopefully you know one day in person but you just kind of have to go out and you have to be proactive at this point um just because you know everybody's working remotely and that is a common theme that i've seen is that for senior engineers it's like they're perfectly fine we're working remotely and they're perfectly okay spending you know not having to commute to the office and and be like right here by the time you get into the office but for junior engineers it's especially hard um but i think the good news right now is that everybody understand the situation and um the the tools have have vastly improved whether you're talking about zoom or webex or some of the other tools communication tools they have vastly improved that allows you to be more active nothing beats in person but it these tools will allow you to at least get closer to that aspect of it and um and i'll give you an example so when i was attending um pi cascade which is a python conference for the pacific north northwest region um they have a social night in a virtual social night and usually those are just kind of so-so like you're really not gonna get much right like you know how do you share a beer virtually but um but they actually have um proximity proximity voicing so for example i would be represented by a dot and they would have this virtual you know auditorium and as i move my dot closer to this group of people talking or closer this people are talking i will actually get that same voice proximity voice of feeling like i'm walking toward this group and um and participate in this group of conversation versus this group of conversation and that is amazing right like i don't know if you've experienced it it works and that's amazing and made that night so much more fun and so much more productive um so so i think more tools will come out like that that help you to uh to to get a better experience virtually but in the meantime just be more active just seek out advices for the more senior people and join community like this one um you know those are those would kind of bridge you over until we could all meet in person again that's some great advice thanks sarah thank you um kind of shifting gears i got a question from ryan newton okay this question is where does terraform fit in in your opinion you know i'm not very familiar with terraform this is something that i want to learn more about as far as i could tell because i have friends from you know even as far back as the time warner cable days that they're uh you know devops sre type of engineers so as far as i could tell terraform is great at launching cloud infrastructure so it's an alternative to whatever proprietary tools that like cloud formation for aws or the equivalent in azure so they bundle that together so they provide an abstraction layer for you launch the infrastructure but once you launch those infrastructure you still have to use other forms of automation tools to like ansible or your python script or you know sdks of various kind to actually operate those services that's that's my very basic you know kindergarten level understanding of terraform but you know it's something that i would like to learn more about awesome you mentioned kafta yes you're speaking at devnet create this tuesday right yeah that's right that's right you want to kind of talk about your talk for create just a quick yeah sure sure so so kafka is in a sentence is what um what confluent would describe as set data in motion so basically of course if you have so typically this is what happens you have a service and you build some sort of service that offers value to people so whether that's a website that's a portal that is a monitoring tool you'll build that and if it's it has a front end like for example it has a website that for people to log in to look at the beautiful graphs that you created or click on the click on the buttons to to execute some kind of task and pretty soon people will be like okay it's great that i could click on it and look at it but how about if i have some programmatic access so you put on the api so other people or computers could you know access it and then pretty soon these other teams will have to will want to access it so now you have multiple people calling your apis and at the same time people will say well your service is great but what if right fill in the blank what if you could do x what if you could do y what if you could do z so you're here thinking to yourself okay i could but there's no point in building these processes these x y and z already existed i just have to make additional api calls to other people's back-end to do it so now you have this full match of people calling your api viewers viewing your tool and you're pulling api from various locations and imagine that being you know multi in this micro services world imagine that being tens fold 20s for hundreds fold and now you become this whole mess of you know you're always trying to negotiate the message format you're trying to you know document your apis and all that so what kafka does is actually to offload all of that for your services so you have this caftac cluster and you have a bunch of producers and you have a bunch of consumers so in this case the these messages are broken down by topics so you have producers produced to a particular topic just once and then all the consumers that needs this information could subscribe to this topic and then in turn they could publish their information to a separate topic and you as a consumer could subscribe to that topic so now you're you're offloading all of that um and now it's more scalable your whole infrastructure is more maintainable so you offload all of that messy into kafka of course this is not magic this is something that you have to go built and you have to go learn but luckily you know there's this project kafka solve the for a lot of people for example you um there's a webpage so if you go to my talk um you'll see a link for um powered by kafka i believe and that is that will list everybody who have used kafka and that list just grows down so much and you can see how uber um talked about their use case of uh real-time matching uh it's uber eats actually so real-time matching between restaurants and how busy they are and how many drivers they have and how much demand they have at any given moment and they have to display those ads and so on so you know not only is this being used as a scalable solution it's also real-time redundant um and uh you know there's a lot of a lot of uh little things regarding like for example if you're published like if you're you know offering some kind of transaction um credit card process transaction is one of your topic then you have to also only process it once right you cannot process your time that you don't get the money you cannot process twice um so that you know your customer would not be very happy about it so you only process once but it's complicated to do that operation if you're just only offering an api and bunch of people calling your api so how do you how do you decide that this transaction as an api gets only processed once right it's very complicated if you think about it however if you offload it to kafka and you just have to set the setting to say you know process once only and bam there it is so so it really offers a lot of different scenario use cases i would highly encourage you to look at um you know use cases by netflix use cases by ubereats and use cases by airbnb so these are complicated scenarios where they offer two-way marketplace which is the most complicated business case you could have if you're just selling a service you're only dealing with consumers if you're doing only selling b2b um or you know something that's a lot on simpler side but if in a two-way marketplace like uber or uh airbnb you're actually dealing with consumers as well as producers so you know it becomes pretty complicated so if you read those cases that you know the ceiling that you have and that's what convinced me of you know kafka is the missing link for a lot of people and in network engineering that means you know you could have multiple devices i mean the true true story right we will have in some of the previous working locations we will have one device and we'll have 10 boxes or more that's pulling smp on this box because they belong to different teams right team a wants that information so does team b but teammate doesn't want to talk to team b and therefore the easiest thing for them to is just to query your box another time so you just kind of add to that imagine that as opposed to say okay i understand everybody wants this information but why don't i just put into a kafka topic and everybody subscribe to that topic as opposed to you know you you're hammering down the box and by the way it's snp is a very expensive process you can actually crash a box if you interrupt it too much right because the control plane actually had to stop and respond and then grab it from the data plane whatever data that you're asking and um and you you know and if you do it frequently enough often enough you can actually crash the box it's like debug ip any anywhere debug ipo i was going to ask you how does this apply to network engineers and that's a great example um so if i'm understanding correctly this is real-time data this isn't like data that's off like an assist law server or something this is real-time data that you can subscribe to in query api to get that real-time data that's right that's right so kafka itself doesn't care your message format in fact it's probably the biggest complaints about kafka is that there's no um there's no message validation kafka doesn't like say say this value is supposed to be integer kafka doesn't care if you send a string or you know if you send a boolean it just happily takes that and it wasn't we won't get caught until your consumer take that data and say oh no this is junk and spit it back um so so that's part of the that's part of the you know i guess features of calf guys that it doesn't really care but kafka because it's being used so often and so widely adapted there's also an ecosystem that spawns around it so there's one you know uh registry that you could register with and say it does the value validation for you and so on but that's you know kind of over the out of scope for this discussion but you're right so kafka is just this cluster that takes care of uh taking your message in make sure it's you know uh redundant you know so you can actually specify your replication factor for your topic so that it's always saying uh i always have an extra copy so what if this server goes down i could still have another copy of the data and so on and so forth so yeah so it's this cluster of things that allows you to store data in not but it's different than database so it doesn't store it permanently usually you have a setting in there and say set this data for it's by default seven days so um you know so you store this data temporarily so let's just say for example your producer keeps on producing message but your consumer for some reason crashed and died so as long as these consumers come back you know within seven days before losing these data you know and go back to the kafka cluster and say hey i want these data back um and kafka will actually be smart enough to say okay the last offset meaning the last message that was probably that was properly processed was this so let me go ahead and send you the rest of them so those are you know kind of difficult problem to solve if you're just using it through an api uh but kafka actually takes care of you so i understand that this is uh very application driven and very kind of developer focused um but these are actual you know actually problems that when you maybe maybe in day two of your automation journey that you you will face so if you're not there yet if you're just trying to get started with python trying to get started with automation just keep this tool in mind um so when you do face that problem later on then maybe you know some some you know light bulb would trigger and say oh you know i actually remember you know duan and eric talked about kafka so maybe i would i would take a look but this is definitely not something that i encounter in day one but it's it's a it's for me it's a missing link for a lot of the projects that i'm working on nah i i see i can think of a ton of value in that especially when you reference the snmp because in my last organization man we had like i don't even know how many different teams that had their own monitoring tool that was configured to gather information from our devices and it never made sense to me because for some reason in networking no tool does everything you need right i don't know why that is i mean the best at the time for me that i've seen was probably solar winds but even solar winds had its limitations you know what i mean yeah yeah definitely not not to mention they're very very publicly exposed uh vulnerabilities so no you're right so it's a uh yeah it's it's not great when everybody is trying to hammer down your networking devices um yeah so it's a it's a possible solution for you and i hope you know my talk would just kind of open up that door for you kind of introduce this possibility but you ultimately host a host a key to whether this is the thing that would solve your problem or not here's a question if you're if you're in an organization where you're the only person doing network automation right how do you how do you go about testing your code before it goes into production what is the process for that so hopefully you have a lab right if you don't have a lab then that's a problem um and nowadays i would tell you it's it's a very different than when i first started so when i first started there's no um there's really no virtualized option so i would go on to ebay i'll buy my lab gear i would test it out on a much smaller scale and you know i don't know if this is still true or not but sometimes you could find um say on launch sites or sometimes you could find a smaller market that you could kind of uh fire walden off you know firewall not in a network engineering sense but like in a business case where you can actually uh you know treat it as a as a as a lab um so hopefully you have that sort of information but nowadays you have virtualized options that i've seen people launching ginormous amount of topology using virtualized hardware hardware plus software that they could test out very close to production environment and um you know i've been in workplaces where we will when we do a prover concept would use the virtualized platform but when we need to you know burn in some of these you know some of these problems would not reveal themselves until the device is under load so we will have a relatively big a relatively close resemblance of production um as a as a lab and then we just burn it in uh for weeks and then we'll just hammer it with uh traffic we'll hammer it with using tcp replay maybe to replay this traffic so hopefully you have that sort of environment to test your automation so you don't you know you don't test in production which is right okay no i haven't but i i could just find the name of it i kind of guess you can see what it does yeah so we would use wind killer to put send that data across yeah so you kind of stress test that circuit and that's a very smart way to do it um i remember you know like a poor man's uh wayne killer i guess would be your sweeping right so yours launcher sweeping so you're a you know a spectrum of packet sizes and over through you know repeated number of times so when you turn up that circuit you could see that traffic ramp up and uh and so on hopefully you know it would just work without errors and and all that but but yeah wind killer is so much better it's a great tool now um continuing on with this this testing the next question i have before i ask you about pipelines is when you were writing your python code for production did you unit test your code did you write unit tests like let's say for instance you're using net micro if you're using rest con for something and you don't want to you don't have a lab and you don't want to send queries to your devices over and over sure so you should mock that mock that query how often did you do that or did you ever do that you know not as often as i should i i have to be honest um it's always a battle between uh putting in new features versus tests and um so officially i would say yes i encourage you to write as much i encourage you to have 100 coverage of your code whether that is you know unit test or maybe that's integration testing for other services or end-to-end testing if you could but i'll be the first to admit that i don't write as much tests as i could um yeah and like you said even when you're mocking it it's not perfect right like the message needs to be exact but in real in real world the message is always uh messy so you know you could have a coverage that's 100 percent that you would do you know coverage you know the coverage stop high and then you show us hundred 100 doesn't mean that you've tested the whole scenario so it's always kind of this you know struggle to have 100 coverage but as a goal we should always strive toward you know testing our goal testing our code testing our topology and having end-to-end reachability and scenario yeah yeah yeah i think what really helped me learn python was writing tests when i started writing unit testing then i realized you know what this code doesn't do what i think it does yeah and once you're oh my god it's kind of a rabbit hole too right i'm sure you could identify as well is once you start saying okay assert equal uh this value versus this value or a certain not equal this value versus this value and then you start thinking okay just because i got a 200 okay doesn't mean it's great right so now i have to check the message type now i have to change the message format and what if it's uh giving me 200 you know but like 201 but it should just be 202 right so like you go down this rabbit hole of you know how many tests can you have you could you could really uh you know you could really go down that rabbit hole and just wait i don't want to say wasted but you could spend days and days just writing tests and at the same time you have very little to show for it right like imagine you go to your manager and say i just spend the last week writing tests look at all these beautiful coverage that i have and that's great but you know these are the features that would actually you know make people pay money to get or that will actually make your job easier so it's always a a you know balancing act for people to to talk about you know writing tests or to actually you know implementing features so i think that's that's why you see a lot of times when a new person comes on board the first thing they do is writing tests it's a great way to learn as you alluded to but the senior people needs to allocate their time on uh you know more value-added like features or roadmap and that sort of stuff i don't know if that's true but you know that that's my my observation anyways yeah my from from my last pipeline it was all about the passing test and the failing test what happened when it fails if you can show both of those then you basically got coverage you know if you got an else statement make sure that you test the el statement what happens when that occurs in your code and you can test that you know outside of that if there is some nuances throw the test in there but you know like you said don't get too deep in the weeds on it because it can happen that's a great point yeah that's a great point so if you so the point really just to have a stopping point right like don't don't uh don't just go all the way down without having uh thought about you know where should i really stop and come back come back guys come back right right right hey do you use um function annotations in your code in your python code no no um not not often i guess yeah i started using those um i think it's really cool because like if you add them into your code and let's say you import that library in vs code it'll tell you what the function like the parameters should be like yeah yeah yeah yeah like type hinting yeah so like type hinting or um like your return type or something like that uh yeah definitely i think i should but because this is a relatively new feature um you know i don't i don't do it as much as i could as much as i should so so i'll give you an example right like um i guess i'm someone who learned how to do things once and just keep on using that way right so when i first started when i learning when i was learning django i i learned about function based views so basically you define a function and you know whatever html template you return but you know there's a newer thing that could save you a lot more time uh it's called class based views so it has multiple inheritance and you have all these other cool new features i i shouldn't say new feature because it's been around for a while but because i initially learned function-based views that i'm struggling to learn class-based views so i think that's my own blind spot is just i need to go out and and actively try to if there's a better way to do it actively practice and uh improve my my code so thai painting is certainly one of them i think with i appreciate you being 100 you know transparent on that because you being a personal engineer oftentimes you know they think you get this high level you got to figure it out and you don't have struggles you know what i mean so i'm sure somebody caught that and is inspired oh no like oh my god like like the title doesn't really mean anything oh hey we have a visitor so people joining our stream um but um no i i don't think title means anything except that um you spend longer time doing this so um no definitely i'll be the first to admit that i don't i don't have everything figured out and i'll be the first to admit something that i don't understand doesn't mean i'm not willing to spend the time to learn doesn't mean i'm not willing to spend the time to invest in it but right now i'm just not familiar with terraform for example or function annotation or i don't know a lot of other things i don't know about but you know i think we should all be transparent and honest about it because uh it's just so tiring to uh to pretend like i certainly don't have the energy to do that nowadays man i i don't i don't either you know there's so much i don't know like i've never had to work on the cloud so i don't know nothing about the cloud you know if you got questions about the cloud i point point you into the directions of one of my colleagues or somebody that i know that does but you know um i think warren buffett said um he says something about not being a genius he's smart in spots so he stays in those spots [Laughter] that worked out pretty well for him right yeah yeah just he did all right yeah he did all right yeah that's cool man yeah i could talk to you all day man i love talking to you um likewise i think the next time you know we have you one it'll be great to bring john i saw john in the chat john capobianco shout out to james yeah hey shout out john sorry i had some technical problems i actually closed my youtube uh commentary but it's it's it's great to see you um there and definitely um one of one of the my new found friend good stuff good stuff um i got a couple super chat shout out to rasta rit thank you he said super late but appreciate the content thank you family um keon johnson thank you appreciate it if you got a question drop it in the chat kenya he says all for the cause awesome guys thank you and let's see yeah everybody's saying thank you this has been a great interview and i mean you dropped so much knowledge and i want to do this again man yeah i appreciate it and like i said i said in the beginning of the hour but i truly mean it that nowadays the biggest gift you could give people is your attention and your time so this is great uh and i appreciate everybody stopping by and don't be hesitate to you know reach out connect i love geeking out i i don't think i can respond to every single message and i certainly cannot um you know if it's out of scope or something like that but i i want to say i appreciate all of you thank you yes thank you we thank everyone for tuning in eric you want to shout out i guess your platforms anything you want to shout out and then close this out another message yeah thank you for giving me the opportunity to do that i mean i procrastinated four years for my podcast if you notice the network animation nerds podcast it was initially launched in 2017 and then just you know life happened and all that but i recently rediscovered you know trying to find my own voice just like duane did uh and still doing that uh trying to launch my podcast i'm really excited about some of these guests that i'm bringing on so duane is one of them for sure but there's also people who have you know co-invented vxlan uh we have entrepreneurs who are successful in the network automation industry so i'm very excited for these guests and that the only thing i'm scared of is i don't do them justice so i'll try not to be that i'll try to you know get as much information and help everybody to peak their brains and peel back those onions so we can all benefit so now we're on the mission nerds podcast um i try to do more streaming with you know the likes of uh all these influencers but also hosted on my own platform for all these podcasts i would uh try to if they agree if the guests agree i'll record them so sometimes they could pull up you know if it's a project they're working on i could pull up the screen and have a better visual and so on so also you know stay tuned for the youtube channel so those two are really the content uh things that i'm working on and also you know working on more content as far as writing so keep tuned for that so thank you thank you all for um for supporting uh and uh thank you for thank you juan for giving me the platform i'll get off my soapbox now thank you bye

Channel: Du’An Lightfoot

Views: 3,525

Keywords: duan lightfoot, du'an lightfoot, labeveryday, cisco, ccna, python, network automation, python course, python walk through, Mastering Network Automation | Interview with Principal Engineer Eric Chou, a10 networks, a10, eric chou, Mastering Python Networking, How do I get started with Network Automation? | Interview with Principal Engineer Eric Chou

Id: xFhFM1DLjJQ

Length: 86min 50sec (5210 seconds)

Published: Sun Oct 17 2021