Conquering Kubernetes with Emacs

[Music] hi everyone my name is Adrian and I'm here to talk to you about combine conquering kubernetes with Emacs and I hesitated a long time whether or not I was gonna spell conquering with a K and get the alliteration going you know but in the end it's mostly a click bTW title we're gonna talk about kubernetes but it's gonna be a lot of Emacs dealing with external processes and what could what role does kubernetes stand in there so let me start quickly why I'm here why I started with this talk I started because I have a specific annoying workflow problem when I started at my current company company was fully on kubernetes and I was really confused I've never used kubernetes before and I was wondering how things worked so just I love show of hands who here has heard of kubernetes all right who here has used kubernetes who here can define what kubernetes is yeah that sounds about right I don't know either but basically let's have some gross approximation of what it is it's a service system that allows you to control a wide range of types of services and basically if you have a ton of servers running if you have a bunch of pods a bunch of things Khomeini's allows you to control the deployments the replicas the kind of environment variable it's very flexible very complex very heavy extremely interesting and you can spend a lot of time digging into it but specifically when I started working with it my question was how do I see the logs for the server's I'm responsible for and it's really simple like so here's a shell let's just clear it up everybody can see Babylon on my screen yeah so in kubernetes you list your pods or servers however you want to describe it here you do cube CTR or cube control depending on who you want to annoy cube control get pods and so that gives you a bunch of running services so I have a local companies running here and I just put some demo pods in there either I have something called hello node and some front-end stuff I took from the kubernetes tutorial and say I want to see some a part of a particular service and now have to do again coop CTL logs and then I have to go in and hi select text there we go if it's painful if it looks painful it is and then put in there and bam I get like a bunch of logs for my radius instance so that's how you typically look at pods look at logs in kubernetes so it's very annoying you have to list for services and you copy/paste you put them in in in another command which is again can be more complex because you can have namespaces and contexts in communities so your comment can get really long and I think that first stuff as basic as listing your logs on making sure everything is running ok you want to make that as simple as possible shouldn't be a trouble for me to look at my logs I'm gonna be less than to do it and when that makes me a worse programmer also I have to use the mouse in the middle like who does that I might as well go back to Microsoft Word to a dating code I didn't say go back so that's a problem now what's the solution the solution is Emacs that's why we here but specifically why why would I make some interfaces for kubernetes in Emacs so obviously I want to streamline this workflow and I want to learn more about communities see what I can do with it there's already a community's extension for you max out out there that's great Primus doesn't work for me and pray won't work for a bunch of people because the permissions are pretty greedy if you don't have full access right to your kubernetes cluster a lot of thing will break so that was my case so I couldn't use that and then at the end of the day we want to bring everything in Emacs that's everyone's goal everybody's clear on that alright so one of the things that I discovered on my on my journey to get my logs from kubernetes inside Emacs whatever key takeaways from that's why you should listen to me so I'm gonna I derived a major mode which I didn't know how to do before I learned about interacting with sub processes in a meaningful way in Emacs and then I found a lot and I looked around a lot in the Emacs ecosystem and how to you provide some meaningful user experience something better than just like what's out there in like shell completion or whatnot anybody ask questions on what we're gonna see pressing question on kubernetes before we get started now so that's pretty much all the company's concept you need to follow that talk now we're gonna go into something much easier which is Emacs Lisp alright so let's talk a little about listing parts the first part is I want to get the list of parts so I can select which one I want to see the logs from so very requirements as I get with parts with some knife processing it doesn't really matter how performance that is I don't that list into something appropriate in Emacs and then maybe I'd arrive at into my own major mode so I can quickly summon it and put it up so what's the command to get buzz I just run it but it's cute control cube CTL get but I run that I have my list of bots and yes in case you wonder I'm gonna do the entire talking in org-mode and try to be interactive and show some code it's the easiest I found so first pass as much as messaging that the idea is we take this output from cube control and we're gonna dump it in Emacs somehow so a first pass is I'm gonna do some dirty shell trick I'm going to do get pods remove the headers remove the top line and I'm gonna keep only the pod name so I'm going to use arc and just keep a first column so this is just some dirty bash tricks to get the work down so I run this BAM I now have a single column of pods now how do I get that into Lisp into a lisp string so there's this fancy command this fancy immersive function called shell command to string where essentially you give it a command and it's going to return the string output of that command so if I run it here I have my long this long string and you see there's a lot of things happening here but mostly I have all my pod list and I have those backslash ends in between which are the new lines in shell so that's now a lisp string how do I turn that into a list list list list try that one it's it's so again I'm gonna use another function it's a split string function split string together list list I practiced so I'm gonna split the string at every new line and that's gonna give me a list list let's do this BAM I have my list you know this if you're if you're very careful at the very end I have some empty string I actually don't care about it like an improper way you should remove it but for now let's just leave it here it's not going to harm us so now I have a nice list and I want to make it interactive I want the user to be able to select one of the components there are multiple ways to do that but what I really like is the built-in tabulated list mode and Emacs so it's a great major mode if you have like a CSV if you have columns of data and you want to represent that in Emacs and being able to interact with it the problem is that the documentation is a teeny bit abrupt so took me a while to figure out exactly what it requires to to work so first you need to define some columns by passing a vector where each element is the name and the width so this is an example I say I'm going to make two columns column one column two they're both going to be 50/50 I'm not totally sure what the units are maybe percent I don't know it works so that's the first step to determining to using tabulated list mode then the rows is a little bit more hairy but you need to pass a list of Lists where the first each element has an ID and then a vector of values and the idea is not particularly useful here so I'm going to leave it as nail why we really matter where we miles is the vector of values so that's every row of it so here I'm gonna have two column column one column two both gonna be 1550 and then I'm gonna have Row one value one row two value two we're all free value free so that's how you define your data and then I put everything together and I'm gonna dump it into tabulated list mode so same code I just copy pasted the code from before I have my column definition make three rows and I'm gonna set my column set my entries print my header and print my list all right let's hope this works yeah because this is also mostly live coding so let's let's all Prairie a bit all right so it works I just ran this and I'm now you can see I have this header that I cannot get to and I have this code with rows and values so that's simple tabulated list mode let's come back alright so now we know how to use tabulate list mode and we know how to get a lisp list of kubernetes pods let's combine the two together so it's pretty much the same code as before just merge together I am only gonna have a single column and I'm going to call bud well I put at width of 100 and then I am dumping in the rows the split string lisp of pods in there and then i'm just printing my tabulated list so let's try that and now here we go tabulated list with the list of pods so I've made three pods in a nice nice format everybody is happy questions on that alright I'm gonna assume everybody everybody got that now so that's great we know how to combine the tabulated list and the shell commands to the string now let's make a major mode out of this and and that's actually it was actually really easy I'm using the define derived mode so the way this works is you say I want to define a new major mode I'm gonna base myself on an existing major mode and I'm just gonna add some new functionalities to it so here I'm saying I'm going to define kubernetes mode from tabulated list mode it's gonna be called kubernetes you pass some duck string and then here's my body here's what I want to happen when that mode is triggered and that's the same code that's printing the tabulated list and then at the bottom I made a interactive function to summon this the kubernetes function so let's try it I'm going to execute this code so from now on I should be able I can test it out immediately so if I do meta X kubernetes but I just created and you can see at the bottom that's my major mode so we now have kubernetes major mode defined all clean I can do the first the first third of my problem in Emacs all right so now let's tackle the second third checking on time all right so I want to be able to list logs so we have a list we have tabulated list I want to be able to choose the robot interests of me and then get the logs but for that part oh there we go well so the requirements are I need those logs to not hang Emacs I need I need sometimes pods of hundreds and hundreds of lines of codes I want to be able to deal with a subprocess and I want to redirect the output of a buffer and there's an extra thing that you can do also in Cuban a disease you can tail logs the same way you would do tail - F you can do kubernetes log - so it would be nice to be able to see live logs in Emacs in an e max buffer and not have not have Emacs hanging because of some Freddie show or whatnot so as I said would command to get loads is to be kuba need cube collogues and then the pod name let's make sure it still works all right it still works so that's my Redis logs as you can see this will write its logo on top and a bunch of logs saying it started all right so now why can't I just take this command and dump it into my shell command to string well I want the knave we work that's in some fine prose it's it's made performance as I said you can have a bunch of these logs you can have 100 hundreds of lines and that's gonna really kill your your session in there so how to dig into how to properly interact with external processes and what actually the shell command to string function codes underneath and so in Emacs the proper way is to use the qolt process function it's pretty simple you pass it an executable you want to run then don't worry about this value then a buffer name which I'm going to call cube coloring logs then another value I don't care about and then from there on it's just a list of arguments so what I want to run is Q color logs and my pod name it's gonna dump that result into a buffer and I'm just gonna show that so let's try it out and see if has the same effect yeah so I just sent into a brand new buffer called indeed cute color logs and I have my reddish slugs in there now I said we also have a dash F option to live for all those logs how do we how do we do that because Cole process will just go over process wait for it to terminate and dumpy the output in a buffer so there's a congress function or sister function called start process which is async so this time you give it a process a buffer and your list of arguments and process is just a unique string that Emacs will use to identify that so I don't know if you've ever ran like some LSP mode or well else like sometimes sometimes if you running maggot and you kill you kill Emacs it tells you Oh Emacs was a process running do you want to exit yes or no so actually what's happening is that in the background that extension use start process function to start a long wave function and Emacs will make sure that you actually want to terminate that so we're doing the same thing here I'm saying my process is going to be tube CTL I'm going to dump the output to cube CTR logs and I'm going to run this function so let's try it so it looks the same the difference is that if I try to kill it now you see at the bottom it's done this is a running process are you sure you want to kill it so that means that if my Relius were to go down I've had any more logs they will actually live appear on this screen right now which is exactly what I want so how do we combine these two functions and make it into into something that I can summon right so this is my code for the function called kubernetes get logs so I'm using the optional argument explain that in a second and I'm saying either use start process or use court process and and that's exactly the same logs that I just the same code I just showed you and the option argument if you're not familiar with it some commands you can just execute them from a key binding or calling them from a max but they have a secondary I want to say effect or yeah secondary effect if you'd use if you control you MX and that's gonna pass and the next an extra an extra argument into your function so here I'm saying if you don't pass if you pass this extra argument if you do see um X kubernetes get log it's gonna be a sync and if you just do MX kuben ease get logs it's gonna be sync and that's something that I think is not used enough but having the function with just a little bit of different behavior I it's nicer to have this optional argument rather than like define the kubernetes get log sink and kubernetes get logs basing which is a little bit too wordy in my opinion so yeah as I said MX convenience get logs or m/c um excu these get logs so let's try it alright so kubernetes get logs alright so this is my sink version i know because if i try to kill the buffer it dies instantly now let's try CU m x kubernetes get logs and now this is the async version and if I try to kill the buffer it's telling me you're running a process would you like to kill it now any questions on how we define this function in sync versus racing yeah so how do we connect this function to a major mode because case you haven't noticed I hard-coded I cheated I hard-coded my poor my pod value in the function so that's not gonna work if my pod name changes which it does all the time every time you restart the pod so we want to connect it to the major mode and there we want again to come back to using tabulated less mode and the function that it provides in tabulated less mode you have access to this function called tabulated lists yet entry which gives you the vector of values which is currently under your cursor so if I were to run this when my cursor is on this particular row of my combination mode I'm gonna get the pod name so that's exactly what I did I just redefined a little bit mokuba knees get logs and I'm saying let's get the pod under the current cursor changed nothing else and let's just execute it so let's see that's work all right so let's test it out so I swim in kubernetes and now I'm gonna do MMX kubernetes get logs and it's gonna show me the logs of this front-end part not have radius logs anymore and that's correct we're seeing some totally different logs about Apache failing and whatever what's new if I go back to Redis and now I do my control you MMX companies get logs I'm now back into my radius logs and I am still acing so my function works so I successfully linked my function to tabulated list mode obviously the downside now is that if I were to call companies get logs outside of my kubernetes mode I'll have a problem it'll raise an error actually let's try it right now I've never tried it greenies get love yeah so it says that the bottom wrong type of argument are a kneel kick because I'm not in tabulated list mode but that's the function that I'm going to exploit let me drink some whether anybody has questions in the meantime wonderful I'm definitely not running out of material anytime soon so now let's talk a little bit about modern UX and modern UX is a very subjective thing but I don't like doing my MX kubernetes and then selecting my pod and doing MX kubernetes get logs or see um excavators get load but that's too many keystrokes and so I looked out there what's an extension that deals with a very complicated command-line tool and has a very interesting and very nice UI anybody wants to take your guess as to what that is it's magic yeah so how does magic do it we have meaningful UI for users to interact with major mode and they use transient so if you don't know transient is the successor of magic pop-up I think it was called which is essentially that little menu that pops up when you're in magic and you decide to push your branch and it's essentially a very nice menu which constructs dynamic constructs command-line arguments for you so we'll see exactly what that is and it's a great project I love it there is one teeny problem of it is I found the documentation the manual a teeny bit too dry so actually I had to read a lot of magic code to understand how transient was used so I'm trying to condense that here with example what's a simple transient function so I'm defining a test function that just prints test function and then I'm defining this transient command which is going to have a test transient title and then a bunch of actions so like okay what is this let's let's just run it and let's see what happens here we go so you see I just created this little menu at the bottom that has the actions that reflects here with a little key binding and a full name so what happens now if I press a is that it calls my test function so essentially it's a nice menu interface which can lets users select functions with few few keystrokes and this define transit commands create this test transient function which you can now call the only limitation is that you have to make sure that your test function is interactive but you can call it from from an X so that's a simple transient to create a bunch of menus now this is not a very interesting one because I'm always calling test function what can we do with it what's interesting with it so for one it's very easy to add switches so if you do the dash F I was talking about that's that's a switch you turn it on and off when you pass it in in the command line arguments so I still have my test function I modified it to get a list of arguments from transient using the transient art function I'm not sure it works I just copy pasted the code from maggot so then I still might test transient so I'm gonna redefine here and I pass a bunch of arguments - s - a and those are just some random switches and so what's gonna happen when I run this is now I have this new argument section I only have a D action I removed everything else and so if I type - s I just triggered the S switch so you can see it's currently highlighted and if I do again like - babe I triggered the second another switch which I guess I should have called second switch I would have made more sense and now if I press D the test function now prints the arguments I was passed and it's it's an array with exactly those full values that I defined so that's very useful to define in command line arguments there is another thing that we can do more complicated is that we can pass some parameter and Zients who can or every can let users enter some arbitrary parameters so maybe I should not show the code before say things it's very before saying things very disruptive but basically when you do if you do what's a good example in the example of tail you can precise how many lines you want to tell so you can do tail I think it's - n equal something-something so we want to be able to collect user input in those parameters so I still have my test function I didn't change it and now I defined an infix argument which is going to be - - message you give it a description I don't know what that does short tag of - M so that's going to be what the user selects to trigger that that parameter and the actual argument that's going to be passed to your function and then I just added this test transient - - message to my list of of transient command and let's run it so I still have my switches that I can still trigger but now if I do - M I now have this L input in there and I can type whatever I want okay so now as you can see I'm locked in - - message called hello and if I press D all my arguments with the user input we're past I think other so that's pretty good I'm I'm a fan of this and in case you're wondering it also has memory so if I do - M again it preserves my last input that's a really nice thing and ctrl G to quit all right so now we know how to define those nice menus which I think makes a lot of sense in the context of what we want to do what should be our transient our kubernetes transient menu so we want to get logs follow those logs with - f we want to be able to specify the tail length maybe I'm not interested in every single message of every publish maybe I only want to last hundred and I want to be able to combine those I want to VSA down the last 50 message and keep tailing so that's exactly what I did we have the - - tail in fix argument which is exactly what it is it's then my transient is gonna have the - F as a simple switch my tail in fix and then I'm gonna have a single action for now its l2 logs and you see I am triggering companies get logs on the L key so let's try it alright that looks pretty good so I have - F I can trigger that I have - DL oh here we go and so I can specify a 100 messages and then I have L to finally trigger my company's get logs BAM error we didn't connect everything together so let's let's try that by this it looks good and that's what we want to do so we now have to go back to update our companies get logs function to take those arguments from transient so that's exactly what I did we are reading both arguments the same way my test function was doing it from transient args passing it the kubernetes transient so now args is now a list of parameters and switches the other thing I'm modifying and making an instead of splitting my logic if you pass an optional argument or not I'm looking if - F is in the list of arguments if you have it you want an async process if you don't have it you want the same process maybe there are better ways to do this I'm not sure and then at the end I'm now passing those arcs but arglist to my function I don't have to worry and it's also pretty neat because if you want to add some more arguments tomorrow to kubernetes if there's other option than - f + tail that you want to add you essentially don't have to modify that company's log function you just have to modify your transient so let's try it alright last thing before we actually trigger this transient node we want to connect such that this transient is easy to pop up when we incriminate this mode and that's done easily by defining a mode map so my mode is called kubernetes mode so Emacs by default if you define the kubernetes mode map is going to assume that that's V came up you want to go by when kubernetes mode is activated so here I'm going to define the L key to my transient so if I do L it's gonna show my transient and from that on if I do ever again it's gonna just show me the logs of a pod under the cursor let's try it alright hopefully this works we try everything out coogan it is alright so I select my pod I'm under my ready spot and I'm not just gonna press L it's undefined wonderful one second [Music] still undefined here we go did not believe I work so I have my transient corrected connected if I do L on top of my ready spot I have the options I just define I do L again that's my ready slopes now let's try some switches so I do L tale and I precise I want only the last five lines I'm not gonna have Redis ASCII logo anymore to do that that seems to work let's combine everything I want to follow host logs and I want the last five logs here we go and so now if I try to kill the buffer indeed it's a running process I am live and I'm telling the last five lines so I finally accomplished via experience I wanted and I don't have to use my mouse anymore which is great let's kill that here we go so with some time to spare let's get to the conclusion what did we see what could be improved so that's sure I'm gonna get some comments on that obviously you should not use oak or some ugly command-line arguments to get your list of butts and while you have all those columns about the state of your pods it would be nice to expand and have more vengers for pod names in your kubernetes mode that's one thing you also want in general better error handling I said that if you summon the kubernetes get logs command in not kubernetes mode you're gonna have a problem so it's try to be better about that removing obviously over hard-coded values allows some customization so I said that you probably want a default tale mode a default number of lines you're gonna tell or people are going to forget and they're just gonna bust very very Mac session and then it turns out it works pretty well with get logs so why not expand and do a bunch of other services so we can can describe pods we can describe services Ivan so that's where I'm getting I even actually expanded the logs to do full deployments from Emacs to kubernetes and then a list of resources on what I just showed you this entire org mode file is a guest just I don't know whatever it's a it's an out there so I'll make sure that it's posted in the in the meetup you can see all the notes I also have a complete code of everything you just saw but hopefully a little bit cleaner with every single communities function I implemented using that if you want some duck on processes it's inside Emacs the transient manual over a bit dry is very complete it's up there and then finally a kubernetes cheat sheet and I don't know how to spell kubernetes all right yeah and I think that's pretty much it so does anyone have any questions what wasn't defined oh it's because I've been running this thing it's because I've been running the presentation over and over and it's probably some collision with a mode map so actually I deleted the mode map and I really find it all right in that case if we don't have any more questions thank you everyone [Music]

Channel: thoughtbot

Views: 18,408

Rating: 4.9803281 out of 5

Keywords: thoughtbot, Kubernetes, emacs

Id: w3krYEeqnyk

Length: 33min 34sec (2014 seconds)

Published: Thu Jul 25 2019

Reddit Comments

[u/ares623]

Came for Kubernetes, stayed for the introduction to transient.

[u/c17g]

Could have been called "An Intro to Modern UI plugin development in Emacs". Very nice presentation, especially on the transient introduction part.

[u/rollc_at]

Link to the gist: https://gist.github.com/abrochard/dd610fc4673593b7cbce7a0176d897de

[u/tdehaeze]

Very nice presentation indeed, I was just going to post it here :)

[u/FOSHavoc]

Brilliant presentation! It's brief and to the point, but you really get a lot of useful information across. Also it's just a great demonstration about how to integrate shell commands very quickly into Emacs. Brilliant!

[u/stribb88]

Thanks, I found the presentation excellent -- very clear, and thought-provoking.

But I wonder: why try to parse the text output of kubectl when you can get the json?

[u/Psilioxus]

Sorry for being lazy, but does any of you know how he has the outline-path displayed in the minibuffer when the cursor moves over a heading?

I guess I could add advice to org-next-visible-heading, but perhaps a setting (couldn't find it) exists?