Building APIs and backend services that run PowerShell - Roberth Strand - PSConfEU 2023

foreign [Music] hello and welcome good morning um apparently there are some AV issues but you guys see this and I got everything set up here so I'm just gonna start so we can keep to time I'm going to talk about building apis and back-end services using Powershell um and um we'll see how it goes I I was up until at like one in the morning trying to get everything to work so we'll see uh thanks to our sponsors uh it's thanks to them that we have a great event here and I hope you're all happy with it I am even though I haven't been paying attention since I've been doing my Demos in my hotel room my name is Robert I'm a principal Cloud engineer at Ernesto 42 yes it's 42 like the history Galaxy which is why we have T-shirts like this um so we basically all just do references to hackers cut through Galaxy internally um we we create uh services around platform engineering security and Cloud native and so on and so forth I'm a Microsoft MVP for uh in the Azure uh category I'm a cncf Ambassador for the things that I do with the cncf so I'm in the Tag app delivery culture of the platforms working group uh one of the maintainers on the opengettos project and I do a whole lot of other things I'm also hashtag Ambassador for the things that I do with teraflow uh which we're not going to talk about today so uh and I might have not offered a slight explicit so I'm going to talk a little bit about Powershell Beyond scripts uh how we can run it as Services uh running Powershell cloud natively and I had some demos um the serverless functions I thought that there was no reason to include as Ben who's gonna be on stage here after me is gonna run you through how to do this so you know why get a repeat on that um I um I'm gonna show off how to do web servers with the Powershell and this typo there apparently um I wanted to do it in a certain way I'm going to get to that we just have to do it manually [Music] um and then the recurring task with kubernetes so um basically like a drop in replacement for running as a service on the on a server or something like that and I wanted to create create a kubernetes operator um but uh time so I was doing stuff in my hotel room uh yesterday and every time I did something that made a container image and put it up and then tried to pull it down again it took like 10 minutes so every time I did a change it took like 10 minutes so bear with me all right post I'll be on Scripts so what can you do with Powershell that is not just running scripts on your computer well you can run it in a pipeline which I think is a pretty standard way of doing stuff these days you can run it on triggers based on you get the workflow you can run it on you know timely triggers Etc et cetera and let's say that's a fine way of doing Powershell um and you can also for instance use this in an API context or backend service context by creating some sort of like Gateway that triggers these pipelines so you can still use that as part of a back-end service um you have software to run scripts there are numerous ones I choose to include script Runner because they are actually sponsoring this event so thank you if you um and they have everything you need to run basically so if that's a service so you can Define your scripts Define your packages click run or have that triggered in different ways and then there's Cloud native Solutions um so you have serverless and Cloud functions whatever you want to call it so your Azure for instance your your Amazon AWS lambdas and so on and so forth or basically anything that runs a container which can be a lot of things it could be the for instance like Azure has its container service and app service also supports containers um or you can run it in kubernetes which is what I'm gonna so yeah okay functions container app service and Azure kubernetes service those are the main things so like I said I cut out some stuff of these presentation so we'll see how we are at the time at the end uh mainly the reason why I did that is because this conference has a lot of API talks um some of these were yesterday uh so uh if people are watching this video that might be something that you want to go back and look at um or if you missed it yesterday um I think besides this one you have the building apis with powers and Azure functions which is going to be here so I think this is like the API room today every API talk is happening here uh and Ben is going to go through that video and that's a follow-on so that's that's a great way of like getting up to speed with I have to do Azure versions and then there's various other talks about consuming or setting up and creating your your various sorts of apis so mainly what I'm gonna do focus on today is a kubernetes demo the reason for that is basically our platform that we're creating this is not a sales page this is just medium forming what it what we're doing uh so what we're doing we're creating a deployment platform is going to be run on kubernetes um but you know we have a lot of infrastructure assists admin people people that are used to creating stuff and running into pipelines and it gets kind of daunting like how to you know teaching people how to program for instance is is not something that we want to do we want them to focus on what they do best so the platform that we're building is focusing on automating terraform or basically anything that we can run in a container so for instance if we if someone has done some scripts for a customer that they think this could be a service they should just be able to put it into container put it into a platform and then I'll expose it through API so you can trigger it uh some of which we some of the techniques we're going to look at today um it's being a witch there we go these last two pictures by the way if you see there's a delay between speakers and when this picture comes up because it's huge like I have no clue why it's like that but it's been like that for a couple years all right anywho so the first thing that I went and did uh well that didn't work oh yes obviously it's a screen mirror or not except for display there we go sex big enough for everyone only in the back everyone cool great thumbs up so the first thing you need to run stuff in kubernetes is obviously some sort of container um it's um we could do this inline I'm going to show that afterwards but um for sake of Simplicity I've created a container that's just based on the official Microsoft uh container image for Powershell this one in particular is using Alpine Linux just because it's small and it kind of is the default I want to but just for clarity also I wanted to use um chain guard images if you haven't heard about that that's that's they're the creating chain guard is a security Cloud native security company they make really small container images that have basic in those CVS so the idea is that it's updated all the time so if there isn't a CVS there's going to get eradicated straight away so you know really secure problem is obviously it's really secure so when I'm doing demos of stuff that might need some rights that they don't you shouldn't necessarily have in a container everything went to hell uh so I just went like okay sure let's just take the lid slightly bigger slightly less secure it's not that it's not secure it's actually pretty fun so basically we're using our Powershell uh image as a base image and then we're copying over my bunch of scripts that I have created and I'm also installing a module called uh code however you're supposed to pronounce that um so I went searching for how like creating web servers with Powershell um and you could just build it because obviously you can run.net stuff so there's um you could just do that straight off but the thing about web servers that they have a lot of things in it that you need to deal with endpoints of uh different kind of middleware etcetera logging um so I was just looking and I found this uh Port Let's uh are you just gonna show it off import is basically like I said a cross-platform framework to create web service that's on Powershell so um if you're looking to do so that sort of thing look into it I I can't endorse it because I've used it for two days or something but it basically supports a whole lot of stuff the main drawback from by using something like this just like in my experience is that this is one person relying on something that's based on one person's work might be a little bit uh risky if that one person just disappears so if I I wish there was like a big project with a lot of different companies behind it but this is what we have and that turns out that doesn't run in kubernetes that easily and like I said everything took like 10 minutes to check so we're gonna run it manual on my computer um so this is sorry bigger fonts there we go so this is basically How We Do poll very easy you create you you run the start Port server and then you can just add endpoints you can add routes you can even ADD full-on web pages apparently and just you know put in whatever you want um if I do this hope everything works I think I'm running now I don't know why my terminal didn't pop up but let's check it out localhost 8080. there you go now we have an API so oh sorry it's really small there you go so this is a simple simple as guess right so if if you wanted to do this yourself you would have to use the.net web server dll thingamajig get it in you know and tweak it to your liking and so on so forth what I had to do is this right so obviously using this kind of framework makes it a little bit easier uh they uh on the web page they have if you want to run this as Azure function they have guides for that if you want to run and create your docking container you can do that Etc et cetera et cetera so there's there's various ways of running this they integrates into IIs it's Etc so they they've done this person has done a lot of work behind this and I think it's uh it worked so you know just didn't work in my environment that's basically my fault any here so that's how we can run a web server um so what um to look at how the deployment would have worked and and kubernetes if it worked so as you can see I use chat GPT or the copilot sorry to write this and I didn't get anything particular good as I had to write it myself anyway but um who here is if familiar with kubernetes ish like 40 of the room so just for everyone else in kubernetes when I refer to pod that is one or a collection of containers so it's like a container group and other scenarios um when I want I could create a static pod which I just like Define and put it out but what we usually do is create something called a deployment that's a abstraction layer that takes care of your entire deployment so that puts out all the parts and that's when we can get stuff like replica so we can say we want 10 replicas and it would create 10 parts and it kind of just do the life cycle work for us so this is a deployment file uh just give it a name I'm in a replicas the selector is basically something for the deployment to know what content or a part is part of the deployment uh I create a template on how it's supposed to look it has a label and then I Define one the one container that I have right now and it could be several containers in different scenarios but you usually just kind of end up with one part with one container this one is called web server it has an image uh it's the the container that I'm building on GitHub I have image pull policy set to always because I was doing the checks and stuff uh you uh probably wanted to be um set to if not present so you pull it down once and right now I'm I'm also as you can see it by tag I'm just deploying I'm just building it manually but usually you would have it set to a specific tag right so if not present would then work because then you could just update the tag and it would actually pull down the image and then in the container I'm running the command Powershell with the file web server and I give it some resource uh limits and requests just just because it's good practice to do and expose the container Port the unfortunate part is I run this and I try to do something called a port forwarding which is just setting up basically a total to the deployment uh and nothing happens and again troubleshooting this at one o'clock in the morning right before I'm going to do a presentation kind of didn't work so we have just a look at manually I will say that I I'm gonna try to get this to work in a proper fashion and I'm gonna put out on Twitter and everything else that like this is this was a solution to this and all the code will be in the PS golf EU uh repositories for for this year so if you are interested anyway let's go to something that works um so I wanted to do a practical example of um of using uh getting some information storing it and then reusing it so I created a script for getting weather information because that's an easy demo to do um so basically what we're doing here is we're checking the weather from open weather map for Prague and I'm referencing an environment variable here for my API key I created that as a secret in the back end it's uh I'm going to delete it afterwards I probably could just show it but you know don't store secrets in kit Etc and I'm using invoke rest method to just do a get call on on this particular URL um and then what I'm doing I'm putting this out into a file uh based on uh and calling it weather Dash and then I'm using this long string to just generate the time of day uh to have unique files um and storing that and then I'll use this in what's called a crone job Crown job if you're familiar with computer terms might sound make sense but basically it's a job in kubernetes a job it's a single run thing you you run it once and when it's done it quits a chrome job does that on the timer so by setting it to just Stars I'm running this job every minute and what it does is is put up a a um a pod that has a volume so this is a persistent volume claim I say I I said I want a volume and I want it to be persistent um the container is the same container because I have all my scripts on it but this time I'm running Powershell file weather with the dash fetch switch that I put in and this is the volume up that mounts this volume to the slash data environment and then I set up my environment variables by defining it as open Weather API key uh and I'm using a secret in kubernetes so this is the name of the secret and this is the key of the secret in the um the value inside of the secret [Music] um this is how you define a persistent volume so I'm just saying um this is the axis type is read write once a storage class standard that's the only thing I have because I haven't added anything else uh it's supposed to be one gig and a host path is something relevant to the node itself um and then we have the claim so the claim that the the pause can then use the claims storage space on this and as I said 100 megabytes is what it gets which should be fine so let's see if this works um so sorry that this is black hyper terminal didn't want to just easily jump over to something else so so I'm just running a cube CTL apply on that file and it's going to take a little bit of time to start doing stuff but if I check I have my crown job and by the way on top I have K9s or canines which is a terminal GUI which sounds weird um to basically just handle kubernetes stuff so instead of having to write Cube CTL get Chrome jobs namespace all those things I can just go in and and select the things so it says that everything is okay and it has run and it's completed if I press L I can see logs um and as you can see apparently it's 296 degrees which I thought was funny so by the way how are how are you feeling is it hot how are you dealing with this heat uh Rob graciously came by before the thing I meant like is that Kelvin and then they checked and yes it's Kelvin so apparently if you go to open weather map API and ask for the temperature you get it into Kelvin which I guess is a compromise between Celsius and Fahrenheit so you have to convert it yourself I I don't know I didn't even think about that so I was just looking at it I was like oh this is almost 300 degrees okay anyway if you go back now we see there's another job that's run so it will do this every minute uh and it basically just runs get the data uh whatever it logs there is what it's put in putting into different Json files and this is all stored in the backend now we want to you know check that data as well so I created a a job called psconf weather demo presenter for some reason uh which again basically just do the exact same thing but instead of doing this every minute it just does it wants and it runs the file without the dot Fetch and if we look at the weather.ps1 file uh the default if you don't have the the fetch switch is to just sort out all the Json files looks into what is the newest one get the content and convert it from Json so it's something that representable but if this was something that would pull it out and give this to a usage to an API then you obviously would just have the Json content but this look more pretty for us so if I do a KFI whether um tourism is Right K is because I have an alias uh if you were wondering about that and as soon as I click uh press the enter button it should create a pod so it's already creating a part um this is uh keep on scrolling there but this is uh the latest data that I got um probably should have had something that actually wrote out the name of the file just to prove that this is what's happening I checked it manually don't worry I think I can see the pressure at least as you've risen so you know good for good for us um so that is how you could uh just create a script put it into container run it get some sort of result store it somewhere at this point like I'm storing this in a uh let's see what I'm at the time um right now I'm storing this as a uh on kubernetes but obviously you could pull data in and you could put it somewhere else if you have a SQL database if you have you know wherever you're storing your stuff in um so uh people all the people getting fetch the data what I wanted to do speaking of things that didn't work is uh connect to the API the kubernetes API itself and do stuff so this is this is what I started with which is why it took so long to get stuff to work but apparently if you're running Powershell scripts and trying to get into contact with the kubernetes API from the Pod probably have to do something to allow it to do so um but basically the idea was just getting the the address of the API server on the Pod uh when you when you deploy a par you have a service account that service account have certain rights through the API by default I should be able to just look at stuff um so um there's the path where the service account places the bar run Secrets kubernetes IO service account folder the service account folder has a token it has a namespace and it has a certificate to be able to connect to the um to the API so this one runs and then just times out um but you can see here kind of like the the idea here is that we we run a get on the API server URL the API for kubernetes API slash V1 namespaces this is basically when you look into for instance let's see this job you can see a top here you have the batch leave on job so um this is basically the the uh the address written backward look at this again and then you know give a token I tried with Skipper certification check that didn't help and it just basically types out but in theory whenever I get the time to actually look into writes on the kubernetes part and so on and so forth you should be able to connect directly to the kubernetes API from within the uh the Pod and and then do stuff this is kind of where the kubernetes operator either he came in or was supposed to be so a kubernetes operator is a software agent that looks at a certain resource in the API so again for those who are not familiar with kubernetes kubernetes basically has an API with different resources but they also have something called a custom resource definition so you can create your own resources and this is how people make services like for instance me you would do a lot of get Ops get Ops works on like these defined resources that you can that a piece of software looks at and then try to to you know consolidate at all times so an operator basically looks at a resource in the kubernetes API and makes sure that that is the reality and then it just loops so this is going to make a great example on a easy way to do an operator through Powershell because you could just easily do a loop forever that looks at a certain resource if that resources change then do stuff doesn't help if I can't connect to the API though great enough so I forgot to say at the start but if there are any questions along the way that's also fine but I think basically we have everything any questions so far at the end no questions ask questions health checks all right so the question was if I've considered health checks on the endpoint so for the pod I am fairly sure there are there are uh it has a lot of functions so um as we can see it basically has everything you would need from a web server so creating health checks uh at that point would shouldn't be a big issue um if you're running things in um in kubernetes if you can get it to work uh basically what you would do is you would have a a an open endpoint that just gives off a HTTP 200 if hits so you can kubernetes make sure that the service is up and running and stuff like that that's one alternative it was a hand there any recommendation on monitoring uh logs uh yeah so again if the container is then running in kubernetes there's there's a lot of tools for that there's there's um there's several log tools if you go into if you look into the CNC if there's and the open Telemetry and and there's um observability tag that deals with all these types of things uh you have but you have Loki from uh from grafana you have a fluently logged there's a lot of login services and and all those things with again um with Prometheus for instance for metrics uh those things put together you can then install for instance like profile not to make a dashboard out of it to show off stuff but basically everything that does logging works because um when you're running Powershell it's just standard out and stand there so the Pod would then surface that and kubernetes knows that this is how uh how you do how you get logs out of these containers so any native solution for kubernetes would work and if you're using Azure functions or something like that the built-in tools but work just as five so what was the first part of the question okay yeah so the question is that in the drawing was a reference to API manager and what was the rest of the question sorry right so would you want to use the API management manager API or or have them directly go against the cluster yeah if there I would say if it's in the same cluster you you it's pretty safe to just use it as is and you would um if it runs in the local context it security wise it's it's so much better because at that point you don't actually need to take uh you know handle all the Securities between all these endpoints uh if it runs in the cluster uh and you give it uh for instance in kubernetes you can do Network policies so you can restrict what pause is allowed to talk with what pods uh you could do that to make sure that all there's no crosstalk and stuff like that and at that point a pod should just be able to connect to whatever um if you have it outside assistant like an Azure function um and you have something in AKs for instance and you want them to be interconnected somehow I would rather go the route of setting a private link and all those kind of things and make sure that everything is happening at the back end and not go in and out that would just make it easier because at that point the API manager would basically just be this is how you connect publicly to the API and then everything else is handled by the platform itself I I would go that rather all right any other questions no well if not I would say that um if you are interested in getting Hands-On again Ben is going to do an actual Hands-On with Azure functions and creating apis um I see that I would have just enough time to show that off if I did but I removed it like I said so if you want to learn how to do apis in a serverless fashion I would recommend sticking around here and yeah so you get a little bit of minute backs and then you know you can wake up get some coffee and stuff like that oh thank you [Music]

Channel: PowerShell Conference EU

Views: 1,052

Keywords: Monad, PSConfEU, PowerShell

Id: 6GS3Z_8fULk

Length: 31min 57sec (1917 seconds)

Published: Sat Jul 15 2023