GPU Acceleration in Kubernetes with Jellyfin (or Anything!) - Intel, Nvidia, AMD

hey everybody and welcome back to Jim's Garage so hopefully by now you're sitting pretty with your kubernetes cluster be it rk2 or k3s and just to be aware I have actually changed the script so that Source IP is now working that means things like py hole and crowd secc that require the source IP to operate can take full advantage of setting the external traffic policy to local but one thing we haven't covered yet is Hardware pass through through and if you're trying to replace your existing Docker setup with kubernetes that's probably a vital part things like jelly Fin and Plex and anything else that can benefit from a GPU you're probably running it in CPU mode which just isn't efficient and might just not work at all so in this video I'm going to take you through how to pass through a GPU into kubernetes and for this demonstration I'm going to use my existing discret Intel G GPU the A380 but before you turn off most of you have probably got a consumer grade Intel chip and that will have an integrated GPU on it now thankfully due to the way in which the Intel drivers are shipped the same package is used for both the integrated and the discret so that means that this process the drivers the pods that we're going to deploy should work no matter what you're using so by the end of this video you should have a GPU available within your cluster that you can assign to pods and get all the benefits of Hardware acceleration so for this demonstration we're going to jump straight into proxmox where I've set up a new VM with a GPU pass through it's exactly the same machine that I used in my previous Baldor gate gaming VM and this time we're going to join that to the cluster we're going to install some drivers from Intel we're going to deploy a pod and then hopefully we'll be able to assign jelly specifically to that node to take advantage of its GPU now because I've only got one GPU I will have to hardcode this pod to be on this node but if you're running a cluster that has all the same Hardware or at least has an igpu or a discrete GPU on both servers great you're going to get all the benefits of GPU acceleration and you'll also get failover okay so I've got a third proxmox node that I've created the same one as in a previous video and on this I've created a new virtual machine now this is a 2310 auntu version the reason I've done that is because 23.10 or 2304 ships with kernel 6.2 which has support for the latest Arc gpus which for me is important if you're running an integrated GPU you should be able to get away with older LTS versions like 22.10 and if we have a look through quickly just to remind ourselves please do go check out my my other video where I go into this in detail you can see here that I passed the GPU through it's pcie 3 and 4 and it's a Q35 machine and I blacklisted the drivers so that this works now I'm going to hop into this machine we're going to run an lspci just so that you can see the device within the machine okay so I'm now logged into this virtual machine and if I do an LS PCI to list all the PCI devices you can see that at the bottom here I've got VGA compatible controller and the audio device audio controller so this is just the audio controller that's embedded onto the GPU which I don't strictly need but I passed it through anyway so now that we verified that the GPU is available to the virtual machine we can get onto the installation steps now the first thing we're going to do is install this into our cluster so I'm going to join this to my existing kubernetes node and and I'll show you how to do that in a moment so for this demo I've set up a new rk2 cluster using the new script I mentioned earlier so as you can see there's three Masters and two workers so I'm now going to add this new machine as a worker node so heading over to my GitHub I've put all the commands that we need to get this node added to our existing cluster and it's pretty straightforward if you've looked inside the scripts for deploying the cluster and it's very similar whether you do this on rk2 or k3s with k3s you can just use k3s up catch up and it's really straightforward but having a quick look here what we're going to do is we're going to make the rku config directory we're then going to add a config file to that directory because when we start rke it takes any configuration files in that folder and uses those values those values are specified here and we need to get the token which we can get from our existing admin machine and we need to set the VIP which also if you've forgotten will be in your config file then I've just added some labels and you can add whichever you want we're then going to install R2 and we're then going to enable the service so let's copy these commands and run them and hopefully we'll have this joined to our cluster very shortly so the first thing to do is to make that directory then we want to Nano or whatever text editor you want to use into that folder and create a file so here it's just going to be called the config do yam now that we're inside we need to copy and paste this text here and we need to populate it so for the token let's make sure you delete all of the brackets you can find the token on your admin machine so somewhere like this this and the script will have put it in the home directory here now when you open that it looks something like this so make sure you select all of this and copy it and then you can paste it into your text editor so it looks like that the next thing we need to do is to change the VIP IP and so in my case that's 192168 3.50 but just check your configuration file the script that you WR and it will have the VIP listed in there so now that looks right for my cluster so I'm going to save this and exit it and then we're on to the next stage so to do that let's become root and then we're going to head over to the script and we're going to copy this command here so with any look we should now be able to run the script it's going to find that configuration file we created and it should be able to join our cluster so there it is going off and finding it nothing to see here different from when you've run my script so I'll skip over this part and I'll see you in the next bit now that that's starting up let's head back into Rancher and hopefully we'll see this joining the cluster so here you go you can see that GPU pass through is now being seen so it should just be a little while before this is fully operational and so now that a little bit of time's passed you can see that it's fully available it's even got the labels we specified and if you look closely it's even on a different subnet within my home lab so now that we've got this added to the cluster we're ready to get on to the next part which is to install the drivers so to do that I'm going to head over to the Intel website but the process should be the same if you've got Nvidia or AMD I'll put a link in the description below for their respective sites but it should be pretty simple to get it up and running so over on the the Intel website and remember this is the same process for an integrated GPU and a discrete GPU we want to head down to the installation now I'm just going to skim over these documents but I recommend that you go and read them because it tells you sort of what the process is and why you're running some of these commands now things like checking that the drivers are installed is a good idea just to make sure that you've got everything up and running and it mentions that the I 915 GPU dkm M package is recommended for Intel discrete gpus that's what I've got also it mentions the 6.2 kernel which I already spoke about earlier and it's the reason that I'm running Luna auntu because it has 6.2 in it so my arc GPU is natively supported now the next bit of interest is where we want to install this package so you may not know but this is going to install some pods onto our cluster to enable it to share the GPU and there's a couple of options we can install it to all of the nodes which would be great if you've got an homogeneous setup I.E you've got the same GPU family within all of your nodes I don't because myell doesn't but this extra VM on a different cluster does so I'm going to go for the next one which is to install to nodes with Intel gpus with NFD node feature Discovery so you can see here that we've got to go back to our good old friend cubec ETL and we've got to install one of these manifest files so the first thing to do is we're going to deploy NFD because we don't have it by default so I'm now going to copy this command here and I'm going to put it into my Cube admin machine so not the GPU one this is the admin machine and this is the one that you'll have run the script on previously so in my case it's the rk2 admin and I'm going to paste that in and I'm going to edit the value with the release version that we want and changing it to version 028.0 hopefully it will be able to run that so here you can see it's gone and created a load of custom resource definitions service accounts role bindings clusters config Maps Etc Great that all looks good so on to the next part so now it's going to create the rules that this tool here the node feature Discovery has to find G us so again we're going to paste this into the terminal and I'm going to change that to use. 28.0 so now that's going to go away and create those rules and we can move on to the final step in the process the final step here is going to create the GPU plug-in demon set and this is what's going to enable us to start sharing this with our pods so again we're going to paste this command in and we're going to edit the version hit return and with any look we should have this in a moment so there you can see that it's applied the GPU plugin so now we should be in a position where we can go and test that this is working so let's head into Rancher and have a quick look so clicking on Rancher oh great we can see that there's a load more labels here so previously we just had Longhorn and worker but now we have GPU Intel device count one that's great because there is one discrete card there it says that it's present it's given it an A series I guess because it's an arc that might be wrong and it's enabled the Intel feature node for kubernetes GPU is set to true so so far so good I think we're on the right lines and back over on the Intel site we can run this command here the cube control which is going to spit out all of the nodes which have a GPU so running this yeah you can see that GPU pass through I 915 that's that Intel driver one that's true or it means it's got one GPU I'm not sure but we know that the rest don't so from my perspective this looks great so to test this out I'm going to be deploying jelly Fin and I'll share this config when I upload the video but the key thing here and this is replicable for all pods that you want to use the GPU you need to add here the resources limits GPU Intel I 915 set to one that's requesting one GPU this will be different you'll need to change the vendor whether you're using an Nvidia or an AMD but the syntax is the same as here so I'm going to go ahead and deploy this and hopefully in the next scene you'll see jelly fin up and running and it'll be using Hardware transcoding to transcode a video file to to a different resolution or bit rate and so now with jelly fin up and running if I check the logs you can see that we've got here available decoders and if we look closely you can see that these are enabled and on the last line you can see the available Hardware acceleration types Cuda vapy qsv DRM opencl and Vulcan so that's good it's recognized that the card can see it and you'll also see here that the container is on the GPU pass through node so it's honoring that limit we put in the deployment so now if we go into jelly we go through the installation Wizard and if we're heading to the settings we can pick Intel quicksync we can choose all of the hardware decoding so I'm going to do them all just because this card supports them all we can re-encode in hvvc if we need to then I'm just going to hit save okay so I've downloaded the obligatory big book bunny because there's no copyright infringement here and I've opened up a second tab of jelly fin just so I can see the statistics here about playing so if I go to the movie now and hit play you'll see that on here if I click the I it's transcoding and it's doing it at 300 FPS and also if I fire up the terminal for GPU pass through you can see here that it's using the video and the video enhance which I think is the encoder and decoder so excellent we now have GPU acceleration within kubernetes for the use of jelly Fin and you can repeat this for any of your containers by specifying that limit so hopefully this now gives you another piece of the puzzle for migrating from Docker into kubernetes with full GPU acceleration for the pods that you require you get all of the benefits that you probably had with transcoding and things that require GPU acceleration let me know in the comments below what you're going to use this for but if you like this video please give it a thumbs up hit that subscribe button and I'll see you on the next one take care [Music] everybody

Channel: Jim's Garage

Views: 2,629

Keywords: jellyfin, jellyfin tutorial, jellyfin transcoding, plex transcoding, transcoding, gpu transcoding, gpu passthrough, kubernetes, k3s, rke2, jellyfin setup, jellyfin media server, jellyfin server, jellyfin kubernetes, jellyfin hardware transcoding, intel arc, nvidia, amd, linux, homelab, docker

Id: YxDaDOzeN9E

Length: 15min 23sec (923 seconds)

Published: Wed Dec 06 2023