Single GPU Passthrough Tutorial - KVM/VFIO

hello and welcome I'm Steve and this is Bland man Studios today we're going to do a full tutorial for setting up single GPU passthrough if your computer is compatible if you're building a new PC for GPU passthrough or vfio you should check out my other tutorial series which has much better advice on what you should buy for a flexible setup that can even use Looking Glass this tutorial is going to be all about giving it a try with whatever Hardware you have on hand so if you'd like a demo of what we're building today I've already posted that here if you'd like comprehensive info on what makes a desktop PC compatible with this tutorial watch that first video in the previous tutorial and the only difference here is that for this tutorial you can ignore anything I said about needing to buy two gpus okay let's get to the setup alright so first let's install Linux and get it ready for gaming on the host OS for this tutorial I'm using Fedora but you should be able to swap out any of these instructions for the relevant equivalents if you're using your favorite Linux distribute I'm going to Time Lapse through the install process here because I've already filmed a more complete tutorial in a previous video Linked In the description okay so now that you have Linux installed we just want to do a few things to get it ready for gaming first I'm going to update all the programs that are currently installed this is a good thing to do when you first install a new operating system the command here is sudo dnf update and then I'll let the computer reboot and now I want to install the proprietary drivers for my Nvidia GPU right now the display is powered by the open source drivers that are installed by default as you can see here when I type in lspci-nnk my GPU is using the new vo driver to get the best performance I want to install the NVIDIA drivers with the following command I've already enabled third-party repos by clicking this button when I was installing Fedora so it should all work on the first try if you have a different GPU or you're using a different Linux distribution you might need to look up how to install the GPU drivers for your specific Linux distro now I'm going to let those install and reboot the system when the computer restarts I can run the same lspci command and see that my Nvidia GPU driver is being used and it should be able to do some gaming so let's install a benchmark program and see if our performance is reasonable I'll download the unigen valley benchmark and then I know I need to install open alsoft which is a dependency that happens to be required to get audio working for this benchmarking program then I just need to make the program executable and execute the dot run file so it unpacks itself I can change directory into the folder where the program's installed and then I can run the valley command to start the benchmarking application let's let it run on the Mac settings and check the results [Music] foreign [Music] not bad I've had better results but 60fps for a demanding graphical application is definitely usable and anyway this tutorial isn't about Max performance it's about being able to run Linux and virtualized windows on the same PC so let's get started on that I'm going to power off the PC and then press f2 while it's booting to open up the setup menu your computer might use a different key but it should tell you on the screen what to press then I want to go and enable anything that looks like it has to do with virtualization PCI pass-through or iommu on my computer that's enabling virtualization here and enabling vt--d here on an AMD platform it could be called amd-vi these features go by a lot of names so you might need to poke around and find the right setting next when the computer starts up again I want to install all the virtualization software we need so on Fedora the command is sudo dnf group install with optional virtualization if you're not using Fedora you'll need to install everything related to KVM qmu libvert and vert manager so let's let those install and boom we're ready to move on next we need to edit our grub config to have some extra arguments so that iommu and vfio are enabled on our next boot you can see here when I installed the Nvidia driver it added a bunch of arguments here so I just want to tack ours on to the end I'm going to add Intel iommu equals on and iommu equals PT this will enable iommu and set it into passthrough mode as you'd expect if you're on an AMD platform you'll want to swap that first one out for AMD iommu equals on then to apply the grep configuration we'll use this command then we're going to add a few drivers to our knit Ram FS using draken I'll create a new file stored at Etsy drakeit.conf.d and name it local.conf then I'll add this content then to apply the configuration we use this command if you'd like a full explanation of why we needed to do all these things check out this timestamp in my other video the steps here are pretty similar and over there I give a more complete explanation of how Linux computers start up and configure themselves now let's reboot and check that everything is working first let's check that KVM kernel modules were installed you should have been activated on our first reboot after installing all that virtualization software to check this run LS mod and then pipe it into a grep search for KVM if these lines show up you should be good next let's check the d message logs to make sure that iommu was correctly enabled the command is D message and then pipe it into a case insensitive grep search for iommu and we can see here that it was enabled right there then I want to make sure those Linux command line arguments that we used correctly took effect to check that I can run cat proc command line and we can see the arguments that we gave included here then lastly we want to make sure our GPU is still gaming ready and using the NVIDIA drivers so I'm going to run lspci-nk again and as you can see here it's still using Nvidia I could rerun the benchmarking program but instead let's get to installing our Windows gaming VM okay so let's download a Windows 10 ISO from the internet and while we're downloading things I'm going to also download the vertio guest drivers both of these are linked in the description next we want to create our VM so let's go to vert manager find that Windows ISO we just downloaded and say yes if it warns us about paths and permissions pick an amount of CPU and RAM for the ram I recommend using half your systems Ram but this is easy to change later so I'll just use eight gigabytes because that's plenty for the CPU let's just put one because we're going to need to edit this later then we'll set the size for our virtual hard drive I'll give it 200 gigabytes and give it a descriptive name and then check this box so that we can do the advanced settings first let's double check that our VM is using KVM that's going to be important for our ability to get good performance next we'll set the firmware to UEFI and chipset to Q35 now it's time to set our CPU topology my computer has one CPU with eight cores and no hyper fitting but I don't want to use all of those cores in our VM so I'll just allocate six if you have a different CPU you might need to tweak these settings to get Optimal Performance but you don't have to worry because you can always change these things later next I'm going to remove my virtual network interface card this prevents the VM from connecting to the internet which is going to allow me to create a Windows VM that isn't connected to a Microsoft account then I'm going to set the install disk to bootable and make sure it's at the top of the bootloader that's going to make it a little easier to install the OS because it will boot straight to the windows install disk then we can click begin and start our VM for the first time I'm going to allow it to capture my mouse and keyboard and then press any key to start the install disk then we just need to run through the process of installing Windows you can click this button if you don't have a product key or you don't have one right now and then eventually it will reboot and it will ask you a bunch of questions I usually turn off all the features that have to do with virtual assistant and sending information back to Microsoft then when it's done you can turn off the VM and we're going to change some settings to make make sure our VM is ready for gaming level performance let's add a new virtual hard drive you can make this one really small because we're not going to use it for much and set its type to vertio we're adding this so that windows can install the vertio drivers so that on our next boot our main hard drive will be able to use vertio for its performance benefits then let's add the vertio iso we downloaded earlier this has the drivers on it and the VM will see it as a CD and then start the VM next use the vert IO CD we added to install the vertio drivers you can check this worked by looking at your tiny vertio Drive in device manager now you're ready to power down your VM now we can remove that tiny vertio drive because we just needed it for help installing those drivers and you can remove the vertio driver disk too next we'll need to change our virtual disk's bus type to vertio for the performance benefits in the main vert manager window go to edit preferences and check enable XML editing then go back to the virtual Hardware settings and on our SATA drive change to the XML tab change the target bus to vert IO and change the address type to PCI then click apply then we'll add the network interface card back because we'd like our VM to be able to access the internet I can do another quick reboot to make sure everything's working and now we'll get into this specific single GPU stuff because here's where it gets interesting okay so we want to switch our display type to VNC and I want to make it available to all interfaces this means that we'll be able to connect to our virtual display of our VM from another computer that's on our local network this is going to be helpful because we're going to need to kill the graphics on our host but first let's start the VM and check again that the display is working and that it's a VNC type and that everything looks good the next thing we're going to want to do is connect to the VM using another computer on our local network for me this just means that both computers are plugged into the same home Wi-Fi but for you this could mean two computers that use ethernet cables to the same router or it could even mean computers cabled directly to each other but the next thing we need to do is figure out our local IP address so we can remotely connect to the VM I'll run the command IP adder and you can see my network IP addresses 192.168.0.132 this is the address I'll use to remotely connect the VNC for my VM which is the VNC server we just configured and tested I'm going to use a VNC client running on another computer and that client is called tigervnc but there are plenty of options and depending on what you have access to you can even download a VNC client on your smartphone and as long as you're on the same Wi-Fi you should be able to connect I'm using an older computer with this awkward squarish computer and it's running OBS in the background but I'll just need to start tigervnc and then type in that IP address and boom I'm using the VMS display remotely on another computer like I said this is going to be really helpful because we're going to need to start killing the host graphics on our host computer when we disconnect the GPU and this will allow us to still connect to the guest but don't worry we're going to get to all of that next we need to go back to our main computer that is the virtual machine host and we're going to set up some hook scripts these are scripts that automatically run whenever we start or stop our VM I grab the instructions from this GitHub repo and this article on the pass-through post both of those are linked in the description first we're going to copy and paste these commands from the pass-through posts hook helper page start with the make dirt command and continue through the one that restarts liver D this downloads and installs the hook helper from pass-through posts vfio tools this will allow our VM to automatically trigger scripts that are in certain directories where the name matches the VM and the event that's happening then we want to create a folder for the begin script and end script I'll use maker Dash p which creates a directory and any relevant parent directories that also need to be created then I'll copy the two starter scripts from jonox tutorial to new files in those relevant directories and don't forget to run chmod plus X on these new files to make sure they're executable then next we're going to need to customize those scripts and start testing them out first I am using gdm so I'm going to uncomment this line then we're going to need to change the GPU PCI addresses to match those that are on my PC here we have ocoo through oc001 I'm going to run the same trusty lspci command to see that my GPU has four devices and the addresses are 01000 through 01004 so here I'm going to edit the text to have the correct addresses so the start script will unbind my devices and so that way the GPU will be ready to pass into the VM then I'll make the same edit to the stop script that runs after the amp shuts down and I'll add the PCI addresses up to the top here in the opposite order these will rebind the GPU and allow me to play video games on Linux when I'm done using the windows VM the next thing we'll need to do is test out these scripts these scripts are going to disconnect our display so the only way to test them is from another computer this time we're not going to be using VNC but we'll use SSH so on my computer I'll run the command systemctl start sshd this starts an SSH server Daemon this allows me to log into this computer remotely and run terminal commands thus it gives me the ability to test these scripts even when I'm killing the display okay so let's log in using SSH from my other computer this is the same as the VNC step in that there are many ways to connect to SSH I like using the terminal on a Linux computer but you could also install putty on a Windows computer or you could even use a terminal app on your Android phone like turmox so I'll log in with my username IP address and password and then I want to trigger the begin script and I will make sure it executes completely without error codes and correctly disconnects my GPU as you can see here it did Kill the display on the computer but it messes up and the command hung here because it can't unbind the GPU because it's still in use so I'll hit Ctrl C to stop that script and start working on fixing those problems I want to make adjustments to make sure that nothing is using the GPU by the point where we're trying to unbind it I can run this command to see that the NVIDIA drivers are still using the GPU I gotta stop these drivers because they're using the GPU and lucky for me there aren't any programs using the drivers so you can see here there are two that are using the Nvidia driver and this one is using the Nvidia set driver and because there's a zero on the line here and here it means nothing's using the Nvidia DRM driver and the Nvidia UVM driver if another program was using these drivers I might need to find out what that program is so that I can kill it as part of the VM start script but we don't so I just need to add terminal commands to disconnect these drivers to my start script I'll also have to do them in the correct order so that way they're disconnecting something that's already unused so I'll write this script to to remove the Nvidia DRM driver and the Nvidia UVM driver and then once both of those are removed we can remove the Nvidia mode set driver and then we're good to remove the base and video driver after that I'll run the script and you can see it completed without error then let's check the end script that runs when the VM stops we'll want to make sure it's able to reattach the GPU and restart our display environment and it looks to be working this is all you need for the VM to be able to get the GPU but only while it's running then once these are all working reliably we are ready to run our VM connected to these scripts and using our GPU so I'll start up for manager open the VM and I want it to be GPU accelerated so I'll add the PCI devices for the graphics card which will be disconnected by the time the VM starts up and then I'll click play the display will cut out and that's fine our VM should be starting but we can't see anything because we haven't installed any GPU drivers on Windows yet and that's why we switched our VM to use VNC and tested the connection earlier on my other computer I'll start up tigervnz and connect the IP address and boom we're connected to the virtual display and the GPU is attached to the VM so I'll just need to browse the internet find the NVIDIA drivers and install them by running the installer and as that's running I can see the display come back and it's showing the inside of the VM we now have a GPU accelerated VM now I just need my mouse and keyboard to be shared with the vm2 and so that's the last thing we'll do here I'll turn off the VM and the display should switch back to Linux where I can log in and edit my VM settings I'll add my USB mouse and keyboard using USB device assignment and we're ready to start the VM again now the display switches back to my windows VM and I'm ready to download a benchmark program and give it a run foreign [Music] foreign [Music] [Music] and we get a pretty impressive score this is likely because the benchmarking program is probably optimized for DirectX instead of opengl which is what it uses when we were running it earlier on Linux and that's it that's all we need for a single GPU setup I can turn off the VM and the GPU will get passed back to the host and we can re-run our benchmarks using the GPU on Linux let me know if you have comments or questions or Corrections if there are ways I can make this tutorial better or if there's anything else you'd like to see in the world of Linux gaming free software or virtualization thanks for watching and don't forget to stay Bland wow I just finished editing and I'm realizing that this tutorial is a lot of work so I'm going to do two things to help you out the first is that I've publicly opened up the blind man Studios Discord channel so there should be a server invite down in the description so if they're if you're running into a problem that's too difficult to debug down in the comments you might want to check that out because maybe there's somebody over there who's run into the same problem or has the answer to your question the second thing I've done is that I've started a blame man studios online store I haven't settled on what I want the URL to be so I'm just going to put a link down in the description too um but basically I've been wondering for a while now if it would be helpful for people to be able to just buy either pre-built or custom computers with all of this vfio stuff just configured by default and working out of the box um so I've set that up uh the prices are going to be high at first because I everything is going to be custom and I want to be able to offer individ individualized support with every product but if that's something you're interested in definitely check that out and give me your feedback down below yeah that's that's all I've got to say so uh thanks for watching

Channel: BlandManStudios

Views: 93,494

Keywords: bland, man, studios, blandmanstudios, stuidos, blandman, kvm, vfio, virtual machine, vm, gaming, linux, opensource, qemu, libre, libvirty, libvirt, kernel, operating, system, operating system, native, windows, windows 10, fedora, pcie, disk, guide, tutorial, part, zero to hero, hardware, and, demonstration, considerations, demo, build, pc, cpu, gpu, guest, host, usb, flash, drive, vt-d, VT-d, intel, amd, nvidia, iommu, passthrough, VFIO, Linux, graphics, card, looking, glass, halo, infinite, Glass, reboot

Id: eTWf5D092VY

Length: 23min 42sec (1422 seconds)

Published: Thu Oct 06 2022