My Proxmox Home Server ... (GPU Passthrough, IOMMU Groups and more)

having your own virtualization server is fun you can host your own personal cloud for safe storage set up a console or gaming PC or use it as your ultimate Powerhouse that is available on any device in today's video we're going to cover all of that from setting up the host server with the open source kvm-based virtualization platform proxmox to passing pcie devices like a graphics or capture card through to your VMS including some details about IO mmu groups to stream your games or applications to other devices with as little latency as possible but before we do that don't forget to give this video a like and if you're new to the channel and like videos about open source and Linux then you'll also consider subscribing thanks already in advance alright let's start off with what is proxmox like mentioned earlier proxmox or in our case more accurate proxmox virtual environment is a free and open source virtualization platform similar to VMware and hyper-ve it features a bare metal type 1 KVM hypervisor which was built with a stripped down version of Debian and the feature-rich graphical web-based user interface for configuring virtual machines and lxd containers like other virtualization Solutions you can create snapshots of virtual machines set up storage pools virtual networks and much much more but that's enough for now let's talk about on what we'll cover in this video first we'll start with the initial setup how you can access the web interface and how you create users and safer authentication methods down to how you can create virtual machines including everything from UEFI settings to pcie and USB passthrough essentially everything you need for your personal server then installation of proxmox can be done in two ways you can download their hypervisor as a standard ISO from the website or install it on top of Debian the latter one is really good for a more exotic setups with a lot of custom partitioning and everything but for most people and even businesses you can just download the eyes oh flash it onto a USB drive or whatever else you want to boot off and head into your server's ufi in my case I used regular consumer parts and there are a couple of things that we need to set in order for it to work properly the first thing that we want to do is to switch to the advanced settings then head over to Windows OS configuration and deactivate secure boot since the proxmox iso does not support it since this is only a virtualization host and not a personal Linux desktop this is a completely fine thing to do next we want to head to the overclock settings or wherever else you find the virtualization settings for your CPU on IMD this is usually called svm and on Intel it's called Intel virtualization technology on Intel you also want to check vt--d and on AMD try to find a setting called iommu and set it to enabled on some motherboards it's hidden under the North Bridge settings with these settings are for will be covered later on in the video last but not least we shouldn't forget to change the put order to our proxmox boot device save and quit and start with the installation the first thing that we need to do is to choose our operating drive to install proxmox on and hit next now we select the country or local time zone and the keyboard layout and also enter a root password which we need later for administration for important notifications about your host should also enter a valid email now if you have several network adapters then you need to choose one which will be used for updates and general connectivity to that outside world since I only have one I'll just use that one next you need to enter a so-called fully qualified domain name AKA a hostname DOT something and a static IP address if you have access to your router then it's a good idea to log into its interface and check if you have any static IP addresses available so that you don't end up with double IP addresses that wouldn't be all that great the default gateway and DNS server should get entered automatically unless you're not connected to your network make sure to check your entire data and finish the installation after the reboot you should see something like this now we can just open up a web browser on any device on your network and connect to this IP address with the port 8006 use root for the username and enter the password that you said earlier make also sure to set Linux spam standard authentication instead of proxmox ve but more on this later before we continue I urge you to try out the menu for a bit to get a feeling on how the proxmox interface works on the left side we have our resources including our host networks VMS containers and storage if you don't like this view then you can change it to folders or pools I'm personally more of a folder fan so I'll go with that in the column on the right we have our settings for whatever we've selected on the left if we switch from our data center to our host then the settings change accordingly for example with our host selected we can access its command line and interact with it let's head down to updates repositories if you didn't get one of their subscription models then it's a good idea to deactivate the Enterprise repositories and add the non-subscription one in order to get updates okay let's move on to storage with disks you see all the disks in your server don't be scared if the lvm partition shows that your operating system takes up almost all of the disk space because that is actually not true it's just the reserved space I'm going to skip straight to CFS since I want to use three drives for a storage pool for redundancy with one parody disk I'll choose raid set one Let's Wait a tiny bit and our storage pool is already ready now before we move any further I think now is the best time to secure a root user with a multi-factor authentication select the data center expand permissions and head on over to two factor in here we can add several multi-factor authentication methods for most totp is probably the way to go and there are some random description and scan the QR code with an authenticator app of your choice enter the given pin and save it next head over to Realms edit Linux Pam and choose this option as required and now every time you try to log in you get asked for your second Factor now you might have already asked yourself what is the difference between these two Realms now the first realm are actual Linux users on the hosts themselves with root for example we can interact with the host shell while a VI user cannot so you can essentially create users which cannot interact with your home and that's pretty neat but it also means that if you want to create a Linux spam user and you need to First create it on the host and then at the enemy proxbox later okay so now the spicy part let's create a VM the first thing that we need are some isos you can upload isos to proxmox by selecting your local storage go to ISO images and upload them from your PC if you want to create a container instead of a VM you can download templates Straight From Within proxmox pretty cool right anyway click on create VM choose a name and hit next now you can select the iso and the reference I'm deliberately choosing Windows since there are a few more things to consider with it by default most templates already set up everything you need and all you have to do for Windows is to select a place for the secure boot configuration and TPM chip and hit next on Linux VMS I also recommend you to already check qemu agent which is a program that allows your host to gather some data like IP addresses from your VM next we choose our operating system disk size the amount of assigned CPU cores and memory and of course our network card after the setup we can now head over to console and start or VM as you can see you can control your virtual machine straight from here some things like copy and paste doesn't work of course but for quick maintenance or setting up the amps it's usually enough now if you want to install Windows and you didn't follow my tutorial so far and selected the windows template then you might realize that there is no disk available for the installation the reason for that is because Windows doesn't come with the necessary word IO drivers which are needed to see the disk luckily we can download them with help from proxmox add a second CD-ROM drive to our powered off VM and upload the iso like before now we can load the driver follow proxmox guide and install the necessary dependencies remember how we added the second CD-ROM drive in the hardware tab you can also use that to pass through a graphics card or whatever else you might want to pass through click on ADD and PCI device as you can see with proxmox8 we have two options here we can add so-called map devices or pass through a raw one raw devices are all of the devices that are available on your system but only permitted Linux users can add them proxmox ve users cannot if you want to allow your proxmox ve users to add pcie devices as well you can map them into groups open your data center go all the way down to Resource mappings click on ADD and enter a custom name I for example have chosen GTX 1080 for my Nvidia GP View and edit the necessary IDs and IO mmu groups oh IO mmu groups remember that you've already seen them in the UEFI and also in the Raw input settings so what's up with them IO mmu groups are the smallest amount of devices that can be passed through to a virtual machine for example many motherboards including my own often group together several devices into one huge iommu group therefore if I wanted to pass through one device of that group I would have to pass through all of them however this is often not that easy as an example my Elgato cam link Pro used to be in a bigger IO mmu group together with some USB chipsets easy eye slots but also my Theta controller that means that whenever I wanted to pass through my cam link it tried to pass through the whole group including set SATA controller and it crashed by VM as well as the host itself not good now some main boards especially higher priced or I'm gonna say tier ones usually have several UEFI options for that one of them would be of course the io mmu groups but this option is available for most anyway however the more expensive ones also offer ACS overrides to split the io mmu groups even further if your main board has issues with splitting up iommu groups and you can't seem to find any more settings in your UEFI then you're in the same boat as I was luckily for us proxmox already comes with a kernel patch that allows us to further divide IO mmu groups all you have to do is to log in with root head into your host shell and open slash Etc default slash crop with Nano and add this line to the CMD line save your changes with Control Plus o hit enter and quit Nano with control X Type update Dash crop and reboot your system now you should be able to select pcie devices which were previously in a different group however this method is not without risks IO mmu groups don't exist to annoy you they exist because very often chipsets pcae Lanes or other controllers often share certain parts like memory for example these devices are being grouped together because they cannot be isolated from one another meaning that if you are to pass through a device that was initially in such a group then chances are that it can still read and write into the shared memory this is a huge security risk which is known as a VM breakout since an attacker can use the shared memory to access other VMS or in the worst case your host system IO mmu groups are therefore only safe if you pass them through as a whole as they are reported by your UEFI since only then can they be isolated but hey don't let me ruin that for you if you know what the device devices that you want to pass through you can access you can use the kernel patch without any worries plus attacks like that are usually very rare and you would also need to catch that malware in the first place for a home lab no worries really oh yeah and one more thing make sure to always pass through all devices with a certain ID these actually belong together for raw PCI you can take the checkbox all functions on one device and it will automatically assign all with similar IDs for you USB devices are really easy as well and after that you can just start to install all the necessary drivers from within your VM if you want to use an HDMI or DisplayPort cable on your pass-through GPU you can also take primary GPU however don't do this if you want a so-called headless system or otherwise the console VNC session might not work anymore you know what I think I'll cover the low latency remote desktop sessions in another video so definitely make sure that you're subscribed to the channel and while you add it why don't you also give this video a like as well right here you can already watch the next video and all that's left to say now is good morning good afternoon or good evening wherever you are I'll see you around foreign [Music]

Channel: Michael Horn

Views: 30,243

Keywords: gpu passthrough, gpu pass through, proxmox, proxmox install, proxmox ve installation and configuration, proxmox ve 8, proxmox gpu passthrough, proxmox iommu, proxmox iommu enable, iommu groups proxmox, iommu groups, linux acs kernel patch, host crash gpu passthrough, proxmox tutorial, proxmox tutorial 2023, open source, uefi virtualization, qemu gpu passthrough, kvm gpu passthrough, homelab, home server, bios virualization

Id: TWX3iWcka_0

Length: 15min 1sec (901 seconds)

Published: Tue Aug 15 2023