Double GPU Passthrough in Proxmox 8!? Play Baldur's Gate 3 and Minecraft On The Same Machine?

hey guys and welcome back today we're going to be doing Hardware pass through so we can do something like this Boulder's Gate in a VM or how about Baldur's Gate and Minecraft on the same PC at the same time all of that stuff is possible as well as doing things like Plex transcoding jellyfin video converter maybe in a Docker VM that also runs on the same machine that you're also gaming on and is running proxmox so how do we do this what's it all about how do I make it work stay tuned because that's what we're going to go through today so what you're seeing here is what's known as pass-through in the example behind me I'm doing GPU passthrough but I'm also doing keyboard mouse and Bluetooth so that I can listen to the audio from that virtual machine and I have input through a keyboard and mouse that means I can play the game as though it was a regular computer and any spare compute that's left over I could spin up a Docker VM and I could run a number of contain this out of it so hopefully that should give you a hint as to why you might want to do this I originally went down this route when I had my two sons and they wanted to play Minecraft now I didn't want to have to go out and shell for a new pc I have an old gaming PC and I'll talk you through the specs of what I have here running in the background in a moment but suffice to say it's six years old it's not expensive I don't recommend you go and buy it but if you could find one for dirt cheap why not it works but this allowed me to give one of my sons a GPU which I bought just a low budget Intel GPU and the integrated GPU as well I could split those up into separate machines so that they could game at the same time so in this video I'll give you step-by-step Guides of how to do this I'll be following the proxmox documentation but sometimes it's good to see somebody actually implementing that and going through the steps so that you can repeat them and before that I'll give you an overview of what I'm actually running this on so looking at my machine don't be off put that it's in a server case I just happened to have it in a 4u server case because I've got a rack and it makes sense to consolidate so in here is your bog standard ATX motherboard it's an MSI z170a so that gives you a sense of its age there's a 6700k processor in here it's got a knock to a heat sink it's got 32 gigs of RAM and a 750 watt power supply and probably the most interesting thing about this is the Intel Arc A380 GPU in here now I know it doesn't have rebar support so don't break me in the comments for that but it was 120 quid when I bought it and it came with a 80 quid game which I sold and recouped the cost of so I thought it was a pretty smart deal now I've never actually tried this build with proxbox 8 and I was quite excited by that because proxmox8 released with a new kernel which supports Intel aux by default now I didn't know if that was going to have any impact but as you can see I have this up and running so now that we've gone through my machine how do we get this working well the first thing you're going to need is a hypervisor obviously and as you've seen in my videos I'm using proxmox now GPU passthrough or passthrough in general is not exclusive to proxmox and basically is a key feature of many hypervisors I just know that proxmox typically is more favorable especially when it comes to Consumer devices because typically there's just a better chance it'll work anyway and proxmox tends to be a bit more hackable I.E it's open source and you can tweak things to your liking then some others for example that are closed Source like esxi now there's basically three major steps that we need in this process the first is to make sure that our host has all of the virtualizations setting turned on you'll have to do that first in the Bios as I show on screen and in my example here you can see that it's vtd and virtualization both of those need to be enabled once those are enabled you basically then need to enable IO mmu groups within proxmox and add it to the grub file don't worry I'm going to show you how to do all of this what that's basically doing is if you imagine your motherboard a bit like a smorgasbord all of the individual components on that motherboard can be split up give or take so when you run a command it will show you which groups each device sits in now you can pass through groups to Virtual machines now the caveat here and this is really important is if two devices sit within one group and you pass that group through both devices will go through that seems pretty obvious but you can often run into pitfalls where say for example you wanted to pass through your nvme controller you might have only one device in there and proxmox is using that controller so when you try to pass that through it either errors or it crashes proxmox so you kind of get my gist what we need to do is make sure that the devices we want to pass through are in their own iommu group and that makes life a whole lot easier if that isn't possible there is kind of a hackable way to do that which I'll show and it's actually referred to in the documentation but it's far less reliable and it's more likely to crash and it also breaks some of the security controls that we have in place to segment devices the next phase after that is pretty straightforward and should be much simpler to understand we basically need to Blacklist the drivers in proxmox so we're going to say hey proxmox don't load these drivers because we're going to steal that device from you and give it to a virtual machine the final bit is then basically to Blacklist those devices so we find the ID of the device that we want to pass through and we Blacklist that for vfio and we just add that simply into a configuration file and reboot the host so those are pretty much the three high level steps so let's jump into actually doing this now in real time so I'm going to assume that you've installed proxmox and set that up and whilst you're doing that you might as well download something like Windows 11 which is going to make this much easier to diagnose but there's no reason that you can't pass through Hardware to basically any operating system so if you wear our sunglasses and head over to the proxmox website you can see here all of the instructions for how to do this but I actually think there's a few important steps that are breezed over or just assumed for the user so keep this in mind as a reference set and come back if you need to but I'm going to give you all of the steps that are necessary on my GitHub site so you can just follow those so let's go through how I did this and I'll explain all of the sort of baby steps so now hopping over onto the server where Baldur's Gate was running I'm going to assume that you've set up proxmox fine I'm going to show you how I ended up with what you're seeing here which is that Windows 11 machine running again so as you can see on here it's just my 6700k and if you click on the Windows 11 you can see that it's got six of those sockets so it's a six socket eight core that means I basically want one chord left because you want to keep at least one reserved for proxmox now I could probably dial this in a bit certainly if I was playing something like Minecraft which isn't a CPU attensive now the key things of note here are it's a Q35 machine that's not the default but when you create a Windows machine it should default to this but just in case it doesn't do put it on here because pcie passthrough is only supported officially on Q35 machines also on here when you create a Q35 and a Windows 11 machine you're going to need to emulate the TPM that's the trusted platform module and that's typically a hardware device that's built into your motherboard and you've probably seen this using things like BitLocker for Drive encryption thankfully those are configured really simply in the create VM wizard next you're going to see some USB devices that are passed through and in the example I showed this was my keyboard and mouse now thankfully we don't need to do any jiggery pokery in the command line for this you could if you wanted but USB devices are generally straightforward you can do that by clicking on ADD and USB device and then you want to pass through a port that typically has the fewest issues here you can see that the only devices on the host are my Razer Mouse and my Microsoft keyboard which I've already passed through the key bit to this video and why you're probably all tuned in is the PCI devices that you see at the bottom now you'll see two and I've only mentioned a GPU being passed through that's because most gpus are actually a GPU for the graphics and a sound card as well so for this example I've had to pass through both of those so that I can get graphics and sound in my VM and probably the interesting bit here is the numbers so these are important because these are the device IDs on our host machine and we'll be getting onto that just a minute when we jump into the command line so let's go and do that so jumping into our proxmox shell you can do that just through the GUI or you could run an SSH terminal in the Shell the first thing we're going to want to do is to turn io mmu on now to do that we simply need to edit the grub file this is the bootloader that proxmox uses and it's part of Linux there is systemd as well so there are separate instructions for that but the default proxbox installation will use the grub bootloader so to edit the bootloader it's simply your text editor in this case I'm using Nano we want to go to Etsy default and Grub and when we go into there we can see that I've added in the default the Linux default area I've put Intel underscore io mmu on now apparently since kernel version 5.15 this isn't required it should be on by default but I put it in just in case because there's no downsides to specifying it the key bit we actually need is IO mmu equals and then PT PT standing for passthrough do note that if you're running an AMD CPU you can swap out the Intel underscore IO mmu to AMD underscore IO mmu and that's all you need to add in here to enable i o mmu so once you've added this simply save and exit the file control load save control X to exit and once you've updated the grub file you need to run the update Grub command and once you've done that you need to reboot proxmox because now it will turn IO mmu groups on and it will enable pass through so once proxmox has rebooted head back into the terminal and I want you to paste this command so this is going to validate the i o mmu is now enabled so running that you can see that on the third line IO mmu is enabled Bingo that's great that means that we've set up the BIOS correctly and that our devices support iommu the next thing we need to do is to actually enable a few modules within proxmox that's going to facilitate the use of device passthrough now for this we need to use vfio which is the kernel module that's required to enable us to pass through devices now again this is a super simple edit copy paste job so let's jump into that now so we want to Nano into EDC modules and once you're in ETC modules you want to paste the following modules we want vfio vfio IM OU type 1 vfo PCI and vfo virq FD once you've added those again Ctrl o to save control X to exit once you've added those again you need to update the init Ram fs and reboot once you've rebooted those modules will be loaded by the kernel and we've pretty much done all the configuration necessary to enable us to do pass-through so now that you're rebooted hop back into the terminal and run the following command this is just going to validate that the modules we specified are enabled and are working properly so when you run this command you're pretty much looking for that first line The Meta driver version in this case 0.3 so that shows that this is loaded and it's running so now we're going to get back to a bit that I mentioned in the proxmox view you know where I showed you the PCI device and it had an identification number that's what we need to find because when you first do this you won't know the ID of the device that you're trying to pass through so to do that it's super simple we do lspci LS being list much like we've done four folders Dash NN and that's going to show all of the devices by name so running that online machine and this will be different for all the machines that you run it on so don't expect it to look exactly like mine the key thing we're looking for here as I said are the groupings and you can see those in the First Column now it's quite typical that some of the lower order groupings I.E things like imou group one have lots of things in that are related to the CPU and the bus Etc and you're pretty much guaranteed to not be able to pass that through because it's going to be used by The Host but thankfully on my motherboard you can see that the groupings are quite favorable so I can pass through O3 which is my GPU 04 which is the audio device attached to the GPU and if I wanted to I could actually pass through my entire USB hub so I could pass through all of the USB devices into this virtual machine so this would actually negate you know where I clicked on ADD and then a USB device if I pass through the entire USB controller anything I plugged into the host would automatically be given to the VM so it would be very much like a physical machine where anything you plug in it becomes true Plug and Play I don't recommend you do that because you probably want to be quite specific with the devices you pass through but I don't know maybe there's an edge case where that's useful the other really cool thing here is if you look at number two not that one the integrated GPU I.E the GPU that is on your Intel chip sorry AMD guys you won't get that on some of the older ones I can actually pass this through as well so I mentioned before I've got one GPU the discrete one the Intel Arc that's going to run Baldur's Gate and I've got one eye GPU left which yeah they're not great but for things like Minecraft it's going to be fine or for things like a Docker VM and transcoding it's more than sufficient to transcode a number of 4K streams in parallel and before anyone tries to challenge me on that I've done five 4K streams in parallel on an HD 530 it works so now that we know the IDS there's a separate command that we need to run to get the underlying vendor ID so that we can Blacklist it and as I said this will instruct proxmox to say hey I don't want to load this device I want to leave this out and that's the way that it leaves it available for a VM to pick it up so how do we do that well thankfully super simple and we take the following command we delete one of the ends and then we do a dash s and that's for specifying the ID we want so in this case I'm going to specify 0 3 0 0 which if we look on here is my orc A380 so hitting return on that you can see that it comes back with a not very friendly number 8086 colon five six A5 that's the bit that we need to remember so as you're going through this and it's in the Shell why don't you just copy and paste this into a notepad and you would repeat this process for any of the devices that you want to pass through bearing in mind they're in a separate IO mmu group or with the understanding that if they're not when you try and pass through anything from the i o mmu group it's going to pass the whole thing through or it might actually break things as a result so your mileage may vary but you should be good if it's in a separate IO mmu group and it's not being directly used by the host machine so in my instance I went ahead and I did this for number four as well which is the audio card and you can see it there it has the same first set of numbers but a different set after the colon and finally I wanted to pass through my igpu which in this case is number two and the eagle idea among you might actually notice that it's this number here at the end of the device name anyway so you don't have to run these separate commands if you want but I find it quite useful to separate them out so now that we've got those numbers what do we do with them well as I said we want to Blacklist them so to do that we simply take those numbers and we add them to a file so to pass through my GPU my discrete one it would look something like this so This command will Echo it will write into the file options vfo PCI the ID so those are the bits that we just found and you would add as many as you wanted in here each one separated by a comma there's no spacers just to be sure and we also add disable VGA equals true and it's going to be saved into the vfio comp which if you remember that's one of the kernel modules we enabled so you wouldn't be able to do this if you hadn't enabled the kernel module so once you've done that hit return that's going to write it into that file obviously you could Nano into that file and just manually copy and paste in there if you wanted but this command is much simpler so with those devices specifically specified to be isolated by the host that's what that command does there's one last thing that we need to do and that's driver blacklisting so this means that the host won't even load these drivers so in my instance I'm using an Intel GPU for both the integrated because it's on an Intel CPU and the discrete graphics card because it's an Intel Arc so for me I only have to run one set of commands it's simply the i-915 driver which is the Intel driver in Linux so if I run this command again it's going to Echo that command it's going to write it into the blacklist.com file and you would add all of the Linux drivers here for your device handily all of those are available so amds and Nvidia particularly are much more popular so you can see on screen that it's just as simple as copying and pasting those into your blacklist.com file so once you've added those naturally you're going to need to reboot your host because it needs to boot up with those devices isolated and without those drivers loaded now if you're doing this and you have your HDMI cable stuck into your machine and you're currently using the igpu you're gonna lose it because it won't load it so make sure that you are going through the web GUI for proxmox otherwise you might think that something's wrong but don't worry the fact that it isn't displaying an output is a good sign because it means that you've isolated the hardware and it should fingers crossed be ready to be passed through to a virtual machine so I already set up a virtual machine earlier that was running my discrete GPU and I showed you that with Boulder's game so I'll stick to my promise I'm going to spin up another Windows machine this time I'm going to pass through the integrated GPU so hopefully then we'll have one VM with the discrete GPU and one VM with the igpu which will give us a myriad of use cases so to create the VM it's pretty straightforward if you've been following any of my videos you should know how to do this already so create a VM I'm going to call this one igpu I'm going to hit next I've already downloaded the windows 11 ISO file but again you could use anything you want if you're unfamiliar with proxbox and downloading do go check out my proximox installation video it will go through all of this in much more detail the important bit is to change the guest type so Microsoft Windows and this should be the bit that turns it to a Q35 Machine by default so yeah you can see that it's created a Q35 machine so nothing really fancy here you can choose where you want to store the EFI and the TPM module as I mentioned earlier we're going to hit next this is totally up to you in terms of how much disk size you want to give it I'm just going to give it 32 gigs because it's just going to be to demonstrate that I've got the igpu pass through and there's some acceleration going on so hit next now I don't have much left in the tank in this machine I've given six cores to Boulder's Gate and I've only got two cores left one of which I need to keep for proxmox just to make sure that it has enough horsepower for the overheads and running the high provider itself not sure how this is going to install with one core if it fails I'll probably bump it up to two and I'll edit that out of the video but the key thing here is we get the VM up and running it doesn't really matter what resources we give it one thing I usually change on here is to put it as the host this gives it all of the CPU capabilities and it makes sure that it's represented properly within the task manager so it will show 6700k Etc memory I'm just going to give it four gigs that'll be fine for this the network I'm going to leave the same this is just a default setup running on a bridge but if you're integrating this into an existing setup make sure that you choose your network adapter and perhaps it's on its own VLAN for security Etc I'm going to confirm that bearing in mind we haven't done any pass-through at this stage so to do the pass-through it's really straightforward we click on ADD we want a PCI device now back in proxmox 7 it would already show what is now called The Raw device so click raw device on the drop down we want to select the device that you want to pass through so I've already passed through my arc A380 so I can't pass that through in this case I'm going to pass through the integrated HD 530 which is on the CPU so I will click all functions and I will click add now hopefully that's all that's required to boot this VM start the installation process and hopefully be able to see the HD 530 picked up within the device manager so I'm going to go ahead and install this I'll edit the installation process out and I'll see you on the other side so now I'm on the other side of the Windows installation and it's safe to say that you can't install Windows 11 Pro with one core I end up having to use four but due to virtualization that's just going to divide up the virtual cause anyway so it should work it's just if they're both hammering the CPU they won't get the same performance as before but the important thing here is that we've got the Intel HD graphics 530 within the machine it's recognized there's no dreaded error 43 which has been the bane of GPU pass through for years thankfully that's not here so for all intents and purposes we now have a machine that has GPU acceleration now I was hoping to be able to show you some output through that HDMI port that's linked to the hd530 but to be honest in some ways there's something to gain from this failure so whilst I can't get an output to a monitor I still can use the hd530 within a virtual machine and that really is kind of a testament to PCI pass-through it's always been sort of experimental and the results will vary between Hardware now there's a couple of things here this is six or seven generations old Intel Hardware so you may have a much better experience with newer hardware and I exhausted all the options I could find but there may be something existing that will fix this I just don't know about it so I will keep persevering but if we just take everything on reflection I still have a discrete GPU that's passed through to a virtual machine and that's allowing me to game and I still have the hd530 available to Virtual machines so I can still use it for things like Plex and jellyfin transcoding and I could technically game on it if I use something like Moonlight or parsec so I could run the game on that virtual machine and I could stream it to another system so I hope this gave you a Whistle Stop tour of how to do pcie passthrough and hopefully you'll be able to follow this process and get everything up and running and who knows maybe you can do the igpu and the discrete GPU I hope so because that's a pretty awesome setup let me know how you get on in the comments below and as always if you like this video please like And subscribe and I'll see you on the next one take care everybody [Music] thank you thank you

Channel: Jim's Garage

Views: 20,704

Keywords: baldur's gate 3, baldurs gate 3, virtual machine, proxmox gpu passthrough, gpu passthrough, virtual machine gaming, proxmox gpu, linux, proxmox, minecraft, minecraft virtual machine, docker, gaming, baldur's gate 3 in a virtual machine, vfio, proxmox passthrough, proxmox usb, gaming vm, gaming in a vm, parsec, intel, intel arc, proxmox 8, how to pass a gpu in proxmox, plex, jellyfin, plex transcode gpu, jellyfin transcode gpu

Id: i_2ZGcm4E_4

Length: 25min 55sec (1555 seconds)

Published: Mon Aug 14 2023