GeForce PCIe Passthrough Enabled! But what took so long???

uh hello old friend i missed you did you miss me too [Music] today's video is brought to you by the manscaped performance package which includes the lawnmower 3.0 along with everything else to take care of your grooming needs put down the disposable razor and get yourself one of these the lawnmower 3.0 is wireless rechargeable and ipx7 rated so you can safely trim the hedges in the shower plus with its skin safe technology you won't end up tilling instead of just trimming also in the package are the shears 2.0 for your nails crop preserver and toner sprays and the weed whacker ear and nose hair trimmer which i didn't imagine myself needing in my mid 30s go to manscapes.com craft computing to get 20 off free international shipping and 2 free gifts that's manscaped.com craft computing your balls will thank you welcome back to craft computing everyone as always i'm jeff and you should all recognize the pc next to me at this point this is my epic cloud gaming server and the original plan with this was to create a server that i could have multiple friends remote into and play games as if they were sitting in front of a local machine spoiler alert after 7 videos worth of tinkering on this thing it still didn't work all that well but i was already working on a part 8 to this video series using the trio of amd s 7150x2 cards that i came across in fact the system was working so well with that configuration that me and five of my patrons jumped on and had a crisis lan party on this server in parts one through six of this video series i was using nvidia graphics cards starting with their enterprise line with the tesla k10 and grid k2 and then moving on to the gtx 690s and then later on the tesla m60s now while i was able to get into games with the grid k2 inside of a virtual machine the lack of an nvinc encoder meant that the video encoding fell to the cpu which meant that the system was practically useless from a remote standpoint also there was a limitation of needing to use windows 8.1 with that driver set as no windows 10 driver existed for that particular configuration next up i tried the trio of gtx 690s that you see in the system right now however i was met with the classic nvidia code 43 as soon as i installed the driver inside of windows to this point nvidia has not allowed geforce cards to be passed through into a virtual machine because nvidia has reserved that for their enterprise or workstation class gpus even though a lot of people end up using geforce cards as workstation gpus i then attempted to play nvidia's game with a pair of tesla m60s however i was met with the same exact problem as i faced with the grid k2 that is if you use the nvidia grid driver and pass through a portion of the graphics card to a virtual machine there is no end vinc encoding on the card even though nvink is still present on the card and you're using pci express passthrough to get the grid card into the machine it disables envincing the driver for whatever reason at that point i waved my white flag and begged team red for asylum with a trio of s7150x2s with the help of sriov as i said in the meantime i have continued to tinker with the 7150x2s in my free time however earlier this week nvidia sent me a major curveball with one of their announcements on wednesday they said that geforce gpus were now able to be passed through to a virtual machine sans code 43 meaning that my gtx 690s might finally work over here on the download page for nvidia driver 465.89 they highlight some new features such as dirt 5's new ray tracing update the launch of evil genius 2 and the launch of kingdom hearts series on the epic game store however it's under the gaming technology that kind of caught my eye first off it includes resizable bar support for the rtx 30 series which is honestly a really cool thing to have if you can get one of those but there's a little footnote right here that includes beta support for virtualization on g-force gpus now let's be honest here nvidia this is not a beta this is not a we're trying the waters or testing things out this is a we finally decided to remove the block that we had arbitrarily on g-force cards that when they detected the presence of a virtual machine disabled the driver inside of windows and linux the whole issue of your consumer geforce gpus not being able to pass through to a virtual machine was an arbitrary limit to force people to buy quadro or tesla cards so they would end up spending more money on essentially the same technology and yes i know they spend more time on quadro drivers and yes i know they have ecc memory on certain models of cards but they're the same gpu cores there's nothing architecturally different so with all that out of the way what is the actual plan for today well i'm gonna fire up proxmox on this 32 core epic system with 256 gigabytes of ddr4 ecc registered memory and we're going to test out a gtx 690 inside of a virtual machine i'm also going to grab a couple other g-force cards and test them out as well because well this is a beta driver so i'd better test them the good news though if you wanted to use the same technology at home is nvidia is not limiting this to only new gpus which has been their mo in the past in fact they have support dating back to the 600 series kepler gpus like the gtx 690s going all the way up to the more recent 3000 series gpus so i think that's it for introductions let's go ahead and fire this thing up and get to work wait hold on i had something for this there we go that's better so here we are inside of proxmox and as you can see i've already been doing a little bit of work with this system first off i have four different virtual machines that i have created and for the gtx 690s i have already configured those for pci express passthrough i've also set up two other virtual machines one for the gtx 1080 and one for a gtx 1650 both of which are physically installed in this server but i have not configured yet for pcie passthrough so that's what we're going to be working on now now while nvidia has made some things easier on the driver side of things there are still a couple of prerequisites we need to handle before we actually do the pass-through first and foremost since the server has an integrated video output we can go ahead and disable any nvidia or amd graphics drivers so virtualization passthrough can actually work now on this server i have already done that but we're going to double check and make sure it's there anyway so i'm going to go nano and then etc mod probe d and then blacklist.conf inside of this file is a list of drivers that will no longer load inside of linux so we've got the radeon drivers nuvo drivers nvidia drivers and i also went ahead and threw the intel sound drivers in there because those were actually linking to my gtx 690 audio drivers and they shouldn't have been secondly we need to disable the video output on these cards just in case the driver blacklist didn't take care of everything or the video output started activating during the bios boot of this particular pc to do that i'm going to type in lspcid v for verbose and then we're going to only look at nvidia cards so nvidia in the grep and you can see device number eight right here is our gtx 690 device number nine is the second gtx 690 we have device 21 which is my gtx 1650 along with 21.1 for the audio driver and then 41 is my gtx 1080. now the gtx 690s have already been done previously on the server the last time i tried to pass them through but let's go ahead and take care of the gtx 1080 and 1650 right now so first up let's go ahead and look at device 21 so we'll do ls pci dash n dash s and then 21 colon zero zero so right here are the actual device ids that we're going to disable inside of linux and to do that there is this handy command right here now i know it's a little bit long i will have a link to the original reddit post that i'm taking this from down in the video description as well as the commands themselves but we need to change the device ids that are listed right here so 1 0 d e colon 1 f 8 2 then a comma 1 0 d e colon 1 0 f a and we're going to hit enter and now our gtx 1650 the vga output on it should be disabled and i'm going to do that exact same thing for the gtx 1080. so lspci dash n s and that is device 41. and hit enter once the device ids are changed and we are good to go once you have all the device ids fully disabled we need to type in update dash ini t r a m f s dash u this will force those device ids to be fully disabled after that you can go ahead and reboot your server and get to passing through your pci express devices so we're going to type in reboot and then cycle power really quick with the server rebooted let's go ahead and have some fun first off i'm going to go to vm 313 which is the one that i've assigned for the nvidia gtx 1080 i'm going to go to hardware and then click on add and go down to pci device on your device i'm going to click that pull down menu and go down to device 41 which should be my gp 104 gtx 1080. so click on that i'm going to click on all functions and then click on pci express as well for right now we are not going to click the primary gpu button we'll do that after we have the driver installed so go ahead and click on add and as you can see i have done nothing else to this virtual machine it is an ovmf bios with a q35 machine type i have not hidden the cpus or passed through a weird id i haven't added the gpu bios to pass through to trick the system into thinking that it booted up with the virtual machine this is just straight up pcie passthrough so let's open this thing up and click on start now that we are inside of windows and i've waited a couple of minutes so windows would actually download a gtx 1080 driver from its own repository let's go ahead and go and right click on the start menu and go to device manager we can see that it has installed a gtx 1080 but like usual we have the little yellow triangle right there and right now we're getting a your computer's system firmware does not include enough information to properly configure and use this device code 35 which is not one that i'm used to seeing but if we go to driver you can see that we downloaded a driver from september 30th of 2020 that's obviously not the most up-to-date so let's download the most up-to-date driver and see if this will work so we're gonna go nvidia.com drivers i'm gonna go to geforce drivers gtx 10 series gtx 1080 64-bit we'll do the game ready driver and search and we've got 465.89 release highlights includes beta support for virtualization on geforce gpus yes please download that let's install the nvidia graphics driver no thank you on the geforce experience i will pass on that custom just to make sure geforce experience isn't there and we'll click on next so here we are back on the virtual machine desktop after the reboot and you can see my mouse cursor is acting a little bit strangely and that's because the vnc console built into proxmox cannot display a physical output from a gpu because they are technically on two different graphics cards one of them is the virtual machine gpu the other is the physical gpu but if i navigate my mouse down to the start menu go up to device manager we can see that i have no errors under display adapters and in fact if i pull that down there's my gtx 680 with no code 43 who would have thought so now that we have the gtx 1080 working i'm going to do a couple of configuration things to make it actually play games or at least render some 3d visuals i'm going to do the same thing to the 1650 and hopefully both gpus in the gtx 690. for you it's going to be like 10 seconds of video time for me it's probably going to be about another half hour worth of work but hopefully when i snap my fingers we'll be back to a very happy jeff watching maybe like four instances of heaven running at the same time oh sorry wasn't expecting you uh we're back and uh i have some success to report that is we're about 50 successful i have not been able to get the gtx 690 to properly pass through and i think that's just due to some oddities of the gtx 690 and the plx chip on board itself but if you turn your attention to the screen right here this is my gtx 1650 running inside of a virtual machine at 2100 megahertz and running heaven at 80 frames per second on medium settings and low tessellation and not only that if i exit out of that one and go into my gtx 1080 machine this is also currently running heaven with no problems at all and on the gtx 1080 machine if we open up hardware info and scroll right down you can see everything about the nvidia gtx 1080 is showing up right here we've got our fan speed we've got core voltages temperature this is as if i was running a native machine but i'm not it's in a virtual machine see was that so hard nvidia as much as i gripe at nvidia for a lot of these arbitrary limits inside of their cards to give them credit they did listen to consumers this time around and in fact it was kind of a sweeping change dating back to the 600 series of cards they enabled geforce pass-through on consumer graphics cards so if you have a linux machine and you want to run a windows vm you can do that now now there are still some pretty major caveats to that first and foremost your linux host still has to have its own video card you can't share the video card between both the host and the guest os at the same time that's called sriov or mx gpu or vgpu and that's a technology that nvidia doesn't want consumers to have yet now if nvidia is watching this i would still very much like to take a look at sr iov on the new 3000 series cards i know the hardware support is there for it you just need to enable it in software there are some use cases out there for like i said multi-headed virtual machines or whole home gaming rigs or distributed video editing machines that while you've sanctioned those off as quadro and enterprise technologies only there's use cases in the consumer and professional realm that could very much take advantage of those now i did successfully get one of the gpus inside the gtx 690 linked to a virtual machine and launched and had the driver installed however it wasn't recognizing the display output but i didn't get a code 43 either the second gpu on the 690 however gives me a code 35 when i install the driver in windows so it's not a code 43 so i guess that's a step up from where we've been before but it's also not working either now i figured this could have been an efi problem as the gtx 690 is not a native uefi card so i flashed on some bios which according to wendell over at level one text were sketchy af and when he says it you know it's serious but i went ahead and did it but to no success for me since uefi failed on a virtual machine i figured i would try a bios boot virtual machine however that is still affected by code 43 thanks to nvidia so it appears nvidia's drivers do unlock the gpu inside of a virtual machine but only if you're running a uefi bios so i obviously still have some work to do to get the gtx 690s working in this system to create the six-headed gaming workstation i originally sought out to but nvidia this is definitely a step in the right direction and thank you for listening to the consumer anyway if you liked this video make sure to hit that thumbs up button and subscribe to craft computing if you haven't done so already follow me on twitter at craft computing to keep up with my daily shenanigans like this and if you like the content you see on this channel and want to help support me in what i do consider joining the patreon or float plane links are both down in the video description because this hardware is really expensive so for now that's all i have for you in this one thank you all so much for watching and as always i will see you in the next video cheers guys oh i missed barrel age beers hope you guys enjoyed mixology march but we are into april which means it is a return to craft beer and i am kicking it off with uh one that looks pretty fantastic this is from lewitt brewing war tortoise 2020 barrel aged edition a 9.4 imperial stout now if you watch the channel enough you know i usually try to theme the beer to whatever build or project we're working on for the day in this case war tortoise is an exactly perfect match for my epic cloud gaming system as uh this is just a tortoise shell with a whole bunch of guns strapped to it much like the three gtx 690s that are strapped inside of this thing if your motor oil comes out like this you are well past your change date in fact i may need to take the head off this and make sure there's no sludge oh man as much as i love cocktails and often i would prefer having a cocktail to a craft beer uh the bourbon and caramel and vanilla and oak and chocolate right on the nose of this i missed you that is incredibly smooth and i don't even want to know the calorie count that's like a chocolate caramel brownie with a little sniffed or a bourbon on the side that's good it's smooth it's sweet it's rich it's thick and there's every single flavor i would expect to find in a bourbon-barrel-aged imperial stout guts glory grim determination appetite for destruction of a robust imperial stout with pronounced notes of roasted barley and chocolate there's chocolate there's caramel there's a little bit of vanilla there's this real deep roast that really doesn't come in until the back end of the flavor but it is definitely present there's a real rich dark chocolate in this one like i said this is definitely a very sweet imperial stout but very very good hello this message is intended to contact you in regards to a legal case filed against your name this call is to inform you that we have received legal notice concerning fraudulent activity against her social security number oh no officer on duty ignoring this message will be considered as intentional attempt of non-appearance uh i got one earlier today that had some better voice synthesis it was a male voice synthesis and it actually had some inflections in it but it was still very clearly uh machine language uh but what was great is they ran it through a filter and gave it an echo so it sounded like he was trying to call me from like a school gym or something uh it was pretty great [Music]

Channel: Craft Computing

Views: 98,254

Keywords:

Id: 12JtV1ou6EQ

Length: 19min 57sec (1197 seconds)

Published: Sat Apr 03 2021