How To Setup NFS Shared Storage In Proxmox

to get the most out of a hypervisor like proxmox you'll want to connect it to shared storage even if it's just a standalone computer you get the benefit of being able to backup all of your files from one central location but the real benefit is when you've got a cluster of servers because if you give these servers access to the same shared storage it means that if you want to do a live migration of a virtual machine from one hypervisor to another it'll go a lot more quicker because the actual files themselves never move similarly if an actual hypervisor breaks another hypervisor can then restart the actual virtual machines that had stopped because it's got access to those files because they're on this shared storage so how do you actually configure shared storage on proxmox using nfs for instance well if that's something you're interested in finding out then stick around and watch this video because that's what we'll be going over [Music] now what you use to provide your shared storage really depends on what you've got so you could have a nas for example a linux computer even a windows computer is capable of providing an nfs share now in my case i'm going to be using trunas so the configuration might be somewhat different but it's mostly the same because the idea is we want to set up an nfs share that proxmox can get access to the only thing is is that proxmox is going to try and connect in as the root user and we don't want that it's not a good security practice but what we can do is we can actually restrict what proxmox can get access to by effectively substituting that root account for a user account that we're going to create so that's the first thing we need to do is to actually create a completely new user account for proxmox so we're going to click on accounts and then users then we'll click the add button now what you want to call this is up to you so i'm just going to call this one pde user because this is the proxmox virtual environment click on username and it automatically gets populated and then i'm going to give it a password now as ever i do suggest using something a lot more stronger a lot more complicated than this in the case of truenas i have to give this user account a primary group it's just to do with the file permissions when a file gets created or edited the file has permissions for the user as well as the primary group of that user so i can't deselect that option and not then give it a different group in this case i've got to do something i mean i can either create a completely new group or assign it to an existing group but i'm going to keep things simple the group really doesn't matter to me it's just a matter of i need to give it access to a folder all i'm interested really in is the user account itself so i'm just going to leave this on the default setting here where it's going to create a completely new primary group now lower down what we've got here when it comes to truenas is that it's going to ask for what home directory do you want to give this user now this is more of a service account it's not a user account they don't need to store anything in a home directory so i don't need to change this setting which is don't create a home directory in the first place for them but the features i'm just going to leave as is but i do want to deselect that option which is the samba authentication it's not really relevant to logging in to truenas over nfs share so they don't need sambar authentication at all so i may as well deselect that option and then other than that i'm just going to click on submit and then that will create a new user account specifically that we can use for proxmox as well as the group although as i say the group itself doesn't really matter to me it only really matters as far as the actual file permissions are concerned now i like to keep my data separate and i want to be able to restrict who's going to be able to actually get access to the data that these proxbox hypervisors are uploading to the nas so for that reason i need a dedicated nfs share and i need to bind that to a specific folder on the nas now how you do that on your system really depends maybe you just need to create a folder to begin with now i could do something you know with truenas here but what i'm going to do is just create what's called a dataset instead because it has some extra features as i'll demonstrate so i'm going to click on storage then i'm going to click on pools now i've only got one pool so i'm just going to highlight that and then over at the end i'm going to select the ellipsis here then click add data set and i need to give this a name so i'm just going to call this one pbe vms because this is where i will be storing the virtual machines the proxmox virtual environment now there's a lot of options that you've got that you can change so the idea is there's a lot of various options that get set for the pool itself when you create a data set as you can see they inherit these options but you can actually change them for each individual data set or folder if you will so for instance here we've got an encryption option now encryption takes a performance hit there's no point encrypting data and wasting compute power if the date is publicly accessible in the first place so there's nothing specifically private about that information anyway so for example computers virtual machines they don't actually have any data at least they shouldn't have any sensitive data on them so for me i'd rather have the performance over the encryption the data is going to get stored somewhere else that could be put into a folder that gets encrypted but the virtual machines i'm just going to leave them as unencrypted so i've got the option to change this if i wanted to but like i said in this particular case i'm just going to leave it on the default which is no encryption and when i say the default that just happens to be because i set the pool itself up with a default setting of no encryption so something to bear in mind if if the pool itself has been encrypted every data set you create will be encrypted by default so these are just my default settings we've got a similar option up here for the compression level so it's inheriting this default option of lz4 as the encryption algorithm i mean i can change that to something else if i want i can even disable it now i've found a particular problem with virtual machines when it comes to compression so i'm quite happy to leave that there but if this has been a folder where i was storing files that had already been highly compressed there's no point trying to compress them again can actually end up you know using up more storage space aside from you know using up more compute power having to do that compression in the first place so again it's a useful feature to have in this case i don't actually necessarily need it to be honest now there is one interesting feature for nfs in particular and that's it the sync option up here the way i've got it set up is it's still on the default setting so the idea is that in nfs a computer sends data to the nas but it then sits and waits for a response back before it sends any more so that gives you data integrity but it comes at a cost of performance now what you do about that is entirely up to you as different ways of handling this for example depends on which is more important the performance or the data integrity because essentially with this set the way it is what's going to happen is proxmox will send data to the actual nas the nozzle then right at the disk proxmox is going to wait until that data has been written to the disk which is going to be probably slower then the nas is going to send back a response to say yes the data has been synced send me more data so as i say that that takes a bit of a performance here the thing to bear in mind is that if you do disable this feature what can then happen is that if the naz were to fall over for example say you have a power outage say that uh enough disks would break or something happened where the data wasn't actually committed to disk you would you would lose the data basically you could end up corrupting in this case virtual machines in the process so that's something to bear in mind is what you've got a wheel which is better to you the performance of the actual um uh the data integrity essentially now the actual virtual machines i'm running don't necessarily need particularly high throughput so i'm quite happy to leave it as this as is but i could change it if i wanted to you know the options there that's the key point it's these are all options that i can choose but what i can do is to improve this more is to actually give the system an actual caching option so what will then happen is truenas will actually write the data off to a cache ready to be committed to the actual drives but it'll be able to send a response back to the server quicker so there are ways around this if you want if you want to improve the performance but like i say i'm i'm not going to actually change anything i just want to demonstrate the difference over data sets if i was just going to create a normal folder i would have to drop into the command line to do that so this is actually even easier just to do it this way to be honest all i would have to do is just give it a name i don't need to change anything click on submit and that's it so that's all i'm going to do is click submit and then off that goes and it actually creates a completely new data set that i can now use to then share out using nfs now the next thing that i need to do is to change the file permissions now whether you need to do that really depends on the system you're using so do check because for windows computer for example what you might find is that at a file level everybody's got access anyway in the case of true nas it's very restrictive and it won't matter what permissions i allow in the actual nfs share if these aren't allowing access to proxmox then it doesn't matter so i do need to change these file permissions so i'm going to click on the ellipsis for this specific data set then i'm going to select edit permissions now because root is the user i'm logged in as then as far as permissions go root is the owner their primary group is wheel so wheel is the group and then the access permissions route the i user over here has got full access the group wheel has got read and execute access everybody else has got read and execute access so as i said straight away proxmox would have a problem that try to upload any files um to this so we need to change these so i could stick to the existing access mode but what i'm going to do is i'm going to opt for this acl manager within true now it's because it it'll give me a lot more flexibility now to be honest i won't take full advantage of it but it is there and because it's a new folder i may as well make this decision now because it's easy to do it now while there is no data there so i'm going to click on use acl manager and it changes the mode over and what i'm going to do is i'm going to go with one of the preset acls which is restricted i'll click continue so we've got two access list entries but you can add more so you can add groups in you could add a specific individual users it's it's very very flexible here now what we've got to begin with is one for the owner in other words whoever we define over here and then one for the group that we define over here but again we could change these if we wanted to i mean you can be very specific but uh a certain user or a certain group where you've got that ampersand at the end or at saying it's referring to these fields over here so the owner at means this user group at means this group here you can also change the permission type from basic to advanced for example that allows you to get very very you know granular in terms of what permissions are available but i'm just going to stick with these sort of settings these are fine for me the idea is this is going to give me flexibility later down the line if i wanted to uh to use them and i'd rather do this now rather than later on now i don't want root being the actual owner of this folder so i need to change this to the actual user that we created earlier so that's the pde user and i need to take a box apply user because there is a button down here to save the changes but unless you take the box this doesn't apply the reason being is that it can have you know significant um impact if you change permissions in a folder all of a sudden you might lock people out so it's a case of this is an extra step just to make sure you do actually want to make the change and it's the same for the group as well i didn't create a like a special group at all um for this i've just got a group that's the same name as the user i don't need a group that's got multiple users in it so i was quite happy for it to create its own group but again i've got to tick that box to say apply the group so i do want these changes applied to the actual folder so those are our permissions if you will this is going to be our new owner and the group for this folder i've got an option down here to apply the permissions recursively and what that means is that if there's any folders within this parent folder any subfolders any files it'll literally go through them and it'll actually change the permissions for them but this is a completely new folder so it doesn't really apply to me there's no nothing in there at all so i don't need to do that but if this was an existing folder that had data yeah you would really want to take that box so these are the changes we're going to make i'm going to click on save so i've swapped over to the acl mode from the access mode and we've also changed who basically owns and has permissions to get access and make changes and so on to this data set if i click on the ellipsis and go back to edit permissions as you can see that's all taken effect we've now got a new user or owner and a new group as well so that takes care of my file permissions now the next thing to do is to share the full route using nfs so here on truenas what i'm going to do is select sharing then i'm going to select unix shares nfs then i'm going to click add to create a new share and then i need to select the actual folder that we want to share out now when it comes to these general options i mean i could put a description in i could also set up this option which is all dirs now i don't need that because the hypervisors are just going to connect to this share and get access to everything through that share this is more of a case of maybe you've got sub folders and you want to be able to give people direct access to the actual sub folders so they don't have to navigate through you know the top share if you will and then work their way through it you can actually select this option to make things a bit easier but it's not really relevant for the hypervisor we're setting up next option is quiet now i'm going to select that and that's to suppress certain syslog messages now if i run into problems i'll come back and deselect this but otherwise i'd like to select that option down here we've got enabled now if you want to actually share this data out then obviously we have to enable nfs sharing so i do have that left on its default setting of enabled then down here we're going to click on advanced options now because i'm setting this up for virtual machines i certainly don't want it as read only but i do need to make some changes here regarding mapping user accounts the reason for that is proxmox is going to be connecting in as the root user now one thing that's going to cause a problem is the fact that like proxmox servers and trunas are using a different password for the actual root account in which case it wouldn't work the authentication will fail but it's also a bad security practice anyway so in that case what i can do in true now is i can remap the root user account i can't actually change that behavior within proxmox itself it's not giving me an option to say use this user account to log in it's just going to use the root account whether i like it or not but what i can do is i can remap the root account to something else and that's why we created this other user account in the first place so i'm gonna put in this other user account that i created earlier so that essentially what will happen is when proxmox connects in as the route truenas is then going to remap that root account essentially as part of the connection and only give it access to wherever this user account and group can go to basic at this stage is it's only going to have access to that folder so that gives me a bit of a a better security practice if you will except for the fact it means that anybody who connects in as root doesn't matter what password that supply will get remapped to this account and get access so that's something to bear in mind so for that reason you have to think about some additional security measures now one option is to put a firewall in front of the nas so that if you want to get access to that as you've got to go through a firewall and only certain computers will and get access to it but that takes a bit of a performance hit so what i do is set up a dedicated network specifically for the storage so this nas has got an interface in a dedicated network and so the hypervisor so they'll get direct access but nothing else will but having said that i still like to add in an extra line in here so that i really at least restrict things to a specific ip range i mean i could do it for specific hosts if i wanted to um i might add more proxmox servers for instance as we go along so for me i'm just going to leave it at the actual network so anything in that direct network essentially is going to be able to get access to this and if they use the root account they'll be able to get access because then it'll get remapped to this user but at least even still limiting the access to that specific folder that we've got because at the moment that's the only thing that this user account group have got access to i mean there is this other option this map all user and map or group but that's more for like anonymous connections if you use that option it doesn't really matter what account you use you're going to get remapped to something else um here we know it's going to connect in as root so we may as well just be a bit more specific but the principle is the same the only thing to bear in mind is that you can't have both of these options enabled at the same time you either remap the root account or you remap everything so the only other option that i've got really here is this security option which i'm not going to use we're not using any authentication i'm not going to get into the realms of kerberos and so on so i'm just going to leave that as is in which case we've now got our share ready to go so i'm going to click on the submit button then that creates a new renfresh share now the whole point about this video really has been about how do i actually configure proxmox to get access to shared storage using nfs now if you already know about nfs and i've just jumped straight to this part of the video it's going to be extremely short because there's very little you actually need to do now as a prerequisite you do need to make sure that the actual server or servers if you've got a cluster i've actually got access to that so if i go to mine yes there's only one i haven't actually set a cluster up yet but i'll go to this server here and go down to network i do have an interface which is in that subnet so it does actually have access to the nas i already know that in advance so do make sure that's already configured in advance but then if we go back to data center and then we select storage because we need to actually create new storage for our actual proxmox system up here we've got a drop down menu which says add so i'm going to click on that drop down menu then i'm going to select nfs i then need to give it an id so it's not in the vms where it has to be a number i can call this something that's going to be meaningful me to me so i'm going to call this nfs vms the idea being that i'm going to be setting this up for virtual machines so when i create a virtual machine and it gives me the option of storage to choose this is going to stand out as being an nfs share basically next thing i need to do is to tell up the actual server that i needed to connect to so i'm going to use the ip address the reason for that is you can run into a problem if you've got a cluster or just a standalone server and you're booting things up from cool in other words all of the hypervisors are turned off if you're actually connecting to shared storage and all of your virtual machines are on shared storage and one of those virtual machines is your dns server hopefully you can see the paradox here when these computers boot up i'm going to try and connect to the nfs share and as a sound whatever it happens to be and they're going to try and do a dns resolution but the dns server is not running yet so they can't do it so when it comes to hypervisors and various other infrastructure devices it's much better to use the ip address so you avoid that potential problem with dns not working so now that we've given it the ip address of the server we actually need to pick the share and that's what it's referencing is the export it's the it's the share being exported by the server click the drop down menu it's telling me what's available so i'm going to pick that one there which is the share that we just set up down here we've got the content so this is where we're going to restrict what proxmox can actually store on this storage now by default it's set to disk image we can change that to you know various options that we've got here so when i click the drop down menu got a big list here the good thing is you don't have to use a control key or a shift key to do multiple selection just click something deselect something it'll either appear on the list or disappear from this so for example in this case i'm going to be using this for virtual machines and i think i'll try it for containers as well i'll keep it all centralized so by default it's set to disk image so i'm also going to click on container you can see that suddenly popped up uh but let's say i change my mind if i click container again it disappears so as i said it's very easy to pick things so in this case as i said i'm going to keep this one specifically for disk images and containers i've already got a separate nfs share that i'll set up later for my iso images to keep them centralized i'm going to click on container here and then click away and that's sets my content for the nodes at the moment i've only got one node but what you can do is if you've got a cluster you can actually click on that drop down menu and you can select it so that only certain nodes within the cluster i'm actually going to have access to this storage it doesn't apply in my case but it's there if you need to use it then we've got the enable option which obviously enables the actual storage so we do want to leave that as is ticked down here we've got an option for pre-allocation but that's only visible because advanced has ticked so i'm going to just deselect so unless you've picked advanced at some point you know while you've been configuring proxmox you won't see that so do tick on the advanced option if you want to make some changes pre-allocation it's similar to what vmware uses in terms of thick provisioning and thin provisioning and it's all about saving storage space basically let's say you've got a virtual machine that you're going to be creating and it's going to have a 40 gig hard drive you've got choices here where you can either reallocate all of that disk storage in advance in other words if it's going to be a 40 gig file you can actually have it create a file of 40 gig so that's good for performance reasons but it wastes disk space if you're not going to use it now in a production environment you should actually have enough storage space in the first place it's not good to use thin provisioning um it's better to actually make sure you've got all that actual space in advance otherwise you run into problems but it's also for performance reasons it's better now interestingly you know if i was looking at some tests that somebody's run the this option here which is default what it does it actually selects this option here which is metadata it only allocates enough space for the metadata full would have allocated the entire space that you asked it to use but the performance trade-off isn't that bad actually it's they're actually very very close in terms of the performance capabilities so i'm going to leave that set as the default the only thing to bear in mind this is similar with you know vmware for example is that if you're using a sand for instance that can do thick and thin provisioning itself you don't want to be using that system on both your hypervisor and your actual sun in this case for example we'd either want to be doing full provisioning on the actual sand and then leaving this one to do default provisioning or we want to be doing some sort of thin provisioning on the sand and this one that would then be full if you mix the two up and they're both trying to only allocate enough it can cause performance problems so something to bear in mind but in this case i'm just going to leave it as the default because that seems fine we then got an option over here which is for the actual version of nfs to use so you've got version three plus various version fours here now i'm already setting that on the nas anyway i've already blocked out version three so i don't mind um you do have to make sure that the two are in sync and it's it's easy to just limit that on the actual nas itself and then just leave this at the default if you want but entirely up to you in either case you can't sell except this one to be version three for example if the nas itself doesn't allow version three like mine doesn't so i'm just gonna leave that at the default as well i'm then gonna click add so what it's then done is it's it's creating the storage and you just saw a brief flicker there where the question mark popped up but it's now actually connected so it's actually connected into the nas and what it means now going forward is that if i want to create virtual machines or in this case containers i don't necessarily have to store them on the local storage now i can take advantage of this nfs share and it means that when i build a cluster and start adding servers in i'll be able to store my actual virtual machines on this shared storage and i'll get a lot better usage out of my hypervisors than if i was storing my virtual machine files on the actual local storage well thanks for making it to the end of this video i really do hope you found it useful if so then do click the like button and share as that will help get the video to more people who might find it useful as well if you've got any comments or suggestions please post those in the comments section below and if you're new to the channel and you'd like to see more content like this then yes do subscribe just remember to set the bell icon to actually send your notifications when new content gets released although i also post to twitter as well as facebook if you'd like to help channel and support it you can actually make contributions through paypal and buy me a coffee i've also got links to patreon and there's also the join membership option for youtube itself patreon and youtube members do have the option to actually benefit from early access as well but above all many thanks for watching this video i'll see you in the next one [Music]

Channel: Tech Tutorials - David McKone

Views: 27,308

Keywords: proxmo nfs shared storage, proxmo nfs share, proxmo nfs export, proxmo nfs preallocation, proxmo nfs user, proxmo nfs thin provisioning, proxmo nfs truenas, proxmo nfs setup, proxmox shared storage, proxmox, proxmox ve

Id: txx0z-4HlSQ

Length: 27min 20sec (1640 seconds)

Published: Tue May 17 2022