TAM Lab 087 - Setup and Use vSAN File Services

awesome everyone thank you so much for joining uh tamlab87 today so my name is bill i'll be uh presenting this uh this is uh setting up and using vsan file services there we go um so quick reminder uh our customers can actually submit session ideas as well so uh if you're a customer checking this out online um go ahead and reach out to your tam with you know your name your your title uh the title of the session and the description uh and then work with your tam to to you know get it produced and uh ready for uh everybody to view online so this is me uh i'm a a tam a team lead uh here at vmware been here about four years i've worked in i.t i've covered a ton of different customers in different uh in different markets um i'm a team lead on the leadership team here the expert and vmux speaker and then i got some certs so okay what are we trying to accomplish today right with this lab on vsan file services so um quick touch on the overall architecture right and then understand some of the limitations about using the solution um and some of the prereqs and then we're actually gonna get into the lab um and actually configure it and show how we can use it and then clean up so uh if you haven't seen this yet this is what the solution looks like i totally stole this from our uh our documentation i didn't create it um file services is made of a couple layers here right so when you enable it it creates this vcn distributed file system on top of vsan and so this vdfs file system is an aggregate of vsan objects so it creates some v-stand objects and then works on top of those it creates a handful of vms these appliance vms or file service vms excuse me fsvms each one of those interacts with that vdfs and um presents uh nfs and smb uh you'll find that there will be one vm uh deployed per host up to 32. um and then at the bottom here you have the file services control path and monitoring right so that lives at vcenter it handles like the deployment of the vms in the service um you know management making sure that um when a host goes in maintenance mode that that vm is uh is destroyed and things like that um and also monitoring and we'll touch on monitoring here in a little bit and so effectively across your vsan cluster you have a single vdfs and it's served by those vms right and those provide your nfs so uh it's a very logical you know build on top of what vsan has already uh and so from a limitations again totally stole this from our documentation um two host clusters and stretch clusters are not supported if you have vsan 7 you can support up to 32 file shares and 8 file servers but if you go update 1 then you get 32 file shares and then your 32 file servers one for host you also are it's not supported to mount the nfs share from an esxi host and run vms uh there's other ways to accomplish sharing vsan um outside of a vcn cluster this is not the way to do it now from a prereq perspective right we got a little list here so we'll run through it um if you're going to use smb or nfs41 uh to leverage some kerberos you need active directory you need a bunch of static ip addresses for the service and the fs uh fsvm node so those virtual machines each one needs a static ip and dns entries uh all of your ips should be in the same subnet we don't want the the traffic being routed um and then you also need a distributed switch uh version 660 or higher uh and then you need even though the vms can run on you know some of the same management subnets or whatever you want um having a dedicated port group for for um for these file services is important because it does enable mac learning and forge transmits if you don't already have those set up there could be implications on an existing port group right so having a new port group for that helps isolate that change but still give it the functions it needs and then if you're using nsx you need to manually ensure that those are going to be set up on the nsx side and then the last thing here is uh if you if you're going to enable this right your uh let's see here with no nsxt in your environment you need four cores and 16 gigs of ram and if you have nsxt you need some more ram so just keep that in mind as you're considering like how would i place this stuff in my environment um yeah all right oh okay we'll move on oh we have a question here in the chat real quick can you co-mingle native vsan and vsan file services in the same cluster yeah so you can in fact that's how my lab exists so i do have vsan and vcn file services in my cluster and i leverage both at the same time uh we also have a question here why does it need to double the the memory for nsxt that is a really good question i don't have an answer sad to say um yeah i'll just leave it at that i guess i could speculate um does the function need a special license you know that's a really good question and again i apologize i don't know what licensed version um we would need if somebody uh on chat wants to just get the new license it just it's on enterprise oh is it on enterprise yep hey there we go thank you enterprise enterprise plus uh no license required oh my gosh thank you so much i appreciate that okay perfect all right so let's head over to the lab um because powerpoint is boring so um let me do this and i know it's not the most exciting way to start the lab um i recorded the setup of this because it does take a little bit of time and for a lab to sit here and watch something uh go through you know all of its paces just takes a little while um so let me go ahead and start this we'll walk through it and then i'll kill the video and we'll go hop into the lab so i did this last night um and so you can see here that we're gonna go create a new port group uh i am on uh version seven so that's above the 6.6.0 requirement for uh the the distributed switch so we're gonna go ahead and create it just you know my own naming convention here um it's a pretty straightforward uh port group setup uh because it does most of the configuration itself so you just need to kind of create it um and here you you can see i'm specifying my uh my vlan since i only have you know a small home lab keeping it at the default was exactly what i needed and now you can see there's hey there's support group um and the policies you can see here that they're sitting at um reject for for all of those and we'll see the two of those change and then going back to the cluster that's my vsan cluster right so we're going to head down to vcn services um and then you'll find a file service there and it's sitting at disabled with like no information so here you go hit enable um and i'm going to pause this here real quick give me one second so uh if we're to go back and play just a little bit we go um so again keeping in mind you know at this point to get to where you need to be right you need to make sure that you um that you have all of your ip addresses and dns set up right it's going to be really important going forward to have that as their prereqs so make sure you have that set up ahead of time you understand how many nodes you're going to have um and that dns is you know created and propagated um similarly if you're going to use active directory right make sure you know um you know the credentials that you're going to bind with if there's going to be any kind of oh you uh that you're going to uh to you know deposit these servers into as well so now we're just starting to fill out uh fill out this information you know that's relevant for setting up right like dns file services domain i tend to use vb.info for everything uh my domain is vb.info um from an ou perspective i'm just going to have a dumpling of computers my 80 is pretty pretty weak bill we have a question in the chat is uh ipv6 or only ipv4 ipv6 as i understand it works for the file the primary file service but everything else needs ipv4 thanks yep all right so there we picked our our vsan file services port group right that we just created it's going to go monkey with that here in a minute now what i like about this is it's let's take a quick second here it's assuming when we put in these ip addresses that we've picked some serial ip addresses right so you know in this instance it's like 175 and then 176 177. so you can have it do all of the work and if you have 30 nodes letting it do all of the work and filling out those fields is going to be really uh good use of your time um so here we'll put that in hit auto fill awesome and then dns now in my environment um for some reason this vcn file services 2 is still being cached or something so i had to just manually go edit that it's still a legit valid um dns entry now one other thing i want to call it here at the top there's a file service agent that needs to either be downloaded and the process will do that um sad to say i've already done it so it didn't capture that part of the process you can either let it do it on its own or you can go manually download the bits documentation explains you know how to get those bits and then how to provide it here so if you have a server that doesn't have internet access to go download it you can go do it manually um so let me go ahead and continue on then so you know reviewing i put some data in here right and then uh let's kick it off so you know it hasn't been asking for a whole lot right like this isn't um overly complicated to provide the information it needs uh you can see here on the left we have a resource pool here now called esx agents and we're going to start seeing three vsan file service nodes in here so let's let that go right you can also see down here looking at progress we're installing the agent itself now well the process is on each of these hosts and the same process for deploying the ovf template will happen a couple more times so let me go ahead and click through so just you know 30 seconds later we can see that we have all of these nodes here you can see they're being cloned which is kind of a nice way to do it enabling the service on the esx host this is what seems to be taking the most time in the process and drs is kicking in and moving some stuff around because i just started up a whole bunch of new vms in here looking at the file service domain being created and there we go let me go back okay so this is where we get off the video about here um but let's take a quick look um at what we've seen so you know before this was all empty right just a bunch of dashes and nothing now we actually have the information we put in so we can understand what the configuration is of of the environment so i appreciate you uh hanging out while we run through that video um let's head over to the actual lab and this is pretty much where we left off right so again you can see the same information on my cluster um let's take a quick look here at the vsan the the node right so it did allocate eight gig vms and four vcpus with quite a bit of storage though it's thin because it's on vsan and if we go look at the um at the port group we can see now that change right so now it changed forge transmits to accept and mac address changes to accept as well recall when we set it up it was set as reject so it did that it knew what it needed to do which is which is pretty sweet okay so let's go back to the cluster because that's where we live uh with vsan and so now that this is enabled there's this uh file shares option that is opened up at the bottom so if i go ahead and open that up now normally this would be blank i wanted to let i wanted to test this and leave some data in there overnight but you know again you can you can imagine that there's nothing in this field you know when you first start it up um what we're going to do here is we're going to create a new file share pretty straightforward process um you know let's call this one tam lab nfs and you can see our options here are our smb and nfs and we have the option of specifying a different protocol so in this instance i'm just going to specify three because why not we also have different warnings whoops and quotas that we can we can configure so in this environment for some reason i wanted to have a warning at 5 gigs and a hard quota at 10. and then what's really nice about this is uh all of the content that goes into this file service right this uh this nfs service here or share is going to have this tag associated with or this label i should say um and so you can help organize understand how that data is uh you know being used and accessed um and if you're used to nfs you've probably seen something like this before where you know you come up with who can access your service um if you do based on ip and some you know ip subnet here you can actually add a bunch if you want you can change the permissions based on a subnet and also do root squash if that's a use case for you for the lab don't really care i'm just going to do any ip and i'm going to move on and then finish in the qa why does it need to change the port group settings oh um as i understand it part of it is um for replication and also if um if there's a host crash moving the um moving the vip around uh the service ip address as well there's probably other reasons that i'm not aware of but that's what you know at least a couple of the reasons that i know of okay so we create an nfs let's for fun create uh that was a version three i'm gonna do a version four real quick it's just subtly different um nfs and this time i'm going to pick 4-1 now you can see it changed a little bit um where if we were here with kerberos then we can actually change um you know some of the authentic authentication options and things like that i'm just going to keep it default here and let's do something very similar file server nfs41 now one of the advantages to nfs41 is like pnfs so rather than having a single stream like with nfs3 nfs41 um can enable you to use multiple streams to multiple uh these file servers in there so in theory assuming you're not in my lab that's kind of storage performance constrained uh you could potentially see better performance using nfs41 versus three okay so we're cruising let's go ahead and hop over to um [Music] to a linux host that i have set up and i want to um let's do this there we go let's actually show connecting and uh and checking this out so excuse me the first thing i want to do i need to copy a couple commands over um to prep the environment so let me get that over here so we're going to create a couple mount points um on this machine nope that's not it that's not it that's not it oh this is gonna be fun if let's see here okay and we're gonna do four one as well so now we have these mount points that we're gonna go ahead and mount um these nfs shares too so it's kind of it's kind of cool uh some of the options well some of the ways we try and make this easier um so i'm gonna say type is nfs i'm gonna make it a little verbose and i'll show you why um if we come back over here back to the the web client or the vsphere client and we select one of these file shares you can see this new option pops up copy path and if we open that up it shows you actually this is the path to that um to that share and you can notice here it picked one of the other hosts right we have three three nodes sorry yeah pick one of the other nodes we have three nodes i picked vsan file two as the appropriate one for this so i'm gonna go ahead and paste that in here and then i'm gonna tell a word to mount okay now the reason i wanted to show have this v in here was to show that nfs clients like on the linux side typically roll through versions trying to match the protocol version to the services being provided so you can see here that this mount nfs amount of type nfs tried version four two four one right version three and then it got to version three protocol 17 and then it and then it worked i'm gonna do the same thing here now with this um nfs41 so again copy path the path is slightly different but that's fine um [Music] i don't need the verbose here um it doesn't get as much this go around okay now you notice the the path changed a little bit so if we're looking up here for the nfs3 it's the file server in this instance vcn file2 and then a path to the folder with nfs4 it's slightly different where you can see it's the file server but then this time it's vcnfs tam lab so um it handles the pathing and the organization of that slightly different but from the server's perspective if we look you can see there's um there's the two mount points here so next thing i want to do is just i mean we mounted it why don't we put a little data over there so in my home directory i just have a one gig like a large file and then i also have the uh the the linux kernel source um so [Music] let's just copy it over real quick that should work so you know having my my uh vsan environment having like a cash drive and like one capacity drive um that lines up with with what i've seen from performance so i was pretty happy to see that that it wasn't a big drop um and i wasn't expecting a major a major gain as part of it so you know pushing those files to the nfs3 file share works and then let's see here do the same thing to go into the four one slightly slightly higher speeds um there we go took a little longer on the um on the smaller the linux source but that's fine you know we're talking four and six seconds versus ten and one for the previous so i don't know i'd call it kind of a wash a little bit for my environment but um that's that now one of the ways this can be really useful for our customers or even our home labs is persistent storage for containers trying to figure out you know we know that's one of the challenges that um that a container any kind of container initiative has is you know what do you do that persistent storage so um you know with vsan's ability to now present nfs um as as these you know the file share it could end up being a really cool tool um for for container initiatives all right so we got nfs down let's uh let's go ahead and get rid of this and we'll make this big again and let's create one for windows actually real quick let's refresh here because we did put some data in there and so now you can see here that we have um in both instances it's the same files they were correctly reporting 1.11 gigs so we can even just see that here i'm going to add a new one smb [Music] so again change the protocol to smb if we want to do encryption because of reasons then you can totally just you know flip the switch um you know i called this out or i didn't call this out earlier but it was available for nfs you do have access to your different vsan storage policies so based on the policies you have to find or you want to define you can also use that to help protect this data maybe differently than you would other kinds of workloads and there is also a new policy that's created here it's the fsvm profile do not modify that's created and applied to these um esx agents right that provide the file services so uh don't modify it anyway so let's go over here bill before before we go on too far we've got a couple of questions in the q a uh oh great um i wanted to see if you could address some of those let's do it okay uh how are the files on this nsf service protected replicated assuming this is more black box than the way vm files are protected by vsan that's a big topic yeah that's that's a big one um you know it's going to leverage partially the configuration of vsan right that you know we know it doesn't support stretch clusters for example so um you know relying on on kind of that um well i guess that geod potentially geodifferent right like if you have a different cluster in a different data center across town or whatever else as long as it's within latency um that that won't apply here um i'm not sure how like uh srm for example would handle that um so that's a really really good question that might be an interesting session for you know a follow-up honestly um and then stephen ling has a can nsf client rpc the tag if yes how to do that i don't know if they can i'll be honest yeah then what is the purpose for the tech just only for the uh recenter to maintenance and management those files stored inside of the nfs yeah so the tags you can actually use because you can add whatever labels you want in here so it's oh it's up to you to define what those labels mean for your organization it could be uh maybe it's project code or application or owner or whatever else but the nfs's client is not able to access the text right for example if i would like to something like a ls or search some file and i would like to use the tag does it work yes as i understand it does not know about the tag it only knows about the data without okay i got it and then the follow another question from from steven is if the vsan file to esxi host were put in to maintain will the nsf nfs service be able to migrate to another host automatically yeah so uh yes it will so if a host is put in a maintenance mode um part of the activities is shifting that um that workload um and moving that ip itself is actually deleted so as you go to maintenance mode that that vm gets deleted when it comes out of maintenance mode part of the activities of restoring vsan functionality um on a cluster with file share enabled um it'll go ahead and redeploy that service node and then pick up where you know it left off i think i think that that answers the the next question was kind of in in the same vein of uh yes yeah yep and you know for the for the the the instances where like pnfs for example um you know can use multiple um multiple servers and that's something that uh smb41 can present um you know you'll be down a path if you will right like a performance path you'll still have access to the data but now it's two nodes in my lab environment it'd be two nodes that could serve you instead of three until it comes back but service will still be there okay okay thank you yeah thank you i appreciate it um so i'm gonna go ahead and finish this real quick um you notice that setting up smb didn't have that network um you know the question about you know who can access it because you know that's something that nfs protocol does whereas smb doesn't it's handled in other ways um so i'm going to go ahead and create the share real quick and let's let that finish and there we go so now you can see it's here in the list it wasn't that hard right we specified some active directory credentials early on that um you know help join it to the domain and then now we're kind of living under domain um domain policy if you will for uh for how we you know kind of manage these things so you know as before when we had you know we selected an nfs share and we see this copy path well now if we do the same thing for smb it's slightly different we have a copy mmc command and then also copy path so uh let me uh grab this copy mmc command right so it's copied and you can see here it's calling uh you know fs management.msc and it's passing a computer to it so let's go ahead and hop over to um so this is a server 2012 r2 machine on my domain um i'm logged in as administrator uh to the domain and i have a command prompt running as administrator so if i were to go ahead and paste this command in here this is very similar to how you'd be handling other file servers um in your environment right so it it popped right up which is great um so we're seeing that authentication happen um through active directory uh you can see we have some shares here like this is you know this is the primary one we can set some share permissions um you know if we yeah there we go if we wanted to we could do you know file sir one file server excuse me dot info spell it wrong file server file server let's try it again we'll do it this way there we go so vcnfs here's my tamlab smb right similar to how we saw with um with nfs41 we also saw you know vsan fs show up in the path um but you know now we have uh we have nothing in here and if i look at open files let's refresh this you know there's not really much going on i can go ahead and create a new thing in here ah damn lab right so we can create files and i still have the same over here on the far left i know that icons are a little small but you know large file and then the linux kernel same ones from that uh from that host or the linux host so i'll go ahead and just copy them in here and we're seeing you know similar ish performance um whoops so let's let that finish up there we go so we moved our you know 112 gig meg file and a gig file over um so that's that's pretty sweet right so with the right permissions in your active directory domain now you're able to give give users and services access to another smb share in your environment so that is that's that with um with creating the shares i want to walk through just a couple more things here real quick oh let's see here um we have a question in the q a is vsan applying dedupe and compression to files on these shares it depends on what your um what your vcn policies are so if you need it too it can um so we're back here at configure um looking at the file shares i'm going to go ahead and switch over to um to monitor and actually before i do that um if we look at this file share health and recall that we set up um a policy that says if you're over five gigs that's a warning right and so that's what we have triggered here right this test environment where i uh this share where i put a whole bunch more data in there right we're over the five gigs um is throwing that warning and we're seeing that here at the cluster level hey you got some file share health issues we go over to monitor we can look at you know cluster level um you know things that are going on i'm going to scroll down to vsan and first thing i want to do is look at virtual objects and we now we have this vsan file shares objects and so you know whereas you know where we traditionally see virtual machines and their hard drives and you know where does that data live and whatever else now we can actually see file shares and whether or not there's issues with the object supporting those so that's kind of nice you know if uh if you have a vsan crash or an issue going on and you know sometimes we see objects that are kind of confused on where they should live or if they're in any kind of error state we see that here for the file shares which i was pretty pretty happy to see we can look at capacity so again we you know if we if we look back at configures like all right we get a sense of what's going on but in the overall relationship of my vsan environment how does this storage um you know play into it and so these files are stored they're considered user objects just like my isos for example so you know we can look here we can click on the donut and we can see here that okay well of the 144.55 gigs of user objects we're using about 20 gigs of files and then last performance here um so now that we have that enabled we have file share as an option so i can hop up here to file share and it you know it defaults to defaulted to this nfs one but we can certainly select a different one like maybe smb oh let's do four one then there we go um and so now we can actually see you know how things are going um you know latency for my lab environment is not amazing uh so i'm not shocked to see it that high hopefully you know an enterprise or more mature lab would have uh better performance there but yeah we can see you know what's going on with iops latency and throughput for our shares over a period of time so if there's any questions about that container workload um you know we do have some uh some visibility there all right any uh any questions comments thoughts about what we've seen so far my last step here is to show you how to disable and clean everything up so i don't want to wreck the environment quite yet if there's any any questions or anything you want me to click on is there a chanzu version of this or um a tensor plug to see what it looks like to leverage file services um i'm not aware of of maybe there's other people on the line mr tokens um that might have a better answer i don't know of a specific plug-in uh versus you know just presenting nfs um you know if you enable vsphere with kubernetes or v6 excuse me visa ratanzu um that may enable some features i'm just not sure what those are yeah my my question would be things like what is that what did the tagging look like on the shares when um kubernetes clusters spun up for example um because i guess when we present it to customers uh it is all done on the developer side pretty much so i would like to see what it looks like on the operator side on our site just just a curious thing yeah well i'll make note of that because that could be a good follow-up as well um so you know how do we how do we build on top of um vcn file services because it's not just it's not just here's a file server but let's you know let's actually talk about what we can do with it so i appreciate that i'm just going to make a note why thank you yeah tons of okay great what's the behavior uh of losing a node well you know that's that's interesting um if you lose a node because you know you can already envision that you're gonna see a delay in um in some traffic right because that that connection that exists between the client and the server is unique to the client and the server um with pnfs it might be a little different um so you could expect that there would be a delay while you know the network realizes that network point is no longer there and then the process or the the service moves that ip around so there could be depending on how sensitive uh your storage needs are you might have to tune your mount points um to allow for a delay uh if if needed um so there would be a little bit of a delay um as those as those things move around but that i think you know i think that's not necessarily unique to this solution as it is it's just kind of the nature of um you know tcp communications to a single endpoint all right well thank you for that question um all right well then i'm going to move ahead and just show you real quick how to kind of wreck this um and uh you know it's our home labs sometimes we need to clean up i know that um it does take up quite a bit of uh capacity for me so first thing i'm gonna do is go to my file shares i'm gonna i'm gonna delete these now actually you know before i do that now you know what it's the lab i'll clean that up manually later uh i'll go ahead and just take these and delete so hey you want to delete all these sure you know obviously you could do it one at a time if you needed to or whatever else but in this instance i'm just going to go ahead and wreck it all right so you can see it's rolling through the different um esx hosts here probably doing one for for each so let's just let that finish um now the next thing here oh hey they're gone cool so the next thing is we're gonna go back into um configure vcn services and now we see file services is still showing as enabled it's as simple as going disable and it makes a note here there's no file share right we're going to clear the config but disabling file services does not remove the file shares all the protocols including access to the shares are disrupted so in theory uh if there's a reason to disable but you don't want to lose your data for i don't know whatever that use case would be you could do that for me i want it all gone so i'm going to go ahead and just hit disable and now it's just going to kind of go in reverse and undo what we did not too long ago so we'll just let this finish the the one part that it doesn't touch actually there's two parts it will keep this esx agents um a resource pool around so that'll continue that'll be a relic as will your um your port group since it didn't actually create it it just leveraged it um it'll leave that alone as well so if you are truly trying to remove it from your environment you'll still have to do a couple additional steps with the pork group and that um resource pool so you know this is like watching paint dry as it's doing its things so i'll go ahead and just suggest like hey any any other questions comments um one questions here yeah um okay for example you create a nfx share and then the people's they transfer a file to the fshare so are we able to uh look at this file transfer on the an nfs nfs share from the storage i mean the server the template the network and the storage that one if you click the storage are you able to find out those files we upload it that's a really good question my understanding is no that you can't know um it's similar to like you know um there could be there could be reasons why you know compliance for example um and so having that within there you know as long as you have access to that network and permissions to mount it then you can go that way yeah now i i already see the answer because of i can see the nfs files notes one two three here so that could be a a kind of vm uh okay can you click inside of the uh ssd or the dvs data that one to see is there any file inside it oh sorry i apologize yeah let me go back um let's open wow so we're kind of at the point where i'm not sure what's going to happen but let's look here because it is in the process of deleting things okay you got it no yeah i got it you reorganize the file there you go is that what you're looking for i mean sad to say yeah yeah it's gonna it's gonna disappear and yeah there's the deletes um so that's the stuff will probably leave fantastic silly okay cool i know i have um i have a customer who's interested in possibly using this for um kind of a cluster quorum if you will uh an smb share um for some cluster file services and i thought okay that's a really interesting use case for it um because you know often with these cluster services you just need a shared file system and there's not a lot of i o that happens with those often if it's for quorum it's really just kind of keeping track of what's going on um so it's a very interesting use case for them um you know they were uh looking at some other more expensive solutions so you know crossing my fingers that they're testing for it'll will pay off anyway well with that we can see it's disabled everything is back to how we were um before just you know a couple little things to clean up but um all in all that's that's it i you know i was very impressed when i started down this path trying to understand this the solution how easy it was to set up and you know the the prerequisites being a bunch of ip addresses uh you know some capacity to run these fsvms um and you know a port group right from a from the vsphere side it all seemed very very reasonable and what you get out of the deal is pretty darn uh pretty darn useful so anyway i guess with that uh i'll go ahead and end the recording uh everybody thank you so much for joining today and you know keep on taming and then we'll see at the next the next team lab

Channel: VMware TAM Lab

Views: 989

Rating: 5 out of 5

Keywords: vSAN, vSAN 7, vSAN 7 Update 1, vSAN 7 U1, vSAN File Services, vSAN NFS, vSAN VFS v3, vSAN NFS v4, vSAN NFS 4.1, vSAN SMB

Id: 4vW6vWhwtu4

Length: 44min 0sec (2640 seconds)

Published: Thu Mar 25 2021