Enable Kerberos on Cloudera 5.16 (CDH)

now let me share your screen so you shared my screen you so friends today will be how to calibrate the claw database requesters if you want to learn how to set up a Hadoop on multi node environment using chlorella distribution we have a course in udemy I'll be sharing the udemy coupon link as part of the description if you actually watch it if you watch the pre-recorded session you can actually see the link in the description and you can click on the link and you should be able to sign up for the ten dollar course to actually provision the service from DCP and the setup chlorella on top of it that being said now let me actually tell what we are going to do today so I already have a seven node setup you can see here I have provisioned seven service from DCP and then I have gone through step by step process where we added all these servers as part of this those seven nodes using cloud as our distribution so first we set up cloud or a manager then we have added all the services onto it now we are at a point where we want to securitize the cluster or camera with the cluster or enable security on this cluster you can actually start here if you go to administration and if you say security and you will see the enable cameras option here don't click and proceed further I will click and I will explain what it is all about at this time and we will come back to this at a later point in time if you click on enable cameras here you see it is the checklist you how to make sure you take care of all these things manually then you have to come here make sure you check these things and then continue without taking care of these steps if you just check these things and if you hit on continue you might end up in a major issue it will be a bit challenging for you to course characters okay so if you look at the checklist the first item is set up a working KDC chlorella manager supports MIT KDC and Active Directory active directory from Microsoft MIT KDC is calbro's KDC stands 12 key distribution center we have already seen core concepts of Kerberos as part of the previous session so it is continuation two to it so when it comes to setting up of Kerberos we have to first identify the terms which we typically use so there will be five stands for cameras five you'll be seeing Elvis I quite a bit when we get into the configuration KD c stands for key distribution centers families pluggable authentication modules it is not required actually but we'll be setting up only further validation because once we set up calbro's on one of these servers which will be part of the enterprise but outside the cluster and then we want to weld it using SH if you want to validate SH service to actually see where the calvo's is working or not then we have to install some Pam related libraries otherwise pathway is not required well miss transfer the domain was which a Kerberos authentication server has the other two or the ticket a user so we will create Velma and we'll be using rel in several areas so well may actually tell you the configuration of dekalb rows where the calbro's related databases there where the log files go so on and so forth we will get into those details as we proceed further now when it comes to components in KDC there is a authentication server and there is a ticket granting server so we have to start to process one is KDC and second one is Cade min one is primarily for authentication second one is primarily to grant the tickets I think Kate minis for granting tickets and KDC is primarily authentication also whenever you try to create tickets first you need to have a principal for that users are hosts only those users who have principals are those who have principals will be able to generate the tickets and able to use those tickets to authenticate and all these principles which are created for users and hosts has to be stored in some database that's where the principal's database comes into picture okay so these are the key concepts of calbro's and when we actually plan it on a Hadoop cluster and mentioned to be allele typically we will not be setting up the kal breasts undies service big data cell one through big data service seven which are designated cell setting up us Hadoop using floral distribution in all case big data cell phone is the gateway to three foreign masters five six seven are slaves outside the big data clusters there will be one server two service because we can distribute to the authentication server and ticket granting service into two different hosts and we can configure the the KDC i'll calbro's okay so either we will have one server or two service depending upon the scale of the organization and also the design which is decided typically most of the production implementations will have the separation of duties between two servers with respect to other indications and ticket granting so we are not going to have two service here will be having only one server and that server can be used to sulk calbro's related authentication across multiple clusters in the organization it can be big data cluster it can be non big data cluster it can be a standalone server also so anything can be a client to KDC that being said first what i am trying to do is i am trying to create a virtual machine there should be aware to create a virtual machine using this as reference let me see if I click on this virtual machine if it show so you can use this create similar option typically we don't need a same level of servers as we have in the big data cluster and the Kerberos server might be smaller server only we will not be having a very big server and also typically the Hadoop admin will not be sitting up the CalPERS but if you set up and understand the concepts it will help you in troubleshooting the issues that's why you should go through the exercise set up cameras by yourselves configure the cluster which you are using which you are building to learn the cloud as a distribution of Hadoop to actually build the cluster how it works distribution to build the cluster and then configure against the Kerberos server which is built by you itself it might not be in enterprise standard but at least you will be knowing what is going on and you will be able to troubleshoot the issue okay so make sure you go through this exercise that being said I'm naming it as KDC let me give you this I have a 2 V CPU 7 points:5 JB memory machine we don't need to add any storage for this except for the boot where the operating system will be there on all these other servers we have 60 GB additional storage so that that can be mounted so the hdfs filesystem ok so we can actually go ahead and say create no need to add any additional storage here that will give me good enough it will take a bit of time once it is up and running we should be able to connect it to this however in this case I will be using big data cell phone itself as a gateway to actually access any of the servers in this setup and hence what I will be doing is I'll be saying SH - I dot Sh Google compute engine then big data server - one so I'm connecting to the Gateway node or the first node in the cluster as we are as I am logged in now I can actually go to the KDC first I'm pinging to see if KDC the listening from here it is listening and hence I can say a stitch - I dot s search Google compute engine which is the private key for which we have the public key on all these servers and then I can say KDC and hit enter now I am in that KDC okay so this is the one where we will be setting up calbro's related components especially from the masters perspective KDC perspective now as we have seen yesterday it is pretty straightforward to set up KDC you just when this install command as root so either you can say sudo and paste that uninstall commander are you can say sudo su - switch to root and then just run em - when I install a Kerberos server kairos libraries cameras works Justin and Pam underscore cat was fine so for KDC we need to care with five servile and Kelby five lives can be five extension is primarily saw declined and whenever we set up a rose on all the Masters will be having web station also okay now it is installing all these wineries once these are installed we have to configure KDC components the location for KDC components is nothing but this one everything related to calbro's will be primarily be here so let me copy this it is still don't bring and sling this office now the installation is done we can paste that path LSI's 1 LTR you can see there is a configuration file and a ACL file you have to open this configuration file and here you have to give the real man it should be specific to your organization ok so my organization is the IT versity and hence I am saying ID worst accom if your organization is some X you can say some XCOM you can give whatever name you want and this name has to be used in just as we configure the things like Caedmon ACL the client configuration etc if you look at the properties here it says where the ACL file is dicta file is it been key tab file so and so forth okay now I can save it and come out of it and then we can open this ke admin v dot a zip file we just have to replace the well name here so whatever the well name we have used as part of the previous configuration spell that has to be updated here once again I'm reiterating the meaning of this is any principle which is added with pin will have all the permissions to manage the principles of assigning the tickets etcetera okay so that is the relevance of this file at the entry in this file and you can have a rule like this is love opening up everything you can create groups like admin several groups and you can actually control the permissions okay we don't need to in the pursuit of glory the cluster with Karros you can leave leave it like this now save it and come out the configuration of the KDC components is done okay now we have to configure KDC server as a client also so we have to have a client setup on the same machine where we have configured service so we just have to copy page to these things and make sure you review the well send domain well so when it comes to domain realm in this case I'm just using this what we say m GCP related domain typically will be having our internal domain so as miny the I diversity dot-com if I want to use the dev domain I have to say I diversity comm here in all lowercase this is the realm name okay all capitals we just named it like this for our understanding purpose you can name whatever way you want okay so you need to ensure that the default realm realms the value in realms you can have multiple realms like this in a complex and also the mapping between DOM and realm should contain the same value with respect to realm these are these are these are supposed to be same and when it comes to the KDC here I have prepared the material with a different set of service that's why we have different names here and if you actually look at the IP addresses of the first names which we are using right now it is a bit different it seems there is a question I will look into that question in a moment so this is the one which I want to use so the name of this area is nothing but KDC dot C C dot IT was to discuss dot internal even this domain realm has to be fixed the entries under domain realm has to be fixed so let me delete this okay now we have configured the client also once the client is configured then we have to create a database and here also we are specifying the realm and the realm value has to be same as whatever we have used so far meanwhile we will actually validate this location if there any changes happened we only see two files one is the KDC dot constant the second one is KD m phi dot ACL and if you review the KDC dot cons there should be a entry for the location of the database I think there won't be entry if we are using default if you are customizing it probably we might have to specify the location of the database also but let me run that command and show you what is going on internally Here I am actually creating the master key for KDC database with the password so make sure you remember this password if you forgot this this password then it will be tough for you to operate so you have to keep that in mind that being said now the database is created you can say LS - LTS and you can see there is a file by name principal dot K ADM 5 and also principal so these are the database related files so whenever you add a principal you will see that these files are updated ok so we will come back to this at a later point in time when we actually add principals now you can actually start and enable calculus before going further you need to ensure that both KDC as well as admin are up and running without any issues for that you can actually say systemctl status Calvi fight KD c and k admin after starting these using systemctl here I have started both these services both the components under the KDC and also I have enabled as part of this startup so whenever system is rebooted automatically these two components also will come up now it is time for us to configure a switch to use Kerberos and then take rough indication that will actually take care of validation also Kerberos but before getting in there let me actually see this quotient okay it's just a statement from DB so we can move on now we can actually validate KDC so here I am using I diversity user to log in primarily so if you actually go here and the username is ITT versity using which I have logged into this machine from Big Data self 1 alright so I want to create a principal for this user so that I diversity is authenticated to leverage the services that are configured with calbro's so here I can say sudo su root and then Kinkaid MnDOT local okay now you can actually create principles you can say question mark and hit enter there are several commands you can add principles to delete modify rename etc write it so we can actually say a drink I'll add underscore principle so add pinkie there alias for Erin Driscoll principle we can give the name for the principle I will be creating two principles once of root so that even root can actually erase rows and also we'll be adding principle diversity so you can enter the password when we add principles either we can add principles with password or with keys okay if you add with keys then you can actually take care of password let's login but you have to provide with the kita file to the client so that the client to copy that into their machine and then they will be able to authenticate without entering the password whenever the client runs care unit command to generate the ticket it will not prompt for such a password if you use the kita approach okay so here we are using password if you want you can also use the key table approach where you can actually take care of the authentication with calbro's without entering for the password while generating that it gets on the client side you will go to those steps also have the proceed further because it is required for us to generate principles with key tab claw data and we will actually go through those details here I am also creating a diversity and I'm giving admin permissions to it and we'll be using IT versity a letter to generate credentials using claw data module so make sure you use the user who have the sudha access to take care of this so enter the password now the principles for both root and I diversity users are generated and both are it means now we can actually say a drink a - Randy host and give the host name so in this case we are trying to configure principles our host so we need to create the principles for both users as well as host so that you any user on those particular hosts who have principles can actually access only when both user as well as principles and sorry as well as post have the principles then from that host using that user the cameras can be leveraged okay that is the key concept here and the host name is a bit different here it is KD seedot seedot IT varsity discuss dot internal okay now it didn't prompt for the password it has internally generated some random key and we have to create key type files for this host okay so we can actually create kita file using this katie add command so I can say Katie add which is kita we had yesterday as part of the last session I I forgot explaining this because I not I forgot but I could not explain this because at the time it was not clear but it is clear now okay so Katie and host Katie seedot seedot I diversity discuss dot internal and hit enter now you can see that the kita files that are generated for different what we say I think these are all algorithms encryption algorithms so for different algorithms the kita files are generated depending upon the algorithm you are using on the client side you have to copy that key tabs well there okay so that being said at the kitters are generated now we can actually go to the next step and we can elope calculus authentication to the server I think in this case there is only one file and this file either updated with all the key tasks using different algorithms so if we copy this as long as the client is using one of these algorithms for encryption it will work but if the client is using some other algorithm then whatever is here then it will not work so that is the meaning of these many entries with respect to kita it is actually creating entries in one file using different encryption algorithms which can be leveraged by client to communicate with the KDC okay as we are done with creating the principles for both users as well as hosts now we can come out of this and then we can a locale prasad indication to the server here we are trying to use a city as a service so that we have to run this command okay and and we have to consider a switch as a client to calvess for that we have to go to et Cie ssh asset underscore config go here and then paste this come out of it after saving and then you have to reload sh d once you reload now you can actually validate that KDC setup using this server itself as client for that you can actually exit from here go to the idea versity user if you try to log in as i diversity user again onto the same server by saying sh k DC so if you are trying to connect with the same user using a sit you don't need to specify the username specific user name is optional so here we are trying to connector the idea was to yourself again on to the same service for that reason either we can say a set KD c all we can actually say i t were city at KD c both means same okay now hit enter yes it is complaining that permission denied so the reason is we have configured the client but we haven't created the ticket so one of the key component with respect to Cal Bruce is nothing but ticket granting server you need to get the ticket so that you are allowed to use the service so here as I diversity usual for which we have the principle already in calbro's we can actually say K in it because it is configured with I will troubleshoot this issue in a moment because it is configured with the password it should prompt for the password but it didn't prompt so the password because there is some bug ok now let me say host name - s and hit enter so this is the host now let me actually say so sudo su and then k admin dot local and hit enter list brings let's say value well read here okay so here we have ID was t / it mean we don't have a diversity so the way it can be done is probably I have to say K in it I to verse t / it mean I hope this might work no so okay there is a typo here sorry for that yeah because we have tagged admin while creating the principal that's why we have to stay say that here now I can say I diversity I'll I can enter the password and then I can say K list to list the principal's so not the principal's to list the tickets and you can see the ticket is valid for 24 us now I should be able to say SH KDC and hit enter okay still saying permission denied because we do not have the principal's all ID was do we have principal celerity was t / admin so for that purpose what we are supposed to do is we have to sudo su to root once again and then say k admin dot local listings now you can say a drink and then I diversity provide the password we need to have that administered it mean you will not be able to generate the keys at a later point in time so keep this as is without modifying and now you can actually come out of this exit from root and then you can actually say k list there is okay there are two I don't know why it has created to anywhere what I will be doing is I will be saying kak destroy it will generate destroy all the tickets now if I say canister it will not show anything now you can say K in it enter the password all right if I stay at itv.com and now you can actually say ass it k DC and hit enter now it has logged in without any challenges so the setup is successful so the calbro's is setup on one of the server in the enterprise now we want to configure our chlorella cluster to use that KDC but to actually setup calbro's up to enable calbro's which is KDC on our big data nodes we have to prepare them so the next step is to prepare level nodes in that protocol cluster so it will be better if you have ansible so that you can actually take care of installing everything simultaneously on all this service into one server at a time yeah if you are new to this content we already demonstrated how to set up a sequel how to take care of this things as part of the main course as I have explained to you earlier those who are joining from the youtube you can actually sign up to our you request for $10 which will be available as part of the description okay that being said first I have to go to the directory where I have created the host file and everything so as my ansible so that I can run the ansible commands to take care of to the action on all the hosts so there's a folder called as set up clusters and you can see there is a host file and if you look at the horse I have from Big Data server one through big data server seven now if I run this command it will take care of installing all the boundaries that are required for a client so it will run simultaneously and we will get the feedback very soon what happened why it is taking so much of time yeah now it is working fine it is installing okay there seems to be some issue big data powerful it failed interesting it it have failed only on one cell well I need to check why it has failed let me try running it once again you you you know it is failing for some reason so let me connect to that big data server for now let me run this command okay there's a typo here you I don't know why it is conflicting so let me fix this I am not sure why it is only same thing finger on this let me see any questions on YouTube meanwhile okay there are no questions there seems to be some issue with the young repository on this let me do one thing let me say I am clean all okay so first of all let me this let me do sudo and then I am clean all let me try running it once again no it is still not working let me see actually all the servers are in the same state I don't know why it is working on all the other servers but not this one you okay I'm - ye mo let me remove these packages Lipke areum Lib kay areum five stars you oh my god it has a little quite a bit of stuff let us see now let me run this once again let me remove Kelby Phillips also and I will land this I don't know how come it came into this I hope it will group the server oh my god it is removing so many things oh my god I think I messed it up let me see if everything is healthy let me go back here so far I don't think there is an issue but why it is trying to remove those many it is even trying to Oh YUM okay let me do this cat etc' a silly nuts let me do this let me say update this is a complete surprise to me so I never ran into this issue I have done quite a bit of installations but I never ran into this issue so let me undo this now you got to the shoe so I it got messed up so let me see what is the issue now almost everything is down I think we might have to stop this session for today I'm sorry for that it's a complete surprise you you so if you're done then we have to discontinue if it comes up then we can actually proceed you yeah on 4 not coming up so it's it's it got messed up shoot I don't know how come it got messed up why we got this issue anyway I think we have to stop for now so we will actually catch up tomorrow if the issue is addressed otherwise if Friday I am NOT available if tomorrow yeah demolished are there so we will continue next week by next week we will make sure that everything is working fine okay meanwhile what I will do is I will share the document you guys can give it a try and see if you are able to take kale because most of this stuff is the based on the coal aspects of cameras only let me walk you through the documentation and I will give it to you so try it out so what you are supposed to be using ansible you have to install w gate and unzip then you have to install something called as unlimited JC ok on all nodes where we want to enable cameras from big on all nodes in Big Data where we have to enable calculus we have to install this unlimited JC policy there is no yum repository for it you have to download first from Oracle then you have to unzip then you have to copy that file into a particular location there are two files that will be there once you unzip one one is local policy and second one is the US export policy you have to copy these files into this location and you can use this ansible command to take care of it then you can actually copy the etc' Calvi five concepts from KDC so you have all be on the KDC first what you do either you copy that to Big Data server one all you actually just to open the CTC calvess i.com and paste the same contents ok whatever we have used earlier when we actually configure the KDC service the client use the same contents and copy to ET c KL b phi dot console on a big data server one where we we have setup ansible and then you have to copy that file on to all the hosts because each and every host is a client now forever KDC and hence we have to conserve client on all the hosts and this will take care of it for you then you have to create principles so here you have to go to KDC as part of the KDC server you have to create a principal well cloud allah sem the reason why we have to add principles sem is because Claddagh SCM is the one who actually start all the services so here we are trying to enable cameras to manage the services in Hadoop as Florida SCM will actually start all the services we have to make sure that Claudia SCM is authenticated through calculus two to take care of it so first you have to create a key for Claudia SCM here we are using password less key because it doesn't make sense to enter password whenever you it try to start all these services on the server because we will be doing it quite often and it should be password less then you can actually generate the key tap file and you have to copy that kita file on the Big Data server one where the cloud data manager is running this test has to be taken care on the server where Claudia's imagine is running so in our case Big Data self one is the one with the Claudia else is running so you have to copy this CMS dot kata which is generated in KDC on to etc' claudia sem service on big data server one then you have to change the permissions also once you change the permissions then you are done with the preparing the nodes in the Big Data clusters all calbro's then you can actually go to that enable cameras check all the points in the check list click on continue then you have to take care of generating the keys and restart the all the services on the cluster so make sure you follow and come to this point by tomorrow if possible if not we will see ok try it out in the worst case you will mess up your cluster as you are in the learning mode if you have enough credits if you mess up you just to rebuild the cluster this time when you try to rebuild the question is thus adding one service at a time I have created a video yesterday where we can actually add all these services in one shot so make sure you follow that I will share the link with you whoever is joining this session live our premium customers and you can actually follow that and set up cluster in one shot if you screw up and if you could not figure out where the issue is but most likely you will not screw up if you follow these instructions for some reason I don't know why I generally use ansible to actually manage all my components why it failed on big data sorrowful I don't know probably I might have given to our team and they might have changed a few things all I might have messed up unknowingly but this is the first time I am starting the service for calculus I don't I never opened up the service for canvas I don't know why it messed up it's very unfortunate that we have to stop in the middle but we'll take care ok any questions from the team yeah so calbro see how to be very careful you have to follow these steps very methodically you have the reason is if something screw up troubleshooting is a bit challenge okay because when it comes to KDC the locks will be only the KDC related stuff if something messed up in the in-between with respect to the typos in the properties file and all you will not be able to terms with that issue very easily so make sure you use the appropriate values under wells and default realm especially on this client file so this is the configuration file for all the clients you need to make sure that the realms and domain realm details are accurate if you fail to do so then you will run into issues for sure and if you don't focus on this and if you travel through it you will not get anywhere so make sure you do the you do follow the steps very carefully and take it and try to implement validate at each and every step to make sure that things are progressing without much challenges ok we will catch up tomorrow bye

Channel: itversity

Views: 2,023

Keywords:

Id: BhPsLwD1M6M

Length: 47min 29sec (2849 seconds)

Published: Wed Aug 21 2019