KEYCLOAK Implementing Custom User Storage Provider (in-depth) | Niko Köbler (@dasniko)

hey friends of key clerk nice to see you again this is des nico of course you all know that you can store user's data in the key cloaks database of course the default settings perhaps you already have configured an ldap server or active directory as in user storage for user federation but sometimes it might be the case that you already have an external data source where you use this data live and you want to connect this to key cloak or in certain cases you want to store the key cloaks users data in a separate external data source today in this video i will show you how to do this with custom user storage providers [Music] when working with the key clock admin ui you have in the menu at the left the entry user federation and with the user federation you can connect geeklook with an external data source and to let key cloak store and manage the users from an external data source so the default data sources built in in key cloak are kebros protocol or ldap protocol for connecting ldap server or active directory server and you can see this is that provider drop-down box is the kerberos and the ldap these are the default built-ins and today i provided a peanuts user provider example for implementing own provider connecting to an api in my use case but it's not restricted to using api you can connect your database to every data source you can connect through any code and now just let's configure the peanuts user provider give it a simple name that's the peanuts api we have a base url for our service that's running currently on aws we need a username and we need a password for this for authentication and yeah this is completely custom configured i'll show you later in the code how you can do this and then you can set some cache settings these are defaults for user providers we just leave it for the default cache settings hit save and our user storage provider is already configured and if we switch now to the users tab and hit on view all users all the users will be displayed which are retrieved via the api and the user federation so you can connect to external user data sources of any kind you can query through an api through an sdk through database connections whatever you like and you see here some members of the peanuts family and uh these have the username email last name and first name and if we go into um one of these users the charlie brown you see als so the email first name and last name uh we see the storage origin is the peanuts api we just configured so key cloud knows this user is coming from the peanuts api and you can also see this in the id of the user this idea is a constructed identity of the uu id of the user provider we just configured with a preceding f column f of federation and we have this federation provider with the uuid 4 dbd or whatever and a suffix of the primary key the id of the external data source so charlie is the username and the id of the external data source of the peanuts api and this will be concatenated to the user id f the uid of the federation configuration and the external id of the user so we can just have a look to to the peanuts api to compare the ueid you can see here the provider id of the peanuts api s4 dbd 0 and if you go back to the users you can see all so this uid is here and if we go to the list you can see every user here is starting with this federation uid because they are all coming from the external database they have also some other attributes besides first name last name email we have a birthday and we have a gender in this example the credentials we let know key cloak that the password is handled from the peanuts api user federation we don't as a key cloak does not handle the password itself and we have some roll mappings and we can see here we have a role named child for charlie because charlie is a child of the peanuts and charlie is in the group peanuts and if we go to another um peanut let's say snoopy snoopy also has some first name snoopy doesn't have a last name has attributes and also credentials snoopy has another role has a role animal and also the group peanuts because they're all coming from peanuts and the nice thing here is we don't need having these groups and roles created in key cloak which the users are belonging to in the external provider as long as we don't want to manage these groups in key cloak or assigning other attributes or whatever so if you go to the groups menu you see there is no group created and at the roles we don't have any of the the child or animal roles in the groups you don't have a peanuts group but our users have a role child and a group peanuts great so um we can also authenticate of course with these users data and you can just log in with charlie and charlie has password and this password will be compared against the data of the external data provider and yeah we are authenticated logged in and we can see we have a user name charlie charlie brown first name is charlie last name is brown and we can do all the things like any other user can do because for key clock it's just a regular user in all the management for tokens for assigning roles groups whatever it's for key clock it's just a regular user like any other user also but also but only the source of the user is coming from the external data source so let's switch to the ide and have a look into the code and compare it to what we have here in our user federation as usual we starting with the provider factory in our case the user storage provider factory of type the the peanuts per user provider which is the actual implementation of our logic so the provider factory as usual has some provider id which gets returned from the get id method we have a create method which just returns a new user provider instance with a session and the current model we have a help text and we have the configuration the config properties so we can build up our own configuration for the user provider and here are the base url username and password for authentication for the api um and that's the place where we configure these um no where is it here we configure the base url basic of username basic auth password so these fields will be auto generated by the key cloak ui if we configure them here in the get config properties you see here the base url username password there's some help text and some type of string and password so if you can if you set the property type or to password you get a masking here in the ui nice feature you can set default values also if you have select box you can set various options if needed in our case just regular text inputs are rendered because that's enough for us so also to the config property definition itself you can configure a validation of the configuration so in our case i'm just testing if all the fields are filled if one field is not filled is still blank i will throw an exception and this will look like this if i miss some field and i try to save all this i get an error configuration not properly set please verify this is the message i'm throwing with the exception and then i can enter the value and i get a success message that's exactly the message here when the validation fails configuration not properly set please verify so there are a few other methods you can override but these are the basics the minimum you should do most of the times you need some kind of configuration whereas the database located with the user service locator or whatever you should validate your configuration of course that everything is entered as as much sensible as um as needed and of course the provider id and the create method to return the actual provider so let's dig into the provider so our penis user provider has a constructor and in this constructor i'm initializing a rest client the rest easy client for accessing the external service via chucks rs and for this i'm retrieving the http client provider from key cloak so that i don't have to configure my own http http client so i can reuse the key cloud internal http client and i can get this from the session get provider and with this http client i can build up a rest easy client and this client definition i have put to the peanuts client and in this peanuts client interface i have defined the various endpoints the resources which will get count in our called in our user provider so we have a list of the peanuts we have a count we can retrieve a peanut by an id and we get i can get credential data and we also can update credential data because our user service also allows to update a user's credentials or the password in our case so that's done in the in the constructor of the peanuts user provider and by the way all this code as in every video from me you can find the link to this code in the video description to a link on the github repository where you can look up the code in detail so then we have several other methods we have the close of course the default method for all these providers and we have the support credential type is configured for so that key clock knows which credentials to handle with this user provider in our case it's just returning that we are using the password the password credential model type is just password a built-in class of key cloak and we're just supporting password with our user provider if you support more than just a password with your user provider you can configure it here and this method just tests if a user when a user authenticates with a password for example if this user provider is suitable for using the password so we're just returning the result of the supports credentials type of this method in here is configured for then we have for the israelite and update credential methods i will come back to these later i will skip these at first and first of all we have the get user by something methods get user by id get user by username and get user by email so these are called by key cloak when working with the admin ui or when authenticating a user these methods will be called from the user storage provider and in our case i just created a common method find user in this realm for a certain id and with this method storage id external id of the key clocker this id is the concatenated id of the uuid of the federation provider and the id of the external system and to get just the external system you can use this static method external id and yeah this find user method here just um gets an identifier which may be the username the email address or the external id but the external id is the same as the username our example and the api is capable of retrieving a user by username or email via the same endpoint um yeah we're just trying to retrieve the peanut by the identifier from our client which we have defined here get penalty by id just regular checks rs syntax and if we get some data back if you don't get a web application exception perhaps because the user is not found 404 then a web application exception will be thrown by chuck's rs we're creating a new user adapter the user adapter itself is a custom class implemented in your federation provider and in this user adapter you're implementing all the logic which is needed for handling the attributes the roles and the groups and all the stuff our user adapter here in this example extends an abstract user adapter class where some methods are pre-configured and so you only have to implement certain methods you really use the abstract user adapter is kind of a read-only user so the default method methods for setting something are already implemented with default exceptions that the user is read only and the rest we can configure here we have the constructor where we set the actual storage id again with these storage id class and we're constructing the uh the storage id for this user and restoring the the peanut user itself in here in as a class member in this class and then we can operate um on this instance of this peanut instance throughout our class when um implementing the get username get first name get last name get email methods which are the special attributes for the user model in key cloak and then we have methods for handling the attributes itself and um important method is the get attributes method um to return all the methods all the attributes to in in one map to a key cloak including the username email first name last name and because these are special attributes um it's handled by keyclock itself to extract them from the whole map so when you're in key cloak and looking at the users at the attributes tab you only see the birthday and the gender because key clock filters out the username first name last name in email although we have specified them all in this get attributes method then you should implement the get first attribute method because otherwise knowledge will be returned as is the default implementation so please have a look at the default implementations of this abstract user adapter class you are extending and get user attribute stream pretty the same as the attributes but as a stream and then you have the option to implement uh methods for handling groups and roles so the get groups internal and the get roll mappings internal methods um you can override and in our case i'm just returning the groups and the roles the external api delivers me back as a simple set or of role model or group models and um i created uh the group model and the role model is an interface you have to implement your own classes and just ditch some simple default implementation of these interfaces there are no abstract classes available of these interfaces and um just a simple implementation returning the name and all the other things are throwing read-only exceptions that these role or group groups the same with the role pretty much the same are only available for read operations not for write operations so from our user adapter back to the user provider class we have the find user method creating a new user adapter implementation in this case i put it to a map first i tried to get the user already of this map if the user was retrieved before just to minify api calls and um yeah then returning the adapter our own user adapter implementation the user model to key cloak and key cloud can handle this user model for us so then further in this class we have forget users count method key clock needs this to know how many users are stored in the external data source to get the the slices how much users should be retrieved from per batch in in the admin ui when managing the users and then we have these methods for handling the user lists or in this case the user's streams because i implement the streams classes of this user provider and yeah we have several ways to get user streams streams of the user model of our user adapter and also in this case i have made a common to user model uh method where i map all of the data and depending on the method which is called i get a call the client get peanuts with search first result max results parameter if needed and then i have this to user model stream i just map all this the list retrieved from the external api to a stream of user models creating all so the the various user adapter implementations or the instances of the user adapters for our use case then you have to also have to implement some methods for the group members and if key cloak is looking for group members if you're dealing with groups and you can implement here a method to get the users back which belong to a certain group i skipped this in this example because i don't need it and so i'm returning empty stream and also the search for user by a user attribute stream um in my example it's not needed so i haven't implemented it just returning an empty stream then the last two methods of this um user provider are just um for um handling uh what happens if a new user will be registered in key clock by by registration or from the administrator if the administrator creates new users and here we can determine if the user is stored in the external data source and then returning a user model with the appropriate data and if not then just return null and the user is not stored and key cloak doesn't throw an error in in such case and we have the remove user method which is called when a user is removed of course so these are the methods for handling the users itself the user data the lists the streams and um the single users that get used by something methods and these um methods are coming from the user lookup provider interface and the user query provider user query provider are the user lists user streams and user lookup provider are the get user by username email whatever so for the user provider you don't have to implement just one interface at most of the other spi's in key cloud you have to implement several interfaces depending on your use case so the common interface is the user storage provider then you have the user lookup provider to look up a certain user a user query provider so for example if you don't need to handle a user through the admin ui and you just want to provide an authentication for this user then implementing the user lookup provider is enough you don't have to implement the user query provider interface this is only needed if you want to see a list of users in the admin ui or through the admin api and the last two methods we saw there at the bottom of this class the ad user and the remove user are coming from the user registration provider which handles if the user will be stored in the external data source and delete it or not so then we have two interfaces left and these are the credential input updater and the credential input validator and as the names of these interfaces say we can validate the credentials which are in entered by the user and we can update the credentials which are entered by the user so if the user wants to update the password we can propagate this new password the change up to the external data source or when the user authenticates we can just validate the entered data the entered credential with the data from the external database so let's start with the input value data the input validator brings the is valid method and in this is valid method we're just checking if the android credential is valid um and verified against the data of the external data source again in this example i'm retrieving the credential data from the external system and if we have a look to the to the api again we get the credentials from charlie and you see here we're getting um an encrypted value so no um clear text password is transmitted here it's just encrypted and we have assault we have an algorithm iterations and the type this is pretty the same data model key clock uses internally you don't have to use the same model you can use whatever you like and perhaps you have to send the password to verify in clear text to the external system maybe that's if this is a requirement yeah you can do it because at this point in in the method the isvalid method you have access to the password in clear text so be very um be very sensible sensitive with this information you get in this input just have to look um yeah you have this input you're casting into a user credential model i have this credential object and credential getchallenge response is the clear text password so please be sensitive with this data don't expose it where it doesn't have to be exposed or if you have to send it in clear text to some other system be sure that the connection is encrypted with tls https um or whatever so here you can compare the the data in my example i'm retrieving the credential data the encrypted credential data from the external system so no clear text password is transmitted and i just compare with the key cloak built in methods with the password credential model password hash provider and the verify method of this hash provider if the password itself is valid and if yes i'm returning true otherwise of course false and nothing else is being exposed somewhere so the same comes for the update credential method the update credential method is from the credential input updater interface and it's the same like isvalid vice versa you get the credential input from the user and then you can create your payload to update the password in the external system so in our case i'm just creating a payload of exactly this structure here and encrypting the password in key cloak with the pbk df2 sha256 algorithm with the default iterations and then sending the encrypted password with the salt to the external system with this method the update credential data so when i have here the credential data the credential data is just a java bean with these attributes it just retrieved before in the israelite method so and uh yeah the israelite method is um called when the user authenticates we already saw this when we um switch back to key cloak and start an authentication or a user and entering the user and the password the iswelt method is called uh the external data is being retrieved from uh key cloak from our provider compared to verified validated and in our case it was valid we can sign in and if we just enter a wrong data then we can't authenticate because validation paid in this case and again signed in so compared to other provider implementations other spi's in key cloak we already saw pretty much code in this user storage provider and the more code you write nevertheless every time you write code you should test it and the more code you write the more you should test it so how can you test all this stuff you just do here i also write wrote some tests of course i have the peanuts user provider test and as usual i write my tests with the help of the test containers key clock implementation and if you saw some of the other videos of these you already probably already know the test containers key clock implementation if not in the upper right corner you see a link to the video of the test containers key cloak project which i'm a maintainer of and in my tests i'm starting a pure fresh key clock container configured for my needs and can test against this key cloak container so this is what i'm doing here i'm just creating a new key clock container from the test containers project importing a peanuts realm and deploying my user storage provider classes which are compiled to target classes and i started in a certain network because i'm starting also another container but before we have a look to the peanuts realm jason and in this peanuts realm jason's just a very tiny representation of the realm it's called peanuts and the user storage provider is is configured already in this json we are importing in our user in our key cloud container sorry and we have the peanuts user provider we just have an id and we have the base url it's an internal url at the api hostname and we have the username peanuts we have a password test and the defaults the cache policy priority it's enabled so that it works and that's all we have in this peanuts realm json and um yeah we have the compiled classes in our target classes of our user provider and restarting it in the network together with a generic container of the image open rp and mock and the open api mock is a container to mock an open api specification in a generic way and together a running server with a working rest api defined by the open open api specification and i have this peanuts api yaml in here which is the specification of the api which is deployed currently in aws and with this definition a container is started open api mock container is started and which uses this specification to create the endpoints um also in the same network and i'll set network lias to this api container named api so that i can call this container via the api host name from key cloak and this is what you saw here in the peanuts realm i'm calling the api through the api hostname because i'm setting it here as the network allies yeah and then the containers are being started from text containers at the beginning of the tests and the tests are being executed just basic tests for um accessing the realms and then log in as user and check the access token if there are certain values in in contained in the payload of the of the token um test log in as a user with an illuminated password then we should get 401 test accessing users as an administrator to query the user to retrieve the user by an id and all this stuff we're doing through the admin ui we can also test here automatic and yeah everything works like a charm and we can be sure to have a working user provider at this point so just start these tests and look what happens just sorting it in the ide and then the tests will be executed make the window a bit bigger so the containers being started you can see it here container is started key clock is starting the api um container is already started this is much more faster than the key clock container itself and then the um various tests um starting to run and as you can see and at the left all the tests are green all the tests succeeding and after the tests run the containers are stopped again so this is all what it takes to implement a simple but powerful user provider or connecting external user data external user storage to a key cloak and having the the user data the users being able to authenticate with key cloak of course you extend can extend it in every way you like so you can add update functionality to update the user's data the attributes in the external data source you can split up the user storage so that you have some data stored or retrieved from the external data store and some data stored in the key clock sum attributes and you can also extend it with synchronization functionality so that the data for the external provider is synced to key cloak also periodically synced to key cloak whether it's all the users or only some users which are all already available in key cloak and so this is for performance reasons because the external system is perhaps a bit slower and does not have the best performance as we can sync the user data to key cloak and back important as the user storage provider it's not necessarily for migrating users you should do this in another way but it's a good fit to connect external data sources whether it's already existing or not to key cloak so i hope you like this video and give me some thumbs up if yes and subscribe to the channel of course so that you don't miss any of my future videos um i'm curious what user providers do you use put in the comments tell me your experience and your use cases and yeah hope to see you next time have a good time see you bye bye

Channel: Niko Köbler (@dasniko) - Keycloak Expert

Views: 29,455

Keywords:

Id: 1UklqPPjcRY

Length: 39min 28sec (2368 seconds)

Published: Tue Feb 08 2022