An Introduction to Elasticsearch (Building Elasticsearch application using .NET 5.0)

hello everyone and welcome to dot net course central in today's video i am going to talk about elasticsearch elasticsearch is an open source analytics and full text search engine elasticsearch is built on top of pachi lucin and apache lucien is a java library which provides powerful indexing and search features apart from elasticsearch in the elastic search ecosystem there are other products like kibana which is essentially a visualization tool for elasticsearch but i'm not going to cover that in this video it is for another video now let's talk about the use case for elasticsearch in which case we are going to use elasticsearch elasticsearch as the name suggests we should be using it anywhere and everywhere we have a search requirement traditional rdbms databases are not very good with indexing their indexing capability are limited and if we create too many indexes it becomes very expensive very quickly whereas elasticsearch is built for indexing and full text search so any search requirement we should be using elasticsearch the first thing i'm going to do is i'm going to install the elasticsearch and for installation i'm just going to use a docker image and this is the command for docker image it's docker pull and then elastic search version number seven or 11.2 if you just google for elasticsearch docker image you can get this command and next thing i'm going to do is i'm going to run the elastic search and for run the default port where elasticsearch is open is for 9200 so i'm going to again copy paste this command and run the elasticsearch and as you can see i'm running on a single node cluster if you are using it for a server infrastructure i would suggest using managed elastic search provided by either amazon or azure because it's going to be much more simpler that way once it is up and running what we can do is we can go to localhost 9200 and if we run this we can see that the docker cluster is started it's a default name docker cluster we could have changed the name to something else and it is up and running now if we want to see what are the indexes available we can just go and do slash underscore aliases and right now we don't have any index setup so for the purpose of the demo we are going to create a set of users under users alias or users index so for doing that i'm just going to use plain old postman so in the postman i already have some users so the url is localhost 9200 and then users is whatever index we want it doesn't exist right now but eventually it will and then underscore bulk is for bulk upload and then here we are going to pass the json object for the users and as you can see it's little bit different in terms of it's not a valid json it is not as you can see it is not encapsulated the second thing you have to remember is you have to provide a new line after the last record otherwise the bulk is going to throw an error and then we will be passing index before every record we send and given that we are using users and index we don't have to pass any index name here now if we send this request it is going to create the set of users and we should get created response back and as you can see there is no error and the users are created and every user has an id it's an important thing but not necessarily we need to know or use every time but there is a use of this one i'm going to show later so as you can see the index is user now if we get back to the chrome browser and if we refresh this now we can see user showing up as one of the index once the data is ready what we can do is we can go to localhost 9200 users which is our index and underscore search if you just execute this it is going to show all the users that we have added and you can see here the showing the user one then user two then user three user four and user five all the users that we added are going to show up here now the next thing what we are going to do is we are going to access this elastic search using our dotnet 5 application and for that i am going to create a net 5 web application and i am going to name it at elasticsearch.demo and for the framework as i said i'm going to use dotnet5 so in this case it's going to be asp.net core 5.0 and i'm going to create a simple web api application once the project is created i'm going to get rid of the default controller which is the weather controller i'm also going to get rid of the weather forecast model and then i'm going to create a new class and it's going to be the user that's the model and for the user we are going to have three properties string name age address the string here for age is not coincidental or by mistake it is intentional the reason is when we added the data we added age as a string so if we don't provide string here while we start using this object the elastic search api is going to fail unless we delete the data and recreate the data within as an age meaning we pass in here without codes this is an important fact that's why i wanted to share so that you understand that whatever data type is inserted through json same data type has to reflect in the dotnet object so once we are done with that the next thing i'm going to do is i'm going to create a user controller and i'm going to create api controller and it's going to be user's controller and here i'm going to get rid of the get it's not important i'm going to keep this cat and i'll change it to a string i'm going to keep the post it's going to be user similarly the response for the get will be user and then i'm going to get rid of the put and delete we don't need it and here for the time being i'm just going to return a new user so that we don't get any reds quickly now that this class is ready next thing i'm going to do is i'm going to install the necessary new get package for elasticsearch so the package for elasticsearch is nest so if we search nest this is the one we will be using and as you can see it's support.net standard version 2.0 hence we should be able to use it in dot net 5 application so i'm going to go ahead and install this once it is installed the next thing i'm going to do is i'm going to go into the startup and here i'm going to create an instance of the elastic client now elastic client depends on something called connection settings so we are going to use that first and for that i have to add the nest namespace so i'll add it a connection settings it has a default constructor plus it has a constructor which takes uri and few other variations now the one which takes uri is where you can pass the uri but if you don't pass the uri by default it will go to localhost 9200. so i am not going to pass anything because that's where my local machine is running at least and then what we can do is we can then add services dot add let's set singleton and here i can use i elastic client and for i elastic client here i can do new uh elastic client and pass the settings as the constructor parameter and if we don't want to create the instance like this we can always register connection settings into the dependency ejection and just expect it to be injected to elastic client but since we are not going to use connection settings anywhere it doesn't make sense just to add it to the dependency ignition container hence i'm not doing that so once this is done now the elastic client is available throughout the application so what we are going to do is first i am going to go here create a constructor and here i'll expect i elastic client to be injected and i'm going to add first the nest namespace and secondly i'll create a field for elastic client i'm going to get rid of the unwanted namespaces now when a get is done we want to return a user based on the username passed so i'm going to keep this still as id but we'll be expecting a name in the parameter so for that what we are going to do is there are two options first option is doing a direct get async which is just getting a particular user based on an id and second option is to use the search i'm going to start with search because that is what we are going to use most of the time search is the feature that we want to use from elasticsearch not necessarily getting a direct document based on id because if we have to do that we might as well just use a simple rdbms here in my example i'm just showing a string as a name but it can be any combination of search criterias so here what i can do is i can do a var response is equal to elastic line dot search i'm going to use the search async and here for the searches sync it is going to be the user because we are trying to search and user and then what we are going to do is we are going to use the lambda function of search descriptor which returns a search request so we are going to say start query and in the query again we're going to use another func define the query terms so we'll say query.term and in the term we're going to say name is the name property and here for the name value we are going to pass the id or we can say query dot match and we can say match on a field and the field we are expecting is field dot name and then query on the value which is the incoming id so that's going to be our response and then finally we are going to say return response dot documents i have to use an await statement here and i'll make this as a sync task of user i'll add the namespace system the studying.task and then here i can do dot documents dot faster default and for this i have to add the links namespace i can now get rid of this return statement so that's for getting an user based on id pass and then finally we can use this value to post but i'm going to do it later first let's run this and i'm going to change it into console mode so once we run this it's going to start up and then we go here to get user id try it out and we can say user1 execute and once we execute we get the index name as null because we have not passed any index name so there are two ways we can pass an index name the first way is we can pass it in the search itself and the second way we can pass it here in the connection setting so here what we can do is for settings we can go ahead we can say dot default mapping for and here we can see user and then it's going to take a mapping descriptor delegate and we can say here we can say what index name we are looking for and the index name is users for us so if we do that we have set up the index at the connection settings level so throughout this project everywhere we can just deal with this index but if we have to deal with different indexes for different queries we have to do it where we call it i'm going to show it in a minute first let's see how this works so now let's run it again and after the swagger shows up we're going to try it again and now here again let's give user one execute and this time you can see the response is showing up as expected now i'm going to show how we can do it in the search async request itself so we can get rid of this code from here and then we can come here and here what we can do is we can do dot index and at this place we can pass the index as users so now if i run it and i execute the same query i should get the exact same result so we can go we can try it out and if i pass user one and execute you can see we're getting the exact same result as before so this shows how we can use the search surchasing function of the elastic client to search on a particular index and as you can see we use the query term and then match to match a query now for the use case that we are using we really don't need to have term we can just use match and we should get the exact same result because we are just matching on a field and we are providing what is the expected name and if i run this now and i go ahead and execute this i should see the exact same behavior as before so the next thing i wanted to show is instead of using a search if we want to use a direct get now direct get becomes little bit complex because for direct get to work we have to provide an id an id is something which is coming from the create response so if i say var response is equal to elastic client dot get async and here i pass the user and then this one also we need to pass a new as you can see we can pass i get request or uh document path of the user so if we do that if we pass the document path we can say new of document path of user and then here it expects an id so again we can say new of id and for the ide we need to pass the identifier of this particular document that we are looking for now here what we need to do in this case is we need to pass id like this so as i mentioned earlier first of all it is complex and then i don't think it's very useful to search by id but i just wanted to show you that this is doable as well i can take the id from here and we are going to pass the id that we saw in the postman earlier but here cannot use document anymore instead what we are going to use is source and as you can see source returns a user object so that's all we need now if we run this application again and once it started if we go we try it out again but this time we provide the id from here instead of providing user one and execute i'm missing the index again so let's go and fix it so here i have to pass the index so after the document path i can take if you see that we get a get descriptor so we can say and this is going to give an option to pass the index that we are searching for and it is the user so let's run again and once it is up and running we can again go try it out pass the id and execute and we get the exact same result as before so this shows the get option and then finally i wanted to show how we can add a value to the elastic search programmatically not just using bulk we can use bulk also but if we want to add one user at a time what we are going to use so here what we can do again we can save our response is equal to elastic client dot index i'm going to use index async user and it's going to be await and then this one also is going to be a sync task string and i'm going to tell you why it is string and then after this is done we are going to pass the value which is the incoming value and then just like before we need to pass the index so we can say dot index and for the index we can pass the users and then finally we're going to return response dot id and this is the id automatic id that will be created when we make a post call so let's try this out so if we go here and run this and once it is started we can go post we can try it out here for name we can say a new name for age 25 and for address new address and if we execute see we got in the response body this id back and we got a 200 because it's created now to validate these if we go back to localhost 9200 user search and refresh this we can see the new name h25 address the user that we added is now visible here and we can see the id is this which is exactly what we got in our response so this is all i wanted to cover as a part of this one this is just scratching the surface of what all we can do with elasticsearch there are quite a few advanced features as well as using kibana for visualization which i am going to cover in future videos if you like this video please give me a thumbs up if you are new to my channel and if you think you are going to get value out of my channel please subscribe to my channel and if you have any questions or queries please leave it in the comment section below and thanks so much for watching this video

Channel: DotNet Core Central

Views: 9,943

Rating: 4.9338841 out of 5

Keywords: elasticsearch, elsaticsearch .net 5.0, elasticsearch .net core, elasticsearch .net 5.0 api, elasticsearch .net core api, search using elasticsearch api in .net 5, create elasticsearch index in .net 5

Id: 9tkrDqMbFMg

Length: 18min 51sec (1131 seconds)

Published: Sun Mar 14 2021