Installing and Configuring Elasticsearch and Kibana 8.x

hello everyone welcome to my channel in this video i'm going to be installing elasticsearch and kibana in a lab environment at home so that i can send my firewall logs to elasticsearch and use kibana to build visualizations and just make sense of the data and see what's happening on my network so elasticsearch uh this is their website elastic.co uh it's basically a free tool a free search engine at the end of the page here it says the products that started it all elasticsearch and kibana so let's search is this search and analytics engine kibana has the tool that manages the elasticsearch cluster and uh you can use it also to look at the data and build the visualizations and see what's happening on the network so i'm going to show you here the whiteboard how my setup is so i'm going to be i have a virtual machine that i'm going to use to build uh or to install elasticsearch and another virtual machine to install cabana and then kibana will look at the data in elasticsearch and then the data in another video will come from my firewall to log stash i'm going to be building this machine and the log slash will use filters to parse the data and sends it to elasticsearch so right now i'm going to i'll show you here i have an esxi servers i have two virtual machines i'm going to use them for elasticsearch and cabana uh it doesn't matter the name here this is this is the one that i'm going to use for elasticsearch and if you just google install elasticsearch with rpm i'm going to open open that and then i have the virtual machine right here and these are the instructions i'm going to just copy and paste so first of all we have to import this key copy paste and then it says create a file called elasticsearch.repo and and this uh directory here so i'm just gonna use nano and then call it elastic search dot repo and then i'm gonna copy and paste this content right here and right now it's gonna look for version eight originates is out so control x and yes to save and all i have to do is run dnf install and enable repo elastic search i'm already root so i'm not going to use sudo so it's going to install the latest version and it will say here elasticsearch installing elasticsearch i'm just going to say yes and then i'm just going to show you here down in the construct in the instructions they can download it and install it install it manually as well but now elasticsearch and version 8 it enables security by default and the previous versions you have to enable it explicitly but now it is enabled and you have to if you want it disabled you have to disable explicit explicitly but of course it's not recommended so now it gives you a password for this elastic built-in user it's the super user and then it will create self-signed certificates that you can use to authenticate clients such as like kibana it will let kibana connect securely to elasticsearch so i'm just going to wait for this installation here it will basically print all of this information on the console after it finishes or similar information okay it's installing 8.2.0 it's almost finished okay see those this information here is basically the same as in in this page so it says it gives you the password of the elastic built-in user and it says you can change the password later at any time i'm gonna continue to enable as search because it's not enabled by default so i'm going to go here it says running last search with systemd so i'm just going to copy this command paste it and then copy this next command and start the service and after that i will be able to change the password and i can use this command here to start a service okay it takes a few seconds to start the service the first thing after i'm going to do is uh change the password instead of using this complex password it is recommended to use something complex but right now for my lab i'm going to put something easy so we'll go usr share elasticsearch bin elasticsearch and there are all of these commands and i'm going to use this one reset password and there are some options i'm gonna do use dash i for interactive if you don't use dash i it will just generate a random password so dash i and then dash u to specify the user and the user is elastic press enter and it should prompt you to change the password this will prompt you will be prompted to enter a password i'm going to say yes i'm going to use something very simple re-enter okay password for the elastic user successfully reset and you can check the status of elasticsearch service by system control status elasticsearch it will say active and running and now you can also check about the running node by using the curl command but now you have to pass the cert which is in the configuration uh directory here let's see elasticsearch certs and it is http underscore c8 or crt and you have to use the user elastic and you're we're curling this the local host so local host and port 9200 this is what it listens on you put the password for elastic and it will basically give you this output it will give you the name of the node the cluster name by default it's elasticsearch cluster uuid and then the version tells you some information about the version and the build okay it will give you this tagline here we're going to go to the configuration file for elasticsearch now so it is in nc elasticsearch and elasticsearch.yaml but before that i'm going to make a copy of this configuration and the same location i'll just call it the ammo backup so now i'll have this backup file as the original file i'm gonna and then i'm going to be working on this file and this is where you change the configurations for your node so the most important ones are the node name so i'll go down here node name just call mine chamber one later on i'll add chamber two and three to build a cluster and path data you can you can make a custom ones here but i'll just keep the uh the default ones path logs and another important settings here is the network host so my um let me see the whiteboard this one into 160 25.100 this is what i'm using 25.100 not request is the ip address of this node last series node and then the network port by default it is 9200 but i'm just gonna uncomment that and that's all i need in this configuration file control x yes and to listen to the 9200 port i have to enable that on the firewall or allow communication to that port so i'm gonna go firewall dash add port 9200 tcp permanent and then i'm gonna use as well udp just open the port and tcp i'm gonna save firewall dash reload now we can check the boards that are open and you can see the ports that i opened 9200 tcp and 9200 udp so this is basically for elasticsearch and now i'm going to be installing and configuring kibana same thing install cabana with rpm and i'm going to import this key i'm going to create this file i'm going to copy this content here it's very important to have or to install the same version of kibana and elasticsearch with an rpm package they will install the same version or the latest version they will also always fetch the latest version so i'm just going to say yes to start the installation and here in the in the documentation it will it will mention the elastic built-in user super user and how to change it and how you can create this enrollment token to get kibana to authenticate to elasticsearch but i'm going to be copying the assert from elasticsearch to kibana and that that's what i'm going to use to authenticate in addition to another built-in built-in user i'm just going to wait for the installation okay i'm gonna run kibano with systemd enable the service so that if the machinery starts it starts combine automatically okay so now kibana is installed i'm going to go to the configuration file it is at see kibana kibana yamo and i'm going to do the same thing also just copy and create a backup of this configuration file okay and the most important pieces here is the server port you can use a custom port when you connect to kibana you have to use the ip address of kibana or the fqdn plus this port server host which is the ip4 kibana and i'm using 192 168 25.120 and then down here this is cervido public base url i'll change this later and i'm going to use elasticsearch.ssl.certificate certificate authorities it is somewhere down here oh i'm going also to configure this username i'm just gonna say password and on elasticsearch i'm going to change the password for this built-in user kibana uses this user to authenticate to elasticsearch and on and also this the certificate so i'm going to look for this setting right oh and this is another important piece we have to specify which elasticsearch host we're going to connect to and because we're using usernames and the certificates i'm going to use https and this is the ip for the elasticsearch node it's 102.168.25.100 and one more where is just kind of plastic surge ssl certificate authorities this setting right here so i'm going to copy this certificate from the elasticsearch node i'm going to put it in this directory in cabana i'm going to copy i mean but it's saved so right now i'm going to do two things so secure copy let's http 32 root uh diabetes of kibana and to this location and before i move it i'm going to make a directory and cabana so i'm going to copy secure copy from elasticsearch search this cert to my cabana instance to this location yes and then i'll use the password for planner and now i moved it is right here and another thing here is i have to change that password and the built-in password in inelastic search so elasticsearch dash reset what was it yeah reset password dash i for interactive dash u and you specify the user which is the kiban underscore system i say yes and then i'm gonna use that simple password of password okay so from the kibana instance i change the password of this built-in user and i copied the certificate the built-in or the self-signed certificate from elasticsearch to kibana i'll go to the settings folder settings again settings file so server port server host elasticsearch hosts that command will connect to the username and password and the certificate is specified right here this one one more thing we have to do in kibana is allow communication to that port 5601 the default one this is what i'm going to be using when i connect to cuba 5601 tcp gonna reload just all just make sure okay this is the port i opened and now i can use this link to this link to this ip to connect to kibana with this port let me see here after i change the settings i have to do system restart cabana service let's see if it failed or not no it's active so now it sometimes takes a few minutes or a few seconds for the service to come up but it already is came up it's our command server is not ready yet so we'll have to wait a few seconds after the service restart and you see this warning here it says uh in the production environment it is recommended that you configure this setting it is i'm going to show you where this is go to the cabana yaml file public base url we'll put the ip address of the cabana instance 25 to 120 with the port save yes and after we save we have to restart it and then i will have to wait again for for it to come back but that's okay this way we won't get that warning message status is active on running so far it's looking good cabana server is not ready yet so i'll just wait and we have a login page so everything is successful we log in with the super user elastic and whatever password you put and this is basically a cabana user interface where you manage your elasticsearch cluster and look at your data right now we don't have any data so and when you can add a sample data just to see how it looks like for example you have sample ecommerce data sample flight data i'm just going to add this quickly it will add the data to add an index i'm going to say view let's view the dashboard just to see how it looks like i can go back let's say last 30 days you can see you can build dashboards of whatever data you have i'm going to send the firewall data my firewall at home to elasticsearch and build dashboards from there so this is all i'm going to put these side by side if you need the configurations so in elasticsearch yemo the node name path data path logs i kept them by default network host the last search node ip the port and this is all by default enabled and in kibana what we changed is or what's important is the server port the server hosts the ip of kibana and then this is just to remove that wording on the login screen tells it last search hosts it needs to know which elasticsearch so this is the ip of elasticsearch and then the username cabana system password and i copied the sell sir self sign certificate from the last elasticsearch to the configuration directory or to this directory of kibana where configurations are and don't forget to open the ports 5601 for cabana or if you want to change it and 92 uh for the elastic search okay that's all for this video thank you for watching and i will see you on the next one configuring vlog stash and sending my verbal data to elasticsearch

Channel: Ali Younes

Views: 73,266

Keywords: Elasticsearch, Kibana, Linux, Syslog

Id: kkrLanotz1I

Length: 27min 23sec (1643 seconds)

Published: Sat May 14 2022