Install ElasticSearch Logstash and Kibana on Windows 10 (ELK Stack) (Elastic Stack)

hey guys in this video I'm going to show you how you can get started with elk stack on your Windows 10 operating system so I'm going to show you how you can install elk stack on your Windows operating system and then I'm going to show you how you can get started and start using elk stag on your Windows 10 operating system now keep in mind the setup I'm going to show you will help you to get started with Aleks tag but in production environment this setup will look very different so first of all what is Alex stack so elk is an acronym for three open-source projects which are elasticsearch log stash and Cabana elasticsearch is a search and analytics engine log stash is a server-side data processing pipeline that ingest data from multiple sources simultaneously transforms the data and then sends it to a stash like elasticsearch and Cabana lets user to visualize data with the charts and graphs and Cabana is browser-based so you don't need any other tool to visualize your data other than your browser also there are many auxilary tools available in addition to this alt stack for example beets and other stuff which you can use with this L stack and the elastic stack is the next evolution of the elk stack so let's see first of all how we can download and install log stash elasticsearch and Cabana on our Windows operating system the only prerequisite of installing elastic stack or elk stack on your Windows operating system is you need to have Java JDK installed on your windows or any other operating system so once you have installed a Java on your Windows operating system just go to your browser and search for elastic stack download and the first link which will appear here will be from elastic go forward slash down low so just click on this link and once the link is open you will see elasticsearch Cabana beads which is also now very regularly used with the elastic stack and then logstash here and other stuff here so what you need to do is for now we are going to just download elasticsearch Cabana and the log stash so just click on the download links of all these three sections and you will be able to see under elasticsearch webpage this windows options so just download this Windows zip file this will be downloaded as a zip file so just click on this windows option same you need to do for the Cabana so just click on the windows option and the same we will do for the log stash so for the log stash you are going to download this zip file option so once you have downloaded all these three zip files I'm going to transfer these files to the folder of my choice so generally I just keep them under my sea directory and I have created a folder called elastic stack here so this folder right now is empty so I'm going to transfer all these zip files there and then I'm going to extract them one by one so now once all these three folders has been extracted first of all we are going to start the elastic search server so I'm going to go inside the elastic search folder and then I'm going to go inside the bin folder and then I'm going to copy this part and then I'm going to right-click on my windows icon and then open the command prompt as admin so I'm going to open it as administrator and then I'm going to CD to the directory where this bin folder is there and then I'm going to start the elastic search dot b8e file so I'm going to just press Enter which is going to start the elastic search server now it might take some time to start this elasticsearch server so just wait for some time and it will be started and if the elasticsearch server has been successfully started you will be able to see an IP address which will be the localhost and the port nine to double zero by default so on this IP address and this port the elasticsearch can be queried now it's important to note here is that the elasticsearch is a rest-based API so to query elasticsearch you need to send rest based query so the simple way to test if the elasticsearch server is working or not is to go to your browser and then just write localhost colon nine to double zero which is the default port on which elasticsearch runs and it's going to show you all these informations so it's going to show you the name of your system and then it's going to show you the cluster name and the version of elasticsearch which is used which is seven point three point two in our case so if you see this kind of response that means elasticsearch is working fine and if you want to query more to the elasticsearch then you need to use tools like curl or postman to send the rest based queries to get your replies from the elastic search now in the next step we are going to start the Cabana so just let this elastic search terminal run and we are going to once again go to the folder where all these folders are there and this time we are going to go inside the Cabana folder so we are going to go inside the Cabana and then into the bin folder inside the Cabana and once again I'm going to copy this part and then open the terminal as admin and once the terminal is open we are going to just the CD to the directory where this bin folder is there and I'm going to press ENTER and here I'm going to just write bhana dot dat file which is going to start the Cabana so here also it's going to take few seconds to start and once cabana is started successfully it will be started at localhost port five six zero one so to check whether it's really working or not you just need to open your browser once again and here we are going to just open cabana on localhost port five six zero one and then press Enter which is going to start cabana and now you can see cabana has been started and you can see all these UI options so here if you want to expand all these menu options you can just use this last option here to call collapse or expand and here you can visualize your data which is there in elastic search using cabana in the form of graphs or other formats now let's see how we can use log stash with Cabana and elastic search so I'm going to minimize this browser once again and keep in mind both Cabana and elastic search are running on different servers so I'm going to go to the folder where all these three folders are there and this time I'm going to go to the log stash and inside the log stash folder once again in the bin folder I will be able to see this log stash dot batch file which is the windows batch file which I can use to run log stash but before that we need to create a corn file in order to give the configuration for the log stash but before that what I want to do is I want to add this log stash and kibana and elastic search path into my system variables so I can open this log stash command from anywhere in my file system using the command prompt so for that I'm going to just copy this part and then I'm going to right-click on my windows icon and here I'm going to go to the system and once system settings are open I'm going to go to the advanced system settings and here I'm going to go to the environment variables and then here in the system variables I'm going to click on the path system variables and here I'm going to add one more system variable which is the path of the logstash bin file same I will do for the Cabana and the elasticsearch bin folders also so first copy the path of the Cabana bin folder and add it to the new system variable and same we are going to do for the elastic search so I'm going to go inside the elastic search bin folder and I'm going to add it to my system variables now I'm going to just click on OK ok and once again I'm going to click on OK so now in the next step what I'm going to do is I'm going to create log stash conf file so I have my Visual Studio code editor and I'm going to create this log stash corn file so I'm going to just write log stash dot scorn now this log stash conf while you can create on any location wherever you want it's not necessary that it has to be there inside the log stash folder it can be created anywhere in your file system now there are three important parts inside this log stash conf file also it's not necessary that you need to give the same name you can provide any config file name to your log stash conf this conte file have three different important parts first is called the input the second is called the filter and the third is called the output now keep in mind that this logstash config file format is not adjacent format it's something proprietary so it's not a JSON format now here two things are important input and output options filter is an optional option so for now I'm going to just remove this filter now logstash config file is going to look like JSON file but it's not a JSON file so you just need to give input and then the curly brackets and here also output and the curly brackets now as an input we are going to provide the input as the standard input so I'm going to just write STD I n and then just space and then curly brackets and in the output we need to provide the name of the technology where we want to send our data so for now I want to send my data to the elastic source so I'm going to just write elasticsearch curly bracket host my elastic search is running on the local host : 9 to double 0 that's why I have written this host here and the port you can also just spread this something like this so it's more readable and once again inside this elastic search you can also provide the index of your data so you can just write index and then this kind of arrow and then in double quotes you can write the name of your index so here you can just give any index name whatever you like so for example I am going to just give the name index for log stash or any other index name whatever you like so once you have created this log stash Khan file we can use this log stash Khan file using our log stash command now when you go to the log stash download page also and when you scroll down you will be able to see all the installation steps here described also so we have created our logstash cons file and then to run that logstash with the config file we just need to give this command so just call the log stash and then with hyphen F option we can provide the path of our log stash dot conf file and because we have added log stash to our environment variables we can just right click and open our terminal as admin and once the terminal is open we can use this log stash command from anywhere in the terminal now so we don't need to go to the log stash bin folder in order to run our log stash come on so because my file is here so let me just copy this part so I'm going to just copy the path where my log stash dot conf file is there and I'm going to just CD to this folder so CD and there is my log stash file located so here I'm going to just give this log stash command and then I'm going to give the path of my log stash Khan file so log stash and then minus F and then the log stash corn file ok so because we have added this log stash binary folder into our environment variables that's why I don't need to go to the log stash binary folder to run this command so I'm going to just press ENTER so now my log stash server has been started but it's complaining that my index name must be in lower case so I'm going to just close this server by pressing ctrl C which is going to stop this server and once the server is stopped going to just go to my log stash dot conf file and I just need to create my index in all lower cases so I have just given this index in all lowercase letters now and I'm going to save this gone file and once again I'm going to go to the log stash command prompt and first of all let's clear this terminal by pressing CLS on Windows terminal and once again I'm going to just give the same command which is log stash - F locks - dot conf file so now it's not giving me any errors so I have some comma separated values data so I'm going to just copy this data which I have and then I'm going to try to give it as the standard input so I'm going to just paste this data which I have copied and you can see this data which I have copied is now sent to the elasticsearch you can see successfully started log stash API and whatever data have copied and pasted here on the terminal it's going to go to the elasticsearch so once this is successful we are going to go to the browser of Kabana and we are going to go to the visualized section here so this is the visualizer section and once this visualizations check section and once this visualization section is open you can see here create index pattern and you will be able to see our index which is index for logstash this is the index which we have given in the logstash config file so I'm going to just give this index in the index pattern box here and it says successful when I enter this index inside this text box so I'm going to just click on the next step and now we just need to give the time filter field name so by default we just need to choose at the rate stamp which is the default time filter field name and then click on create pattern which is going to start creating the index pattern for your data and once this is done you will be able to see that our index is created with the name which we have provided which is index for logstash in my case and you will be able to see all these default values here so all the data which you have sent from your log stash command is going to go under the message name here so to visualize this data just click on discover here and then because you have only one index which we have created you will see the index name here and the time at which this data is gone into the elasticsearch so you can see the count 14 which is sent at the time you can see the bars of the count of data which has been sent into the elasticsearch from this input so when you expend this section here you will be able to see under the message whatever message which we have sent using the standard input so for example 5 1 2 1 and then this address is there so I'm going to search for 5 1 2 1 in into my data which I have sent so you can see 5 1 2 1 and then this is the address which I have sent which is clearly visible here also so this is how you can start using the Elks tag on your Windows operating system in the next videos I'm going to show you some more features about elastic stack so stay tuned and I'm going to see you in the next video

Channel: ProgrammingKnowledge

Views: 212,113

Keywords: ElasticSearch, Logstash, Kibana, ELKStack, ElasticStack, Windows 10, Filebeat, scalability, reliability, pipeline, Elastic Stack Tutorial

Id: 8iXZTS7f_hY

Length: 19min 46sec (1186 seconds)

Published: Mon Sep 23 2019