Part 1-Start Apache kafka on Linux- Ubuntu OS, Apache Kafka Configuration, Start Zookeeper & Broker

hello guys my name is vithet i'm a data scientist and this is my second video here in my first video we installed the ubuntu operating system so in my comment box i already given the link you can install the ubuntu operating system before start work on apache kafka today in this video we will discuss how we can complete the prerequisite before to start work in apache kafka how uh we we would required a few things here we would require the java we would require uh to set up the uh log paths for zookeeper and kafka broker and then we can test the f2 if the zookeeper is running successfully and the broker is running successfully because without zookeeper the broker cannot work so let us start it today so here uh this is a window it's my ubuntu operating system so i'm just simply opening the terminal port once i go here i'm simply into the home window and i'm checking whether i'm having the java it's showing the command java was not found so we we do require to install the java first so i'm installing it um simply mentioning for uh apache kafka we would require at least version or jdk so we can have that so it's starting now so installation is going to complete now on the other hand uh we can open our browser also okay i am simply typing apache kafka um we would require to download the apache kafka whatever whatever the latest uh version available we would require to download apache kapka libraries so you can open this link kafka dot apache.org so and here you can see the download kafka button just click on it here you can see we have a binary download scala 2.12 so let's download this one here are the links going i'm checking whether i have downloaded it before or not so just checking that yes i have it so you can download it from after click on this link okay so let's see whether our java is complete or not yes it is let's check the version of the java and here yes our java jdk is has been installed on this urban operating system that is very good now let's move to the download folders let me clear this before and let's move to the download folder and here you can see we have this kafka the download file the libraries whatever the libraries are available let's unzip those let's write command tar xvf and the name of the file kafka underscore to 0.12 3.0.0 point tcc and yes it's going to unzip now this will be unzip in the same folder if you see that you can see here we have this ng folder right let's go there let's take it or first we can do one thing let's rename it it's a simple like kafka okay and let's cut this or copy this and put this on the home here okay so once we put this we can see it has a bin and config and lips and there are many other files available here so what would what we will do now let's check what is the path of this file and it's coming as home slash with it right so home slash with it slash kafka and we have these folders okay let's go to the command prompt again let's clear it let's go to the home window again and if you check here we have a folder here called kafka right uh let's use command ll it will show you some of the best uh one of the uh file here called bashrc here we can set up the path of our uh bin file basically and now after that further we can update the zookeeper and and and broker file so let's type another command nano dot src okay so if you come down so just go to the end of this file here you can your you will write we are just setting up a path we are nothing that's going to do this time just mentioning the same path which i told you it's a with it under that we have a kafka under that we have a bin file colon and a dollar symbol and path okay just uh with it upcoming yes just use ctrl s ctrl x okay and just uh let's close this window either or we can check cd kafka yeah so uh if you go into the kafka it is having uh bin config clips and many other folders right and we have set up a path they are till bin so whatever the file inside the pin those will come in directly these are called basically dot sh file sh file and here you can see there is a windows folder also so these are somewhere like in windows we have a batch file and here these files are basically uh the sh file is for ubuntu i mean the linux operating system so if you see one of the file like kafka topics dot sh this uh this batch file is used to uh is used to create uh the topics basically if you see here this zookeeper server start we use this best batch file to start our zookeeper server if you see this kafka server star dot sh file we use this file to start our broker so um you will get the whole idea about the apache kafka one by one but before that in this video we are just completing our prerequisites uh okay guys so we are here just check the path again i think we had some mistake in that file in path basically okay let's correct it again and that we know like it is home with it slash kafka sorry it would be for slash slash spin folder right ctrl x and if you try let's clear this window and i think it's the this is a normal command where you can see it's it's working fine or not found let's open this in a new window cd kafka folder and if you simply type kafka topics dot sh that's mean and if it is working you can say oh yeah java is stalled and this path is also working here so now uh we can see uh under this bin we have many other uh these are the search file and if you see uh the config folder we have the properties file like server dot properties and the zoom keeper dot properties so let's change those like first we will change the zookeeper dot properties i mean what we are going to do this time we are just setting up the correct log path here so if you see uh we have the kafka folder here and inside that we have to create a folder where we'll we will have all the data the data basically to maintain the uh the backup messages backup or data backup basically to for the kafka so we have a data folder under the data folder we can create two more folder light one is a zookeeper folder for zookeeper purpose and another one would be the server let's suppose okay so if you see the path of this file it is a home with its kafka data and we have a zookeeper right so let's go to that and let's edit it okay let's come down and here you is you see the data directory is a damp zookeeper we have a zookeeper but before that we have a home we have a with it then we have a kafka slash data right this is your folder so just change it by the client port is 2181 for zookeeper just control x control x right the same thing we will do with the zookeeper the server dot properties and if you go down you can see the broker id was zero there and here we have to again change the path so use the kafka log slash home slash with it slash slash data slash server okay ctrl s control x this also change let's clear this and let's try if our kafka zookeeper is working fine or not dot com start oh sorry let me check what are the files here i mean uh so guys uh you can see the zookeeper is already running here and as i told you like the port is 2181 which is inside the zookeeper.properties now we can start our uh broker here so for then we can write the command like capco server start what is config.server.properties so let's see if it is running successfully or not it is required the zookeeper should be run must be run and here is the broker id 0 which is mentioned in the server properties and yes it is working now it is connected as you can see like that port for the uh broker basically the kafka broker is nine zero nine two um in my next video uh we will start to work how we will create the kafka topic this was the part first in next video we will create the topics we will create the producer and on the another hand the consumer producers will send the messages to the topic on the topic and the consumer will consume those messages using that same topic so this was the part first video where we have the all prerequisite how to set up the kafka how to set up we required the java also and how to start zookeeper how to start a broker if you required anything if you like this video please like it or any comment please like please mention the comments in further videos we'll start the producer consumer and then we will have consumer groups then we will have replications and partitions things so you will learn everything one by one so whatever i have done practically i will fully explained here in my videos so thank you so much for this time and we will meet in next video if you require the ubuntu server you can go to my first video where we first set up the ubuntu server operating system even you have a windows operating system on that you can install the ubuntu operating system please watch my first video and then you can set up the kafka server here and further we can move ahead the stream thank you so

Channel: TechMaster

Views: 30,134

Keywords: Ubuntu OS, Apache Kafka, how to start apache kafka on linux os, configuration - setup kafka, apache kafka zookeeper, apache kafka broker, start zookeeper and broker in kafka, create kafka topic, create kafka producer, create kafka consumer

Id: tFlYoEJsT2k

Length: 16min 23sec (983 seconds)

Published: Wed Jan 05 2022