Kafka Installation without Zookeeper || Kafka installation with KRaft || Chapter 1 || Day 7

in itself is a very interesting if you're coming directly to the to this video I'll suggest you go to the theory part where I have explained how it is possible without the keeper to install and have the Kafka so uh just let's jump to the installation part so all the setup and uh the setup file that we downloaded in the previous video uh from this location from this location it is similar it's just that we have different set of commands for this one so Kafka with these are the steps that I have to do so let's go ahead and do this to config directory here there is a separate folder for Kafka so just uh close all these tabs broker.properties is there uh similar some of the configurations are similar that we have seen previously uh in when we were discussing with zookeeper controller properties and server properties these things are there so there is no mention of Zookeeper here so these are the Kraft specific only so let's see what the steps are there so these steps are for Mac actually so if you go uh let's try to do this on Windows how it works so coming here so just uh Ctrl C so that it is stopped the is the previous installation uh Joe keeper and Kafka so now just clean this one and let me expand it a bit so that it is visible so Kafka cluster.id um okay it is referring to bend directory but we need Windows actually dot slash this one [Music] so yeah I think uh this these command lines are for Mark machines so on Windows they are not working as expected but I we can get this work okay so what it is doing actually it is uh calculating a Kafka cluster ID and this Kafka clusterity is a refer it is referring from here only okay so I can do that I can calculate it like uh normally and then set it wherever it is referring okay so let's see CLS uh then uh windows and then what is the command Kafka storage when those windows inside that Kafka configure Kafka storage dot part it is there yeah stories not bad and then command line I have to pass random uoid so if I hit it it will give us some random ID it has given us the uh this random ID so what is the next command next command is format log directory so this is the command line that I have to give here dot slash then windows and Kafka storage dot bat then for Mark and then this is the dollar kapha cluster ID so we have to pass put ID we just created and then config file C config craft see dot slash config Kira server.property so this server.properties we also have to do one change before it is executed actually so coming here so we had created one data directory when we were installing with the Zookeeper so let's delete it for now in This Server dot properties so look for the logs dot dir so same change I have to do here as well so data let's do it and then hit this unrecognized option it is saying unrecognized option so what is the problem cluster ID I can see maybe it will uh accept it in double quote foreign T also I missed that yeah formatting this uh data it is saying this version 3.4 so see this Kafka is data folder is created method properties so this is the ID that it was changing and bootstrap checkpoint file is there so the next step is that we can start the uh this Kafka server start directly so the notice here is that config file that we are referring now it is from kerav here this server.properties okay so just clean this up and slash Bin Windows uh Kafka start server start this is the same bat file same file is there uh just that we have to change the config Kraft server.properties that is it so if we hit this it should start our Kafka server we do not need zookeeper here registering this one controller all that log is coming okay so it is cannot invoke Melissa replay verses because the return value of server shared was a wrap manager is none okay this is a strange uh it should have not come actually uh I'll pause the video for some time and we'll look for the errors and see how we can fix it so we'll continue in a while so yeah coming back to this one so I posted this error that was coming here on stack Overflow I looked around whether I can get the solution of this one I posted this on this stack overflow with the option I get got one answer actually it is not Kafka isn't tested on Windows can you use WSL too instead wsl2 means you have to install the Linux on Windows itself so it is supported and tested on uh Linex machine if you try on Windows it will give us errors so for till now in this version I have not got the solution of this one how to start it but whenever I get it I'll keep the thread updated how to fix this one but for now we have to live it with it that on Windows it is not possible so just to show you the demo part I am going to jump into the uh jump into a Mac machine and show you uh this in Kraft mode how we can start it so this video is like gone too long due to this one uh but yeah it's it's good to see because we have got this error right so let's jump to the Mac machine quickly and see this is startup so once it has started actually whether it is in Kraft or zookeeper for clients producer clients and consumer plugs it doesn't matter what you are running but it is going to work fine so just to show you the demo part I'm going to install this k-raf on Mac machine separately so the setup is same that we had on Windows machine so I have opened the same file on the intelliser ID itself if we go to the config so broker controller server dot properties inside server.properties I just do the same change that we did on windows so that we can see the folders in here directly so this is the only change that I have to make it even on the mark it is very quick to do same quickest same quick start guide I am going to follow Kafka with Kraft so just copy paste this one see the output of this one just uh try to print this most pieces it is just a random ID okay so the next is copy paste this second command that we had it's formatted so now we should see that data folder here so Kraft meta dot properties so this ID is here wct the similar the same one right next command is directly we can go ahead and see this [Music] next command is actually to start the server directly so if you see it will start the server for us uh Kafka yeah all the default properties you see right all the things are working properly here everything just to make sure it is working fine quickly let me go ahead and create a topic put some message and do that we'll be doing these clis and all that later on also but look quickly just to make sure because this is we are in Mac machine on this Kraft board just wanted to quickly show you that other files you can see combined logs and all that we'll be talking more about this one so next uh next one is the creating a topic and doing all that stuff so just go ahead and copy this we'll talk more about these commands what things are there but for now just copy paste these things so that we have desktop so topic it created if you want to see the listing of the topic how many topics are there in the system I can go ahead and list the topics by changing this bit of command uh so quick start the name of the topic is uh here by mistake this create should not be there uh list should be there actually okay yeah you see uh you can see this uh I'm sorry a font is a bit uh small here uh you can see this uh quicker start events right it is coming right right down here okay so if I have to push some uh event into this one uh you can have the description come on also also you can see these things so let's write some data into this one um it is going to give us the console for producer which we can use I first message you can see I'm typing something here my first this one so let's have consumer for this one where we can consume these events read these events open one more tab here like the console so you see consumer dot sh topic and then quick start events is there so it is going to listen from cookie star this is the flag to show that it is going to read events from beginning and this is the bootstrap server localhost 9092 where it is running so let's produce new events from here so something new yeah these new events are coming over here so you see all that new message are read properly here and the respective log files is created you see the data Kraft combined logs are there so I'll talk about more these folders uh in immediately next video where we are covering the Kappa CLI okay so we have covered both Kafka uh installation on Windows half consolation on Mac Big Joe keeper and without zookeeper I hope you found it useful uh so see you in the next one man we are covering the CLI part

Channel: Codefarm

Views: 3,462

Keywords: apache kafka, kafka basics, kafka introduction, kafka for beginners, kafka explained, learn kafka, kafka architecture, kafka use cases, kafka setup, kafka messaging, kafka topics, kafka producers, kafka consumers, kafka offsets, kafka partitions, kafka replication, kafka broker, kafka cluster, kafka applications, kafka use cases for beginners, understanding kafka, getting started with kafka, kafka interview questions, kafka interview preparation, architecture of kafka

Id: aH3-UjIrnRs

Length: 13min 9sec (789 seconds)

Published: Sun Apr 30 2023