install kafka on windows 11 | Install Kafka 2023 | install kafka on windows 11

hello everyone in this video we will see the installation steps for Kafka and how we can easily install without any fault in our Windows second thing is we will see the basic operations of kafa how the producer and consumers are sending message and receiving it one after another and for those operations and installation we need to First understand why there is a Kafka and what the need of it so in the real world application if there are two servers Let It Be server one and server two which are interacting over one another always there is a storage system which are available in between for understanding let understand that if there is a database which is storing the messages and server 2 is using those messages after reading it from the database then there is a interaction happens from server one to server two now the question arises what's the need of database why don't the server one sends the message to server 2 directly the reason for that is if server one keeps on start sending the message and if there's no database and server 2 is not available at that moment then there are chances that message will be lost for those scenarios to handle the message lost scenarios it is better to store somewhere and then send it to server to or server to will start consuming it from the storage storage device which is available now for understanding if there is a database which is storing those messages instead of any messaging service like kafa the problem arises when there is a large volume of data which are being produced and which are being consumed from large pool of customers if there are millions and millions of customer which are actually producing or which are actually consuming those data the chances of the database to go down is very high because the throughput or you can say the read and write operations which are performed in any particular system is called as throughput so if you talk about the database the throughput is very low so the read and WR operations are not performed very easily at very less moment and through high volume in the database so then comes the picture of the messaging cues like Kafka so Kafka is actually fall tolerant you can say and gives high throughput and stores and receives large volume of message through consumer so consumer can get large volume of data in the Stream or pipe format where the message will not be lost because it's a fa tolerant and the read and write operations can be performed easily from server one to multiple server and high throughput can be given at each moment now we have understood why there is a need of any storage devices and why there is a high throughput and high volume uh storage devices are required like Kafka or some other cues now let's understand the architecture of Kafka so when Kafka we talk about they are divided into multiple Brokers and these Brokers are actually kind of you can understand as a separate tables or separate segment of databases which are there in the cka and these Brokers are further divided into separate topics so if I talk about any particular broker broker one is segmented into topic one and topic two likewise broker 2 can be segmented into multiple topics so the segregation is required because different pool of customers can read the data from different Brokers or different topics now these topics are further divided into partitions so we can get the data from the granular level of three level like partition topic and broker now we have understood through different segment we can read and read and write the operations and give get high throughput so now comes why it is a fault tolerant so if you talk about fault intolerant it is always backed up by zuker so zuker comes into the picture and Luke keeper tells I'll work on the worker slave pattern so if any of the slave server gets down or workers slav gets down then we will pull out those slaves and make it as a worker so this is how the worker slaves work and the Zookeeper is backed up and gives kafa support of F intolerance so high throughput fall intolerance and high volume data is easily received now let's go and have the installation in our system of Windows and then we'll see the basic operations in this video we will download and see the basic operations of kka for that I written a FJ kka in my Chrome and I went to this k. ap.org and these are the basic things which are written like 80% of all the for companies uses is the scfa and some of the basic properties I mentioned here so if you can see these are the performance parameters which are available first and most prominent thing is the throughput that is most of the read and write operations can be performed in very less time so which is not available or which is not performed with DB and this is highly scalable and this stores the stream of data safely in the distributed system and can be used easily with through high fa tolerance so now let's go to the download Kafka section here I will download this file this tzz file and extract it in my D drive to be mentioned the other Drive which is not a C drive and I went to the Wi section then windows and you can see zke keeper server. start.bat file is there so I'm on my windows so I'm using bat file and this is the Z keeper. property which is available in the config folder so two parts one is for for starting the bat file in Windows section of Bin second is the properties from config so I going to Windows of Bin and then start at the server dobat and use the config folder for Zuki c. properties as you can see the error has came for this only I mentioned that this is other Drive which is not a C drive so for running properly you should not use the C drive instead not use the D drive but other way that is the C drive should be used so now I pasted in the C drive and I will extract it here again one problem will come I will show you how you can solve it so now I went into this folder again same path the windows and zookeeper server start.bat file then config for zuker properties so I'll went to the main folder and go to the command prompt and here I will use the same command go to the bin initially I went inside this go to the bin poter and then I'll go to the windows then I'll go with the Zookeeper server start.bat file then I will use a config from the config folder that is z keyboard. properties now again this will come so if you use very large folder sections one inside the another then there will be a problem in starting the Kafka so instead let me delete it and first of all exit it from here delete this folder and use the minimum folder which is available so that the Kafka can be used for run so I am extracting the file here instead to a folder so extract here so one folder section has been uh reduced now I went to this Kafka I'll reduce the name from this to Kafka only and inside this Kafka I will show show you again the things are there only I'll start the CMD and now if I'll use the same command then it will easily run my CFA for that first start the Zookeeper and config then go to the zookeper properties no now you can see this has already started so first thing to use the C-section that is C folder and never use a very large folder sections one after another instead it will be very less folders so that Kafka can easily run the files now after this let's start the server. properties server. properties is used for running Kafka now we have started the Zookeeper now we'll run the Kafka for running Kafka go to the bin then again the windows then Kafka hyphen server hyphen start. pad file for running Kafka config to be used not zookeeper but instead server. properties so as you can see CF has also started here so for running Kafka we should first start the Zookeeper to add 2181 then run the Kafka now we are again at the Kafka uh main website we'll first use the topic and we'll first try to create a topic so for creating a topic I can show you that I can again go to the CMD folder and let me show you how we can create the topic for that we'll go to the windows then go to the cfar topic. pad file for earning a topic then hyphen create then hyphen topic then my topic is that any name which you will give to the topic so I've given my topic then hyen bootst do server then Local Host so at which my Kafka is running so it is 992 so as you can see bootst hyphen server so hyphen server is there not a full stop so I'll will change it to hyphen server so after this my topic has been created Created topic my topic so after creation of topic we will try to publish some certain data in the producer add this file and try to consume it through consumer. file bad file so in the next command prompt I'm am going to Kafka console then producer so console after that producer the producer will produce the data we'll have to go to the section where we will find the data which is available so I'm going here to publish the data so it is saying go to the topic which you have created using the same syntax so topic and this is my topic so again I'm using this name only my hyph topic then again use the same syntax that is bootstrap server and Local Host 9092 so this is what I'm writing here and bootstrap server and Local Host 9092 so as you can see the console is already started it is saying to write anything it will be published to your subscriber so I'm writing hello so hello developers and again I'll do the enter you are on Play Java channel so this is how the message are being published in the Kafka so this is a publisher now let's create certain consumers for creation of consumer synx will be similar instead that Kafka hyphen consumer will be used Kafka hyen console hyphen consumer so this is producer and again in this window section only I'll use CMD and I will try to use same producer and consumer one after another so that it will be clearly visible when I'm publishing in Kafka the messages are being read through consumer so I'm using the same topic my topic then again the bootstrap hyphen server and again Local Host 9092 so my console of the consumer has started so whatever message I will write in the producer it will be displayed in the consumer so let me write certain message before that let me arrange this one after another so I'm writing here hi so you can say High has came but this is not clearly visible let me expand it little bit and whatever I'm writing it will be coming here so hi as you can see this a proder so you are on again you can see consumer is writing you are on Play Java channel so consumer is reading just after the other so this is how producer and consumer are working one after another I am producer and you are consumer will be coming in the consumer CMD so this is how producer consumer are working by through Kafka now let's create one more consumer so that you can see that whoever is subscribing to that particular Kafka topic all of them will be getting the same data whenever Kafka gives a stream of data so again I'll use this here so that things will be coming on the same page for this producer and consumer so I'll use this same command as the topic is same so I'll write Kafka console consumer bat then topic then the name of my topic is my topic only and bootstrap server then I'll use the Local Host 9092 where the CFA is running so Local Host 9092 so again I now I have two consumers so these all are subscribed to the same particular topic so whatever data I'm providing in the topic will be coming to both of them at the same time so as you can see I written CLS CS is coming I'm writing this syntax the syntax is coming again I'm writing let's write some certain data that subscribe b as you can subscribers are coming are listening all the messages are being sent rapidly so as you can see producer producing and these consumers are there which are consuming the data this is how the basics of Kafka works thank you all for watching the video

Channel: TrendingCode

Views: 773

Keywords: Apache Kafka installation guide, Kafka quick start guide, Setting up Apache Kafka, Getting started with Apache Kafka, Kafka ZooKeeper integration, Kafka configuration options, Kafka use cases and examples, Kafka command-line tools, Kafka best practices, Kafka documentation, Kafka video tutorials, Kafka online courses, install kafka on windows 11, install kafka on windows 10, how to install kafka on windows 10, how to install kafka on windows 11

Id: i7aeZcFhRAA

Length: 15min 8sec (908 seconds)

Published: Sun Sep 10 2023