Real-Time Live Soccer Score Streaming Application Demo with Reactive Spring Stack

my name is Adam guinea I work as a principal engineer in a truck sale today I will try to share what we have done in one of our most recent projects where we have used reactive spring stack for rip lat forming re writing one of our legacy applications that was not performing very well we were not happy with it and by using reactive spring stack reactive reduce reactive kafka we have come up with a brand new application which is performing quite well since March this year I'm gonna go over select what we have down there I have got a couple of slides to share and then I will make a demonstration of the application from production real-time and then I will dive into some pieces of code to show you what we have done basically how we have used the reactive spring stake does anybody remember this slide from Tuesday the first day's main stage keynote from Lea this is basically why I am here because I like sharing what I know what I learned what I do what I gain and it was really like reflecting my feelings what she has said with these similar feelings the same feelings I have initiated a spring meetup group in Istanbul almost three years ago and we are now like 4,300 members and we have been organizing sessions on spring technologies spring stake in Islam woman in Barcelona in Kingston we also organized twice spring one two activity in Istanbul where we had 300 plus attendees local attendees in the spin-1 truth just fevers on my company to Excel is a telecom operator mobile operator the biggest one in Turkey but having a vision on the transforming to digital world transforming to digital services we have messaging application we have TV live and on demand TV the service we have music platform and we have online magazine platform and there are others in total we have 50 million users in the countries where we operate the scope of this session will be as I said a demonstration of streaming features in our brand-new application it's going to be a live coding on reactive Spring Redis we will do simple things but that will reflect believe me that will reflect what we have done all in the project to come up with this brand-new application I will I will also to write some code for streaming events through Kafka using reactive spring but it's not going to be a deep dive into reactive stake Kafka in this and I'm gonna share at the end in my last slide related sessions that has happened so far that has been shared in the conference there are 12 related sessions where you can follow when you can go into details of these topics and we only have 30 minutes so what we have done in the project is we are streaming live real-time data of 200 tournaments football tournaments of 3,500 teams from different many different countries with the project we have managed to send accurate push notifications to our users before we did it one of our users could possibly get the push notification of 3 to 1 score before 0 1 so that wasn't really [Music] in English situation in good shape and now with this all these engineering ray platforming when there's when they when a goal is scored before the ball is even taken out of the goal we can deliver the push notifications to the users that's a really big gain big change out of only for virtual VMS not too too much capacity we can easily reach 1 million concurrent users for streaming real-time data before it wasn't really bad it wasn't it was really bad every now and in especially in the big games we would have crashing the platform no more crashes as I said this is not comparable to how it was what it was before and we can deliver more frequently ok now it's time for the demo I'm going to show you the application and I'm going to show you the streaming features of the application ok this is the application we have news we have a live score information very deliver the score changes as I said if I click on this button I will be locating to only those active games that's currently being played before this client would Paul every now and then every 3 seconds every 5 seconds and it would require something like 400 500 K kilobytes of data for each request and that would cause the crash obviously what I will do now I will connect to the production server just like this application now I have to streaming connections to the production servers as you can see the data is being sent only if there is a change in one of the games and this application mobile application here it will get the result from the change very few bytes of data and it would reflect the change on the mobile application that's basically how we could manage to solve those problems if I go to one of the applications here sorry one of the games that's already finished I will be able to see the events here as well this is no more streaming because the game is over there is there is no make sense to stream here again and one more demonstration is here it's the same infrastructure unconnected to the server with another streaming connection streaming socket where I will be listening to the commands that is this is kind of a chatroom you can think about it I will open the application here and go to the same game I will write a message and I should see it on both application and the console and that's going to be the end of the demo okay I'm connected to the same page here from say Pring one we should be able to see the message coming here this is my message and the same message is streamed here as well hi from spring one this is the end of the demo now I should go to show you and implement it together with you how we did it have you used the spring stack well before going into that part I'm going to show you one more thing which is all right it should be there I have got two repositories in my github account one of them is the first one is the project of today's session which contains only which focuses only on reactive Redis and reactive Kafka but I also had a chance to share this project in Barcelona in spring IO 19 as well where I had the chance to do this in two hours that was a workshop I had a lot of time so I could go into details of other features that we used in the project like cloud contract and cloud config service registry gateway stuff the second project is going to be the kind of full set of being cloud negative reactive application you can refer to these repositories now it's time for coding okay what makes an application reactive I'm sure you have seen this right who has seen a reactive application has implemented a reactive application okay not that many okay so I'll just briefly go over it it's springboard application no changes there we use Springwood starter parent again but this time if you want to have a reactive web application you need to use spring boots started web flags not the spring boot starter web this is going to boot a native server where your application will be running in a non-blocking fashion in a reactive fashion if you want to use Redis in a reactive way you need to include starter data Redis reactive dependency and similarly I will use spring Kafka and reactor Kafka dependencies for accessing Kafka in a reactive manner so that I can stream the changes the events to my clients there are a couple of configurations here for Redis and for Kafka I'm not going to go into too much details I don't wanna lose time there but just to mention what I need is a connection factory letters supporting reactive stay and reactively this template for storing saving a match entities and querying them I'm gonna need it I'm gonna need some serious laser but you can go over yourselves currently the application does not do much I have the basics I have controller I have mappings I didn't want to spend time on here but my rest controller endpoints do not actually return anything for example this streaming part is just returning an empty empty flux empty stream I'm not going to see anything and this finding the entities from Redis and persisting the content the the data in Redis is also empty I'm gonna what I'm gonna do today is I'm going to implement this part so that you can have an idea on how you can use a reactive spring let me run this application once I need to check if my Redis is running yes my Kafka's zookeeper is running everything seems to be fine when I run the application my demo scope will be I will query the matches with ID 1 which currently does not return anything because there is no implementation I will save the data of a change of a game which also doesn't do anything and what I will also do is I will start a stream for following the match with ID 1 but it doesn't do anything because it's an empty flag so it opens and closes immediately I need to implement code here now let me first start with a Redis I also show you what I have here in Redis I will delete matches I should have nothing here and then I will start persisting data in Redis all it takes is the template you remember I showed you using the template i get i get a hash operations object reactively and then I will put this data in this key which is matches the name of the collection and then the hash key of the data that I persist in Redis witches matches IDs in set ring and the match object match entity itself currently this mono is returning me a boolean like saying that Redis was successful or not true successful false not successful it's not like that but I don't go into there even if it's false it does update your data I don't know how why all I need to do is I want to change the scenario to just to show you how they play with the efflux and mono streams what I get from Redis is a boolean what but what I want to return is Oak not okay oak knock you know I just it's a silly business but still can be business what I will do if this a boolean is true I will return okay if not I will return not okay I should be fine I'm restarting now I have dev tools that should do hot deployment but I think there is a little thing with the kafka so I will keep on the safe side for the sake of demo I will do restart now if I go and submit again if I post the match information again I see okay here and I should see my data being populated in Redis you see one H get matches one I see the data that I have posted serialized in a JSON format JSON data format and processed it in Redis and I get the response back what happened here is when I call this put method it's not blocking any more until it leaves the layer of my application the thread will be this will be will be will not be with my thing anymore my work anymore and as soon as Redis returns with the response possibly another thread will take the job and continue processing from this moment on with the mapping okay so let's see if we can also get it we can return we can clear it from Redis it's the same hash operations but this time get from this key matches with the hash key ID converted to the string representation let me restart and see if we can get our data back with this gate query I'm gonna ask him for I'm going to query inform to match with the ID one it's a bit small but not there's nothing much than this actually ok now I can query a item match item with ID 1 if I look for true it will just return nothing because it's an it's a mono stream with no item in it and empty item because there is no data that Redis is lettuce part is this much I mean you can you can obviously add other functionalities other features if you need but this is mainly what we have done so simple that simple we have been we we store a lot of information in Redis because we use Redis as our main data store considering the performance and it's basically this much of information this much of things we have done now I should go to to finish my my hands-on coding I should look at this part because um I try to start a stream between my client and my server application but it doesn't do anything it just opens and closes a stream so what I have to do I have to work on my controller here which is returning an empty flux instead of returning an empty flux what I will do I will connect to a kafka receiver and then I will start getting the events which are pushed to Kafka topic here and then I will play with it because I get a raw string from Kafka I need to play with it I need to convert it to match entities I need to do some control subjects because I should not deliver all the information to all the clients because for example in my demo I was looking at only one match only one day only one topic it doesn't make sense to send me to send to my client all the information all the data that would result in problems so I need to make sure that I'm doing the right thing to begin with what I will do I will first get event publisher which will return me let's see what it will return to me it will return me a connectable flux of servers and event of setting as I said it's it's simple to use string a while communicating with Kafka but this connectable flux has a special feature I could not use Kafka receiver that received which is also returning my flux you can connect to this flux only once that's why I wrap it wrap that stream in a connectable flux so that I can open as many requests as that come from micro vines maybe 10,000 maybe 1 million so connectible flux is the key term here you need to change this flux kafka receivers flux to n connectible flux so what I have here I have got a string of servers and event but what I want to do I want to convert this to a magic object because I'm dealing with matter objects so what I will do match JSON sit ring to match which is set drink servers and events data payload body that has been arrived from kafka to the right from Kafka and now what I have I converted the flux the entities in flux to different type it was server sent events now its matches so what I have to do I also have to convert this match back to a server sent event of matches because that's my promise that's my contract I I said I'm gonna return server sent event of matches so what I do match to server sent event match I could make it simple this 2 semicolon and then the name of the method I could did but now seems fine I should be able to see something going on at least my clients should be able to keep connected because it's not an empty stream anymore it's a stream which will keep which will be kept alive as long as one of the sites does not kill the session does not kill the connection ok I have it but let's see if I will be able to get the data streamed to this client I'm posting again with the ID one but probably I will not see anything because I didn't bind the the event to Kafka I didn't put it in the topic of the kafka I'm gonna do it now which is again the service in the save match details right just after putting the content the data in the Redis in Redis what I will do I will also put the contents and the content the event to Kafka for this I will use Kafka sender that sent which will expect me a stream publisher that can be a moment it can be flux mono means I will send only one item so that means I might send more than one items to Kafka which is our match object and then once we do this we get back what do we get we get back a flux of responses Kafka results so what I will do since I know I sent one item I can say that I'm interested in the next item because I'm sure there is there is nothing else and what I have now I have here a center result what happened to my request to Kafka and I need to convert it to boolean because here my promise is I want to stick with radiuses return type I want to reach I want to generate a true or false to indicate that this was a successful or not successful operation what I do tis and the result that exception if it is now to me it's my business if it might not be the best the most true thing in the world but to me it's like this and then I will return this I will return [Music] okay I should put these things mono that just match Kafka sander let's not try to understand too much let's just try to use it here Kafka sander that said ma know that you oh yeah match to send the record I missed this part because I cannot just send a match item I need to wrap it in a sander record that's what was required actually I have only three minutes left let's restart should be fine we will see alright again streaming for item ID is 1 and let's post it again yeah there it is we have we are notified with the with the events with the Delta information of this game with ID 1 but now I'm gonna challenge myself I still have 2 more minutes I can do that this one should not get it right because I was I'm looking for I'm watching the match with ID 2 I should not get it so what I do I do another iteration of my application and here when I get the match I'm gonna do a filtering because all of the events passed through this flux here I need to filter out some of them which are not relevant so here what do I have match and then I need to check if the matches ID is equal to the argument ID here that is passed to my restful method here ok let's restart should be fine I guess I will have half a minute left again since the application server has restarted connections are lost this one is 1 this one is 2 and now if I send it this one doesn't get this one is it I sent item 2 just to make sure that everything is working fine if I send 3 nobody gets it right should get it ok this one doesn't this one doesn't that's I think pretty much all I wanted to share with you I hope you enjoyed I hope you find it useful and it helps [Applause] you

Channel: SpringDeveloper

Views: 13,964

Keywords: Web Development (Interest), spring, pivotal, Web Application (Industry) Web Application Framework (Software Genre), Java (Programming Language), Spring Framework, Software Developer (Project Role), Java (Software), Weblogic, IBM WebSphere Application Server (Software), IBM WebSphere (Software), WildFly (Software), JBoss (Venture Funded Company), cloud foundry, spring boot, spring cloud

Id: OmaF0WkBH2g

Length: 27min 16sec (1636 seconds)

Published: Wed Oct 16 2019