Camel microservices with Spring Boot and Kubernetes (Claus Ibsen, Denmark)

welcome everybody I'm chosen to this camel micro-service talk so yes I was here in seven years ago in 2011 back then it was 100 people in one room and Montel at a time I'm really placed to see how the community and the conference has been going strong since so congratulations to that thank you so I am a software engineer from Red Hat I've been working on camel for 10 years nine and a half as a full-time developer I wrote two books on camel camel in action second edition was just published three months ago and it's a little big pool so camel is easy just read the book and I'm from Denmark so what is camel and what do you use it for that's a very good question so cameras is used for system integration integration so camel is that little piece that sits between system and glue them together now from a developer point of view camel is categorized as an integration framework or integration toolkit here are the five highlights of camel so you go about using camel to build integration where you use Enterprise is practice and those are known as enterprise integration patterns which I'm going to talk about in a moment now camel comes with more than 200 components out of the box and you use these components to connect to many different systems it can be anything from mainframes to legacy systems to more traditional enterprise systems Oracle database IBM WebSphere messaging systems using DMS mqp rabbitmq it can also be streaming based like Casca and also traditional soap web service restful service and of course also file based FTP and secure file copies there's also for social media with Facebook LinkedIn and Twitter and so on and all the way to more modern settings with Klaus so he can you know use camel with Amazon Web service Google and Microsoft Azure and they're also no cloud service you can access like Salesforce and s AP and ServiceNow and a whole host order components so more than 200 and it keeps growing now the third thing is that Cameron comes with support for data mapping and transformation out of the box that allows you to transform new data between different structures so you can transform from Java to XML and vice versa - Jason llamó and also more traditional with fat files and comma-separated files and so on and there's also support for a number of industry standards in healthcare and finance now when you in camel you define how systems are integrated using routes camel routes and as a developer you develop these routes in either Java code or XML and finally camel has support for wrist and ap eyes so what is enterprise integration patterns that's in fact a book with that same title that book is it's the equal size of this one so about 13 years ago Greg a hobby and Bobby Bulls who are the authors of that book were a consultant that went from client to client and help with integration problems and I saw the same problems over and over again so what they did was to write down using knows about these problems and how to solve them and after time they got so many nodes that they thought hmm maybe we could turn that into a book and surely we should be blessed for that this is the enterprise integration patterns book it's a highly recommended book and you may say it's the integration Bible in this book we have recipes or solutions for common integration problems we are on the slide some of them there's one called content-based router this is a very famous pattern so it's more like a java if else thing you can also fill the message you can crowd misses in a dynamic way you can send a message to a number of recipients you can split misses you can aggregate and so on and so on the book has 65 patterns camel implements those patterns and some additional patterns which we're gonna see later in this talk so what about camel routes so here is a very basic example of camel you want to integrate two systems a file system with a messaging system using DMS and it's very very basics it's just say from file to DMS and in XML is similar way how to define that now let's use a little more advanced example this is in fact still quite often used today so you need to pick up a file and split that file line by line and each line you need to do some sort of missus transformation and that information you need to send to another system maybe a message queue and in this example they're using a passive action queue this is a missus program so you can you know use for patterns from that book and here's how you can implement that in camel using Java for example so you just say a phone file and then to direct your name then you want to split and then you tell camel how to split you want to split the missus body using a tokenizer that is using a newline character so it will be split on line by line then you want to transform that data to XML in this example but this example is picking up files that has some sort of custom in-house format it's not a standard format so you need to implement how to transform that and you provide that to camel and then the result of that transformation is sent to a missus program with just one line of code so in a certain we have four patterns that maps to four lines of Java code that you know it's camel can run that is very beautiful now what is the high-level architecture of camel well this is it one picture so in the center you have camel context so what is camel context well that is camel it's the run time camel so the idea is that you have this camel context then you can add camel routes to it and in these camel routes you can use these Enterprise patterns where you can do messaging routing transformation validation in Richmond call your own business logic and so on and so on and on the bottom you have camel components which you use to connect to all the different system to kind of like to speak to the other systems that is the camel architecture now where can you run camel well I say camel you can run camera anywhere well camel is based on Java so everywhere you can run Java you can run camel now because camel has been around for 10 years then you know it supports both the older way and the more modern way probably the other way around it flip for you guys so in the beginning where we are building more traditional applications that runs on application servers then camel works beautiful there but now today when we're moving towards a more micro service and container based solutions then you can also run camel in that environment and this will be the focus of this talk now camel also connects to everything or a lot of things I said that camera comes with 200 components out of the box and on this side I tried to Kairos those into four different categories in the traditional enterprise systems the public cloud and assess but pay attention down in the corner I have IOT so camel also worked with the IOT world with small embedded devices hey you may say the camera really run on these small embedded device like a beacon or something like that or maybe on this Smart Watch no it does not because I said camel is a Java technology so it requires to run on Java and Java itself is a heavy a bit heavy right the footprint of Java is to to come too much but where it makes sense to use camel in IOT world is in gateways so all these small embedded devices connect to some sort of gateway that is a lightweight computing note that may run Java and you can have camel inside and then you know these beacons on whatnot they can connect to that gateway using lightweight protocols like cope mqtt which has support for that out of the box in camel and then gateway can then route that missus data to a back-end system where you can further process that data so that is also a very common approach of using camel on so you can see it spans entire way of integration today now I would like to summarize what camel is using pictures so camel is this integration tool box which allows developers to use these tools to build integrations and you as a developer you can build you know integration solution using you know best practice those are the enterprise integration patterns then you can define camel graphs how to integrate the systems where you can use these patterns and your custom business not again not not and then you have all these chemical tenants you used to connect to different systems and all that together is camel okay so what about running camel in the cloud or in the containers so this is where it gets a bit more interesting let me just take sip of water sorry for the pause so when we start to move from a more traditional way of where we have been building more monolithic application to a more micro service way then we start to split up our bigger application into smaller pieces and those are known as micro suits and then gradually over time as we split up the original monolithic application fades away and these microservice become more and more independent and they also become networked and distributed so now we have a different situation with distributed computing and that is actually quite harder than what we used to have before so before with the monolithic application it was easier I could be easier to manage monitor and troubleshoot it because if something goes wrong we can check the status of the monolithic application is it up or down now when we have many different micro services that run in a distributed computing fashion they are distributed they run on different nodes they are communicating remotely all the network and so on now in this situation on the slide we have just three service a B and C so if a calls P and P calls C and something goes wrong where is that error how do you troubleshoot that and how do you figure that out that is where it gets more complicated so what we need is a set of extra features or capabilities to support running micro service in this new world of containers and here on the slide we have those it's a shape of 10 and the Greek word for 10 is thicker so this is a deck account so we can build micro service that has api's and you can have multiple micro service and you want to know integrate the micro service so micro service should be able to discover each other and those API is and invoke those API and you also want that cluster that can scale up and scale down your micro service so it should be elastic and so on and so on but where do we get these capabilities from well we get them from kubernetes so kubernetes is this you know contain a platform or container or gustation engine that gives or some capabilities like companies makes it easy to discover other service it makes it easier to invoke existing service in the cluster and cubanía certainly makes it easier and manage scaling up and down service what about additional features so you can run what is it called a platform as a service on top of kubernetes one of those is Red Hat OpenShift about their orders in the market what they typically provide is solutions around common needs around continuous integration continuous delivery for example around pipelines so we can have like Jenkins pipelines and whatnot but you also need centralized logging and monitoring a solution in for the cluster and in recent time and new concept has emerged and become more popular it's called the service mesh and one of the products inside the services ecosystem is called East you and it kind of stands out so east you tries to be a solution to manage your service and connect your service in a more secure and reliable way so we get some resilience from East you something about security and tracing and so on and the final piece is about API API management so when you start to have many api's in your clusters then hundreds and thousands you know how do you manage and galileo's it can be rather complicated so what you want to do is to be able to sort of have an api catalog so to speak so develops can browse the catalog and find you api is made they may have a one-stop page where they can read information about the api's how to access the api's and while the contract of that API it could be adjacent Swaggart documentation etc also when many api's and you may want to have sort of like a solution for user management so they can sell sign up for exiting api's they can have handout tokens api tokens to access the api's and maybe you need a way of revoking it the items that has become invalid or security issues and so on so the host knew of problems that an API management solution can help to solve but where is camel he sits there but where is in on that slide well camel is embedded inside your micro service as where you find camel so here we have different Microsoft's using different technologies and I have said before that camel is database so camel can only run inside the Java based micro service so on the slide it will be spring boot vertex Wi-Fi swarm etc you cannot use camel bit go anode and other things like that ok so when you start to have micro service that are using it and doing integration the camel then it's you know distributed integration and so here on the sides we had Tree Service a B and C and we are only using camel where it makes sense where you need integration capabilities and where could that make sense for example it could make sense to use camel on the edges for example in service T we need to integrate with some external system that runs outside the cluster and maybe that system is a Oracle database so there's a camel component for connecting to a database data series etc but it may also just be an old-school FTP server right so there's a common component for that and so on zone there are 200 and so common component so that that's why it makes sense another point I wanted to make is that now we can take integration where it's needed so camel is a lightweight integration toolkit of framework you can embed directly with your Microsoft's and run it directly in that container where you only need it now beforehand in a more traditional way we had more like an application server and more like a centralized application server or maybe even since like ESB not longer the case with the container walls so you can run it where you need it on now companies when they start to look into micro service architecture style and this movement around DevOps and been more agile and so on all that is in the integration space at a coin into a single concept called agile integration these I have three Keith capabilities that you need one is distributed integration this is where you find camel this is the bread and part of camel this is variability integration flows do so where you reuse those Enterprise patterns and so on in the center you have containers you want disks out native contain a platform where you can run your Microsoft and camel applications in a reliable robust scalable highly rables infrastructure and then you have api's as well so distributor integration containers and api's now what about some of the best practice of good practice around running camel in the cloud or on containers well we want to be small in size so camel is very lightweight around 4 megabyte the common core and the intention is that you just piecemeal the can of components you want together and then embed that together with your applications or micro service as at this fat jar kind of style and then you can run those using popular runtimes with spring boot vertex warm and carafe or whatnot or even just standalone camel you know what being small in size is very important in as a practice in the container world now another practices around being stateless no favor building micro service state are stateless they are easy to manage they easy to scale up and scale down and so on but if you need stayed then camel has components with different state solutions for example the memory crates you can have camel integration between finish best hazel caste and so on but also for store solution but to see Seagal components streaming with cast guns I want and many Ron stateful applications on a kubernetes then kubernetes also come with a concept for that which called a stateful set not a practice around running micro service in containers is around configuration management in kubernetes there are two concepts for that one is called conflict map and the other one is called secret now spring boot and camel has great support for that and so here on the slide I'm trying to illustrate that so you can externalize your configuration and provide that as part of the company's platform down in the corner there is a Yamal file that has a single configuration key there's a key called fallback beta value saying I still got no response now that information can be injected into the runtime container by the kubernetes using two approaches one is using environmental variables and another one is using files so there will be a file on the container file system with that information both of those you know out of the box are polar by spring moon and camel now sometimes you want to store more sensitive data like passwords and certificates and so on and those are secrets and those are in fact also possible to be inducted on the same approach using either environment variables or files but when it's files is not stored as a physical file at this node anyway sort of like only in the memory so you can't really it's more secure in that way now yes now a lot of practice around running camel or microservice in containers is around fall torrents camel has supported that in two areas one is called client-side retries and another one is run circuit breaker and that's one implementation in camel for a very popular library called netflix districts this we gonna see later in the demo now when you have distributed integration of that calls is out over the network then things can go wrong so one thing is that you can yeah we have here on the slide it's a Hello service and the client want to call that now if for some reason something goes wrong then you can instruct the client or camel to try again client-side retries so in camel there's something called own exception so you can say own exception and then the kind of exception here we're just using exception so it's a very generic then we tell camel to try up till 10 times with one second delay and in the second on the slide you can see okay the first tree invocation failed the fourth time it worked so everything is okay right awesome well not always because now imagine that you have many micro servers of clients that calls the downstream service and that downstream service is under stress here on the side that downstream services called relevance so all these upstream service call it and because the relevant service cannot handle that load it keeps failing but declined on your ass or has been instructed to keep calling you so you know that's not a good situation and that situation is called a thundering hurt problem now a funding hurt is like a big group of cows you know angry cows that come running to watch you there's no way really to start that you really if you get that situation so there are other means in in how to build tall fault tolerance at the waist and we're gonna see that in a moment not a great practice you really need on container based technologies around health checks it becomes more apparent now because the it it's more like container platform it sells its levering these health checks to know the state of your applications so it really needs to know if your application is ready or healthy and so on now in kubernetes there are two concepts for that one is called readiness probe and the other one is liveness problem there's a distinction between them and it's somewhat important to know what it is out of the box spring boot comes with a great support for health checks spring boot actuators and camel has an implementation for that so it comes if we don't come up with spring boot you get that out of the box on the screen you can probably see there's a camel health check SS up so the entire health state check is up and for that one so it's good now when you have and so why is there two different distinctions in the company's platform one is called readiness probe on your Mons Linus so readiness probes are used to know when your application is ready up and running because application may need time to initialize especially Java technologies you know the Java VM is a little heavy on the container platform he needs to time to boot up spring good and Tomcat and so on only when there really boot up and ready you want to signal to the cluster that you're ready that you can take traffic whereas liveness probe is something that is keeping check by the platform to know hey are you still alive are you okay you know so that's buried maybe after it's ready and handling traffic you know something can go wrong in your application so you may say I'm no longer live and then the kubernetes platform can react to that you can a dispersed form is based around this self-healing concept so it can actually kill containers and whatnot start a mob again and so on based on these probes so one point as well is around these enterprise integration parents is - any less a difference on using them in the cloud on containers the book was you know poppies before we had containers really before talkin companies no there's no they just just as beautiful on traditional ways but also Microsoft's and containers so there's no difference at all so you can say for you stable on containers but camel comes with a number of additional patterns for running micro service on in in the cloud or containers I mentioned a bit before about fall turn ins there was something called a circuit breaker one implementation that is history it's a very famous one from Netflix so you're gonna see that in in the demo another pattern that was recently added to camel is the saga pattern I won't go too much in detail about that but it's more a way of solving the distributed transaction problem with sagas sagas is a way of supporting long-running actions where you can sort of have ways of compensating things so an example will be sort of like a booking system by you booked a flight ticket and hotel you can't really use traditional two-phase commits and whatnot so you can use the saga pattern to implement that and then have ways of compensating for example if they're flat tickets booked but there's something wrong with the hotel you and you can make a cancellation for the fry cook and sew and vice-versa one thing you also need around microwaves in camera in the cloud is around distributed tracing so before we may have this big monolithic application and you can sort of how do you attract a business transaction so to speak in the bigger application it was more easy but now you have many different micro service how do you trace that business transaction across many different microservice and there are in the open source well there are different products around that one is called steep cane another one is open tracing and camel has you know components that integrate with those so if you go back to the deck account slide and see where does cattle provide some capabilities that we can use well we do have certainly around in invocation to invoke micro service in the cluster outside the cross the stairs camel components remember I told you there were more than 200 also run postilions no even though at least you is coming around it's still a new product a camel has clients ivory ties it can make sense to use them because it's in Java code you can implement it directly there and so on and there's also the circuit breaker from history which is a battle test implementation ok so let's move on to see if the Wi-Fi works and works so I gonna do a little basic demo sorry it's about the hello world example so we have a simple service it's a Hello service it's running using Wi-Fi swarm it doesn't really have to just Java technology and 16-bit cause the service continuously it's running on spring so to kind of service so in the interest of time and so on of course I already implemented that and if I execute I call the service moment web browser I can see a response here if the Wi-Fi box says when I called the service a scale of response in swamp size hello from and then there's something that is excellent the host name of birds running and the intention is because we're gonna scale up and scale down and so on so the intention is of course you can see a different host name as we play right now I can also call this service from my shell so have a little script here call hit me and using hell so I can just execute that one and it calls that one right not nothing fancy right now now what is about the client-side application I have it here so this is a camel application ability that's a single-camera route it uses a timer so I say from timer here at equal thousand so that's every second we're gonna call a service and we want to use the circuit breaker from history and the service is calling using its people to call the service name is called hello swarm and then if something goes wrong we have our own fall back where we says a fixed response and then we lock that that's tested that's the essence of that one okay so if I go to shell here I have in my cluster I have it here cube get parts it's already deployed decline history I can tail the lock from that one swarm says hello from hello and it's the same response every time because there's one instance on one pot running with that service so now to build fault-tolerance application in the cluster I'm gonna play a bit and we gonna see what happens right so if I go to the web console so I'm actually running kubernetes using OpenShift yeah it doesn't really have to be it's just because I was given this job at Red Hat summit and at Red Hat we need to you know use but it does make sense because it has this nice button up here so it's easy to scale up a service so if I click here and you starting to scale up to and we should see down here that this guy is going to call the second one and yeah now it called ok I'm going okay mm-hmm okay okay so you didn't seen anything so now here we have the client calls the service there's there's two instance running so we have this low Paddington thing you can kind of see there are two running and can you kind of guess or know which algorithm is being used round-robin that's a very good answer but we'll have to go back two years then it will be correct today it's actually random so Google found out by the do some analytics on the clusters and burn out and found out okay round-robin seems like the natural way right but they found out there's some sort of predictability in it so you can sort of give this cascading influence so it can actually be bad so random is actually the best one apparently but you can choose and you can also implement your custom if you want now there was this probably in the beginning I'm not sure you saw it well it says nobody want to talk to me is there anybody that has any idea why this happened because I was actually scaling up an application in a secure way in the cluster at old Cuba needs please go from one to two and why the hell can you not do that in a safe way you are you're you're in the right path so to speak so yes you won the book just by saying one word right he is correct this is very important so Java technology is not super fast and not spring boot or while first one so not even cameras fast but no just kidding but it's not super fast so when we scale it up to two companies spin up a new container and then it starts this radius probe and if the redness probe just say yes I'm ready it serves traffic but the Java T of M is booting up while fast bomb is not really ready it doesn't have them create the HP listener yet so you get a connection refused and the client I called it with the Health District Lopez it detects that error and toss to fall back so there is a park or problem now this is the hello swarm application so I am a bit lazy there is a tool called fabricate maven plugin that can take a maven project abilities as a docker image and the kubernetes manifest files together so you can deploy directly in the cluster i used that but there is a bach in an older version of it so I'm using version 3 3 5 that version has a bug around readiness probes so it does not include that out of the box if I upgrade to a newer version it will include that redness probe and it will work so if I had that readiness probe I can easily scale up and down my swamp service in a reliable way but nevertheless let's try something else I just go back to one here and let's see here where it was it up here I'm gonna tell this guy and down here I I can get my pots and I can of course you know if I delete it then you'll say okay what you're telling Cuba need is to delete a part Benny knows this their service is gone and so on what what if it crashes you know how can we sort of simulate that something is not no longer access in the cluster without companies knowing it so we can go a bit back mostly if I can because it's darker I could do with docker PS and this is actually a little hard to find on this one but let me try see if I can find it the process for I'm on spring boot hello swarm here it is and then is this is probably the right one don't do this normally so I'm trying to kill the process that is running the hello swarm I'm not sure if I got the right one oops yes dog a kill I kill something yes now said nobody wants to talk to me so I actually killed like crashed something in the cluster kubernetes detected it it was no wrong running now it will start up that one and if we can see it probably not I might have this one hasn't been refreshed so what what you can do as well in the class is to get an overview of your circuit breakers and there's a web console for that called hysterics and we should be able to see on the graph if something goes wrong so what we can do is to simulate that problem again so I'm just going back here get and just going to delete it instead this time dilute hello swarm and now it's deleted and this guy should start to trigger so now the circuit breaker is open it's red so now you can see it's red right it does take a little while to get used to the concept of the circuit breaker open versus closed but if you're not colorblind you can see that red is really bad like and green is good so now it's back again but the circuit break is closed closed is good it's like closing a bridge or something then the data can flow if it opens then you know you not so that's historic for you so let me go back to the slides we should have a little time to QA and as well so discussed at eight minutes so when you get started on building microservice and integration on containers parrot I come from tradition Java background so when beforehand when we're building integration application and so on for Java EE or whatever it was heavy Java based you know you know that Java technology you build a ball file or something and handed or two up skies and they run it on an observer or something like that and the touch points I had was really around that speck or whatever all this content thing was not in my face now with containers there's a lot more to learn and I'm sorry but this triangle you see here is lists some of the concepts you're not really supposed to read it from from the slide it's more like you can go and download the slides offline in media from your browser and get solve like inspiration some of the concepts you need to learn we saw a bit about self-healing we saw that the company's closer can do that but there's also for api gateways for Cillian and for service discovery dashboards and dispute metrics and so on there's many concepts that you need to master and learn a bit more about when you developing micro service for the containers and thank god there's gonna be number of books with explained that to you there's a good book of very good book written by a colleague of mine christian poster microsoft's for java developers it's all like a gentle introduction to how it is to build no java microservice it does not cover camel a body does cover how to build in the spring good applications on on containers and so on it does have like a natural way of you know try this at yourself so it's a good way of learning in its freely downloadable now if you really want to learn the concept of containers and then this there's this great book called cuban ears in action unfortunately it's not free the author marco probably needs some royalties and the published certainly want to have a bit of dollars but you know today there is actually a 40% discount code for Java EE folks so they posted that on Twitter unfortunately not I forgot to put it in here but you know you can also find it on Amazon this is a highly recommended book on on kubernetes my book you got is free but the other guys you can order it from mening the publisher there's a discount code called camel 3:9 and it gives 39% of discount easy to remember in the past they have also had order discount code for different amounts there used to be has been a 50% camel 50 sometimes they forget to revoke them they are not so I see good so you can try maybe camel 99 98 97 until you find it some of them actually works so but camel screen on this is sort of like it always works at least to my knowledge you used to work for the old book and also for the news I haven't heard about it doesn't work and just some links where is the slides it's on my github account just find me on dubs Klaus and there's something called camel riders in the cloud and that's where you can find the stuff the camel website is currently on going to be more migrated to be a more modern look and feel and documentation so don't be you know scared if you go to the camel website and thing is from 1990 something actually the website looks like it did 10 years ago sorry for that but we are working on that so in after the summer break know there will be a more modern website if you the middle thing is really about okay if you have a collector says what the hell is camel you went to this store and can you tell me you can just point into that one five minutes wait get a coffee it's great sauce for camel is and that's the time for Q and a we can have a break Oh so how does Cameron compared to mule camera off no so they're like three I would say three well known integration products in the world today apache camel spring integration mule they've been around for almost the same time mule is a traditional ESP platform it started at is more it's not like a toolkit it's more like an app server so tom can't it better and you have your integration way recently they were acquired by Salesforce so it also have a doubt in the view of that product because Salesforce is not in the integration space they don't really care about that it's more about getting your data onto that cloud or together with Salesforce and you can be a component for getting in there but you know that's the mind they don't yet have mule does not have the own record but still the true spirit of open source you know a passion cameras from a pacer Software Foundation it's been the neutral Red Hat does not have a control over it know what those other companies their multiple companies that have employees that are paid and then work on it contributors from other companies and volunteers and what not is true to 100 and free and open sores and and having this great license called Apache License mule does not it has their own weird license so I'll rather compact camera with spring integration which are more like their toolkit to to kitten and mule is like a a PSP integration platform kind of thing that just got bought by Salesforce that has a cristóbal future other questions so oh oh so the question is what is better in camel than spring integration so don't take my word for everything because I'm biased against camel right I think both of them are lightweight toolkit I do one other thing I said before camel has the true open source spirit neutral whatnot and you know it's a really focus for as on the pin technology or for different companies spring integration is probably one of many spring products that is not as important to pivotal than some of the others but they're both live X camel has a lot more component so Spring has only X and cameras a lot more about use boat if you want camel can also run without spring nose for some people that's actually a matter they are not you know spring shop or something but in common can run in any kind of environment like but give it a go both of them but camel has the new book even though it's big so one last question or awesome I think I will be around here at the conference you can find me in the hallways or not you can also reach out to me on Twitter or LinkedIn or email or whatever you want to do I'm happy to you know take questions or not but again try to get involved in this container world it's not only camel that is necessary there but many many technologies are moving to what that wall is it's a it's a hard wage many new concepts to learn what is something that be almost you know if we stay away from that you will be left behind I'm sorry okay [Applause]

Channel: jeeconf

Views: 15,070

Keywords: JEEConf, Spring Boot, Kubernetes, microservices

Id: I5jdLFdWi6w

Length: 45min 19sec (2719 seconds)

Published: Tue Jun 05 2018