AsyncAPI & Springwolf - Automated documentation (and more) by Raphael De Lio @ Spring I/O 2024

[Music] I think we can get it started 12:31 already um I'd like to get it started today I'm going to be talking about async API and spring wolf but I'd like to get it started by giving a little bit of context of how we got to the actual scenario and and um the first thing I'd like to talk about is this architecture the three Tire application right this is probably the most universal architecture that you guys have seen out there probably the way you guys explain how a software an application works for let's say your grandmother or friend or cousin who wants to become a software engineer as well so we have for example the presentation layer the application layer and the data layer very simple thing um the thing with this kind of Architecture is that usually when you're asked to make changes we're asked to make business changes right and when we make changes in this kind of architecture we're supposed to change all of the layers most of the times right so if you have to change something in the business you're probably going to have to change something in the front end something in the back end and also maybe something in the database but the question is why do we organize our applications like that so this is a practically Universal architecture but why is it Universal right and what I'm saying right here is not only things that I came out of my mind right uh I'm actually paraphrasing Sam Newman in the book building uh Mark microservices and one of the things that he says is that the reason why organize our applications like that is could be because this is popular so we've seen this before everywhere so we just uh we're inclined to copy it or it could also be because this is the way our organizations are structured so our teams inside of our organizations are structured and there is um even a law to describe it which is the conways law right so if you guys been working with software development for quite quite a while or maybe even today you probably have worked in companies where you had like a front end team a backend team a database team DB admins right this is how most companies use it to organize their teams before and one interesting thing is that last week I was watching this movie it's called Big have you seen this movie from 1988 staring Tom Hanks and this interesting thing there are two interesting things about this movie actually the first one is that Tom Hanks Tom Hanks he's a kid right and in the beginning of the movie he makes a wish that he will become big and he becomes big he lives he Li his life Turns Upside Down and well he needs to find a job and he finds a job as an IT consultant right and the first thing that happens is that he starts working he's really fast and his colleague his first day tells him hey you got to slow down are you trying to get us all fired from here this is the first thing the second thing is that well he goes to the toy store it's a toy company he goes to the toy store he meets the founder of the company there they play together and he's promoted into vice president of the company and the guy who is the Rival of him in the movie gets jealous and says hey how does he even get here I mean he was just a nighty guy right so that's how traditionally we were seeing inside of the company just the IT guy who is there not very important right like 20 years 30 years ago but these forces have changed and aspiration around software development has changed we want to ship software much faster than ever before and we need to make it easier to make changes in our code base so it means that software nowadays technology nowadays is in the center of the companies is the heart of the companies if you have banks for example traditional companies that have been there there for a century maybe two centuries nowadays they rely on software as the center of their business and their customers not only banks but other applications as well have been have had the software or their applications as the first point of contact with um with uh with the company but the thing is before we were choosing cohesion of Technology instead of business right and now these forces are changing us to choose cohesion of business functionality instead so in order to do it we also need to change how we group code and this is forcing us to rethink the way or organizations are structured which in turn is forcing us to group people by the way we break business apart so if in the past we were breaking our application like that so presentation layer an application layer and a data layer nowadays we decided to start breaking our application like this multiple business lines one next to the other and by organizing our code like this we can now have dedicated multidisciplinary teams working on each slice of our application so we have a product catalog service an order Management Service a payment service a shipping service and a notification service in this case right and also these teams they are autonomous enough to decide if they even want to have an application layer a a presentation layer or a motor layer so it depends on the team maybe they say okay I just I I need a relational database I work better with the document database maybe I don't even need a database but the most important thing about organizing our companies like that is that now when business ask for a change so let's say I need a new notification to be sent when an order is made this change is going to fall in one or two maybe more thingsam but not all of the teams right so only the ones that are responsible for this part of the business however there's still a problem because our communication is still coupled so we've been trying to decouple our teams our business lines but our communication was still coupled right so when you had a monolith the services inside of the monolith they would communicate with each other and they would block each other but now that we have the services apart from each other we want to avoid this blocking communication and as you can see here we have the order management team notification team send a notification for me so when an order is created the ORD order Management Service needs to contact the notification service and say hey send me a send a notification for me so basically uh it needs to have knowledge about the notification service and how it works right but we want to change this Paradigm as well or or shift this way of communication as well and you want to move towards non blocking communication so instead we want our order Management Service just to say hey an order has been created and the other services those that are interested in it to react accordingly right so I have the notification team now saying hey that's my queue I see that an order has been created I'm going to send um a notification and this is today the leading uh software architecture in microservices right so we're talking about event driven Comm ation event driven architecture who has worked with event driven architecture before M nice nice all of you so just a recap almost all of you so an event is a factual statement about something that has happened in your servers other microservices can listen to the events they're interested and react accordingly if they're interested in them the microservices that listen to the event do not need to be reachable at the time the event is emitted so our communication is decoupled now and the microservice that is admitting the event doesn't need to know anything about the microservices that are listening to the events decoupling the communications between microservices and the distribution of responsibility we see in our event driven interactions can mirror the distribution of responsibility we see in our organizations trying to create more autonomous teams so that's what we're leading towards too we're trying to create more independent and autonomous teams inside of our organization so we can move faster and we can shap software faster a quick recap on of this context right so our software architecture is usually driven by the way we organize our teams we need to release faster than ever before and this has forced us to reorganize our teams and our software architecture and this Force moved us to more decoupled teams architecture and communication so it move towards microservices with communication driven by events and this is kind of the actual scenario right however um even though the goal is that we can change our software freely so we have parts of a software that we can change in the of other teams parts of our service is still coupled to other services right so we still have contracts we still have events and if we change those events we we also need to communicate the other teams that these events have been changed so they can change accordingly because we're going to affect them and that has lead us to that has led us sorry to async API has anybody worked with the Sync API just a few hands had people heard of it before nice are you here because you're curious okay cool are you here because you're just hanging around nice I'm very honored I must say because you know that Josh Longs is speaking in the Next Room right and you guys are here so I'm very honored that you're all here thank you so much but before I continue I'd like to make an analogy right because I'd like to talk about my mom actually you see I lived with my mom for 18 years of my life in the same house and we used to talk every day and it was a blocking communication right so she was in another room or maybe she was in the same room she would ask something we would talk about something and we would wait for each other's response but I moved out of the house when I was 18 I moved to another country later on and now our Communications is most of the time as synchron they send messages to each other right on WhatsApp IM message things like that and the thing is I being the son of my mother for 29 years now right and for my dad as well and I've been married for a couple of years already and with all of these people who I almost leave every day together I still have issues of communication and then how can we expect that if we have issues of communication with our family our spouses how can we expect that our communication with our colleagues at work is going to be perfect people who we have been working with for one year five years something like that right so we need better communication that's why async API is here so async API was first was created was started by this guy frendes Spanish there's a lot of content about him online as well podcasts that's how I learned more about him he created um a Sync API back in 2006 when he was working at at hitch and one of the things that he says in his podcast is that he was suffering from a problem and many other people were suffering from the from the same problem he found out later on right he even gives an example that he was working at a company where they had so many topics in in their CFA broker that they couldn't even understand why those topics were there anymore so people would ask hey why how is this topic being used who is using this topic they would check the consumer groups for example they would say that the consumers were like test one test two nobody knew why these topics were there there were topics with duplicate data and he wanted to avoid these kind of problems happening right besides that he also wanted to make it open source he got several Pro uh uh proposals to to get it bought but he declined because he wanted it to be open source he wanted it to be like the open API of the async apis but in a nutshell it's a communication contract between senders and receivers and well how does it work first of all we start by defining a file so it can be a yo file or Json file so it start by defining the version of async API right now it's 3.0 a lot of breaking changes were introduced in this version then we have five root elements info servers channels operations and components and I'm going to be creating the file with you right now so the first one is the information right so we have information from our system the information have all these elements to play with so you have title version description you name it so here for for example we have an API called Pegasus the version of this API is 1.0.0 the description is that this API provides information about bookings have a contact it's the ego team let's hope that the the teams are not restructured the next year somebody tries to find ego team it doesn't exist anymore right you have some tags so this is the p as a system and you have external docks leading to some docks with outdated documentation that's easily how it happens in our companies right then we have servers so the servers are the servers that our API will be communicating with we could be it could be like as I I put here the production server the staging server to Kafka Brokers we could have two different production servers you could be reading from a RAB TQ and Publishing to a Google Pub sub for example it all depends on your own implementation this is an specification right so you're free to play with according to your needs so we have two servers you can see that they have the the description host protocol the first one is Kafka secure so the first one has security they have TXS the first one doesn't have the second one sorry doesn't have security after that we can Define our channels so I defined three channels here so the channels they also have address title description messages so the first channel is flight bookings so you have the the the address of the channel in your in your broker could be your Kafka broker RAB TQ broker um you have the title of the channel the description as well the servers it interacts with so you could have some channels that interact with one server but not with the other especially if you are dealing with multiple servers in production and then you have the messages that can be produced or read from this from this Channel and the same thing for the second channel the passenger checkings now that I Define our channels we can think about our operations and in in a Sync API 3.0 they they added a breaking change so they change it the action as you see there so you see that our operation is actually telling us how we can use our channels so we can read from our channels we can send messages or we can receive messages from our channels and this is application Centric so we're looking at it from the perspective of the application right so if you look at the first one for example send booking confirmation the action is send so it means that this API is sending messages to booking confirmation topic and if you look at the last one receive passenger check you see the the action is received so it means that our application is receiving messages from this topic but then you also Define other details the title sum summary description again the channel That this operation is is working on and the messages after that we have components components can be almost everything that you can Define inside of your async API file and is everything that is reusable right so you could Define a message for example that is reusable across your uh a Sync API file we will Define it here and then reference it elsewhere after we Define and and for example a type of component is schemas right so these are the payloads that could be over your messages so you have two examples here the first one is the booking confirmed two afro schemas in this case they didn't have to be afro but in this case they are they already defined here and the third one is right here so the passenger checked in as well and with this schemas we can then create our messages that are referenc in these schemas as you can see there so we have three messages again and the schema could be right there but if but the good practice is to make it reusable so if another message is also going to use this schema you can have it defined um elsewhere as well so the message it has a name a title summary and a and a payload this is also a breaking feature that was not a breaking a new feature actually that was introduced into async API 3.0 and it's about the reply request pattern so this pattern is basically when you receive a message you read a message from one topic and then you publish a response let's say into another topic right so they also added a way that you can specify in the operation that you're going to be receiving a message from this topic but then a a response is going to be produced to this other topic and could even be dynamic so you could receive the massage the the topic where it's supposed to be published as a header from the message that received for example and it's also supported by a Sync API but let's let's get done with talking and let me show you async API studio now because if you get all of this information that I showed you and put it on async API studio so studio. async ai.com it's offered by them so I just pasted all of this information here and it came up with this beautiful UI here with all the information that I can share with my colleagues at my company or even for future reference or on boarding or anything else so you can see that we have the information about the system the external docks the servers and finally the operations the channel that he acts on if it's a send operation if it's a receive operation have the payload of the message as well and also the messages here in the end and the schemas all right but a Sync API um it's not just about this specification it's about tooling as well so they also offer parsers and other tools that you can use to build your own applications or validators or document generators or anything else and one of these generators is actually oh sorry forgot you come back spring wolf so spring wolf is analogous to Spring Fox right so you have async API as open API and spring wolf as spring Fox and I don't know much about the history of spring wolf what I know about it is that it was created in 2018 by the first guy here stav Shamir and it's inspired by Spring fox has been downloaded more than a half million times it's compatible with spring AF spr Cloud stream it gets monthly releases and it has a strong and open community so I I know that these three guys uh rather than St Shamir also him uh are the ones that are contributing the most nowadays they're very open I talked to to uh Timon back for example I asked for I I opened some issues I um in their in their project as well and they quickly uh answered and implemented the issues as well so they have a strong and open Community right but what does it actually do so first of all it automates a documentation generation right it's boring to Define your async API file by hand so how it would be better if we could just generate it automatically so you can see that it it's done by scanning relevant annotations in our application such as CFA listener RIT listener sqs listener or even their custom annotations as well as the async listeners then it extracts information from this uh from a code and builds the the async API file for you right so it means that it's async API compliant so it aderes to this today Sync API specification it uses their standard structure and format and also improves the understanding of your application but the two things that I like the most about spring wolf are the ones that I'm going to talk about right now the first one is the interactive web uui so they also offer a web UI that you can spin up right from your spring boot application that has easy navigation you can explore everything just as I showed you in the Sync API Studio as well and also enhances understanding and finally I think this is the one um Fe this is the feature that I like the most actually that is massage publishing right from the webui so instead of going to a khu for example or your rabbit meq manager you can just right from the UI send messages directly into your application with the examples that are generated automatically for you so I've already talked about some of the benefits but yes it reduces manual effort it improves collaboration fast uh on boarding in your company and enhances testing and makes developers happier by uh being able to understand their software better and also also to test the software easier how does it do it this is basically um repetition of what I I had already said and finally let's take a look at the code so in order to get it started with spring Wolf the first thing you got to do is add this implementations in your Gro file or Maven or Maven file so the first one is the the plugin here I'm using spring wolf Kafka but it could have been using RAB TQ or cloudstream or another one and then I also have the spring wolf UI that I want to use to uh see my documentation and interact with my application as well so if I open now my application. yo you can also see let me zoom in a little bit can you see it is it good okay so you can see that this information that is a static the in the the information from your API and the servers you also Define it here so it's not picked up from your application automatically because there's no way of doing it but besides that you also configure the message publisher from the UI right here so you can see that you can Define the serializer I'm using the a serializer here and also the schema registry right here so after I done it I created some a files representing our schemas so we have booking cancelled booking confirmed and passenger checked in and then I created a listener here very simple one just to exemplify so we have this annotation here kafal listener what spring wolf will do is he's going to scan our code he's going to find this annotation and then he's going to create the API sorry the async API file automatically from there but not only that we also have the Publishers and for the publisher since we don't have an annotation that we need to add for it to work we need to use uh spring wolf custom annotations here so you annotate it with async Publisher and then you define the details of your operations inside of it so you can see here that have the channel confirmed bookings the description the servers that it's that interacts with and also the same thing for the sending send booking cancelled here another thing that you could do is also enhance the this the the details of this listener by adding The async Listener annotation here as well let me run this application ah sorry there's one more thing so you can see that I added these two files into to this exposed package because for the sake of this example I also added these two listeners so you can test and you can see that our messages are actually being published but I didn't want them to show up in the in the async API file so what I did was the ones that I wanted to be exposed I added to this into this package and I told a Sync API to only look into this package as the B as the base package so if I run my application right now and it works and go back to our browser and open our documentation so I came into uh spring wolf async API dy. HTML this is what you're going to see let me close this one so you see that we have the information of our API we have our servers the servers that the API interacts with and the channels where the API operates right so have basically the same three channels so this one the the API will be receiving from this Channel and sending to these other two we also have these schemas here but the features that I like the most here are this on so we have already a message generated automatically for you an example you cannot see from yeah and also you can publish it directly into your application right from here so if you come back you see that the messenger was created in our system here but not only that also if I come back and I can also download the Sync API file from here I can download it uh either in Jason or yo and if I copy it and put it on asyn kpi Studio I should also be able to see it here and you can also see that I have the servers the channels the operations the messages and these schemas That's How spring wolf works and then you may ask okay what's next and the thing is they have a strong and very open commun right people are contributing I think this is a nice project for you to start contributing as well but first I'd like to invite you all to try it out so you can you can check spring wolf. DAV and you'll fall into this website they have a demo just like the one that I showed you here uh that you can play with and also how you can easily get it started with it but also it's a nice project to contribute if you guys are into open source or even using your company because every issue that I open so for example they didn't support Avro uh files before Al schemas before December but then I asked them to I open the issue I tried to to help them with that and investigate and in January it was already there released so they're releasing every month and recently we started working with topic patterns in arafa listener it was also not supported but after I talked to them on their Discord server they also released a new version by the end of April that is now supporting um topic patterns in in Kafka listeners so that's what I have about spring wolf now a little presentation about me my name is hael deu I'm a software consultant at CIA in the Netherlands I'm also a knowledge sharer I have a YouTube page a medium page also part of spring Builders the new spring Community where people are sharing knowledge as well and I'm also the cotl Dutch User Group organizer back in Amsterdam and that's it thank you very much for listening [Applause] okay was much much faster than I expected 30 minutes only nice do you have any questions yes yeah so so another way that people use the open API uh specifications is also to generate the spring code from the specification like not the other way around is there a library like that for for acing kpi as well yes there is the there is a generator for spr spring files actually but it uh I saw it yesterday in C1 version 2.6 so async API version 2.6.0 and from what I've seen it hasn't been updated for five months now but there are and maybe it's also an opportunity to start contributing to this project as well thank you questions yeah uh thanks for the talk um I was uh like wondering two things uh one was the question like uh who uses this topic which was like kicking off the sing API thing um so uh the question of responsibility is still you need to write it in the application properties so this team uses or provides this endpoint so it's not really Auto automatic in a way is it uh so if you look at the code for the Publishers it's not completely automated but for the listeners it is because for The Listener all you all you need is the is The annotation that you already need to make a Kafka listener and spring W fill just detect it and generate the documentation for you but for the publisher since we don't have annotations here that are required by Kafka by Spring Kafka to work then we indeed need to add a sync publisher annotation that is provided by uh spring wolf to to get the the documentation generated for you okay cool uh second question sorry it's just one more thing but the message will be automatically detected from the the method signature yeah and also the oh no no no the channel is is provided here sorry okay thanks um second question is about web UI and uh listeners is it possible to listen to messages like using this kind of connection no I'll just publish yeah okay thanks but maybe you can add this feature I think there thei needs a a ramp up as actually so uh regarding spring Cloud stream does it work you don't necessarily have those annotations like C listener yeah um have multiple types of I I have tried with I haven't tried with Cloud streams but they have support to it and I I believe them that it works but in their website as well I think the demo also shows that they also have examples in their repository if I open spring wolf. daav and go to the demo I have many examples here and also I thought we had the one from cloud stream maybe there is but yeah but it's not showing up here but in the in the repository there are the the examples that you can that we can check how it works thanks sorry can you repeat thanks for the talk uh my question is is not uh open API the API first idea I mean you write the the specification and generate the code yes by using spring wolf that's the idea if you start with a Sync API those generators that I said before then it could be API first okay but from my use case I I prefer to go this way so that's why I wanted to talk about spring wolf yeah but it's possible to create yes yes it's also possible but as I said the the project from what I seen hasn't been updated in five months so we better check it out before start using it this way okay thank you I uh I wasn't aware of this project and I'm seeing a lot of great potential to it so thanks for the presentation uh my question is how does it deal with things like deprecation and version and schema EV Evolution like how does it convey information that this field won't be available one year from now or you know things like that that's a good question I don't I don't have an answer for that yet I I haven't got to this part of my project yet where I have to deal with it so still a good answer thank you yeah thank you no more questions okay yeah so at our company we use a swaggerhub to collect all the AP open API definitions into like one Central Hub and I just checked on the website and they said like knew there is now an async API integration so I got an answer to that question but do you have any experience with it is it working well is it integrated well uh I I haven't I don't have experience with it but I saw in their in their website as well that they have integration with backstage so maybe something similar yeah okay thank you very much everybody [Applause]

Channel: Spring I/O

Views: 1,629

Keywords:

Id: DylvTW_ia4Y

Length: 35min 29sec (2129 seconds)

Published: Wed Jun 26 2024