Integration Testing Serverless Architectures, EventBridge and Beyond... - Sarah Hamilton

[Applause] we're off to a good start yeah so today i'm going to be talking about integration testing service architectures so it's actually really great to go after julian because i'm going to be really zoning in on eventbridge now he's given us all a good introduction to messaging um so a little bit of intro to me so i'm a developer at theodo i'm sure we've all heard about that now um i've been a developer there for about two years and over the time i've worked with a range of clients varying from one week to about six months and touched many different tech stacks but it became apparent quite quickly that i really enjoyed working with serverless mainly because when people came to us with an idea it was often the quickest way to go from idea conception to actually launching a product so i could really focus on writing the code rather than the underlying infrastructure last year i also became an aws community builder which many of you may have heard about and but it's a programme for people who are really enthusiastic about aws and it provides mentorship and networking opportunities and it's not just to do with serverless it's also includes like data science and different different topics as well so i would encourage you to have a look at that at theodo we also head up the serverless transformation blog newsletter podcast and the meetup which i've featured on in a couple of times we really have like the most up-to-date um resources there so do check it out i think we've all been intro to serverless quite a lot here so i'll um i'll whip through this quite quickly but what does it mean to me so to me it means that we launch quicker we pay less and we scale instantly and so as i said a lot of people come to us and they have an idea and they want to they might have limited budget and limited time to get something to market quickly i really think that serverless allows us to do that so i can see that the gif isn't actually moving it says explain this to me like i'm five but yeah how do we achieve a serverless architecture so of course we leveraged the cloud so um aws azure gcp um i tend to focus on aws mainly because it's kind of what what i know um and also it holds the largest market share and just so we're all kind of on the same page i know we've touched on these services um but throughout this talk i'll be touching on s3 for our storage step functions for workflows dynamodb for us for our database lambda for compute sqs for our messaging and then we'll be going right into eventbridge so i guess the kind of start with the problem we all kind of know of like the famous monolith um so kind of traditional architectures might be that we start with this beautiful code base and then the business throws extra requirements at us and the engineering team gets stressed because if we add this one thing here then it might break this thing down here or a new developer comes in and thinks we can just add this feature but not realizing that somewhere else there's a dependency on it and we end up with bugs and everyone becomes very stressed and the business wants things quickly and the engineering team builds up tech debt but at theodo we like to use event driven microservices which i know we've kind of touched on as well here um in order to kind of avoid this problem so that when the business comes to us and asks for more and more things the engineering team is actually quite happy to do this and what this means is that when they come to us with an extra requirement either it will easily be added into one of our microservices that we already have or we'll be able to spin up a new service in no time and um and everyone's happy just to dig into that a little bit further um to dig into like exactly what microservices mean to us so each one kind of runs as its own app so we'll use a workshop called domain driven design we have lots of resources on this um on the serverless transformation blog so do check it out later but using that workshop we actually define domains that can be um independent from each other and we use the term loosely coupled a lot but effectively each application within the system doesn't actually need to know about each other so if we take the classic e-commerce example we might have an order microservice and then we might have a payment microservice and each one doesn't need to know about each other and also if you end up with a really large application you can actually have different small really effective teams working on each microservice and each one will have its own data source and it will deploy independently and then the key thing is that if one of the services does fail the rest of the system hopefully doesn't come crashing down if you did do it correctly so there's many advantages to working this way and a key part of it is scalability and at the heart of all of our serverless applications at theodo we tend to use amazon eventbridge as our eventbus so and just to quickly kind of talk us through this um in this diagram you can see that the event producer in the e-commerce example might be the order service which will produce events and put them onto the event bus and so the event might say order created and it might have some metadata within it and when it arrives on the event bus according to a set of rules it will send that event to one of the consumers which would be another microservice such as the payment service to then pick up on that information and use that information however it needs to and so the key thing here is that the consumers don't need to know about the producer and vice versa and so that goes along with the um deploying independently and failing independently and therefore being able to scale so i guess to kind of put a bit of context um to a client that i worked on it was a video conferencing client that i worked on for about six months and this was at the beginning of the pandemic last march and as you as you're probably aware it was video conferencing so it kind of went from just having a steady flow of people to just going boom and they had um on-prem servers and essentially the website couldn't take the load and they weren't able to take advantage of the market at the time so we were kind of brought in as serverless experts to try and help with this and try to take advantage of the market and so the challenge that we were faced with in march of 2020 was to scale to 250 000 users to train up existing developers on the project who were all skilled in.net to release quickly to take advantage of the market at the time and eventbridge was the key to success so to briefly touch on this before we go into how we actually tested our um our event bridge we had the original system where the code base was over 10 years old so we obviously were not going to do a rewrite of that in a matter of months and so we need to come up with a clever way to make this work and scale so we did a migration and eventbridge was at the centre of this so in the new system we had the event bridge and from the old system we were able to pass events into the new system such as a user signing up with their metadata to be used by the new microservices that we were spinning up and so in this example we had a microservice that was called chat so a chat part of the app and we built that up really quickly that skilled really well and we were able to then replace the old chat so that it was scalable and that was kind of how we ended up working towards this new scalable system so to get into kind of the main part of the talk about how we actually test these serverless architectures you also can't see the gif on here but it kind of shows how you can um have all your unit tests working really well but then when it actually comes to the integration test you realize that when things are actually put together they might not work so well um so traditionally i think most of us i hope know how to write a unit test um personally we tend to write our applications in node.js and we use a test runner called jest and so that is what we're going to be talking about in this um in these slides however it can all be translated into whatever language it's just a testing strategy that can be um implemented in a different way so before we dig into how we did it um there's kind of the question of do we test on the real infrastructure or do we test using mocks and stubs so we tend to test on the real infrastructure so like i guess it's kind of like why not um we want we want to test on something that's exactly the same as production and so we tend to test on the real infrastructure um when using mocks and stubs we might do this for like a failed test case something that's not normal so we would need to mock that out because we can't rely on that just happening um there's also assumptions that we might make when um using mocks and stubs that could actually end up causing our tests to pass when actually they shouldn't so we test on the real infrastructure and how do we do this well um we actually put this into our ci process so when we open a pull request um we'll spin up a test stack and we do this using cloud formation and servless framework but again as we've talked about before you can use sam and cdk um whatever suits you best and we'll give it a unique name so ci27 is the example here we tend to name this after the ticket number that we're working on um so that it's unique because all of your cloud formation stacks they need to have a unique name so that you don't override the top of someone else's and then once it's been deployed and at this point we effectively have a copy of the production site ready to go all of the resources spun up in aws we'll run the unit test followed by the integration test and followed by the end-to-end test at the end of this we'll then tear down the stacking cloud formations this is then deleting all of the resources that we came up with and i mean the reason why we do this is firstly i think there's a maximum of 500 stacks that you can have open in cloud formation and secondly you're just paying for resources if you don't tear it down so of course we tear it down at the end of the ci process and therefore it's really cheap to do this as well so to actually touch on the testing strategy now we'll take a really simple example so we've got our e-commerce example again we've got two microservices we've got the order service which has a lambda which does some processing um which then emits an event onto the event bridge the event bus which might be order created as we've discussed and then um due to the rule that we've defined in eventbridge then the second microservice picks up on this does some processing within a lambda and puts an invoice into an s3 but of course this can be applied to any um any use of eventbridge this is just a simple example that we're taking here and again just to be clear we do have um we do have an article that outlines all of this and i'm going to be showing a few code examples as well but all of this is in our articles online so yeah no need for you like taking taking notes so firstly um i'm actually gonna go back so just to outline here again the end to end test would be that when the lanza is triggered that the invoice arrives in s3 which is is a good test um however if this happens to fail you wouldn't know that the whether the problem was in the microservice one or the second microservice so we want to split this into two integration tests and perform these separately and again because each microservice is effectively in its own code base you need to put each of the tests into their respective code bases so they're not kind of running across code base it's one test in one service and one test another service so firstly we want to assert that the event was fired so we want to trigger our lambda and which does some processing within it and then it emits the event onto the event bus now to assert that the event was fired we actually introduced something called an event interceptor so this is something that isn't part of the actual infrastructure this is something that we add in for the test because you can't simply just check that the event arrived on the event bus using our test runner so we use sqs in this case um to actually pull on the event bus sorry um so we actually add a rule to the event bus that sends the event to the sqsq and then using the test runner you can pull on the sqs queue to assert that the event arrived so if the event arrived in the messaging queue then of course it arrived on the event bus too so that's how we did that we kind of had to be clever in doing this because at the time um we'd kind of written our code and then we were like oh we need to test this and we wanted to do it on the real infrastructure not using mocks and so this is how we came up with it and it's actually a really nice way of handling this case and we open sourced it so um so this is great because i'll go through through code examples of what actually is going on behind the scenes but what you'll actually see is that the client code that you'll be able to write is actually really short because we created a library um for people to add this to their projects so the first integration test um so this is written in jest but i'll talk us through it very quickly um so we again were asserting that the event was fired and that it arrived on the event bridge so firstly we're invoking the lambda and using their aws sdk so the lambda from the first service and then we are waiting for that to come back and call in this get events feature and then we're checking that um these events actually have the source that we expect order created and so i have this meme because it's like what the hell is going on because we have this get events and to have event with source now this is all coming from sls test tools so we'll dig into that now but i guess why i wanted to show this um is that the test is actually really simple to write from a client perspective so firstly before we actually write this event we do have this before all so we're building we're essentially building the infrastructure beforehand and this is all happening in sls test tools so before each um so before each event before each test runs um we make sure that we have all the infrastructure in place which basically means the sqsq on the event bridge and so you can see in purple is marked the client code which is kind of the part that everyone will be writing and then sls tools is just to talk us behind what's going on behind the scenes so firstly behind the scenes we're setting up the sqs client which of course isn't part of the infrastructure that you already have in place um and to do this we use the aws sdk and we basically just give it a unique name and spin that up and then we add the eventbridge rule to it so as we said between the event bus and the consumer which is our second microservice um or in this case our sqsq we have the rule to make sure it actually sends it there and so that's setting up the eventbridge rule using the put rule from the aws sdk and then we make the sqsq the target of the eventbridge rule and so you can see that simply we've just added a target using the sqs on which is just a unique identifier for the sqlsq and then we give it a target id which in this case is probably just one but each one needs to have its own separate number and so now that we've actually set up um the infrastructure in the build process what we want to do is call the get events get events method which firstly although there's quite a lot of code here i'll quickly just break it down so we use the receive message on the sqs client using aws sdk to get the messages and we define the wait time as a certain number of seconds and anything above zero means that you're long polling because everything's asynchronous so we need to give it a bit of time to long pole on the sqs queue to actually get the message back if it was zero seconds we would call it short polling and it wouldn't be asynchronous and then we returned the results so it's just the event that arrived on the event bus and subsequently the message came and everything in between is just deleting the message off the queue just to clean up after ourselves and just lastly sls tools is actually comprised of a lot of assertions to help us out so these are just just extenders so you can see that on the bottom line of the client code we're expecting that the messages that have come back actually has an event with the source that we expect order created and simply this is just a jest extender so we pass those two items into it and we re we return a pass or a fail um based on whether there is an event there with that source or not and so that's the first integration test done and you'll be glad to know the second one is actually a lot simpler and so won't be going on for quite as long as that one so for the second one we want to assert that the event was actually received from the event bus so that the so that the event actually uh pushed the event to the correct microservice and that in this case our invoice ended up in s3 so how we do this we inject the event onto the event bus using one of our sls tools helpers and then we just check using jess that the event actually arrived sorry that the file actually arrived in s3 and so this is the client code so we published the event which is just injecting the event onto the event bus and we just um give it the same event that we had in the first microservice of course we can't actually use the real event that happened in the first microservice because all this code is stored in the second microservice and then we wait on that to return to see whether um the s3 bucket actually has the objects and that it received with the correct file name and again it's another assertion so to have s3 object with name equal to we just pass in these parameters um and we use the just extender to return a pass or a fail and of course i'll be sharing these slides with everyone afterwards so that's both integration test complete and that is how we test um eventbridge and in sls test tools we have the range of assertions and helpers um to really help us with our development many of our clients do actually use it and find it really helpful um i think one guy said that it saved him about a week of work by just writing these tests so if you are using eventbridge which i hope that you are in your service architectures then this could be really helpful and we also welcome contributions because it's quite a young project at the moment i'd love to have assertions for um like cognito dynamo and basically the whole of serverless which would be great um and you'll be added to the contributors list so it'd be really cool if people got involved and yeah i'd just like to say thank you for listening and if there's any questions then yeah to ask

Channel: ServerlessDays

Views: 152

Keywords:

Id: f0X1LlHKluo

Length: 21min 55sec (1315 seconds)

Published: Sun Nov 07 2021