The cleanest way to use Docker for testing in .NET

hello everybody i'm nikki in this video i'm going to show you how you can make it extremely easy to work with one of the most challenging parts of integration testing and that is dealing with the database because unlike unit testing where you can just use an in-memory provider or mock the database calls in integration testing you actually want to make those calls of the wire to a real database and if you're using empty framework you should not be using the use in memory database approach you should be calling the real database and if you're not doing that your tests are not really representing the reality of what your system will do so those are bad tests but in this video i'm going to show you how you can make it a breeze and we're going to use docker for that if you like a lot of content and you want to see more make sure you subscribe bring the certification bell and for more training check out nickjapstas.com now just a quick reminder that i will be running my in-person workshop introduction to effective testing in cshapn.net in a bunch of conferences this year for now it's going to be ndc oslo ndc sydney dotnet days in romania ndc minnesota and nbc london all the data on your screen and you can use the links in the description if you want to get a ticket and come and learn with me in person and now back to the video and speaking of that this video is only a fraction of what i cover in my full length from zero to hero course on integration testing on my website i'm gonna put a link in the description if you wanna master integration testing i highly recommend you check that out it covers everything you'd want to know about integration testing all right so let's see what the problem we're trying to solve is so over here i have two projects i have my customers.api and my customers.api.tests.integration realistically you would also have like unit tests and other stuff but i removed all that and i'm only keeping integration tests just to narrow the scope and if i go over here in the database and i expand you can see that i have a real database for this api over here and if i go ahead and i run this api and i use postman to file those requests this is just a crowd api and i can go here and i can say make me in the system as a customer and it will do that and actually if i go here and i refresh then you can see that i am indeed creating this postgres real database running currently on localhost as a docker container over here and i can go ahead and do other stuff i can list all customers and lo and behold nick is here i can get a customer by id here you go i can update a customer by id over here so i can do that and then i can go ahead and delete a customer and that's it okay deleted and if i refresh that the data is gone so this is the system we want to write integration tests for and i actually already have written in the creation test for this system now for those integration tests i'm using the web application factory which is spinning up an in-memory version of my api but it is a real api and i'm using that to do different stuff for example check if a customer is created when the data for the creation is valid or check that the customer returns a validation error when the email is actually not a valid email and those are just a few of the tests i would have in my system and if i go over here i can go ahead and run some of those tests so create customer controller tests both of them are passing or i can go here and say delete customer control tests and those are passing as well it is actually spinning up an api it's calling that and it all works fine however those tests as they are they don't really customize the core project meaning that we are in fact calling the real database if i go here and i expand this as you can see we have some test data remnants over here which i could delete after each test execution but that implies i need to have a database running at all times that other systems might be using or what do i have like a test database i keep up for my tests to run and how does this work when i'm doing continuous integration and i'm running those tests in a pipeline and how do i deal with everyone running it on their own machines it gets a bit like in fact it gets really really clunky now over the years i've tried quite a few approaches to combat this issue and most of the best ones actually involve docker docker allows us to create a database specific for the test execution in isolation so every time i run my tests i can actually create a database either through the compose or through the mechanism we're gonna see now and target that database that then we can clear down and we don't even have to worry about deleting the data and the reason why that's gonna be great is because if you wanna parallelize your integration test execution which can be a very tricky thing because when different things do different stuff in the same database you can have collisions like the following one for example if i run all of them and these are configured to run in parallel within the scope of each class then as you can see these tests fail because the other tests that run left data that these tests cannot delete or can't deal with and they fail and we don't want to do that we can be in a situation where we're speeding up one database for a set of tests only then these tests only run sequentially now to solve that problem i'm going to use an open source nuget package called test containers and you know how it is i'm going to put a link in the description if you think this containers is a cool nougat package go on github and give it a star it really means a lot to the creator of the library and all i need to do to get started now with test containers is search for test containers over here it should be somewhere over here yeah just test containers and i'm going to add this on the integration project and now what i can do is i can go to my api factory over here which is the web application factory and i can now modify it to take into account the database that i spin up per test execution so what i'm going to do is i'm going to say override configure web host and in the full course i show you how to do other stuff as well in here but for now we're only going to focus on a method called configure test services and this will give us access to the service collection over here the dependency injection services and we can now customize them the first thing i want to do is if you can see over here in my database code i am adding an idb connection factory which creates the connections for my postgres database so what i want to do is i want to say services.remove all and i want to just remove all of type idb connection factory now you technically don't have to remove it and the reason for that is that when we register something on top of it if you only resolve one through dependency injection you're always gonna get the latest one however it's way more safe to go about it this way where you remove things and remove all the doubt on things that might happen and instead we just say add singleton over here and the singleton of course is the idb connection factory but over here we want to add a new mvgsql connection factory over here and this requires a connection string now i know how a postgres connection string looks like it looks something like this so i'm going to copy that connection string for now we're going to see how we can change it and i'm not going to change anything of these parameters except for maybe the port let's give it another port for now i'm going to explain why and now it's where we're going to use the test container so i'm going to say private read only test containers container yes i know the name is a bit weird i'm going to call that a db container and we're going to create a new test containers builder over here and the type is just going to be test containers container for now and i'm going to say that i want this to be with image so we're using a fluent api over here and the image will be postgres latest for those tests realistically you would pin this to a specific version of postgres but for my test it doesn't really matter then i'm going to provide the few environment parameters so username password and the default db goes here then i can specify the port binding so this is the internal port and this is the external port which i'm using over here in the connection string as you can see and all of these things actually do match with what i configure here and then i also want to have a wait strategy and this basically tells the code to not proceed with anything that we want to do without having an okay from whatever we define in here for example the container is up and running in this case we're just going to say weight dot for unix container because we're using unix containers and as you can see you have a bunch of options over here but we're going to go until port is available 5432 you want to use the internal port here and then you just say build and that's it we now have our db container now how do we start it in the beginning of the tests and we stop it in the end of the tests well this is where the iasync lifetime of x unit comes into picture so x unit allows you to implement the iasync lifetime interface which adds a couple of methods initializating and dispose async which will run per test execution but because now we have it ineffectively what is a collection going to run per class that has tests in it so each class with tests will have its own database effectively and this is a great compromise between isolation of tests because you don't want to go one per test but if you can limit it to a collection which is the same scope then you can do stuff with it internally to make it run faster as well so all i need to do here is turn this into a sync and say a weight db container start async and that's it and the disposer wanna say stop async and turn this into an async method and that's it i'm gonna also say new here just to not have any conflict with the i async disposable and basically that's it what i can do now is i can go to the docker containers view to show you that hey there are no containers running over here and i can go to the unit test view and these are my integration tests i can say run and i'm going to be very quick because i want to show you how the containers will spin up this is one that test containers creates and this is the one we create for the test and you can see my tests have actually passed and in a second those containers will disappear and here you go they're gone so the containers are spun up just for those tests and then they disappear and you can see any data you want during startup and you don't have to clear them out because the container won't come back in the future it's going to be a new one every single time it's great for throw away containers however this approach has a fatal flow and you might have seen it already because i'm using a static port which will be shared across every test so if my tests are concurrent between different collections then when i run all my tests then only once it will pass and the rest will fail due to a port error and i can actually fix that by making it so i have a random port but then you don't know if the port is available and sure i can write code that checks for that but then i have to write more code on top of that and it gets a bit tricky but i can have something like this private int port equals random dot shirt dot next and i'm just gonna hope that there are available ports between ten thousand and sixty thousand that i can use to demonstrate my point and then i can take that and i can use it over here but of course i need to also be able to instantiate this every single time so i'm going to move this in the constructor instead and this should now work and i can also pass it down here here we go and now with all that if i go ahead and i run my tests then as you can see they're running and hopefully they all independently passed but there's a lot of moving parts in this equation and i don't want to have to deal with them really so what's the solution well test containers actually has this amazing set of databases and other services it supports natively so i'm going to go ahead and delete that and i can now say new test containers builder instead of going with a test containers container i can say post grass ql container and i can replace it there as well and now all i need to do is say database and this now has a special object the database configuration as you can see there's a bunch of them the one we want is postgresql and we can specify to overwrite the image if we want as well in here in my case i don't want to and i can go ahead and provide my database username and password and you have other things you can provide if you want to like the port and whatever but i'm going to leave the port open because what this means now and if i go ahead and build this i'm getting this object back it means i can go ahead and delete all that connection string and instead say dbcontainer.connectionstring and this gives me a connection string for that container and the port will be random every time it's not static in our case meaning i don't have to worry about classes and available ports and anything all i'm going to do is run it and it will run and as you can see all the containers are spinning up right now and we have them being created tests are running and now they're finished so the containers will go in any minute now so as you can see this is a very elegant solution and i really like that they have these native databases that allow you to just support any of them for example as you can see you also have couchbase oracle redis couchdb mongodb mssql mysql you have all the popular ones over here and for the ones that don't exist there are third-party packages that you can use on top of this so it's a very nice and brilliant way to dealing with a problem and i'm personally using this package myself if you like what you saw highly recommend you go and give a star on github for this nuget package i believe it's brilliant and i hope the developer expands it to more databases and even more things that we can use for our projects so that's all i have for you guys video thank you very much for watching special thanks to my patreons for making videos possible if you want to support me as well you're going to find the link description down below leave a like if you like this video subscribe for more content like this hitting the bell as well and i'll see you in the next video keep coding you

Channel: Nick Chapsas

Views: 83,426

Keywords: Elfocrash, elfo, coding, .netcore, dot net, core, C#, how to code, tutorial, development, software engineering, microsoft, microsoft mvp, .net core, nick chapsas, chapsas, dotnet, .net, dotnet docker, docker dotnet, docker in .net, docker c#, docker test containers, testcontainers, docker containers for testing, c# testing containers, docker containres, running docker from c#, contrilling docker from c#, docker and .net, The best way to run Docker containers for testing in .NET

Id: 8IRNC7qZBmk

Length: 13min 36sec (816 seconds)

Published: Mon Jul 25 2022