jOOQ Transactions with Kotlin Context Receivers

hello it's Duncan the last couple of episodes have been pairing with Lucas Ada to learn about the Duke SQL mapping Library we ran out of free consultancy before Lucas was able to explain how to use database transactions so in this episode I have to work that out for myself having done that I listened to the spider sense that is telling me that we have logical issue in our persistence write a test to show that I was right and then fix the test with a bit of a hack but one that might stand the test of time Lucas left us with a working Duke item type the implementations of this items repository but without working transactions now that's less of a problem that would have been with exposed because when we load we're doing it inside one SQL statement here but we're still left with stop which wants to run two things inside one transaction it wants to load and save the items in this one transaction block so let's look at how we can get transactions working within Duke this would be easy to mess up so I'm going to write a test for it and I'm going to put it inside the Duke items tests as a test just for Duke at the moment so we'll add our test fun transactions now what we want to be able to do is write a test that fails because we aren't using transactions and transactions are all about running things in parallel so a sort of thing we want to be able to do is say in a thread and that's creating and starting a new threading content we would like to take our DSL context and use it to save our initial stock list and now in that same transaction we're going to wait a little while so I'm just going to thread sleep in here for now say half a second now while that thread's running and in a different transaction we'll make sure the thread is running at all so it's just very sleep for say half that time and then we're going to say what happens if we load so we say DSL context load and if we have transactions then they're working then that should be the empty stock list and not the one we're saving in this other transaction up here so this should be I think I call it null stock list is what happens if you load that's assert equals I'm gonna run all the tests in here and see what we get okay that does fail and it fails because we expected well we didn't really expect but when this passes we'd like to see that we get no Sage stock list effectively if we ask the question before this transaction finished now we don't have any transactions here at the moment so this isn't working before we go on though these sleeps aren't very helpful what I think we're going to do is we're going to create a cyclic barrier and this is a way of communicating between two threads so that they both end up at a synchronization point so we've got two threads so we'll have a cyclic barrier of two and that means that in here we can say cyclic barrier await and if I say cyclic barrier awakened here then both threads will wait at their respective await points until both of them arrive and then they'll both be let go let's run and see what happens there and so that has the same effect okay how do we use transactions in Duke well here's the documentation and it says that we take a DSL context we've already got this one here and we call transaction on it which gives another configuration that we can use to create a different instance of our DSL so I'm going to copy this and now back over here we'll paste it into here this was our initial DSL context and the kotlin equivalent of that Java is a Lambda here that takes a TRX of that configuration and is this block here and that configuration is dukes and then we want to use this configuration to give us another DSL so we say Val transactional DSL is RT r x dot DSL and if we use that in here let's see what happens okay we're not quite there we're still seeing this modification and I think the problem here is that we're waiting here and here but as soon as this code is let go it can save and exit the transaction before this load down here gets to load so I think what we're going to do is we're going to wait twice we wait in this barrier again and run that then it passes actually it passes though because this thread is always stuck here so we're going to do now is we're going to await on the barrier again and show that if we do which lets this one run then we get our initial stock list by loading technically I think we should also make sure the thread has finished so we will remember the thread and we'll do thread join here and that will make sure that this transaction is always finished before we load here check that okay we have however made quite a few changes since we saw the failure without the transaction so in order to check that in here we'll not use not the transactional DSL context but the normal DSL context and check that breaks in the way we expect does fail and it fails because we're seeing the same stop list in this initial load here replace that with our transactional DSL context and the transactions are working as we expect what I think I'm going to do now then is to implement the same test but without in transaction block here so I'm going to take this and duplicate it we're going to call this transactions Fike and here what we want is instead of this DSL context transaction we want to say items in transaction and we'll use our items save initial stock list now that's not compiling and the reason is that we don't have this context of i o we need a problem we can fix by saying that this is a context IO function because if you remember we made our items contract then did our items contract with this i o resolver that means that we can write context functions for our tests okay returning here we'd also like it that we use the items methods in here so in here we'd say items dot in transaction and we would say that this is items load will be a success of null stock list that's because our items interface returns result types here and similarly down here this should be initial stock list run that and it unsurprisingly fails okay then let's see the whole of that test and move over to our Juke items at the moment this is using no TX which our way of saying that we don't support transactions but you may remember that in case of exposed that's here database items we created this exposed TX context so let's copy that I have a Duke version of that this would be a Duke DX context and I think I'm going to start using ooq in lowercase now if we had one of those then our Duke items would be items of a Duke DX context and that would make this a context that took Duke TX context and both of these functions would need two transaction context to run now we need a Duke TX context to pass to this block so we can go to the base DSL context from this Duke items and say transaction and that's the equivalent of the code we've got here which is the one that takes a TRX which is configuration now inside here we need to create a Duke TX context this is our TX context maybe with a t x and that's going to be a Duke DX context and inside here we don't want this transaction we want the transactional DSL context so that's going to be a cell context that's the one we get from TRX dot DSL that's equivalent to go down here and now we can pass that to our block this almost compiles the problem is that DSL context transaction we can't return out of it's not an inline Lambda not even kotlin but going back to the jute documentation we can see here we've got this transaction result method which allows us to actually yield the result so let's go back and we'll change this from transaction to transaction result in which case we didn't return out of here but returned it from here which makes this into an expression I'll be there we'll run the test to find out now our first issue is we haven't fixed the tests so we had this Duke TX context that is going to be the type of our test now we're able to compile but we're still not quite working and the reason for that is this unused DSL context here at the moment in Duke items when we save we're using a DSL context which is the one passing here but that's not actually transactional we can fix that by making this a vowel and to avoid confusion I'm going to call this transductional DSL context and then that is what we want to use in here and in here and because of the wonders of context functions this transactional DSL context is being taken from the context of gptx context which is passed into the block here is that all right cross your fingers and we're good I think I'm going to have a quick tidy up and check it in before it has a chance to realize that it could fail I think maybe we will rename this with the lower case and the same with the tests and now let's run all of our tests and we seem to have some issue with rename let's just have a little look I suspect that we just need to rebuild and now we run this problem where I think files with the same classes but with different capitalizations aren't properly recompiled I think in the past that's been a question of deleting the build directory let's just see where that gets us because there's everything to rerun Duke to rebuild its code generation all good phew and finally I think I'm going to remove this transaction Spike because we've taken the code out of there and put it into the production Duke items and that commit Implement transactions in Duke items good just for fun I think I'm going to try running this code against exposed so we'll go to our database items tests and paste it into there as well and see whether that works there it does that's nice I'm gonna have this duplication in the two tests which is a bit irritating we can put that in items contract as a not test method and then in our Duke items we can say that this is an override they're called super transactions we could open up there see whether that works and it does and we will do the same thing in database items tests but in here we'll just add a fail to check that we are running we are good delete that and everything's Blended and now commit that as test transactions in exposed good before we say we're done for the day there are a couple of little things in the back of my mind the first is that I wonder whether or not we can save an empty stock list we're returning empty stock list before we saved anything but can we save a stock list with nothing in it after we've already saved a stock list in order to find out I'm going to duplicate this test and this is going to be it can save an empty stock list so we're going to save an initial stop list show that we can load it and then instead of just removing some items we're going to remove all the items I'm going to save an empty list I'm going to save that and then check that we can reload it and run the tests and lo and behold they fail and both the Duke items tests and the database item tests fail now the reason for that is that a clever scheme to have database columns for the last modified we would save all the data for an item with this common last modified from stock list means that if we have no items we don't go into this for each loop at all and we don't save anything that's a bit embarrassing really I suppose at least it shows that I'm being honest when I say I haven't planned the solution weeks ahead we are actually doing test driven development here but what can we do about it that we could solve this problem with two tables where we have one table for every save and another table for the items in that save but there's there's quite a bit more complicated we could solve it though if we had an item that didn't really represent an item it just represented the fact that we saved something so the sort of thing I'm thinking of is that in here if we say if our stock list is empty we do something else and we need to make that compile and for the empty case of the stock list we can insert into items a special item now we don't have an item because the stock is empty so we need to make up values for it and if we made up an ID of no items saved we'd need the stockings last modified the name would be this is not an item say the quality we have to give it some sort of quality that is an INT and probably a non-negative one let's use int max value and the cell by date we're allowed to be null so we can put that in there I know if we can detect that that's the thing we saved we know we should be returning an empty stock list so let's have a look at that code now we left this in a bit of a mess let's first of all rename some of these things this is not items this is last modified two items this is not last modified this is last modified to item now I think we can say is empty is if we have a last modified item the second one will be the item and if that has an ID with a value of the value of the thing we put in the top which was no item saved just looking at proof of concept at the moment here then in here we can return a stock list and if it's empty empty list else that thing are we feeling lucky well let's run the tests quite compiling here uh I will to continue our little chain of question mark dots try again and now our Duke items tests passes and add database items the one we implemented exposed doesn't well that's a start now let's go and see if we can tidy that up a bit we go up here I think we should be able to create an item that represents no items it's effectively what we've done here so let's say vowel often we call this a sentinel so this would have an ID is an ID of no item saved and I'm going to double bang these they return null but this is a constant so if it ever passes or continue to do so now the name is going to be a non-blank string on our this is not an item let's take that our sell by date is null and our quality is a quality of int max value because you know the qualities are non-negative so we can't use minus ones and again as this is a constant I think it's safe to use a double value now we're set up with that I think we can abstract what it is we want to save so we say Val to save is if the stock list is empty then it's going to be a list of just our Sentinel item otherwise it's going to be our stock list items that allows us to State this out of here and the action here is now to save for each try running that still passes for Duke and now in our load I think you would call this actually first and last modified item and then we can say is the first last modified to items second thing equal to our Sentinel let's try that good and now I'm not going to claim that's a nice solution we may come to regret it I don't know it has the advantage that there's strings in here that if they get into our UI they should be noticed and somebody will report them I think it may be good enough I'm going to make that private tidy that away and I'm a little suspicious about these so I think we should say question mark colon error invalid ID this is a valid name this is invalid quality we don't have tests of those but that's just a little bit of better Diagnostics if they do go wrong and that would allow us to inline that the arguments on separate lines at the end names the arguments here inline that inline that and that making this an expression body run this test one more time good good and I think this may be the time where we say that maintain two database items isn't worth a while we have to choose and on the whole I'm most impressed with Duke so I think I'm going to get rid of exposed now we're not using either aluminum construction so that's all right let's go to our database items tests delete that and similarly with our database items delete and lastly if we go to our Gradle build file we can take out exposed from here and build and run ah that seems to crush the compiler and actually as I think of it we're still using exposed to create our database schema so I think I'm getting ahead of myself let's undo that roll those two files back until we some way of building the database cross your fingers so now we have a bit of a moral problem with the database items I think we're going to go to the database items and say it's override can save an empty stock list for that one so we'll go to database items tests and we're going to override can it's not going to complete for me is it oh well can't save an empty stock list take all of this and make it empty which means it needs to be open can we finally run yes we can so let's commit that as save and empty stock list in all but exposed all right second thing I want to look at was around spanning transactions so here in the test we wrote earlier we are running two transactions one that can't see changes and one that can't see changes what I'm interested in though is if we were to take this items in transaction out and that one then what will happen what might happen is because this is all one transaction you will never see any changes from this one let's run it and find out well that's passed and that means that it has seen the changes I think what's Happening Here is we're seeing changes are committed to the database from this transaction in the middle of this one this transaction is able to see things that have been committed by this one it's not truly serializable when I don't at the moment is how to make that not happen I don't know how to set the transaction isolation novel for Duke one of the reasons I might want to do that is because having a transaction isolation level of serializable is I understand it quite useful for event sourcing I don't think it's unsafe here but I'm still learning about the effect of different isolation levels and it would at least be nice to be able to play with it and see that I'm right that that's why this is behaving as it is in the meantime though I think we'll just undo that leaving us where we were when we committed last time well as I just realized a few minutes ago if we're to use Duke but not expose we're gonna have to find a replacement for exposed create missing tables and columns because at the moment Duke is reading its schema from the database and the database it's been created by exposed the grown-up way to create schemas for databases is by migrations where we specify the database we want and then how to migrate from the first version to the second and third and so on doing a bit of research I think we'll be using Flyway for that and Duke appears to have good integration with Flyway but I have quite a bit of learn there if you like to see me learning that then please subscribe to the channel and if you've enjoyed this episode then I think you'll enjoy the book that I wrote in that price called Java to Cotton in a refactoring guidebook details of which are in the show notes below thanks for watching

Channel: Refactoring to Kotlin

Views: 988

Keywords: kotlin, context receivers, jooq, transactions, test driven development, refactoring, databases, postgres, postgresql

Id: GLYL2bkNPjM

Length: 22min 37sec (1357 seconds)

Published: Fri Apr 21 2023