Getting creative with jqwik generators - Advanced property-based testing in Kotlin

hello it's Duncan last week had a lot of fun using jqu for property based testing of our tempin bowling simulator it found a bug will B A minor one in the construction new games this week we look at checking properties of games that are in play this starts simply enough but we soon run into the problem that because there are two balls per turn we can't represent a game is just random PIN counts if we knock down eight pins with our first ball we can't knock down three with our second in the end I have to introduce a data model for testing which is separate from and quite different to our production interface by the end I'm left wondering whether this is just an artifact for testing or whether it's actually a better model overall we finished the last episode with a number of properties about creating games the first one is is that a game with no players isn't playable and in order to test that we pass any non- negative integer into any number of frames and try and build a game and for any non- negative integer we should get a completed game if this expression returns false for any input then the test will fail another property is that if we don't have any frames when the game isn't playable so for any list of player names a frame count of zero will give a completed game we also test for any list of player names and any number of frames that are negative we can't create a game so that should fail with an illegal argument exception and then finally we test for lists of player names and valid frame counts where valid frame counts here is anything between 1 and 10 that if we create a game with that list and that number of frames that it is a playable game and since that episode I've discovered we can add an annotation to a list parameter to say that we don't want an empty one so let's tighten things up a bit and these tests help us to find at least one bug in our game creation code but we're not here for the game creation code we're here to actually play a game so let's see how we can do that with the property based test in order to do so we need to find an input that jqu can give us that represents the turns of a game and so I think we should write a property and the fun is going to be we'll keep it simple let's say if we roll 24s the game is over well once that's true 24 isn't very interesting but 20 small pin counts is so we can say roll 20 small pin counts ends the game so what should this give us well we want for all and I'm going to call this small pin counts that we'll call pin counts and that's going to be a list of pin count and as we saw last time we need to write a provider whose name is small pin counts so we've got some here what we're looking for is something called small pin counts that returns an arbitrary list of pin count now looking at the documentation we can derive this from another provider so if we have fun small pin count to just return an arbitrary single pin count then that could be int. any from 0 to 4 and we can map that creating a pin count from it so that compiles and then we can say here that this is small pin count. list and we can say of size and there are 10 frames but there will be two small rolls in each frame so we need 20 rolls in total okay so far so good now we need to apply those pin counts to a game so in here we can say V initial game is a game we'll give it one player red and this will be 10 frames and now we want to apply each of the pin Counts from here to our initial game and the way that we can do that is to say pin counts fold given our initial game and that takes a block here we can ask to fill it in where the accumulator is the current version of our game and as we did in the other tests we can play the pin count to the game so we say game. roll but we can't do that because our game doesn't have role we need to cast down to a playable game so we say game as playable game that will allow me to roll with pin count and the net result of all this should be the end game and if we are right then the end game is a completed game that's worth a go let's try running it good think we can probably inline the initial game get rid of the default frame count and then to give us some confidence let's print the end game to scorecard so we can see what J cookie is doing for us on that and here we are you can see that there's one game there 2 three one two and so on all the way to total score of 36 and I guess there will be a thousand of those random games which is pretty smart let's take that momentum and ass set something else about small pin counts if I duplicate this then the scoring those small pin counts is easy as well so rolling 20 small pin counts should score the sum of the pins so getting rid of our scorecard we can assert something else about that end game which is that the end game dot lines map it frames to scores now each frame has a score so we're interested in the final one last and this thing will be a score for each of the players but we've only got one player now what is our expected score for that player well if we take our pin counts and we'd like to sum them by it. value so that's going to be the score as an INT let's say expected Gore as int and now we can check that scores is a list of the expected score as in or put another way that scores do is the expected score as in but we need to create a score to wrap for that let's just see whether that's true it seems too good but I should have been taking notice of that yellow because that's saying this is unused we need to return it meaning that we will in fact return a Boolean sort of surprised that jqu didn't warn us about that but we still appear to be good and I think it probably is a tighter test to say that the scores is a list of that because that also checks there's any one of them good think we can make this a bit tighter by putting the score around there just calling that expected score in there and this would be a nice little utility so let's pull it out as an extension function so this is going to be scores given a game we make this through receiver we can cut that out of there and put it as a generally applicable nice thing one more it would be nice to do the same sort of thing here if I take this thing as a variable then I think this would make a good method which is roll all and it's used out there as well which is kind of the point if we make initial the receiver so we can take that again out of our test and put it as a top level utility good one thing is interesting is you might think that it would be nice to take a playable game here because after all you can't roll any on a non-playable game if we do though you see this cast here gets a bit weird I think to fix it you have to say this as playable game in here no that's not allowed yeah as I say it is a bit weird so I'm going to punter now and just say that's game roll all go back up and have a look so now this is very simple 20 small pin counts is game roll over the pin counts is a completed game and once again actually this is returning unit so that jqu doesn't have anything to chew on just going to have a look at what the tests actually say property test roll with small pin counts end the game it has run that test I should think that jqu if you have a function that returns nothing considers it a success if it hasn't thrown I guess it's probably good enough but maybe we should put in the Boolean here and now we can tidy this one by inlining that and run oh there we go as if to prove things well I guess we can just make this a single expression with an equals better you may have notice that the tests are now taking quite a time to run we're up to 1.5 seconds in total and it just needs to be the truth that property based testing is going to be a bit slower it begs the question then whether we should have two separate properties here or not we could combine these two into roll 20 small pin counts should end the game with the score the sum of the pins and then take these two tests and combine them into there the problem is that if I do this and one of them isn't true I don't know which one isn't so for example if we were to say pin counts drop one that would fail but our error is just that it failed we don't really know why maybe though with the knowledge that I don't have to return anything here I can get the best of both worlds I can say in here use expect that game roll or pin counts is a completed game and scores is equal to list of the expected score now if I don't do that and I don't return anything and let's just put end game back in there let's see how that fails it did fail and we got our strict diagnostic out to tell us that it wasn't a completed game that looks like it's the best of both worlds rather nice and in fact it would be even better if I was to say expect that end game and put a block in here so that I can say both those things is a completed game and has scores and we should be able to see that both of those fail although ironically for the case we're taking here if you look it's all gutter balls the score is zero anyway we don't want to test against all but one of the WR pin counts that will do us that passes but still lets us know in which way it fails when it does which is the best of both worlds and means that we don't need a separate property here for checking that the game is ended Good okay then thinking it through I think this property is probably a line with no strikes or spares scores the total of the pin counts and maybe a completed line and I think also we might be able to just inline this so small pin counts is any int from 0 to four map to pin count and then 20 of those is that good it is so I think that's worth a check in with starts to property test playing games Splendid now a game with those strikes and Spares is a pretty boring game obviously we want to test strikes and Spares but the problem is how do we generate a valid game since W scores a strike we have one few roles in that game so we're going to need to find a way of generating a list of pin counts that represent a valid and complete game to demonstrate the problem let's duplicate this test and try and write there are no errors in a valid game so this would be valid pin counts and scoring will be difficult but we should still be able to roll all the pin counts so let's get rid of that and that so we can just say that thing now what would valid pin counts look like we've got small pin counts valid pin counts will be anything between zero and 10 so let's go and try and run that you see that it's failed we've got an legal doent exception which suggests that we failed somewhere in our roll all yes in fact you see the dotted line there let's see what we threw you see jqu has been rather helpful here it said well I originally threw this thing that was a six a five a two and so on and so on and I guess sometimes we're going to roll things like maybe a nine and a three where you can't roll a three after a nine because there aren't enough bits of wood to knock down Jay quick has onone looking for a simpler way of expressing the problem and here it is it's all zeros followed by one followed by a 10 well once I've knocked down one pin I can't knock down 10 so it has found a nice simple sample but this comes down to the problem that we generating pairs of pin counts that aren't necessarily valid in a game another way of putting this is that a list of independent pin counts doesn't represent a game of bowling so what to do well I wonder whether there's mileage in the idea of a turn so in Bowling players have a turn if they score a five in a turn then there are five more pins that they can knock down in that turn but if they score 10 in a turn then that turn ends so let's pull on that thread and see where it takes us what I think we'll try is instead of going with valid pin counts we'll go with valid turns so we'll say this is valid turn and it's a turn and it turns out that for list there's also at size so we can try a size of 10 turns and instead of a pin count we want a list of I think I'm going to call it valid turn and so this is turns and we want to roll all the turns okay what's a valid turn let's create ourselves a class for it and we'll put it in here and I think a valid turn is a data class that has two rolles so has vile roll one which is a pin count and it may have a role too it won't if it's a strike so the second role is nullable and to keep us honest I think we could say that in that in it we check that roll one plus as pin count roll two and that may not be here let's try that the other way around let's say roll two if it exists plus as pin count roll one now remember the Plus plus pin count produces a pin count and a pin count can only be 0 to 10 so this will check that the two rols can't add up to more than 10 which is true for a valid turn except for the final valid turn okay how can we create an arbitrary valid turn let's cetch that out given what we have we'll say fun valid turn and this wants to be an arbitrary valid turn now jqu has a combine that allows us to combine other arbitraries so we can say combine and give it a valid pin count we don't have one of those yet but let's just go and pretend it exists and two valid pin count come back to the Block in a minute and valid pin count was the thing we took away from here so it's this thing here that is valid pin count is 0 to 10 mapped into pin count okay when we combine we're given two arbitary pin counts so we'll call that R1 and R2 for roll one and roll two and we want to return a valid turn from that now we could just pass in R1 and R2 but if we do we risk falling foul of this in it we could have a roll one that's 10 and a roll two that's 10 and that wouldn't be a valid turn so we'll have to think a bit and what I think we'll do is we'll say when if roll one is 10 then let's just call that a strike and that will be a valid turn with roll one and no roll to if we' got two pin counts that add up to more than 10 the first one isn't a strike we're in trouble but we can say R1 + R2 now that would gives a score so I think we'll take the value out of that and if that's greater than 10 then we need to make it valid and we can do that by making a valid turn of R1 and R2 is going to have to be a pin count of 10 minus R1 do value and I lost my greater than 10 somewhere that basically means that if we have got a strike but they add up to more than 10 they will make them into a spare and in the other cases we've got a valid turn of R1 and R2 not entirely sure about that logic but let's go forwards anyway up here we said that we wanted valid turns so we want something like this which is a provid duplicate that move it down here and this is valid turns is going to be valid turn list okay the only thing that's not compiling is game roll all turns let's create one of those on game and it will look a lot I think like roll all down here so let's move it to be with its friend and this is not pin count its turns and to make this work I think what we need to do is we need to turn our list of turns into a list of pin counts so we can say pin counts let's give it its type so that we made to get it right this is going to be turns and we could flatmap each turn if it had a two list what would two lists look like let's put it into valid turn there it is it would be an iterable pin count that's good and we could Implement that by saying return list of roll one and roll two dot filter out the null ones and in fact intell says oh that's clever isn't it look it could see that roll one and roll one weren't null but if we make that roll one and roll two filter not null and now it tells us that we could do list of not null okay so a valid turn can give us a list of pin counts we can then use those to build a list of pin Counts from a list of valid turns and then finally here we can return this dot roll all pin counts that compiles well it doesn't actually because our two roll alls have the same signature go and find one of them here we go and we know we can solve that sort of problem with at jvm name and we'll call this roll all valid turns that compiles yes it does okay let's go up and remind ourselves what we were doing so up here then we were saying that if we take 10 valid turns and roll all then we expect to get a completed game that seems like it's worth a go let's run H that has failed let's see why so we'll bring up the results failed after one try and this is what it failed for it failed for 0000000000 and then a one and a 9 if we move down we can see that actually it fell the first time it tried and that had a one and a nine there as well and jqu has then worked its way back to work out that actually none of these things mattered at all the only thing that matters is we rolled a one and a nine in the final frame and that's a problem because that's not a valid turn in the final frame because we rolled a spare and should have had an extra turn aha so we've got some work to do for the final frame and I think we'll back into it slowly by saying let's take nine valid turns and then another valid final turn where we only have the one of it so we'll call this final turn and that's not a list it's just a valid turn now if we do that then we can roll all turns plus the final turn which I should have named final turn and there and we can give ourselves a provider for this that for now at least doesn't score a strike or a spare in the final frame so let's give ourselves we've got valid turns let's say here we'll say valid final turn and that will combine a small pin count uh which again we don't have or got rid of with another small pin count and the small pin count was this thing so again we'll make a method out of that we'll call that small pin count and then valid final turn we want to Mark with at provide to make it used I think that stands a chance of working so let's try it well it passed once and twice let's run all the tests Ah that's quite pleasing possible to get a bit blaz about this I think we'll just check things by saying again printland endgame to scorecard to see the sort of things that's actually happening run that and if we go to our test results and say there are no Aron a valid game here we can see lots and lots of games being played and there are spares there's a strike and the only thing that we haven't really done is Bowl anything special in the final frame we also haven't check scores but we have shown sort of independently that our production codes logic's idea of valid frames matches our valid turns how then can we do better in the final frame this last final turn here let's see what we're currently using for valid final turn here it is oh the same code AS valid turn but with small pin counts that make sure that we don't score a strike or a spare in the final frame to cover all final turns though we will have to allow for the eventuality of a strike or spare so I think maybe what we'll do is we'll give ourselves a valid final turn as a type so this would be valid final turn and a valid turn always has a role one may have a role two in our final turn and it may have a role three so that will be an optional pin count now the the constraints on a valid final turn aren't as simple as that on a valid turn so I think we just get rid of this in it for now but two list should add in the RO three and now to supply one of those to our tests this will be an arbitrary of valid final turn and we're going to want it to combine three valid pin counts so that's going to be a valid pin count a valid pin count and a valid pin count which means there will be three parameters to our Lambda okay now you're probably going to see my eyes go up that way as I look for the example I prepared earlier so when R1 is 10 and R2 is 10 that is to say we have two strikes in a row then this can be a valid final turn of R1 R2 and whatever R3 is because all the pins were knock down the first time and the second time so they could all be knocked down or whatever the third time now if R1 do value is 10 which say the first one is a strike and our second R Value Plus our third R's value so we're going to sum those together is that greater than or equal to 10 and if it is then we've got the same situation as we had here but just with the second two rolls so we need to restrict the number of pins that have fallen down the second time so I think that is a valid final turn with R1 which is a strike a pin count of R2 and a pin count of 10 minus r2's value so related this will compile but before it does we could to ask the question the first two have covered a first roll strike what then when r1. Value Plus R2 do value is greater than equal to 10 well that's a spare in the first two roles so this will be a valid final turn with R1 as a pin count the second one would be a pin count of 10 minus r1. value and because that's a Spare the pins will be be reset so we can use roll three and then finally we have a valid final turn of R1 and R2 but we can't use R3 because not all the pins were knocked down in the first two rolls R3 well that looks like it almost compiles I think maybe this one here should just have been R2 that means that can go oh right let's reformat and what we're doing here if you like is we are finding a way to map random numbers into valid turns in a way that makes sense and hopefully gives a good distribution we could for example have discarded any cases like this one just sort of tried again but the nice thing about interpreting anything where these two add up to more than 10 as a spare is that we'll get more spares and more spares are likely cause us more trouble they're likely to find edge cases okay I don't think that should compile because I don't think that we should be able to add valid final turns to turns I think that should be a problem up here valid final turn is a valid final turn but because using annotations the compiler didn't know the type was wrong so in order to make this work I think we have to say that this rooll all valid turn well we can either change it to work with any and then downcast a turn or valid turn or we can pull a common super class out of valid turn and valid final turn or we could make this a list of list of pin count and we could say that valid turn and valid final turn instead of having two list were a list of pin count quite like that final solution because it means in here we could say that this implements list of pin count by this expression here that allows us to get rid of that and here we are list of pin count by this expression so we get rid of that and that maybe just reformat these make it look pretty and having delayed a gratification there is a compil error somewhere now there we go so roll now a that's odd I did think that the compiler might interpret turns plus final turn as a list of list of pin count let's pull it out and all it says is any's let's go and check the types again valid final turn is a list of pin count valid turn is a list of pin count well maybe this plan wasn't so great after all but let's go back and see what we can do maybe if we said here this is turns as list of pin count that gives the compiler enough of a clue so maybe we'll just roll that into there if you'll pardon the pun okay so the big question is now does that work let's play running with tests oh I guess it was staring me in the face this should be as list of list of pin count plus the final turn as listed pin count I don't know I'm beginning that was a bad idea let's back up okay then what I've done is I've extracted this turn interface that has a rols property valid turn is a turn and its roles are all the nonnull rolls valid final turn is a turn and its rol is all of its non null rolls and that allows us down here to say that rolling all list of turns is turns flat map the rolls are the test and having run the tests I fixed this here where I did had to have R3 and that led some invalid final turns but if we run this now and look at what we are generating there are no errors in a valid game and you see we started here with a run of gutter balls then we have spares and Strikes and so on and in the final frame we have some frames that are just misses we have some that just two BS a strike followed by spare here H I'm not sure that's rendering right is it interesting I think we found a vug there but not from a failing test but only from inspection because we were generating random combinations here's a pucka spare and here's a spare followed by a nine and here's a spare followed by strike and so on so we're definitely generating more interesting final turns than just the raw randoms would suggest armed with valid and valid final turn I think we might be able to play a game with more than one player as well so we could duplicate this and say there are no errors in a valid two-player game and if we just duplicate this then we put in the commas this would be turns one final turns One turns two final turns two and we can take turns one plus final turn one and if we bring that out into a variable then I think we can take that and zip it with the similar thing for the second player so that's turns two plus final turn two and then we need to flatten that ah that doesn't work cuz zip gives us some pairs so I guess what we actually need to do is we need to not flatten turn but flat m map list of it. first to it. second evidently something is still wrong let's have a look at the type of that of course not list of it first to it. second but it first and it second then the type will be right few and I'm going to call this inter Leed turns and now we need two players to make this work we'll just check that it does fail if I try to play a game of one player and there we go we completed the game too quickly we'll find out what with but I guess any game at all really so that's legitimate and but we add in Barney as a player and run we are good going to take that out to make everything run a little bit quicker I think there was one place up here where was it I think we could take small pin counts and give it a size in here so this would be at size 10 that would allow us to go and find small pin counts and allow the corer of it to to select the size not doing it in here but I note that's 20 not 10 are because these are pin counts not our frames Splendid and it is one more print than there and let's check in with property test for valid turns commit I find it interesting that we've introduced a type here this turn in order to facilitate testing we didn't identify turn as a thing when we're trying to find types in our domain but it seems quite natural here where we're testing and makes me wonder whether if we'd introduce turn as part of our production interface we could have made more States unrepresentable I know I keep on saying this but it feels like we're nearly done but we do still have some testing of the scoring to do uh in particular I think the only scoring we're testing is this here plus the special case in here of the maximum score is 300 not sure that I've yet got any really good property based tests for scoring so next episode I'm going to ask AI assistance for its help either devising the properties or actually devising some tests so if you'd like to see AI assistant meets jqu then please subscribe to the channel like this video so other people can find it and maybe even go out and buy the book that open that price called Java toot refactoring guide book details of which are in the show notes below thanks for watching

Channel: Refactoring to Kotlin

Views: 396

Keywords:

Id: mLpVw0h-hE4

Length: 37min 10sec (2230 seconds)

Published: Fri Mar 15 2024