Learning Golang: Introduction to Benchmarks

hello my name is mario welcome to another learning go video in today's episode we're going to be talking about benchmarks as usual the link to the code will be in the description of this video so feel free to check that out what i have right here is just an empty go module if you're not familiar with modules i will be leaving the link to that video as well in the description what we're going to be doing we're going to be testing something really really simple uh how fast or slow are the different ways to concatenate two strings and go with the different packages that are included in the standard library only so this is going to be pretty much uh really really simple so we have um in order to use tests for in order to use or build benchmarks in go you have to use the testing package like similarly to what we do with your regular tests that you can build using the standard library the biggest difference is that in order to make those available through the test command you have to prefix them with a word called benchmark so benchmark and then the name of your benchmark whatever you decide to call it and what is going to happen let's call it xyz for now and it's going to be receiving a b which is testing b so it's basically the benchmark that is going to be implemented in go now for benchmark to work there is basically an implementation via a for loop that you have to implement so uh and equals zero we have uh n gray less than then n oops and then n plus plus so what this basically means is that depending on the configuration that we pass in via the cli when we're running the go test command this is how many times this benchmark is going to be running inside what we have here is the actual code that we want to test and measure now like i said we're going to be using the different ways for testing or driver concatenating two different strings and you may you know may or may not be familiar with this but there is a way to do it using the form as printf uh one using the bytes package and one using the strings package so we're going to be using all three of them just to see which one is the fastest according to the configuration that we're going to be passing in it's not going to be anything overly complicated you just have to do uh you again like i said concatenate two strings so sort of like concatenate two strings um using what we have so let me define a string here in the case of fromp spin f we're going to do in str equals what we had before again this is sort of like the naive way of defining values so we're going to have what we had before and let's say a next we're going to be copying these oops i had a typo here so benchmark so everything seems okay we're going to be doing now bytes the bytes package which will be bytes write the string or are you buffer bytes buffer byte buffer again we're going to initialize a buffer here bytes buffer and write the string right string a and finally uh oop of course because i'm going to using i'm using the same variable name that's incorrect so write the string it receives a you know a string and it writes it to the to the buffer and finally we're going to be taking the little of this we don't need any of this we have the strings in the strings package there is a topic called builder so we go to benchmark strings builder we can call it builder strings builder again this is scroll down a little bit so we can put it in the uh in the view so builder right string and then a so all of these should be compiling nothing other than that out of the ordinary so we have three method three methods two benchmarks one using thumb spring f by this byte buffer and x things builder now the way this works to run it in go you have to use the test command now the difference between this and when you're on your tests and then you need to pass in this argument called bench you need you can specify a regex to indicate what benchmarks you want to run for this case we are going to be just running everything for the sake of uh demonstration now one important thing to notice when you're running the tests your pen max benchmark rather is that this is this minus eight but it's really not a minus the the the dash is a separator that indicates how many cpus are available in on my computer in this case which will be eight uh in the context of go so there is a way to differentiate that because you can also pass in how many cpus you want to use because there is also a way to indicate if you want to run this in parallel now one thing you are noticing and i'm going to fast forward the video after i talked i finished talking about this is that from spring is taking so long and we're going to be using that for refactoring and i'll show you i will show you a tool called bench stat that it allows you to compare results from two different benchmarks and determine if what what uh implemented afterwards or before that is making a difference so let's watch a little bit and let's see how long it takes i will be back in a few so running that test actually took one minute two minutes and i obviously fast forward the video so you have to be waiting here for two minutes just waiting for the thing to complete as you notice uh what took the most time was the uh thump implementation using the spring f sprint f sprint f so that's clearly an indicator that you shouldn't be using that package and that function specifically for concatenation and creating a new string now the way we can read this from the output is that there are obviously the three benchmarks that we implemented in our test file each one of these lines indicates how many times the benchmark was executed which basically means how many times this for loop was called or or rather the value that we have in here i will show you a way to explicitly indicate that via the cli and what comes next is actually how long it took to run each one of those operations in average so in the case of thump it took 13 130 000 nanoseconds per operation and in the case of bytes buffer it took five nanoseconds in the case of a strings builder it took three nanoseconds like i said we can actually indicate account which if you notice i put it in the readme so you can say hey how many times i want to run the benchmarks how many times for each one of the iterations that i'm going to be using there are two ways to do this one you either can specify a a time like value like 20 s 20 seconds or you can specify a time or rather an amount like 20x like it would be 20 times so if i run this again with the bench time [Music] which would say i want to run it i don't know 100 100 times 10 times because i know the from um you know what and because i know thump is taking the longest let me comment it out for now so we can see the order to run so let's say 1 000 times 10 000 times all right so if i run this again what is going to happen is going to be running those iterations those benchmarks uh ten thousand times this time instead of using the time based logic that we defined before as you notice now the values are a little bit different so that's why well a little bit higher and the reason being is that for the benchmarks to work more or less better you have to have and have them running for a while so that's why by default the count in bench time is using time instead of using an amount so if i run this i say one second it's going to be running the iterations as many times as possible but by the duration of the input that we passed in in the cli which will be one second so that's why you see a more or less a realistic result that i'm trying to show in the benchmark now what else is important in this there is another argument that we can use called bench mem and this is going to be coming into place when you're trying to decide whether uh it's okay to have this implementation of of the code that i'm trying to benchmark whether it is i care about maybe space or i care of about the um how fast it is so if i run this again with one second let's run it for two seconds and i indicate bench mem what is going to happen is now it's going to be including the memory that it was used as well as the locations and how much time it took like before so in this case the bytes buffer it uses two bytes per operation they are not all allocations and the string builder is using eight bytes all right i was reading the left side six bytes per operation and still no allocation now this is a way to determine hey how much um can i can i decide which one to use because if you notice this one is taking its lowest but is using less space and this one is faster but it's using more bytes so that's the decision that you need to make depending on the algorithm that you're trying to do now how can we go about doing this the other way around let me show you another tool called uh the tool that i mentioned called bench stat and for doing that what i'm going to be doing in this case i'm going to implementing be implementing a method called concatbench marks this is the name it's called main that's incorrect benchmark so benchmark benchmark we have a function concat and it's going to be using the one the way that we know for sure is the slowest one for concatenated which will be using return from sprint f and doing the concatenation and again this is just again a silly example but i want to show you the simplest way to to implement this kind of things these benchmarks so now that i have this i can actually take what i commented out before and just change it concat and just run it concut a and b and another cool thing about when running the benchmarks is that it's also similar to what you can do when running the tests and you can specify a name or write a run and you can say i want to run only benchmark concat and let's say two seconds it's going to take a while so let's let's put yeah let's use the default value which will be one second now this one is going to take also also a while and now i have my you know what i made a mistake it's not run it's called actually um uh is it run now run is for test it doesn't matter don't worry about it so there is a way to explicitly indicate which one the benchmarks we want to run what is important here is that now i know i want to run this test specifically and i want to say hey i want to compare the logic that i have right right now which will be using thumb with a more efficient way for example using a strings builder or maybe the bytes uh the bytes build the bytes buffer implementation depending on what i decide to do and then compare the values that i had before to what i have now and see if that makes any difference if it's improving the change or not so how can we go about doing that and for that like i said we're going to be using this this tool called bench stat that you can install by running go install bench stat and then latest it will install it in your go bin folder depending on where that is i actually using the m dear m which is a tool that i really like i will be leaving the link in the description as well that allows you to sort of sandbox each one of the binaries depending on how you want those configure in your project so i have this right here everything seems to be fine i have a bin file which is the bench stat and the way it works with bench studies i'm going to be talking a benchmark okay i'm going to be taking what i have right now i'm going to be running it again and then i'm going to be comparing it with the new implementation that is going to be using the strings builder the strings package and the builder type so let's do that now so i have to run benchmark concat and what i'm going to be using is a command called t and what t does is receives the value that is coming the the the output that is coming from the previous command and it prints out that by default it is not a library but also you can save it in a file so we're going to be doing that so we do a t-current and it's going to be running the configuration that i have here uh oh one thing that i forgot to do will be to run this let me comment that out remove this we don't need this anymore and we're going to run in we are going to be running this for five times and the idea is like like i said before we have a bunch of different results that we can compare afterwards if i run this only once it couldn't it's not going to give me probably the best results that i want so that's why i'm trying to run the thing in the benchmark multiple times that's where count comes in so now that i have this if you remember with a stream builder we have a strings builder we can call it this b we got b we're going m return of course i had the same issue that i had before with the test so builder builder it's a builder string return and what we're going to be doing next is red builder write string a builder registering b this seems to be compiling i run this again now we're going to call it new now with neo is going to be creating a new file obviously and and we're going to be using the two files to compare the result we call bench stud new and current we pass those in as arguments and we can see the result that we have right here so in the case of concat you will notice that because i run it multiple times that's why you have this but you can notice how in the end it's giving me different results depending on on the times i was executing it but if you notice what what you can see i mean right away is that if you compare the actual results that were coming from go test you'll notice that the nanoseconds are lower to what i had before it means it's running faster is obviously allocating less memory uh there are three less three allocations this one is using eight bytes compared to 34 so there is a huge improvement the cool thing about using bench studies and actually outputs more human friendly ways so to speak in a way that you can you can share this with your peers if you wanted you can say hey the new implementation is now 400 times there is a delta 400 percentage so there is a significant improvement right there which is also right here you can see so i like this one the tiny tiny tool that you can use as well you don't have to you can literally just look at the results and say hey this is obviously faster than the other one so this is benchmark the only a few tips that i want to give you before we go is that when you are using benchmark the benchmark the benchmarks try to also consider only benchmarking the things that don't have any io operations like for example connecting to a database or accessing a database or maybe connecting to a remote data service or whatnot try to only benchmark as much as you can things that you can have in memory in place in in in your system when you're trying to to collect those metrics because it gets it could you know give you so many different results depending if maybe there's latency or issues similar that are unrelated to the actual code that you're trying to test and make sure that when you're testing the benchmarks what you're testing is equivalent to what you're trying to describe for example to give a more concrete example would be you if you're comparing a result that happened to be using three cpus in the past well use three cpus now so they that you can compare those results and those make sense in the end anyways thank you for watching i hope you find this useful i will talk to you next time take care see

Channel: Mario Carrion

Views: 323

Keywords: golang, golang benchamrks, golang benchmarks, golang testing, golang benchstat, golang bench testing, golang performance

Id: u6dpEuJ7tB8

Length: 18min 19sec (1099 seconds)

Published: Tue Dec 07 2021