k6 load testing on Kubernetes using the k6 Operator, with Hans Knecht (k6 Office Hours #34)

hello everyone and welcome to another k-6 office hours i'm nicole van der hoeven and today i'm live from portugal i've got two guests on today the first is someone that you may recognize floor would you like to introduce yourself yeah sure hi i'm floor i'm a technical program manager at k6 or funnel labs uh and yeah it's not my first time in the office hours but it's always a pleasure uh so thank you for asking me the third time yeah awesome and we also have hans connect can you introduce yourself too yeah sure um so i'm hans connect this is my first time um uh i work over a company called mission lane but we use the k6 operator and our pretty heavy uh contributors back to to that to help it grow yeah you are the the big the biggest external contributor really of k-6 operator for sure [Laughter] we try to help out help out and help yourselves in the process exactly that's the whole thing so what do you do at mission lane um so i'm a principal site reliability engineer so the most of my job involves helping developers write software that is reliable and scalable and useful that actually works properly things like that and so i spend a lot of time on kind of developer experience engineering effectiveness how do we give teams tools to to be self-sufficient and like make the right decision when they're when they're building software for for machine learning okay and how does k6 fit into that how did you get started with the case yes so michigan is a startup we've only been around for like two or three years um and we as we were as we were growing people started to obviously see the need to load tests but also see the need to continuously test in kind of our lower environments and um and they started to talk about like oh we're gonna go try to use chain meter we're gonna try to go build our own testing framework we're gonna try to do you know some of these things that happens when you're having that conversation um and the big uh problem with all of those tools that are doing about yourself is that you kind of have to manage your own fleet of servers that actually run these tests for you actually manage this and it can be a really big pain to do that you have normally companies have to dedicate entire teams to like go run jmeter servers or go run locus servers etc um and so i started to hear kind of some conversations about this and i was like oh that sounds like we sre should probably help solve that problem let's go find a tool that that does this where all of our stuff is on kubernetes um and so you know our primary driver was let's go find something that is kubernetes native like that works in kubernetes really well the scales and it can be self-service which is um started doing a bunch of research and i found um the blog post on the k6 operator and i was like oh this looks really promising teams can go write their own tests put it in kubernetes and have test runners spun up run their tests for them and they don't have to talk to sre they don't have to talk to a testing team they don't have to do any of that they just you go write the test put it in there and it works the the same way the whole way through right if they want to run k6 locally to run their tests to test it they can do that and then they put it into the cluster and the exact same behavior is kind of um kind of drawn there so that's how testing fits in is that they need to do all of this and so we said okay let's go build and provide this platform for them it's interesting that it was sort of a build or buy type of decision but then in the end maybe not by if you use open source exactly right like we got to say oh and that was again a really big driver right like we could have looked at um other things like we could have built a chain meter operator we could have gone and bought a testing thing and and the benefit of open source is that we could say oh look they have an operator k6 housing operator is ready to go oh here are a few features we need to add right let's go contribute those backs give them to kind of the broader community and help out there yeah so i know that j meter because i've used j meter quite heavily in the past as well it does come with a distributed load testing mode for free but it's also not on it doesn't have a kubernetes operator so that is still still a hurdle and we also already have when you're talking about building versus buying we do have k-6 cloud which is the the sas platform that's around k-6 that does do distributed load testing on the cloud for for a price but that's not always that's not always an option for everybody and one of our main philosophies in case x is that we want feature parity as much as we can sometimes that isn't always possible there are some that are in one and not the other and vice versa but case x operator was something that we that um simon aronson actually started because we wanted to make sure that we're not pushing people to to sign up for for k-6 cloud if they don't really need all of the bells and whistles yeah and for for us it it you know because we did look at k-6 cloud i was like and and other cloud products as well that do similar things but again um we're a credit card company we we can't really ship off or hook people into to kind of our kubernetes environment we needed to maintain enough control over who's allowed to hit these end points what type of authentication um and hooking up the vpns into you know k6 cloud for example or doing over the raw internet to kind of expose those endpoints um that was kind of a non-starter to kind of have that so the ability to take something that is like kasich's cloud and put it in our environment and run it inside of our kubernetes environment was really valuable to us to be able to handle really all of those all of those concerns right yeah that's a really good use case for it um you definitely the best way to if you're if you're in an industry like the financial industry you have to be very careful about security and it is easier if you control everything yourself yeah and there's also more control over the infrastructure so i could also see that maybe i know that a lot of people don't want to to rely on a load generator that's spun up in aws or something somewhere that they they that other people theoretically might have access to if it's on cloud or something sometimes if you have on-prem load generators that you already can use it might be worthwhile just to use those rather than to pay for another cloud service and it also in our in our um experience for us it mattered a lot that developers could write the the kubernetes like the tests themselves next to the application code you could store the load test in the exact same place deploy it out using the exact same process that you would deploy your regular application um and you know like i said really have it be part of that same cicd process that we run through for our application can also manage our load tests and our performance testing and our api testing that happens via via k6 yeah that is such a big thing for us as well because we're all about it being developer and tester friendly very much so much more frankly than any other uh platform so um and then the other thing that i don't know if this was the case for you but i also see people going towards operator instead of cloud when the system under test isn't publicly accessible was that a concern for you as well yeah a lot of our a lot of our stuff yeah a lot of our stuff is not publicly accessible a lot of our internal apis that we want to be able to load test and do things aren't and so it become you know again exposing that out would be would be a big deal okay out of curiosity what do you use for your ci cd pipeline uh yeah so if you if you can't talk about it no absolutely yeah so um we use uh jenkins to build stuff for the ci part and then we use um argo cd um as the um cd part to actually deploy out manifests and manage that um for us and there's actually even a little more caveat there we actually hook up k6 and a tool called flagger that uh that's also an open source tool that does um canary deployments for you automatic deployments and so we use k6 to generate production-like traffic in staging and dev to tell developers whether their canary will succeed in production as well so that they can catch bugs much earlier in that process so yeah we we used kind of a combination there that is super interesting okay is that an integration you built yourself then sorry i'm sorry nicole i consider like no no no no it's um we simply leveraged the the happy coincidence right that like if you um if you if you write good k6 tests right they actually test your application and actually test your api and then you run them on a schedule instead of running them just ad hoc whenever you deploy something out if you were to run that test every minute for example or every three minutes all of a sudden you're generating production traffic like you're generating like a baseline level of load that runs through your system um basically 24 7. and so whenever a developer then goes and does a deployment and that you again we use flagger but you can use any number of other tools to do this i know argo has argo rollouts for example and some other tools but no matter what it is right whatever you're doing to then compare your metrics and look um because it hooks up to prometheus which is very convenient for us as well you hook up and you look at those and you say okay cool is my success rate staying stable is my latency changed all of that and you can see that in staging long before you get to production or in dev long before because you have k6 pushing out all of these tests and the more tests people write right the more teams write these tests the more endpoints they hit the better and better your your non-product environments actually represent your production level traffic um which becomes really valuable later on cool interesting okay well there's another thing that i i can think of too that i would use casex operator for and it's not available in k6 cloud right now but if you in case x operator you can specify the image of k6 that you want to use so that means that you could also use an extent or extensions with k6 a custom build of k6 that that you use to run your tests which is something that i personally really would love to see in cloud but hasn't quite gotten there yet do you do you just run vanilla kasich's or do you use other extensions yeah we we have hooked up the prometheus extinction we had it hooked up and then it became part of core so then we didn't need to actually build our k6 anymore but um and we do know like we've seen there are a few others out there um i know there's the chaos testing one that gives this a few other extensions um we haven't quite gotten there because nobody no no developers asked for them yet because at some point i'm sure a developer will ask for them and then it will become um important for us though the ability to actually run our own image also involved can we control um like side cars and can we control um other other things that we might want to put in there to handle a service mesh or to handle authentication or other things that you can't do in in k6 cloud right we want to build a docker container yeah it has k6 in it but it also maybe has some of these other tools that we need to use to be able to roll um to actually test our stuff effectively yeah okay um i will also put a link maybe we should back up a bit and talk about what k6 operator is yes do you want to go forward hans sure um yeah the k6 operator is a so there's a pattern in kubernetes that is called the operator pattern um it can be described as say as basically trying to take uh operational tasks that normally a human would do and translate them into code and so that code watches for events and reacts on objects things like that and um simone previously built obviously the kasich the original k6 operator and wrote a lot of that initial code that allows you in kubernetes you run an operator and it listens for a k6 test object and so you create a regular yaml spec file a kubernetes object and you can upload it and the operator will handle um running multiple instances of your code in parallel for example to actually run your test it will handle what namespace you want to run them in um or what region or what you know type of node you want to run them on etc so you don't have to maintain that and so the k6 operator basically just does all the maintenance tasks of of running k6 in your kubernetes cluster for you why was it so important for for you and your team to find a tool that could do this on kubernetes as well because you could have very easily just deployed it outside of the cluster um well because we uh all of our the rest of our stuff is in kubernetes um so for us we're basically 100 kubernetes shop we use it for virtually everything else and so um deploying you know even if we deploy really big pods pods that might have 20 gigs of ram to do a load test for us um doing that and taking advantage of the kubernetes orchestrator um to actually handle those tasks for us was was really important because it frees us up if we have to manage a bunch of virtual machines in either gcp or or aws that actually run k6 that's a lot of management overhead that we don't need to do like um amusingly a developer for example tried to stand up a thousand pods to run a k6 test because he wanted a million requests per second and so he just submitted the test and the k6 operator handled it just perfectly fine like i didn't have to do anything i didn't have to touch anything um he got his thousand pods and they ran in his cluster and he broke his stuff um he found out that he could not request a second but um okay that's really important that was really important to us because we're a very small sre team we're three or four people we don't have time to manage a bunch of test servers um and we don't want to hire a bunch of people to do that and so the ability to say oh yeah give me a thousand of them and like that you got a thousand of them and it worked like that that was really valuable for us yeah and the whole the whole load tests or tests as code movement is all about streamlining it and starting to think of testing the same way that we think about code so if your load test can be written using the same the same programming language that your application code is then why can't it be stored in the same place and deployed in the same place and managed in the same place it just makes sense that if you're already using something why don't you just have everything work with that like testing doesn't have to be this separate activity right so i'm definitely a big proponent of that as well i'm going to try and demo don't worry let's say try because this is yeah i'm i'm traveling right now but we'll see how how we go uh there's nothing like a like a live demo on location is even i don't know so much respect for you you know right i mean i'm at the next level if it doesn't work well then i i'm sorry why you have a great excuse as well so i want to kind of show how how to use it uh first you've got to clone the the case x operator repository which i did post in in the chat as well and then you do a make deploy so you do have to have make as well but um i this is so i've got a i've got a cluster and i've got a script um my cluster is on digitalocean not on my local computer because i think it's going to take off if if i have it on here um this is a pretty simple script there just there is just one scenario and i think if you've watched for anyone who's watched office hours before i use this all the time this is my my pokemon one it just sends a single request to my cluster and in this case it's it's trying to look for pikachu it's trying to fetch details about pikachu the pokemon and floor before you ask i don't have my pikachu ears right now i'm sorry it didn't fit in the 10 kilos of my carry-on luggage okay um and then it does i mean priorities priorities yeah right um so those two checks one of them is to to see whether the response is in http 200 and the second one is to see whether the body of response contains the word pikachu and then there's just a thing time just to stagger a variable think time just to stagger the requests a bit okay so the first thing that we i have to do um is to create a config map and i'm going to use um the ter the i'm going to use it i'm going to call it poketest and i'm going to use it using the file this is the script that that i was just showing so you're missing an equal sign you need to do from file equals test.js i also was in the wrong wrong deposit wrong folder um but i think i was able to do this before as well yeah let's see what what happens yeah nevermind config map created okay and then i created this yaml file which you basic i just copied this from the the github readme i removed some bits that that i didn't need called it poketest uh let's say you know i mean it you have to try a little bit more than just one right because that's kind of the whole point of distributed testing um but then i'm also specifying the the poketest config map and the script that i want to run so then go back here and i should just be able to apply this why is it not finding it there oops okay oh i probably should have also showed the pods that are up okay so so here's all the pods that are running right now and we see the k6 poketest stuff that's running already and uh that's actually pretty much it already i'm gonna get k6 yeah so this is the one that i just started 24 seconds ago and they're already running and i think the script is supposed to just run for three minutes yeah and it should be 10 each and a gotcha for this is that in the parallelism this is you have to do a little bit of maths because um this says five instances of the script so if your script is specifying 10 virtual users like i am here then it's actually running 50 virtual users in total so that's just a something to to watch out for because um there's also they don't talk when you when you do parallelism when you put more pods in there then they don't talk to each other so they don't know exactly what the other one's doing and this has ramifications in terms of thresholds and in terms of aggregating the the metrics which i think you were talking about earlier as well hans uh yeah no it um that is the the i will say one of the significant features are kind of lacking from from the k6 operator is the ability to aggregate um all of the results and all of the metrics from a single test run right now each pod is considered to be its own test run which means that test run you just dig isn't actually one it's five um which can really kind of make um viewing those that it can make that a little complicated and and make that a little bit of a pain um i know there's some there there's some open tickets on how to potentially solve that um and there's some open conversation about how to best go about doing that um but but yeah i think that's a significant kind of limiting factor how do you suggest folks uh sort of solve for that now ah we solved the work around yeah we solve for it by um shipping the metrics to prometheus and then aggregating on job name for example so in prometheus when you assign when you use the labels to actually go do your query you can aggregate on different things if you don't aggregate on pod name but you instead aggregate on job name for example you'll actually end up with um all of your metrics aggregated for for an entire job and it's it's very convenient like that yeah so yeah so how do you how did you react to the acquisition actually it's i i think it's great it just means more resources for k6 and we already use uh we are very heavy users of prometheus and grafana um writ large and it for our stuff we use it for all of our alerting we use it for all of our monitoring we use it for all of that stuff um and and so uh you know i'm happy to have i'm happy to have them show support for k-6 come pick you guys up and it means that i just get to keep helping out and i know it will keep being supported so what were the things that you added to to k6 operator oh gosh hold on let me uh [Laughter] commit history exactly just just the favorites um fan favorites yeah so fan favorites for us was probably um things like custom [Music] custom annotations um and labels um that was really important the ability to have your your different pods that kasich's create have the correct labels and annotations because that's what we use to enforce policies um in our kubernetes cluster most people use labels and annotations to do that so the ability to pass those down and let those be created properly um and and you know make mutating mutations to to those specs um let's see i wrote a lot of tests for it that was actually really fun that was one of my first forays into ragging tests for um for goling and for operators in general um so we wrote a lot of tests to solve that um and then oh the other one the other fan favorite is actually that um you can do it per namespace so what happened uh when i first started when we were first looking at it you could only create the job name was um had to be unique on a cluster level that meant that no two developers could ever create a k6 object called test for example um the operator would get confused by that um and it was a really simple fix to make it look in the name space rather than globally when it was doing things and making comparisons but that really pushed it into letting every developer be really self-service and really be able to kind of scale that out and say um because our teams basically own their own namespace at mission lane so every team gets a name space in which they get to do work they run their own pods and services in there uh and so having to do cross team coordination about like what the test was called that that was going to kind of be untenable but now they can it's obviously in the name space all they have to do is you know they they have their tests that they're running and and it basically you know just handles that properly for them within the namespace um and that was a big deal i mean also naming is hard i mean naming is very hard yes uh what is it there there are three three hard problems caching naming caching naming errors uh yeah um yeah so yeah that makes sense um what are some of the other limitations obviously it's kubernetes only which actually for you what sounds like it was uh not the yeah um there are like there are definitely some outstanding um um things and limitations um they're especially around service meshes right now it supports istio really well because istio is what we use or envoying is what we use and so i wrote the support in there but there are definitely other service meshes linker d um raw pure envoy um hashicorp you know console things like that that that we can definitely add support for and build off of because those with something like k6 that does a lot of network calls obviously being able to integrate into the service mesh is a really important part of that when you're on k6 um there's also again the the aggregation part if you're shipping to k6 cloud or you're shipping really anywhere besides prometheus um that allows you to do custom queries but like if you're shipping to datadog for example if you're shipping to k6 cloud the aggregation doesn't work um super cleanly um and yeah there there's we could definitely make it easier to install the k6 operator like you had to download make you had to clone the repo you had to do like i think there's definitely some how do we make the initial usage of k6 operator um a lot easier a lot easier to do a lot more approachable right yeah to be fair though i was able to to figure it out just today actually so i mean it can't have been too difficult since i'm not too i'm not an sre so it is a lot easier if you can just do like how you know grafana for example if you want instagram and prometheus you simply do helm install prometheus or elements or um and even for the operators right home install grafana operator um you can get those and it would be you know there's definitely some room to see can we do the same thing here for k6 that's that's a good point that would be way easier are there any issues where you're like i would love to see this fixed but i can't do it uh for whatever reason and you would love someone else with maybe a different skill set to pick up um so i i don't actually write tests so i want to be really clear about this i have no idea how to actually write more than the very rudimentary k6 test because i don't i don't write like k-6 tests very often and so there's probably a there's probably a lot of like i don't have a specific feature in mind but like there are probably a lot of usability things from like a how do i write the test in the config map better how do i like integrate with options that we're passing in or arguments that we're passing things like that that that probably need to be solved but again i i actually that part was less important to me because i want you to write the test i care a lot about how you run tests and then i told the developers here you go and they were like oh we don't have to run our own servers okay we'll learn to work with kind of kind of whatever whatever those other things are like environment variables i think is one we don't pass any environment variable you know some of some of those things that that don't happen um but yeah yeah you use and usability actually the other one is kind of like a ui like if they're right now there's no ui for any of this right you have to use k6 cloud or grafana or something else um if there was a k6 operator ui but i'm not a ui person so i don't even try to touch that like i can don't look at me it would be a really cool feature to be able to look at you know look at the k6 operator see what jobs people have submitted see what their current status is like you don't even you don't have to do any of the controlling part right still leave that as a kubernetes object but just like be able to see what was actually happening what's going on um maybe read the logs or something like that like that that would be really convenient that would come from someone with a very different background than me because i don't i don't write i don't write e-wise one of the other benefits of having k-6 in kubernetes is that you can also define resource limits because a lot of times people don't think about monitoring the resource limit the resource utilization of the load generators and they just look at the application servers right but it will it could potentially invalidate the results of your test if you're over utilizing your your load generators as well or if you happen to request loads uh or write tests that require a lot of no load and then break other people's applications um in your same for example or yeah yeah yeah we we have we have run into that where developers will write it oh okay keep scaling forever and they won't recognize that they're using all of the memory on the node for example or i'm a cpu and uh kick people off or or do other things again uh the same person who who wanted a thousand pods um a thousand instances of his test uh did that yeah so is this for the person pods is that like a common thing how much load are you testing with no no no no that is that is very extreme um okay the the story that was given to this person was to um break our api like a specific api we want to break this api and so they threw out we don't know how much it can handle but maybe a million requests a second so he wrote the load test that you know it's pretty similar to yours like just made a 200 request like you just make a request to the api and it did every pod um or every node had told it to do a thousand virtual users um and then he went out and requested a thousand pods of that thousand uh virtually used so that he would get a million requests per second running through this um and yeah again it worked um he broke his api uh people got very upset at him he actually it was a little bit of chaos testing because they actually saturated our network traffic inside the cluster as well on the node um uh yeah so so yeah being able to define resource limits becomes very important when people decide to get in their head to just go break their api rather than simply load test their api yeah yes one of the the things too is that with k6 i think that there is less of a reason to even need distributed load testing in the first place more so than other tools because k6 is because of the capacity so i think powell our cto he did a test that said that you could run up to 40 000 virtual users on a single instance of k6 now of course that's like a very simple test right but still the fact that it got up that high is pretty amazing yeah and and i think that like that that is definitely a place where like the performance that comes out of k6 unlike uh like i don't have to give it very much right i can give it like 200 banks of ram and a very small amount of cpu and it can handle running those thousand virtual users that he was that they were making right that's like our default it's like 200 um to to give to a k6 test and that's what he used so he requested all of this and and it's incredible like the performance you get out of a single single k6 instance and so quite frequently our developers won't run more than two or three or four um of these kind of pods or in parallel right run three or four more than a few of those um because the performance throughput is is really good um and again the you know the the for us a big use case is actually less about distributing out the load test across our entire cluster and more or simply about like the operator pattern of managing your resources for you like that again if you work in kubernetes all the time it's like it's hard to um overstate how how important like that that principle really is to like yeah if my developer wants to run a thousand they can if they also want to run 10 they can but we don't have to do anything to support the difference in those in those use cases right yeah that's actually interesting i'm going to change the title for this afterwards because i must admit that when i looked into k6 operator i always thought of it as a way to run distributed load tests without using k6 cloud which is as well but i was going to say for you it sounds like that's more of a secondary benefit and the primary benefit is being able to manage it the same way you do all your other resources yeah like we we could we could mandate that every test ran in multiple regions or multiple zones we could mandate like all of this and the k6 operator would be really good it's spreading that out right spread it out across a bunch of different nodes don't saturate one node um but where the operator shines is when you know if you're talking about just managing one k6 instance that that's really low level right like that's really low level of effort to actually maintain that 1k6 instance to your point you can get really good performance out of it where where the k6 operator then really shines is when you take it from oh this one instance to i want every developer to on demand ragged test and deploy that test out and run that test i want every developer to be able to or every every person who interacts can write this and manage it without having to talk to whoever owns that k6 server um like that's where the operator really shines is you get distributed load testing sure it's useful um but you also get the management of an entire testing platform basically um for free like just works and which is really useful because it means i don't have to do it um yeah yeah so where is your i don't know sorry if you mentioned it earlier but where is your application actually located do you have a data center or something or uh we run the cloud yeah we run on gcp in the kubernetes offering i was thinking that because there are some financial institutions that are very or even like governmental defense departments or whatever sometimes they don't want to run on cloud at all and so you could use k6 operator entirely on-prem with nothing no cloud infrastructure so it's a little bit more flexible than k-6 cloud which yeah has to at some point traffic has to go on to the internet to get you across yes yep yeah another limitation of kasich's operator though is as you said you're talking about wanting some sort of ui this is definitely more of a technical setup you know than than being able to use k6 cloud or some other turnkey solution it does involve a little bit too to get it going and it it does like absolutely um and it involves a lot of like as an sre team we had to think about developer experience when when doing this we don't want developers to have to um so for example you wrote two files to write your test but you could comment if you want to write a um a a test that runs on a schedule for example you might have to write three or four to actually define the cron job or define the k6 test to find the test file itself that you're going to use to find some permissions to allow it to to run properly um and so it gives a much more technical setup to try to go through that things and so as an sre team we had to work on can we build a helm chart for example or some type of customized tool that allows you to as a developer to just say oh here's my test file i want to run it this often here's the name space i want to run again go do go run k6 um and but that required investment from our side like engineering time to actually go do that compared to something like k6 cloud where you simply get to upload your test and run it for example or like that so it's a much more um kind of especially when you're first iterating and you're trying to find what works in your test and what doesn't work right it's a much more much easier experience to just go to the cloud and click the button and run your desk type of thing yeah even with uh test data you would there i think that you probably that's probably a very common case where you need some sort of test data so you're not just making the same requests every time but that also adds a little bit more complication thresholds not being evaluated at runtime was a big thing but i think that you solved that by well actually you just don't use thresholds at all and you use i assume you use grafana for monitoring and alerting as well i mean thresholds in the k-6 in k6 yes so we we we get around that so one of the one of the big limitations is like if you write a threshold test and your test actually fails um the k6 operator will not evaluate that it will simply retry to run your test over and over and over until it actually succeeds um which can be a good or bad thing it depends on if you want to ddos yourself but um if you so what we do is we actually just tell developers not to write the thresholds and instead what we do is we will monitor test failures overall as to um like when you write the prometheus metrics out for example you still get how many requests have succeeded you still get metrics around all of that and so we put that into prometheus and then alert off of that um and then in the same vein obviously because we use it for we'll call it continuous testing not just distributed load testing but like actually continuous testing all the time 24 7. a lot of our alerts um obviously come out of our regular alerting channels right like oh my application is throwing lots of 500s because of my k6 test because i didn't update it to match the new spec um i know something is going to break in production i know that this is a breaking api change for example or things like that but a lot of that for us comes out of our regular alerting channels that we have tied into prometheus and grafanga to begin with yeah and one of the other limitations that i wanted to talk about is that the d you can't select um well i guess you can but it's difficult to select the geographical zone that you want that has to run on do you care about that at all for your situation um not particularly um we don't particularly need that function like we don't particularly need to run in like australia versus here in the us versus other places um e it is it is more complicated again it goes back to the to your earlier point of being a more technical setup right you you have to go label all of your nodes wherever they are with a particular annotation or label that allow that matches up to a geographic selector and then you have to pass that in to say oh this is what i want to this is where i want to run um we did write a feature in in k6 that allows you to specify um what's called affinity and anti-affinity which is what lets you select this so that that does exist but it's definitely a lot more technical of a setup you have to make sure all your labels match or all your annotations match um to actually roll this out kind of everywhere and be able to do that rather than again in k6 cloud you get to just choose what region you want to run your stuff in and then it works um very convenient like that yeah okay so you can just use affinity as well um all right it's just it's just still not as as easy um and then you were saying i actually thought that it doesn't work well it doesn't work at all with k6 cloud but you're saying you can you can actually output results yes you tried that at all uh i have not because we don't use k6 cloud but i do know uh people have opened issues and we've helped them solve the issue of you can actually output to k6 cloud if you so in the event that you want to run your test locally but you want to put all your actual test results for example in case it's cloud you could do that and use k6 cloud as your ui um you absolutely can do that because you can pass arguments to your to your test that certainly support it to be able to handle um where you're outputting um so i i haven't personally tested it but i have helped people solve their problems they open it so i do know it works yeah okay that's good but i the the big caveat there is obviously that you can't aggregate your test runs um in k6 there's nothing that tells the k6 api or the output that says all of this is is one test run in quite the same right way that you do with prometheus and you can query on shop name for example so i'm just curious because you've done all of this in company time in your free time okay so uh so is the k6 operator the only open source project that you've worked on under work time or is this like a thing at michigan lane where you yeah no um it is we we work on a lot of things i've i've put time into flagger to make flagger work the way we we wanted it to we've put we um have put time into glue which is an ingress um operator um we have put time we we try to contribute back like one of our kind of main philosophies is we're gonna go find an open source product we have we have lots of engineering talent in-house we try to hire really good people and so if we can go find an open source product that fills eighty percent of our use case or ninety percent of our use case we can then go contribute back that last ten percent or or twenty percent um and so you know that's something mission lane has been super supportive of is is when i if we go find a project and we're like oh it matches everything except this little part or this thing right here um they're happy to you know have have me just i make it ticket on our internal side and say this is what i'm working on to go solve this problem for us and then i'll all we we will run it internally so we might fork it and run it internally and then we'll just contribute the code back so that we can keep using it until it gets merged in for example that is super interesting do you also sort of include that in your oprs or i don't know what kind of sort of performance it's just it's just something you do yeah expectation we'll call it the like the expectation is that like if you find something like that you're you're able to contribute yeah you finally do fix it uh and so do you use it at all when you're when you're interviewing new talent or you're trying to attract new talent that you know like that's a sort of selling point or um yes no no we definitely do um and it's definitely something like i know there are two of the people on our team for example have have done similar things they've they've contributed to various open source projects on while they've been on on the team or written about um how to use it to contribute back documentation whatever whatever that is um and that's absolutely something like the ability and the freedom to to be able to be trusted to go do that and having the um like the extra work space in in the schedule and have you know have people understand that instead of going and building our own testing framework it's more efficient for me to just come contribute back to k6 for example and and to do to do some of that um and that you know kind of broadly extends to other places i know that um there are multiple people who broke who have left mission lane to then go make like to then go work on on projects that that they started working on while they were at mission lane then like they'll go work for the open source project for example or um go do kind of a thing that like isn't at all related to missing lane but they found really interesting that they found this as a thing and because they were working on an open source project or a problem they were trying to solve um but i know at least a couple of people who've done that and so yeah i love this this is music to my my ears but nicole knows this like i actually think that that is a good cue to also say we're hiring if you like contributing to open source projects and maybe you're already using k6 why don't you check out the jobs on our page or grafana grafana's got a bunch of stuff as well um it's a cool place to work i say it every week but it doesn't make it any less true i i have nothing but good things i have nothing but good things to say about everyone i've worked with acoustics so absolutely go go go do that be really good yeah oh that's that's really good um i just to wrap up as well and i will leave floor for last because she didn't really know this is coming but what is one tip tool or technique that helps you do your job like it could be something just something that you use every day or it could be like a whole life philosophy if you want well that's uh that's a really big question um i i try to live by the mantra of of a five-minute rule so if something is going to take me less than five minutes i do it right now rather than putting it off um and that in general has made um even even tech work like pretty easy like i i know you know i know that if i'm gonna take five minutes like i'm just gonna do it right now and even if it's painful even if it's annoying even if it bothers me um just take the five minutes do it right now and then um yeah just yeah um there you go that that absolutely makes sense i don't know if you're familiar with getting things done the methodology by david allen but he advocates uh something similar he calls it the two-minute rule so a little bit less time but he has this whole complicated flow chart of basically how to process new tasks or or whatever and his his way of thinking is if you can do it in two minutes do it now and then otherwise it goes into your traditional your capture process and it gets processed later but yeah that's that's a that's a pretty good one i feel like i know what you're gonna say nicole which the the tool you're gonna mention no because i have already said obsidian so i i am trying not to mention it actually it's funny because in the last couple of times that i've done this tom was was there as well and he was like oh i already know what you're gonna say nicole so people are saying it even before i say it so you know for those who think i talk about obsidian too much other people are mentioning it no i am hola he's an awesome source of performance testing stuff as well in english and spanish but anyway my thing is completely different it is an open source tool that i use daily not necessarily for testing i think i think i would use this no matter what i was doing it's called bitwarden and it is a password manager it is open source secure you can self-host it i i self-host it so that's even even more secure that they do have like a sas thing where you can just pay them to to store it for you but um i store it myself and it has like it's cross-platform ios android mac windows linux it is awesome it's one of the best things that that i've done to get a password manager in general and then specifically to get bit warden because you can have like organizations and organizations just like a group so i have one for myself and my husband we can share passwords back and forth and especially when you have this delineation between work and personal stuff as well it makes it really easy to to manage all of the logins so that you have a unique one for for each site i use i use beforehand oh yeah i use bit working on all my stuff it's my password manager and amazing i love it it is so so good so much better than anything else i've ever used and i've pretty much used them all like one password lastpass dashlane everything i just love bit warden so that's my tip nothing to do but with note taking so there okay well you know fine no no you're you're right i always do talk about it you talk about it a lot but uh i don't know it's good um i don't know um for like a techie thing that i use a lot is orbit which is mission control for your community so i can i can find some community insights from there i really like working with orbit i think it gives me a great insight uh uh and i i sort of really enjoyed setting it up uh and with daniel who would normally be your sidekick for today nicole uh i'm working on an integration for github for orbit so that's that's fun as well and so that's something we can give back to the community or if we're talking about open source anyway uh a technique that i use a lot or every day and my kasich's colleagues know this is during meetings i knit a lot because otherwise otherwise so it it helps me stay focused right because otherwise i might try and multitask and it turns out i'm not that great as multitasking uh and this you know helps me focus uh and uh provides my family with all kinds of warm and cozy knitted things so that's uh that's an added benefit we have two knitters in k-6 now um mihail who has also been on office hours is also a huge knitter he's always knitting during meetings so that's really cool um one thing one last thing that we want to mention before we go is thank you for putting that in floor we've got observabilitycon next week it is free to sign up i can't really say what we're going to talk about except that we're announcing something pretty big for k-6 not as big as an acquisition okay so that was like the big news of the year right but this is a new this is something new a new function i feel like it's a game changer you can totally say it's a game changer yeah and we probably already kind of hinted at it in different office hours and mentioned it in the past but we are officially announcing it in observabilitycon i am going to be talking at the keynote um there's a little bit for for kasich's and also i have a we have another talk where our ceo robin gustafsson and as our software engineer alhayev duchenko and i will be talking about a couple different things new features to k6 and also things that are going to come for k6 in the future we are doing load testing but we have always wanted to expand a little bit with grafana we've added observability and we're always interested in basically meeting the needs of of testers and developers everywhere so yeah i think another thing that is people should look out for is there's a community session about k-6 and grafana as well by matthew churchill who's a qa architect at vonage um so definitely you want to check that one out as well yeah good good thing to call out there's actually a few i think there are two two companies that that are not us that are still talking about k-6 which is pretty cool their use case for for kasich's in particular it's always nice when you have your users you know volunteering to talk about you that is awesome and super humbling thank you clearly thanks for being here yeah i was going to say clearly i should have just submitted to speak observability kong i clearly missed that [Music] i probably should have thought to let you know now i feel like it's a it's a it's an annual thing right so it's not like you know you missed your shot forever yeah no no i'll do it next year it'll be fine yeah and also these people yeah i mean there's a wish list now for the operator so you know helping and contribute exactly come say hello oh so hans this is your opportunity to to briefly rattle off some things you'd want help with there is a whole slew of open there are currently 14 open issues uh feel free to just show up and uh pick one um there's some unit testing ones there's some custom data um to do data management there's um obviously the ui there's aggregation there you know come come hang out and uh there there are lots of there are lots of issues to solve um um but yeah awesome awesome is there anything else that you'd like to plug before we wrap up no no i'm i'm yeah i'm good to go [Music] if anybody has any questions you can hit up hands on the his twitter handle below i'm sure he he's super easy to talk to he's also on the k-6 community uh slack so that is a much better way of talking to me than on twitter okay because i it's like a once a year thing that i sign into twitter i'm on k6 uh slack like that's on my regular laptop i'm i'm watching that all the time so feel free to just reach me there twitter is twitter is like special occasions yeah exactly yeah yeah oh so that's really like that's when you know you've you've had too much of the kool-aid when you're on the community slack more than you are on twitter that's wow exactly okay uh flora you have anything else to say i i'm all good thank you for inviting me on the office hours nicole of course thank you both for for coming and and chatting with me for an hour on on a friday um despite you know me telling flora uh there's a chance that i might get disconnected thank you for showing up on short notice anyway floor to to save my butt [Laughter] uh have a good weekend everybody and we will see you next week in next week's office hours where we will talk about that announcement i mentioned have a good one bye

Channel: k6

Views: 255

Keywords:

Id: IJ0uQgn7gI8

Length: 59min 8sec (3548 seconds)

Published: Fri Nov 05 2021