The New Best Way To Search for Values in .NET 8

hello everybody I'm Nick and in this video I'm going to introduce you to a brand new type adding net 8 that makes searching for matches into a data set extremely efficient both in speed and memory in fact it is the fastest thing you can use right now for this specific use case and Microsoft is using it extensively in both net and hpet core to make them way way faster now even though the use case for this is fairly narrow there are many many things actually you can adapt it into to greatly improve your performance this is something that completely changed the game when it comes to this specific use case and I think everybody should know that this is the thing that doet now supports in this video I'm going to explain what it is and how it works and we're also going to compare it with some other versions of searching for values into a data set if you L of content and you want to see more make sure you subscribe for more training check out my course is on doet tr.com all right so let me see what I have here I have a simple cons application running in net 8 and what I'm going to do is explain the example I'm going to be tackling here and the use case I want to focus on in this particular example is trying to check whether a stream string we have is a valid B 64 string meaning it only consists of B 64 characters it doesn't have to be encodable or decodable all we care about is do you support the 64 characters of a base 64 string now just as a recap these are the characters supported on Bas 64 from A to Z all capitals and then from A to Z lowercase and then all the numbers from 0 to 9 and then plus slash and also the padding character that equals character and that is it so what I want to do here is say that I have some example text and let's say that the text is this is this a valid base 64 string in this case it is because it only has characters that are included in this array and they're all base 64 characters if we had for example the Hat character or dollar sign those are not supported so they should not be a basic4 string now a very quick and dirty approach of implementing this would look something like this where we would have a e is BAS 64 a method and then we take this input over here and I can say console. R line and print out the is base 64 and if I do that and I run this application then I'm going to get false in this use case because we have these characters that are not supported and all I'm really doing is I'm just iterating over the string because strings can be iterated because there are collection of characters and in fact I could use if I want do a 4 each loop as well or even turn it into link but I'm not going to go into link because performance characteristics change with link and if I didn't have these bad characters and I had something like this for example then this would be a perfectly valid B 64 string it's not decodable but that doesn't matter it still consists of Bas 64 characters now if you wanted to go a step further you could actually improve this quite a bit in fact I could use spans to do the following say that East Bas 64 span and then return the example text as span and span as a read only span of characters allows me to do the following I can say index of any except or I can say contains any except but index of any except the parameters in this array over here and then if that is minus one then it is BAS 64 meaning there was no match so if I use that over here as you going to see very quickly it does return true but if I go over here and I add some parameters also characters that are not p64 characters I'm going to get FAL so this will be both more efficient and faster and you can write this in so many ways for example you can say contains any accept meaning you don't even have to have the minus one check but those are just a few of the approaches you can follow now all that is fine and it's functional and it's very fast but something that completely changed the performance game was added in Dark Night 8 now before that real quick I'd like to let you know that are Black Friday discount on do train.com is now live you have until the 27th of November November to use discount code Black Friday 23 to get 40% off any of the courses and 20% off any of the already discounted bundles so this is your once a year opportunity to invest in your learning and learn anything you need to thrive as a do developer from unit testing integration testing we have clean architecture DDD check out our courses link down below use Code Black Friday 2023 and two things our EF core is not included because it just launched so that just gets 20% and the code will only last for 500 purchases per cost so you might want to hurry this discount has actually been in Early Access for some days now to my patreons and our mail list so make sure you subscribe to our mail list as well if you're going to get these early accesses to discount codes now back to the video okay so what was added well I'm going to instead of use an array over here of characters I'm going to just have a string representation of the characters so I'm just going to say base 64 characters is just this string over here nothing changes with the two approaches other than the thing that we're going to use hasn't an overload to accept a string which simplifies how to deal with this because then it means I can actually have a constant string and that comes with its own benefit so what's the thing that was added well what was added is a new type called search values and I'm going to use search values of characters now you can have set values of characters or such values of bytes which makes the use case a bit more limited but since you have characters and bytes you can use it in tons of places and this is one of them so I can say base 64 search vales over here and say search values. create to create my search values and then I'm going to pass down the string I want to create the values based on and of course I could use a character array as well if I wanted to that doesn't change there is an overload to accept a read only span of characters which means that both the character array and the string have implicit converters to convert it to what should be accepted here it is just that you can actually optimize further if you use the string and now I have these base 64 sets values of T type characters which I can use here and I can say let me just change this to E Bas 64 without any subtitle and then say return example text Dot same thing as before I'm still going to use it as a span but I can now say index of any accept and pass the search values and I can do the same thing as if this is minus one then there was no much meaning it is base 64 so if I do this the API doesn't change as you can see but we now are accepting search values of type T not a read only span then if I use this as you can see over here I get false because I have the little hat character but if I remove it and I run it I'm going to get true so your code is very much the same the only thing that changed is that you can now use this constant string to represent your characters and pass them down into the search values and you want these search values to be casted once for your application so you want to say static read only and just store them somewhere and then reuse that value everywhere this will become way more clearer when we move this into the benchmarks and before I show you the benchmarks I want to show you that Microsoft is using this quite a bit to match HTTP characters to use the HTTP rule paer to pass the whole string to deal with headers path string things that are used extensively in hot paths in aspnet core will use this as much as possible and in fact if I go into the runtime itself not just AET core you're going to see that this is in many many many many places in fact they're using it for a very similar use case to match asky letters and I'm pretty sure they're using it for Bay 64 as well but how much faster is it well let's take a look at some benchmarks okay so what I did is I brought in benchmark.us a static read only field to cast the search values because the magic of this is inside this create method of search values that's where all the paths of actually optimizing searching for values in that set are in here so you really want to call this create method just once for your search values and reuse it everywhere and then I have two strings one being a valid base 64 string and the other one being an invalid base 64 string and the invalid character is very intentionally in the middle to show you how this value would scale in fact thinking about this I should probably also have a third value which shows you scaling as well which will include just this value but many times let's say five times why not and let's see how that compares and then I'm using the set values approach as you saw over here in the example before then the span approach which will still be very very optimized but it's not using search values it's just using the array directly and then the naive version where I'm just looping around the characters and checking if it is contained or not so what I'm going to do is I'm going to go back here say return early over here and then Benchmark Runner do run benchmarks and make sure that this is in release mode and I'm going to go ahead and say run my benchmarks and let's wait and see what we get back all right so results are back and let's see what we have here so as you can see the EB 64 search values 1.8 nond and then 1.6 for the one that doesn't contain the parameter and then 2.5 for the one that doesn't contain the parameter but is 55 characters long now look how that compares to everything else of course we have no memory location everything is very optimized but we have 15 Nan for the one that does match it in the middle and then 32 for the naive approach then 26 for the one that does include it and 50 for the naive approach and then 49 and 253 so you can see that the scaling of search values is insane it is so so fast that nothing even comes close even the very optimized with only span V can't compete because of that search values. create method that method would go ahead and optimize for so many use cases and so many paths and Microsoft will actually keep adding things behind the scenes as St to has already talked about to optimize these search paths even further and if we go into that create method you will see that search values is very optimized to the point where it takes into account the length of the values themselves if it is one then we do this if it is two then we do this if it is three this and then if it is more we optimize for further for four five and so on then it's vectorizing as well if they can with hro acceleration and so on so as you can see this is an insanely small type but extremely efficient which if you have a use case for you really should be using this is something I will be using quite a bit but I really want to know your thoughts in the comments down below what you think about this and what's a cool use case that you might have something like this let us know so we can all learn well that's all I had for you for this video thank you very much for watching and as always keep coding

Channel: Nick Chapsas

Views: 67,091

Keywords: Elfocrash, elfo, coding, .netcore, dot net, core, C#, how to code, tutorial, development, software engineering, microsoft, microsoft mvp, .net core, nick chapsas, chapsas, dotnet, .net, c# new features, c#, .net new features, .net core new features, .net 8 features, c# 12, searchvalues, .net searchvalues, .net search values, search values, C# search values, C# searchvalues

Id: IzDMg916t98

Length: 11min 4sec (664 seconds)

Published: Fri Nov 24 2023