A Complete .NET Developer's Guide to Span with Stephen Toub

[Music] [Applause] hey friends I'm Scott Hanselman we're back with another deep dotnet with Stephen to how are you doing sir I'm doing very well how are you Scott I'm living the dream living the dream um so what else can we do how deep can we go I think a great place to go would be span span of tea which is amazing because there's two kinds of people in my opinion there's those that know all about span of tea and love it and there are those who are using it and it's changing their lives every day and they don't even know it exists because it's just pervasive throughout the platform exactly it absolutely is so when I one of the first hard things when you do computer science is you learn about your types and then you put them in an array and everyone's computer science teacher or software engineering teacher had a metaphor mine used buckets and they literally like lined up buckets in the row and we would that's the zero withth bucket and things like that we all have written if we grew up in a certain gener we we we wrote a mem copy we would you know manually move arrays around but maybe we should start at the at the beginning of why an array of things is is hard or interesting that's a great idea yeah I uh and I love that we'll be talking about span because um you know as many folks who are using net know it has gotten significantly crash faster across the board over the last decade uh huge improvements every release release over release you know racking up big improvements in throughput or reductions in memory consumption or startup or whatever it might be and there are a lot of reasons for that um sometimes it's you know Innovation happening in the research world coming into the platform sometimes it's fundamental improvements in code generation or in garbage collection sometimes it's just sort of quantity you know the massive number of open- source contributions uh coming into the release but there have been a few really critical turning points or revolutions almost um where some new feature comes in and just changes how we go about things dayto day and at that top of that list is span um and it it really has changed what we do how we do it and how we think about it so yeah let's start with array so to your point um let's say that I just have a a simple method that uses an array so I'm going to have a static in we'll call it sum and this is just uh taking an array and we're going to sum up the values in a simple fashion so I'll create a sum I'll return the sum and then I'm just going to for each over each value in the array and sum plus equals I right it's your very basic Loop and as we talked about I think in the link part one or link part two video this for Loop will actually be lowered by the C compiler into a really nice for Loop for I equals z i is less than array. length i++ and then Su plus equals I some Z right WR performant code here we're going to focus on performant code but no one needs to worry about line six will be lowered is the right things just happen and we can see that the right things just happen and I'm going to take a you talked how deep can we go I'm going to take a really really deep dive off the cliff here and we're going to go straight to look at the assembly code um just because you know I I don't expect everyone watching to understand this particular dialect of assembly code that's not the point the point is uh we kind of get a a sense for how this high level code gets translated down and then we'll be able to see as we make changes how that impacts sort of the quality of the assembly code now when you say assembly right now we're in idiomatic C then there's compiler generated C then there's compiler generated ilil then there's Jitter or aot assembler which is processor specific are you talking about I'm talking about the processor specific code so I'm running on Windows on a 64-bit uh on x64 architecture um and so you would see different code if this was arm 64 or if this was Linux you know the differences in other the Assembly Language or in the calling conventions or whatever but again none of that really matters um we're just going to use it as sort of a starting point so there are a variety of ways that I could get the disassembly for this function and then pros and cons to each um one very simple one that I've alluded to in the past is I could just copy and paste this code over into uh Shar laab Shar laab iio so if I just paste this here I've got my sum function and we can see that I can look at the C I could look at what the iil for this function looks like and I can also ask it to produce assembly code so this is basically taking the C compiling it down to iil and then running it through the just in time compiler in this case on x64 using the this particular version ofet and generating some assembly code so that's one way I can do it um I could also hit F5 or control F5 and attach a debugger and look at the assembly code in debugger um that's another way to do it um however there's a really handy uh oh I can also use benchmark.us diagnoses so you you we saw in the previous video slapping memory diagnoser on my Benchmark if I just add disass assmbly diagnoser that will also tell benchmark.com before and after you can look at a diff between the before and after and see what was going on um but there's a really interesting approach that's also super valuable and that is we can actually ask the jit itself to tell us what assembly code is generating um and we can do this by let me just write a little while loop here let me just say while true uh and all I want to do is I just want to invoke this function a bunch of times in order to get the jit to spit out what I wanted to spit out so I'm going to create an array doesn't matter what this is so just do a numeral range 0 to a th000 just warming it up is this the profile guided optimization where you get it to run it a couple times and it goes oh I finally know that that's the best most optimized one exactly we and we'll walk through exactly that part so um uh now I can build this we can see I uh it'll build and it spits out a path to my okay and there's no way we can see that unless you control all that was was just a link to a gotcha link to a d yeah so um I'm GNA come over here to my console are the fonts big enough I think so yes sure um and I'm going to set an environment variable that environment variable will be net jit dism and what this is doing it's telling the jit hey I'd like you to actually generate and show me the assembly code and I can then give it a pattern for what functions I want it to uh show me them for so I'm going to ask it to show me the the assembly code for anything that included some in the method name now I just run you notic a whole bunch of stuff just got spewed out wow I just killed it because I had an infinite Loop but if you were talking about um how it can generate sort of multiple levels of you know so the first time it ran here we got what it calls is tier zero this is sort of I'm not going to do any optimization I'm just going to generate code as quickly as I possibly can but I'm instrumenting it with I'm sprinkling in some tracking and the code here doesn't look particularly good like this is a whole lot of code for what is what should be a really simple thing right um actually uh right we talked about last a couple of episodes ago that you can have it good fast or cheap pick two right and this one was fast and cheap instument it so that it might get it good later exactly so then if I scroll down to the bottom we see now we have a tier one compilation there's a lot less code here and the jit is basically this is the optimized version of the code with whatever learnings it had from a previous version um and we you know we see it's a tiny little thing with 29 bytes and there's all this other information that jit is able to tell us because we're not just disassembling the bytes of the code in memory this is actually the jit showing us you know its view of of what it's producing so the kind of the internals of what's Happening um so this is another way that I can get at that and I use this all the time super useful however um the one of the developers on the jit team the just in time compiler team at Microsoft harnessed that same output and wrote a really cool extension for visual studio called dismo uh dis asmo uh and if I just clip up my uh so you can you can get it from extensions manage extensions in Visual Studio I just put my cursor here and I do control shift d this opens up this other window over here where it automates that process so it built and then it loaded up you control scroll in there does it respect font sizes there's this little open and vs editor which opens it in something that does so I see cool um oh wow so uh I can all of that work for you effectively it automated what we just did manually at the command line yes with the exception that it didn't actually run the code it built it and then it ran it through the jit but it didn't actually tear up so you can see here on the top right it says full Ops so this was basically disabling tiered compilation and just jumping straight to the kind of the do it right the first time and take your time and get it right um so this is really cool so now I can click on any function and I can just say what is the assem code you know control uh what is it alt alt shift d I always have to look to see what my fingers are doing CU I kind of learn it and then I forget fingers know your muscles know but your brain doesn't know exactly so this is very tight assembly code and the the most relevant portion of it is this part here this is the loop you can see it's a loop because when I click on this label you see that there's this jump greater here that's jumping to this label so this is part is just kind of doing this and that's that tight inner loop um so it does the comparison on line 2 three and then the result of that it's like okay I did the compare is it greater than or less than 24 makes the call and if it's greater than it jumps back and starts it all over again exactly and if you don't know assembly um not to you know spend a lot of time talking about AI but one of the things that things like gp4 or you know chat GPT or Bing chat or gemini or whatever your favorite thing is they are actually really good at explaining assembly code so because they've seen a lot of it so if I just highlight this code and I um and I bring up uh GitHub co-pilot so I can say ask co-pilot um explain in detail what this uh assembly code is doing um it'll walk instruction by instruction teaching you what each of these instructions is doing where it's putting stuff and nine times out of 10 it's it's not only right but it's like Clairvoyant in guessing what you know what what was happening wow and really giving you a sense for so if you're ever looking for like you paste it into chat GPT or whatever your favorite thing is but um yeah a super great way of understanding the assembly and I want to call something out we've had a couple of folks in the chat not just a little bit but occasionally fil are saying you know why is Stephen using co-pilot are we trying to is this an ad this is a 90 minute long advertisement for co-pilot it is not this is how you code this is you use light mode this is your Visual Studio this is your real computer this is just how you work and you discovered in doing this that it happened to be good at assembler well because I wasn't sure what was going on in one little region and so I asked it oh that makes sense it makes sense because there's 50 60 years of assembler out there that makes sense why why a large language model would be good at that exactly so um the the key part here is there is this really tight Loop where you know we're just incrementing the um the you know incrementing the iteration variable we're checking to see whether it's still in bounds if it is we're coming back uh we're uh adding the next value into our sum and we just keep doing that until we're done storing it all into eax which in this calling convention is what's returned out of the method at the end so that eax here is some okay so um really really tight code for our array and that's great arrays are great but you know let's say I have a th numbers and I only wanted to sum 50 of them so what do I do I I probably have to pass in some additional information here like I need an offset and I need a a length let me ask you this I I I know where you're going here I want to I want to make sure that we don't lose anybody who might be wondering I I you know who may not be speak assembler but they may go okay I can see where the the increment is in the assembler I can see where the compare is but where is the array like can we visual ual the array because I can see on line one where you just kind of like without mentioning it made an array with a th in from zero to a thousand but where is it so this array is stored on the Heap so there is an object sitting on the Heap that is a th integers long plus a little bit of upfront material around how long is the array so there's like an integer that says a thousand followed by those thousand integers basically and then I have a local on my stack I have a local variable which I've called array doesn't what I call it doesn't matter but it's just some some space on the stack that's storing the address of that array now it's storing it in a special manner this isn't just a pointer like you would have in C or rather it is a pointer but with some special stuff s uh capabilities surrounding it and that special capability is that it's tracked by the a combination of the jit and the GC because that that array sitting on the Heap um if I create a bunch of objects and some of them get freed and then I create more objects and some of them get freed I can end up with fragmentation kind of gaps between these objects and then I got to allocate something new and I don't have enough space to represent my continuous new object because it's all these little gaps between these objects so these these things can move the GC can move them to defragment the Heap and push them all together which means their addresses can change and so when that happens the GC needs to go and update my local that's sitting on the stack with a new address inside it let me let me tell me what you think about this analogy because when I used to teach this I described at the at the end of Indiana Jones when they put the Arc of the Covenant back in the giant C in the big giant Warehouse right it's full of boxes it's in a giant giant Warehouse full of boxes and they're all the same they're containers but there's somebody up front with a clipboard which has a list of the location of each of the containers and if you came in with a thousand containers there may not be a slot for a thousand but I could fit zero through 150 here I'll write it down I'll put at 150 through 200 over there so the Thousand containers get stored but there's a couple over in the corner here and there's a couple there a couple up front but I'm going to you because I'm clipboard guy right and then and then if someone comes in and does a big clearing out of all the old stuff and leaves a whole bunch of holes opportunity opportunity you can just like go to the front and just push everything together to the back right upboard got it okay that makes a lot of sense good deal so so this reference here and this will be important later as well this reference uh is basically a pointer that can be updated by the garbage it's a managed pointer it's being managed by the system now in terms of how that gets into this function like what is array inside this function we're passing some argument into this function and that means the function needs to know where to go to get this array and so there are things called calling conventions which are an agreement between a caller and a call E for where the caller is going to put stuff and where the colli is going to get stuff and by default Windows x64 this uses a calling convention where the first argument is going to be passed in the the ECX or rcx register so if you see here this is basically that register is storing the address of the array and so this this check here where I'm moving something into the r8d register this is loading whatever is stored at offset 0x8 from the pointer it's loading whatever is at that memory location into r it so happens that what lives at that memory location is the length of the array uh so it's loading the r length of the array into R it's checking to see whether the length is zero and if it is zero there's nothing to do so it's going to jump down to the end and we're done if it's not zero then it's going to enter our tight our typ effectively what became a do while loop uh here so that's that's what where the array is yeah yeah that rcx register is really interesting because it's a part of x64 but it's extending how we used to do things on x86 with e a backwards compatible extension which is really interesting and we'll get into that another day yeah so if I come back in here let's say instead of doing passing in the actual array let's say I wanted to pass in only the first 50 elements of the array that I wanted to sum well I could do this weird dance where I would say copy equals new int 50 and then I see array. cop address a chunk of it you want to like say I only care about these this middle bit here right and then pass and copy but that's that's a big expense if I have to create a brand new array copy the relevant portion over like I I don't want to do that so let me ask you this are is would go ahead and put that back control Z that back and make it larger did someone on line five is there an implicit for Loop in line five or does it address that is anyone hunting or searching or do they just they go to they go right to 50 but they still make a copy of it no one's no one's searching the array in that context no no so this is this is just doing a mem copy this is just saying I want to copy 50 elements from array over to copy and then I'm going to pass that of that sub region which may or may not be fragmented we don't know that that's hidden from us right so this is in my array of numbers I've got a thousand numbers but I only want to sum the first 50 of them so I'm going to create a brand new array I'm going to copy the first 50 numbers from the original array to my new array and then I'm going to pass my new array in but I'm pointing out that the 50 could be in 50 different places or could be three different places that's all being hidden from us as well uh well in this case they're all contiguous at the beginning of the ray I'm copying I'm copying from uh array Subzero to copy Subzero the first 50 elements of the array and putting that into the fif 50 First elements of copy I guess I was confused thinking that again you what's on the Heap and what's on the stack and trying to think about if the Heap has become fragmented oh I see well the the array itself the the Thousand elements of the array are always uous they they have no they are intrinsic intrinsic types if they were not intrinsic types could they become fragmented no they would always well that whatever is actually stored in the array itself are always contiguous okay cool but if what was stored in the array was just a reference to something else like a pointer to something else the pointers would all be contiguous but they could be referencing things anywhere in memory okay I'm with you now okay so you grab the 50 so I could do this but that's really expensive I don't want to do that so if you see throughout kind of older apis in net you'll see things like taking an offset and a length right um uh and then I could go in here and I can change what I'm doing in here I could say I'm going to end at some value that is do that just say for in I equals offset I is less than offset plus length I ++ and then I'm going to sum whatever is at array sub I so I'm iterating over just a a particular portion of this array so I can do that um that'll give me the the right answer and not here I could just pass in zero and 50 um this should be array again and but if I get the disassembly for this now and then I'm going to ask for a diff which I'll bring up over here it's one of the reasons I really like uh this tool you can if we look at our we focus on our our Loop which is I'm focusing on the loop because it represents the bulk of the processing if this was a million elements we're going to be doing this thing a million times you can see in our Loop that we now have a few additional instructions going on here we have another Branch now another comparison happening inside the loop to make sure that the array access is still inbounds because this isn't that super clean pattern of zero to array. length that the jit can easily process this offset in this length could be anything I could have passed in an offset of 2,000 even though my length was only 1,000 long so the jit is having to do an a bounds check to make sure that what I'm trying to access is still in range in fact you can tell anytime there's a bounce check happening if you go to the bottom of the method and you see this Telltale pattern of calling core info help range check fail this is a a helper function that's that the jit will call when a bounds check fails and this will actually end up throwing an index out of range exception so if this bounds check fails you can see that it will jump down here to this cold code at the end of the method uh for which it will end up throwing an exception and I highlight this because just by I just wanted to get a little bit of the array I didn't want to do the whole thing now I'm paying a cost for that and it doesn't seem I should have to do that but but I am yeah it's a significant tax yeah of per element having to do this additional Branch so that was with uh array and an offset in length but there are other things that I might want to pass in here as well so what if I wanted a uh a list of int right uh and maybe then I also want to have lint uh maybe I also want to pass in a subset of that list list offset in length you know and we were taught in net which you know for a lot of us happened later in our careers we were taught just treat lists like an array they're they're very helpful they're so useful yes there's a little bit of overhead but we were told not to really sweat the overhead should we um in general no you shouldn't uh the overhead is minimal um if it was a super super duper tight Loop like there's a few extra instructions we can see that here so if I were to just again disassemble this so we're going over list offset like like there's not a whole lot of code here and we're still doing the try to zoom in as much asize uh oops wrong window uh there we go so you know again we have a a relatively uh a relatively tight Loop here um we do notice there's a couple couple Compares in there couple Compares we had the one that we saw for the offset and length there's one more now because we're dealing with a slightly higher level of exraction the list on top of the the array but again not a huge not a huge cost the bigger cost is well if I actually wanted to have a single implementation that was able to deal with the contiguous memory of an array and the contiguous memory of a portion of an array and the contiguous memory of a list and the contiguous memory of a a portion of this list what what do I do like what is the one function that I write that allows all of those things to be expressed once because I could make four copies of this I could have one for you know list without offsets and one for list with offsets and I could have another for if this was just back to my array uh you know this is obviously this isn't compile because my compile things but just showing like all these different copies of the code to achieve um kind of writing it once being able to use it for multiple things now some folks might be watching say well what about interfaces like isn't that what interfaces are for and sure I could write a single method that took an ey list and because I and I could I could even you know just have it be an ey list but then the caller needs to create a new ey list for the the relevant subset so I could if I just took one that was an Offset you know list and offset in the length and now I look at the disassembly for that uh we we can now see that this pretty tight but all of the work is actually happening in this this interface call and so there's now overhead associated with this dispatch every time we're going back to this we're doing this interface dispatch right you got a tight Loop and then an interface dispatch in the middle of the tight Loop middle of it that does God knows what exactly and and so you know I okay so I maybe I I I I'm fine with that cost but now what about other what about other kinds of inputs what if I got um a what if was doing interrup and I got a pointer and a length from C or from rust or forever and I wanted to process I had some uh some zoom in on the left side there for me sorry yeah uh or or as um as is being suggested to me to he what if I just had a a stack alic so if I if I had a pointer here and a length and I wanted to somehow say I want to sum all of the values uh that are stored in this pointer up to that length do I now need to write a whole sum function that's using unsafe code to process those pointers it becomes this um this debate between performance and maintainability I really want the one implementation that I can give arrays and lists and stack out data and data that I got from interop I want to be able to write that once and have all the efficiency but I don't really have a good way to to do all of that and kind of have my my cake and eat it too why why isn't it an array like is array feels intrinsic like it's a bunch of stuff in a line but I feel like we know too much and there's there's there's hidden implementations like I want something that's going to hide parts of things from me but I also want a low-l type well I could you know if I had my if I had my array here UHS what that whatever I have doesn't matter what I actually have in it um I can this is just data in a line and I can in an unsafe block I can say I want to get a pointer to the first element of this array right but now I'm writing C but now I'm writing C now this sum function becomes uh int star pointer int length and now I have friends are like what and as soon as he said unsafe you've now made a foot gun uh unsafe is your foot gun and you're basically saying good luck I right unsafe is is disabling the the what the jit gives me in terms of things like bounce checking when I you know I can walk off the end of an array and the jit says whoa whoa there and throws an exception the moment I try and do that with a pointer have at it have fun go enjoy your AVS and your your you know security vulnerabilities um so we you know we try and avoid using pointers wherever we can and we reduce the surface area that we use pointers down to a minimum but if if the only way that I could write a function that handled arrays and pointers and lists and slices is by using pointers well then I'm kind of forced into that world so this is where span comes in span is that thing that lets me have my cake and eat it too uh if I just go back to the code that I had out here I'm just going to go back to my regular for each Loop and I can just say I have a a span of int and I'll say for each uh in value in Span sum plus equals value um and I can if I look at the the disassembly for this here's my I'm want to zoom in uh I I'm learning uh here's my tight inner loop notice no additional unnecessary branches and yet I can call this function with my array I can call this with a new span around my array that accesses just the first 50 elements I can call this if I have uh some pointer that I stack alect I can call this with new span int uh pointer 1000 I need to put this on safe block for my my pointer usage like I can use this one implementation for arrays for portions of arrays for FS for lists if I want to anything that has kind of a contiguous piece of memory I've now written my one function and I get the nice code gen for my one function but I can use it with all these different things so the span is this great unifier that acknowledges that it's more than an array but it's an array within the context of this this world we have this net runtime exactly you know when we were when we were talking about async and await I I think I talked about how one of the the great Innovation with task wasn't so much an innovation as it was just a recognition that having a a single thing that you can take all of your async functions that are all doing different things but they all generate a task then all you have to then the only thing you need to tool is that one task doesn't matter that I have a million different implementations producing a task I can write helpers in terms of task I can write a language feature like async and a weit in terms of task it's this unifier span is the exact same thing for contiguous memory um uh computer science the joke about like what is the three the two things about the two hard things in computer science or are naming things uh what is it off by one errors and uh I know naming things I don't remember what the third one was but span is a freaking awesome name when I when you find a name like I don't know if this happened if you were in the room but there must have been a time when there were a bunch of people with a whiteboard and a thesaurus trying to figure out what to call name was yeah no span a concept of a span exists in linear algebra right it's the span of a set of vectors but it may have been obvious but a span for me like it's spans like a like a bridge spans space it was a word that's not array but expressed being more than but also the space contiguous space across the bridge of something I just love that because now that I get it and it has a great name you didn't just name it Foo it it clicks for me it expands the space it's also happens to be nice because we use it so much that it's nice that it's this relatively short concise thing we can easily express because this now does show up everywhere it is it it is it pervades the entire set of core libraries that make up.net when we say span we don't say span Cally we say span of tea span of tea span of tea is it absolutely married to the concept of generics generics made span of tea possible Right is there a object or is always span of T it has to be a type that is specific not just so there's no non-generic span there's only span of te and read only span of te there's also this could have also been uh read only span of te is a great Point actually um and so there's basically these two ones that and we'll we'll under we'll get to kind of the core of the difference of what these are we're going to actually Implement these and we'll see what the difference is but there's a very very small difference between the two of them Main can I write into it or not yes it's span if not it's read only span yeah like right like my floppy drive has a jumper and I pull the jumper and now I can't write to the dis exactly and and you can see I'm passing in a span here but there's an implicit conversion from span to readon span that is the pulling of that jumper right so now in here if I were to try to write to span of zero it'll say what are you talking about you can't do that even co-pilot knows oh wow that's cool um uh yeah so span is this great unifying force and it's allowed us to do some wonderful things um not only has allowed us to consolidate these place where we did have multiple copies of code paaths but it's allowed us to take um code that we wouldn't have otherwise been able to express well or efficiently in managed code and do so so for example array. sort was implemented uh all throughout framework it still is and before Net 5 I think um it was implemented in C or C++ in the core runtime so you called the ray. sort you would end up going into native code and that means pointers all over the place and that's primarily because a lot of the the code where we were we were doing stuff um it it was hitting these inefficiencies where you know there'd be extra bounds checking and whatnot and we we really didn't want to afford that on this really core critical code path that's needs to be really efficient of sorting um but with span we were able to take a large amount of this pointer based code from C and basically lift it out paste it at a c fix up a few things in terms of span and 99% of that code now is fully safe using span and readon span there's one little place I think one function in the Middle where for a critical Loop where the bound the jit couldn't today detect that the bounds checks could be removed we left that one piece using pointers in C but everything else is just Spam and then array. sort is just calling into that spam based code uh but then we also were able to okay now we have array. sort in ter we have span sorting in terms of span let's expose that right so now we have public apis in terms of span for sorting and now anywhere you get a span from whether it be interop or a list of t or uh on the stack or whatever you can now use that functionality and there are these KnockOn benefits where you you you take it for being just in terms of arrays now you make it in terms of spans you maintain your efficiency you maintain your safety but now you can use that same functionality in all these different places um it also allows you to do some really really really cool things so for example I can write um readon span int uh and I'm going to use the collection expression syntax that was introduced in C2 so I can just have a set of numbers here um and what this is saying is I want to just construct a span with three numbers in it 42 43 I think I meant 44 doesn't matter 40 I can't type 40 44 um but the really neat thing here is this isn't allocating an array this isn't just putting this on the stack this is actually recognizing all right you have some contiguous data here you want to be able to reference it by something that you can't write to so this data is immutable and we could actually blit this data into the assembly itself so just have it just live in the data portion of the assembly and when I create a readon span around it I'm literally just referencing the data directly in the binary okay so hang on put make a regular just int array with those three the way that we would have done in the old day like no above it just you know in uh you know so you've got you you're you're in class it's it's whatever 101 and you can make a new in in this case here that what do we know what what does the compiler know on line four that it doesn't know on line three that can make that such a more powerful the compiler doesn't know whether you're going to do this but we hinted that we are this yeah I can do that and so you this will end up being basically stack allocated space instead but the fact that this is a readon thing that is intrinsically known to the just in time known to the C compiler known to the just in time compiler it say okay you're never going to change this I can just put it somewhere uh you know where just blit it directly into the binary and then there are things that are built on top of that so if I said um read only bite and I used uh hello Scott using the utf8 uh syntax that came in C 11 so this is a utf8 string basically the C compiler is generating the utf8 bytes from this string and is blitting those directly into the assembly and then this span is just pointing directly to that data in the binary okay this might be a dumb question uh but that's the value that I'm providing these conversations like you know in in in Unix world there's a thing called M protect it's like a system call to go and say that here's some memory pages that are read only and protected and there you can't write to them you can't it's basically schmod for memory I say schmod chod uh you know basically you're saying chod minus X does Net have an idea like in Windows you can say virtual protect to protect a region of committed pages to Virtual address space is there anything you can do to say that that's not only read only but it's like protected at the operating system level to be prevented to be written to uh you can that would be up to the runtime to do the runtime um does do things like um ensuring that uh anywhere that jit wrote code it switch switches to be U you know non-writable once it switches to being executable and things like that but there's nothing that's exposed in managed apis to to a c developer you'd have to P invoke to to do things like that okay interesting all right so we got read only span of bytes here and it's UTF bytes and they're in the assembly they're just hardcoded and that's okay because it's reasonable to be hard-coded yeah and so there there are these really cool things you can do or um let me just comment this out you know one of the take something like string um strings Inn net are immutable objects we really really don't want people trying to use unsafe code to modify the contents of a of a string uh you can do it sometimes and it might work but it's a really bad idea to go and mutate immutable things using pointers and stomping over data that really should be immutable and so you know we for a long time we wanted to have an API that would allow you to create a string and at its beginning of its lifetime populate it with something but then switch over to a point where okay now it's immutable now you're you're done writing to this and you know we contemplated exposing uh you imagine you had an API like create that took a delegate say uh let's say you took a an action of char array or something along those lines um and uh then you could call this create with array and you could write into this array to populate it with whatever data you wanted the problem is if we were to actually pass in that mutable string here and allow you to write to that mutable string you could you could store this somewhere you could have some static field static Char array and then here I could say I'm going to stash this thing away oops array equals array and that would violate what we were trying to achieve because now you have this m things sitting out here that you could mutate later and we don't want to let you do that but with spans spans for a reason we see very shortly can't live on the Heap they can only live on the stack and that means you can't have a static field containing a span you can't take a span and and store it away somewhere um which means there actually is now an API on string called string.c create where I can say let's say I want to create something that is 34 characters long I'll create one that is ID followed by some guid so two characters for ID followed by 32 characters for a guid so I'll pass in the actual guid here and then I can say uh I'm going to get a oh I had these backwards span good so now this is basically creating a string that is immutable at this point but in here I can write to span which is pointing to the data for the string so I could say something like um ID as span copy to span and then Guidry format uh into uh span. slice two and I want to write as just the characters so I could do something like this to basically get ID followed by all the the hex characters of my my guid into my string and I can be sure that no one can take this span and stash it away somewhere and later mutate my my string uh because there there's no way to do that you can't get this span out of this call back and so it's allowed us to expose these like that where do I file that away as an application developer because like string.c create is allocating a string that has its sides determined ahead of time right you've got that a priori and it's going to be one Heap allocation it's not going to do multiple allocations and also string.c creates got access to private data that we can't see and you've gone hidden that in that kind of I guess that's in that closure uh so typically with you know strings and net are immutable so any modification is going to make a new string Y how many allocations happened here and and and what got modified and what didn't get modified one allocation for the string that was 34 characters long and basically the runtime is saying okay strings are immutable but as part of constructing the string containing this data I'm going to temporarily give you access to its backing buffer to write the data directly into it knowing that you won't be able to do so later because you can't take the span that I'm giving you and cash it away it's so it's allowed us to expose additional things like that and we we're constantly discovering these new benefits that span and read only span it's interesting because it's almost like I would have probably done that in an un I would have probably thought about unsafe but this is like kind of unsafe but it's probably fine this is this is guaranteed by the runtime to be safe yeah and using unsafe with pointers do not do like it's very much not so uh yeah so you we only have about uh 15 minutes left I think what I'd like to do now that we've sort of talked through the benefits of span is actually Implement span because as it turns out even with all the benefits that we've seen it's incredibly simple what it is um this one of those things that we were like ah we should have done this years ago kind of yeah um but it took a bunch of steps to get there and so you know for example since C 1.0 we've had ref right so if I have an integer I can have some function that takes a reference to an integer right not I can then call this and say refi and what I'm being passed in as we talked about with kind of arrays and then the pointer to it I'm passing in a pointer to this location yep and if here uh I were to you know if I were to console .ti line I and in here if I were to set I to 42 would I run this code and eventually my window should pop up somewhere uh probably another monitor we can see that you know the the I out here even though I initialize is zero I was passing in the address and uh then I set it to 42 in here because I'm writing to what was at that address not to the address itself not to the location itself um and I get my my 42 out um so we've had this concept of ref for a long time but this was the only place that you were able to do it kind of as as part of these functions but then over releases of C you started to be able to use ref in more places like I could also have uh a ref return value and then I could say return refi and now this function is also able to return return references and I could have locals I could have well hang on when you return that ref eye isn't it all just the same eye like you've got ref now one two three four five six time five times six times this is all they're all the same one you're all just saying that all those refs just to not touch that eye and make sure it's the same one and avoid right they're all they're all pointing to the the same address gotcha um but the the critical Innovation for for span was being able to have one of these not just as an argument uh not just as a return not just as a local but as a field um because having it as a field basically means that I can then store it along with some other state so imagine here let me just control Z control Z back to Glory that's what we're saying right uh so you let's let's call this reference and imagine I also accepted a length right that's kind of all I need to know about a contiguous region of space I know where something starts and I know how long it is and then I kind of just want to be able to pass those two pieces of State around together because if I can pass them around together then my sum function knows where it's going to start and it knows how long it's going to go and then whatever I can get a reference to if I can get a reference to the beginning of the data in the array if I can get a reference to the beginning of the data in a list if I can get a reference that is just a pointer um because I can actually treat pointers as references in fact we can see that uh if I have some unsafe code here and I write instar pointer equals I'm going to take the address of I then I can also have an INT I ref and say ref star pointer and now I have a reference a managed reference for this that same pointer value now I can use this function with that reference and that length and do whatever I want and so if I could just put those together as a little tupal basically and pass around that tupal that's what span is in fact if we just Implement our own I can just say uh I want a a readon struct we'll call it my span of T and I want it to have a couple pieces of information I want that length and then I also wanted to be able to store a ref so I'm going to say I want a private readon ref to a T and reference now you notice I'm getting a little bit of a SLE here we don't just allow these references to be stored anywhere we uh we want these things to be forced to always be on the stack so I can't put them in the Heap and that means I can't just have this be as part of a class or a struct that might be as part of a class so we now also have the notion of a ref struct which is just a struct that is allowed to contain references but then that also means that now anyone consuming this if they try and putting this part of something something else that thing itself has to be a rift struct this is span that is literally span there's helpers and there's there's helper functions but that's all it is my uh uh buddy of mine uh dougas crockford uh came up with Jason JavaScript Optical notation and whenever you ask him like Hey did you invent this he's like no I discovered it yeah it was always there is this one of those kind of situations where it was always there it just got disc pretty much um and so you know we can start you know there's obviously operations associated with SP so we can start adding those but from a state perspective if you go and look at the source source.net or inet runtime this is what you will see seriously and that's it so U let's let's add a few things so let's add a Constructor so I want to be able to take um let's say I want to be able to take a reference to a single item so I'll just say reference I didn't want my T here then I can just store my reference and I'll set length to one and now out here I can say let's say I had my I uh 42 and then I want a span which is new my span of int refi so now I have a span that refers to the single element contiguous region of memory for I and if I had a span indexer which we will add in just a moment and I came in here and said equals to 43 then if I printed out I this would print 43 um I could add another Constructor that also took a length now interestingly this Constructor doesn't exist on the real span because it's very unsafe passing an arbitrary reference and an arbitrary length you could you know if I had if I had up here had done uh thousand I'm passing in a reference to this I but saying this is a thousand integers long I've just walked off you know walked off the valid memory right so this doesn't exist on span but it does exist we put unsafe stuff in places that is clearly marked as unsafe in some fashion this exists as an API called memory mar. creat span and if you go look at the implementation for memory marshal. creat span that is the implementation for memory marshal. creat span um so you know we can we can build those kinds of things um we could go in here and we could have another one that took an array so I could say my span and I want this to take an array um well we know the length here is the array. length and we should probably do some argument validation throw if null array um and then what am I going to do for reference here well I kind of want to get the the first element of the array and this will work unless the array is empty in which case this will blow up um but that's basically what it's doing there is a dedicated API that you can call for this called memory marshall. getet array data reference which returns basically ref array Subzero but if it's empty it knows how to handle that um and so you know we can just kind of build up our span like this so we've had some we have some Constructors let's go add the things that actually make this useful so I want to be able to have uh something that well I'll deal with the syntax in a second um it's going to return a t for the moment and we're going to say this takes an INT index and I'm going to have a getter um now I need to do a little validation so I want to validate that uh this index is within within the bounds I think we talked about this last time when we were talking about link a really easy way to both validate that index is with less than the length but not less than zero is to cast it to a u so that it would wraps around co-pilot knew that as well and that's what it suggested I do um but uh and so I could just here return um then using this uh method unsafe add this is taking the reference and it's just doing pointer arithmetic it's doing pointer plus but in a way that's trackable by the by the GC so now I have my implementation and I could write a Setter as well or I could just make this return a ref using the ref returns that we we just saw now I don't need a Setter because I'm just returning the reference to that that location that I can write into so if I go back up here now I can actually write what I was talking about I can write span Sub 0 equals I don't remember what number I selected and if I print out I I get the 54 from having written through this spam um and you know or we can build um uh we can build a slice routine slices are super valuable slices is just I have my reference and my length I'm just going to either shrink one side of it or Shrink the other side of it or Shrink both sides of it so I can write for example I want to return a new span I'm going to call it slice and let's just slice from the beginning so kind of you know move the beginning up a little bit again we we'll we' want to do some validation so we could say if uh if the offset is greater than the length throw an argument exception and then we're just going to return a new span and I'll let it write it for me where we're going to offset we're going to increase our reference by the offset and then decrease our length by the offset and now we have slice and so out here if I were to come in and if I were to say my span of char equals new let me give it some Char array hello world I say while oh let me also make a little public property here public in length while uh space .length is greater than Z and console. right line span sub Z span equals span. slice Subzero so I'm going to print out the first character and then I'm going to create a new span that's missing that character and then missing the next and missing the next and so on and if I run this oops uh oh what did I do hold on got those extra ticks there yep um I did something wrong oh I passed in the wrong value here yeah you can't slice zero I can but uh it it just returns itself um so now I if I look at my console over here we can see you know we're getting exactly what we expected we're just changing the beginning offset as we go um and that's span and now we can write readon span and literally like 30 seconds I'm just just going to copy and paste this and you're going to set a jumper yep I'm going to do a search and replace my span my readon span and now the only difference here uhuh is this needs to be readon is that serious that's it now this saying where we have a reference to a readon T I can't I can't write into this thing now there is one other critical difference that we don't have enough time to go into but there is there is an extra check that's done as part of the span Constructor that doesn't have to be done as part of the readon span Constructor and it has to do with array covariance uh or with array variance rather so there's actually one more check here in the Constructor that says if uh type of T is not a value type and array. getet type uh does not equal type of this then throw an exception um and if we want to talk more about why that is we can talk in another video about it uh but that's it that's that's what these things are and um you know and your your point though that it it needed things things need to be quietly built to make this happen you know you needed like you you've got memory Marshall array data reference down there you know you've got uh the ref Concepts you've got read only as a keyword these things needed to kind of quietly exist before it could be discovered in such a clean way once you all did that though how hard was it to convince everyone and and how did you all convince each other like you know we could just start PL removing stuff from underneath and making net faster was it was there any push back or was everyone excited to start ripping and replacing once we had it it was a no-brainer really and and we you know there there were always discussions about do we want to add X it would allow for for y and we still have those because you know the incremental value that you get from adding something incremental is it worth the the return on you is a return on investment or not but at its core it was a no-brainer and using it was a no-brainer now we had a lot of debate at the beginning about is this exactly what we we wanted because there are other ways that you could do similar things um for example you could say that arrays are spans and spans are arrays spans aren't just a view over a portion of memory maybe you know maybe they are the same thing and that would have a lot of different um benefits and a lot of different costs associated with it but once we landed on this as the place to be a lot of other stuff just happened and in fact we actually got span before we had the C in C the ability to write ref and so we had we built sort of special recognition in the runtime to be able to write this without actually having it and then once we got the capability in C we swwa it out yeah and it was very clever it feels like to make span array like but not try to pretend that it's an array was a smart thing because you're making it at its own type everyone can live their lives and choose to write their apps and only use the things in the BCL and the Base Class library that they're used to using lists and arrays and things like that and and span will just quietly do its work but then people who are highly highly per focused can choose to uplevel span and make it part of their lexicon and they will be successful as well so really everybody won when span of te came on the scene I think so and you know it requires a video like this to kind of understand the concept and how it fits in but once you do when you care about it it is invaluable fan freak tastic de.net digging into to span of tea thank you so much for the gift of your time and thank you audience certainly for the gift of your viewership if you like this share it tell the people and let us know in the comments what you like what you don't like and we'll try to keep putting together more of these we've still got more videos uh to come I know you've got a ton more in you and soon we're going to start introducing other friends from theet team as well uh to get into deeper areas that we don't uh know about thank you so much [Applause]

Channel: dotnet

Views: 34,818

Keywords: .NET

Id: 5KdICNWOfEQ

Length: 62min 47sec (3767 seconds)

Published: Mon May 13 2024