Json Parser in Functional C# EXPANDS the mind

welcome to the recording YouTube channel my name is Anton and today we're going to be building another Json parser however we're going to be using functional programming if you've never seen functional programming before or you would like to know more about it this could be a very good video for you to get an actual idea of what functional programming is and the actual thinking that goes on behind it not much to say here other than if you're enjoying the video don't forget to leave a like And subscribe if you have any questions make sure to leave them in the comment section don't forget to check out the description follow me on Twitter twitch I also have a c-sharp course that is out if you would like to know c-sharp as I do I highly recommend you take a look at it with that before we actually take a look at the code and start writing anything I want to highlight three three things that I'm going to be looking at the first thing is a declarative approach declarative approach in my mind basically means I wanna express myself and say what things are not how they happen for functional programming this is actually the most important the thing to follow I think most of the functional programming languages that you will ever learn will have a declarative style the feeling that it gives to the code is the code kind of unrolls and blooms and grows rather than you're trying to place things in correct spots for it to work another effect that it has is you're always aiming for this concise declared thing of what you want and that concise thing kind of always stays there as long as you are capable of using the syntax to express it and then you're gonna build out all that tooling around it in order to get that concise expression the second rule that I'm going to be following is immutability so no assignment operator and the only time that I'm going to be using an assignment operator is to say that there is some kind of variable here although I'm not going to be changing its value for other sides of immutability I'm going to opting in for things like immutable array immutable dictionary and I'm gonna be ignoring any potential mutations that are happening under the hood the last and the third thing that I'm going to be focusing on are functions I don't have much to say about functions other than they're very flexible and you can put functions inside functions to change or configure what function is doing because it's a parameter in parameter out and the inside of it can be whatever you want it to be there is so much flexibility to be had here and hopefully this video will show you how that flexibility can be realized so again three things declarative immutable and functions let's go here I am inside this am solution as I've used before I have a Json parser over here which was a very procedural approach I've basically just copied it across to this Json font over here to have my functional approach and then test the fan I copied over the tests to Target this new Json font so we have all of the tests written out currently if I go ahead and run my tests they should all be passing because we just have duplicate code everywhere what we're going to do is rather than thinking I'm going to be going step by step and taking character by character we're gonna try to express what we're looking for or what we're trying to see so if you say I want a personal you are going to want to say there should be a value null here if you're gonna try to see an object or an array you want a description for how an object or an array looks like and that description should be in function so we get a declarative approach and functions and the functionality is kind of like backed away in it but the functionality itself is also going to be declarative so the function that we have over here object parse let's place it at the top here and we're just going to mark it as a delegate this now gives us a definition that perhaps can look something like this where this parameter is going to take some Json and let's say for now it's going to Output nothing all right this is just the area that I'm going to be working on and marking this as read-only having a delegate like this basically brings functions to the front rather than you working with objects or anything like that so this is going to be really useful in that regard also just abbreviating that something like this exists because this is going to get quite big the reason is going to get big is because we cannot just return an object we will need to return a little bit more than an object with functional programming you cannot really really have a counter that is going through Json one by one because we're not keeping State we're only having data we're performing mutation on that data and then we have the result here we have Json input we want to try to parse something out of it and then we're going to have the rest of the Json that is still left after parsing there may be some there may be none the way that we say this is that look yeah we can have some kind of payload and I'm gonna have a tuple for this but the rest of it is going to be some Json so rest you can't have or actually maybe you can have rest just not a capital letter right so this is the value that we have parsed and then there is going to be the rest of the data if the parser is going to fail I'm going to return a nullable so for things like let's say white space we're not going to return any value however we will advance the amount of Json that we returned to say that we parsed it now speaking of the value the object that we have over here I don't want to say that we can have a nullable object or one object I want to say that it can be empty it can be one or it can be more okay the way that I do this is I use an immutable array and then I want some kind of object and before we really get to these objects I want to have my own notation that we were really missing in this first part so let's say we have some kind of Json value no braces here we can then have a Json null which is going to be a Json value and the immutable array is going to attempt to return parsed Json values we can then do the same for false Json false Json true and actually these are not Boolean values these are just like two different tokens in Json we then have number we then have string we then have a ring not Addy and then we have object okay so for a number I am going to make it hold an integer value for string I will make it hold a value of string for array will will hold an immutable array of other Json values so this is still kind of like the internal value and then this is going to have an immutable dictionary from string to Json value and again this is value the values for these three can be inferred from the type so I'm not going to put them in there I'm just going to say that look yeah we can have Json true convert to Boolean automatically through an implicit operator or just have a default value property that just evaluates to the same value here are models here is our interface for a parser and we can build up many parsers here is our let's say an initial parser and what we really want to return is a Json value in the end the parse function is going to accept a string this can use the parse function over here and we're actually going to be evaluating the string looks a little bit crooked we will remove the read only the parts over here will be parse value we will use parse value from here this may be null and if it's not null we want to return the value and we are safe to just grab faster default all right and the Json value that we may return is no so if it fails to parse you're going to get a null in the end all right parse value can be private and now I'm just gonna do a quick change for all of the tests here in order to accommodate this structure so for things like getting the Json value here what I'm going to say is assert is type A Json null and that's going to be pretty much it here this I will change into parse true remove the parameters same assert the type Json true here I'm gonna pass true remove the expected because we don't have the value duplicate this have a parse false expect this to be Json false remove the unneeded parameters and mark them as facts and then I'm going to apply these sorts of changes to the rest of the tests okay and cut the video here so I fixed up all of the tests here we have an object with two properties first one is null and then a nested object and then we have an array inside this nested object and then an object inside of that array here we go for the first object we make sure that a is null and then B is the object that object should contain an internal array and then internal array should contain the object and then I'm checking all of these properties in between as well all right so all the tests are translated hopefully all of them should be passing so none of these tests the tests have n and let me pull this up these are going to be the tests that we want to primarily try to fix so our first case is to try to parse null let's drop down the test come back to the parser we're going to duplicate the parse value and we will say that this is personal and this is primarily where our work is going to be happening one thing that we want to take or to do with this part null is actually assign it to parse value because what's going to happen is the result of parse over here might be null right so what we want to do is say something along the lines of this so either parse null or parse true in case the result here is going to be no and for it to be null we actually need to pass that string and place it into null so we're going to try to personal we're then going to try to parse true and that's going to be the general flow or the structure for this application right or for this parser so formatting everything and let me take all of these data types put them at the bottom here so they're kind of out of the way I will actually keep it at the top so for the declarative style of programming I want to say right let's just bend the whole function and I will say that I want a null to be right there all right that's how I'm going to parse null all I need is a require function that is going to return a parser so let's start off with that so private not a good start I try to type in public again uh private static parser or rather parse and this is going to be required this is going to accept a string or a sequence I will rather just name it sequence because we are expecting a sequence of characters here okay and I don't need the D over there cool so this code compiles this code doesn't let's say I don't know if that's possible we can return null right all code compiles now nice what we actually want here clearly we don't want to know we want a string Json that is going to be coming in here and we're doing something with the Json so again we're immutable we don't want to be assigning things to anything we want to take the first character of Json so let's say something like this and say is it equal to the sequence or rather if we're even checking it like that right we can just do starts with sequence and if it does then all good so not start with a starts with if so let's return a list of so immutable empty with a band or add new Json no so this is a little bit verbose I'm gonna give myself a additional function right so immutable array of a Json value this is going to be a result where I will have some kind of Json value and it's going to evaluate to this okay so cut this off we'll say result a new Json no and now we're cutting a lot of characters there right so V over there and then for the result over here if we did indeed manage to parse this we want to go ahead and for the sequence length just what I wasn't meant to do that a DOT and a dot just basically take the rest okay I'm not sure why I wasn't being helped out there but otherwise if we didn't manage to parse it we want a return though let's go ahead and run the null test so personal let's see if it works and it does okay if we have things like space here it should start failing and I mean I can run old test all tests so they will all fail or should all fail and if we have something like this uh only this one is failed okay so coming back to here this so far is working pretty good can we do the same but for parse true so just slap on the require here do true however now we have uh maybe two things that we want to substitute maybe one actually just the value over here really so Json value V something to return so let's pass the V here we're gonna say new Json null here we're gonna say new Jason true let's rerun this and there we have two tests passing so Jason Nolan Jason true uh this is kind of sitting all the way in the corner but then oh this test I didn't translate it correctly that would have been hopefully wouldn't have been a epic debugging session so horse troop bar is false we then want a false here and this is going to be Json false let's take our parser and we will add it to the list of parsers pass the string here on a really good note here what you really want to do is kind of say look I don't want to be passing any of these strings I kind of want to say all of these uh put them together through some kind of custom defined operator and that should just be the parser right so you want to say something like this but you know maybe something that looks like this something crazy right but in c-sharp we don't really get anything like that so we parse null false and true we don't parse number yet so let's get our parse number going over here we will slap it on to the chain here and let's take a look at the test for the numbers so we have a couple uh we essentially want to read a character Perhaps it is going to be a space perhaps it's going to be a digit but we we want to ignore the spaces if it's something like this this is clearly just one or effectively could be invalid Json let's say let's not worry about the potential spaces there we just want to say we want a number if we come back to here kind of like we've made require over here we want to say ours a digit let's say something like that right so character by character we want to go through this we're gonna get some kind of Json we're gonna double check that the links is more than zero right so we have a character and the character that we're currently looking at is a digit so since last video I've actually learned there is an is digit function all right so Json this current character that we're looking at is a digit if this is and as I'm writing this I'm just kind of noticing I'm getting ahead of myself I don't really know what this should look like or I don't even need to create something like this without saying how do I want to express myself here here and here we were capable of saying there should just be this thing here however now we're saying that we want to keep taking a thing or keep parsing a thing while a condition is true and our condition is is digit so let's maybe grab this condition for now and just say it how it is so keep taking while some kind of condition and then by the end of this we really want to transform it to some kind of value that looks like this right so in our case this is going to be a new Json number and we need to kind of put the value into here so keep taking while maybe let's rename this to something like capture all this digit and then we can say remap transfer spit out or maybe something like two so once we are done capturing stuff we want to take a value and transfer it over to this Json number over here all right so you can say that I'm looking at my parser and I want it to look something like this I'm going to be capturing digits and then the result of that capture I'm going to put it into a Json number also two doesn't really read here if it was here so captured 2 and then this function that would be pretty good I think really remap here is going to be really good so remap whatever we have parsed onto this thing and that should be good so we will need a couple of parsers here let's get a remap going the parameters that it is going to need is well another parser so let's pass it a parser and then the second parameter should be a function we're always working with Json values however the results of these operations is going to be this immutable listing so let's or rather a mutable array so let's take this mutable ring this is going to be a funk of a mutable array and then this is going to be remapped to a t maybe Json value will be a little bit better so here is the actual map let's return null in the middle here the capture function so I did start writing this over here right this was is going to be the capture function so with the capture function in place we still have the is digit here and the signature for the function is going to be Funk character in and then Boolean this is a predicate so while we match this predicate let's keep parsing if I put this current character that we're looking at for the predicate if we return true we want to parse the next thing we already know how to get the next thing and we keep doing it by well doing something like this and hopefully the slice array syntax over here it may be a little bit alarming am I constantly allocating all of these strings it's not good not a good look right so perhaps instead of string we will work with read only memory of character okay place this over here replace this here and maybe we want to create an alias for this so using this is just going to be the Json that we're going to be parsing over here make sure that we have the full reference and now we can refer to this as Jason Jason slice you know just make this a little bit shorter the string over here as long as we capture it in memory all good all right for the start with here on the memory we have a span so that still works and now yeah we hopefully have removed a couple of allocations there uh the character at a position again we can grab it from the span on the Json memory here what we kind of want to do but well not not we really don't want to do this where we say okay once we're executing uh let's go ahead and create another function and for this function we're just going to pass the well the result into it on the first glance it may look like it will work again remember once we have the parser we're invoking it we're getting that Tuple thing this becomes a little bit uncomfortable to work with so instead what you want to generally do is create a nice recursive function that is going to aggregate all of the things and return by the end from here so basically uh let's do something like this we're going to capture a string we can say that we're going to capture a Json value but for now let's say we're going to be capturing a string because we're going character by character the second thing is going to be this Json and this is going to be rest as this is going to be the result and we'll say aggregate characters okay we will be aggregating them into a string Builder as beam and all we need to pass in here is this Json okay let's import the string Builder give this a little bit of a body take the whole condition place it into here so again we can still return turn null because this is going to be questionable the capture we're no longer calling capture we're just reiterating into ourselves and then as B we can nicely append the individual character and this is partly me ignoring the fact that this is this big mutable thing so aggregate character let's go ahead and call this function over here we're going to pass the string Builder into there and the Json value this will return us a couple of things and for trying to essentially unpack this let's switch open this up if this is null let's go ahead and just pass along the null if this is a value worth worrying about so we have a string and the rest of the characters let's go ahead and turn this into a result maybe a new Json number string although we perhaps want to be able to capture other characters so maybe keep this a little bit more generic so we could say Json a string instead of a Json number but wouldn't it be nice if we could just say spit out a string and then pass the rest okay so the string here doesn't really fit the bill here if we do make this generic so let's say t Place T here here and here this still works however this is now going to start shouting that we cannot basically fit the bill of the parse type okay so we want an immutable list of Json values and this is where we can say the parse function can return anything so now not only can we build parsers for Json values we can also build parsers for all various other things so let's add Json value in here to say that this is a Json parser or a Json value parser all of these things are going to change the same remap all of these again let's not think too much we're just gonna say look everything is a Json value parser kind of like it was before although now we want to say that this is a parser of a string so now this is actually going to fit the bill uh remap actually doesn't like this all right perhaps what we can do is say look remap you're going to be accepting a generic thing but you will in the end return a Json value that is what we want you to return and you will get an immutable array of T so the value here is now an array of strings we will try to grab the first thing and we'll just do it this way because if we didn't manage to parse anything we're never going to enter this all right so something like this and we already should have the safety from not trying to enter this function if there is ever going to be a null here right remember if something fails we say no and null is our way out of the parser and let's remove that space over there and hopefully well uh I can I can remember how this code could be confusing to look at so we use it something like something like this here's a function parameter into a function which is going to in turn return this function which is going to go into here so in the end we are returning a function and then there's also a local function which is being called internally all right still I would say very readable understandable code if you take this code base and you accidentally open it whenever looking at something like this before this is going to be an absolute nightmare okay otherwise if you understand functional programming uh you're in good hands with remap what do we want to do again kind of just trying to get into the groove of functional programming here if capture is going to be a parser that in the end gives us a string again the function that we're generating we're really trying to stand in between of what's Happening inside of there so this function right here is going to return the Json value but remember that we're accepting Json so we are accepting Json we are going to push this through this parser which is going to give us some kind of T but remember that parser has a nullable of this Tuple so again we can use a switch case statement where we say did we receive a null if so just pass through we don't care otherwise we are going to have this value with the rest of the Json again I want to return this Tuple where the value I'm just going to go ahead and remap it and then return the rest this almost works we need a result I believe and there we have it I could probably even turn this into an expression body all right so uh let's bring the tests up we are going to be taking a look at numbers here not bad everything is failing so we have a null it's failing to parse something where basically encountering an old and we're exiting up the chain so for uh the number well I guess what what is the debugging experience in this situation like I get you want to kind of start from the middle so assuming we ever reach capture we're going to enter this function we're going to create aggregate Char and then once we're actually executing the Json so we get some kind of Json it's a number 10 we go inside of here we created a string Builder and we pass the number 10 okay so we're inside this function and we start iterating if we have anything to parse and it matches the predicate go ahead append it to the string Builder and then return the rest I'm going to format this a little bit because the space over there and the interesting thing about the string Builder is we never really build it so we never really do with the result uh when do you actually want to return the result well right here so as soon as we stop parsing or we encounter something that we can't parse let's just go ahead and return it so we're going to build the string here and we're not going to advance our Json this may look slightly different here where we need to make a decision R saying no characters is that good well we're trying to capture value so we kind of require at least one thing so instead of string a let's say that we want a string with a length that is more than zero okay so this is this string of our rest over here if if we didn't parse any characters we have effectively failed so re-running the tests uh everything except the white spaced one works so if we remove the white space that works but that would kind of defeat the purpose right we kind of want to support the white space and this is gonna again go back all the way to this idea that we're being declarative and we want to say uh look I want to capture a digit but perhaps I want you to ignore things like white space maybe if we even have a new line over here if for the 60 now I'll say that there's a new line over there right so I think that should fail as well and obviously I put some code that doesn't compile here so rerunning the test again I think something like that should pass I think it passes in the browser you can pause the video here open F12 and do json.parse and check if this indeed does work I think it does so we're essentially trying to do something like this and maybe do something like this so a remap should really work with the final thing and really maybe what we're trying to say is uh remap this custom parser which uh well I didn't mean to move that that is going to do this specific thing I remove the comma over here so here we're missing three functions we're missing an ignore function which should basically spot one of these characters and if it does bottom it just traverses over the stream and the parser should really just aggregate these parsers together so for the ignore character let's start here uh where does a good place to place this and could this be an expression maybe I could let's turn this into an expression expression body uh let's say ignore for the syntax that I have chosen so Char array this is going to be optional characters let's put a little bit of space here what we're doing over here is we're saying if we get a Json that has a length of more than zero and optional character contains this Json character that we're looking at so span zero let's remove that over there if indeed that is true again we want to do something like this but we've already found out that we don't want to do something like this we kind of want to pass along the rest of the thing so indeed we will need to have this internal function here let's grab the Json value that's what we're going to be returning in the end uh the I think there is kind of no point trying to understand how to re-jumble that just get rid of the whole thing and move this into here string Builder we shouldn't have a string Builder we should just have a Json that is going to work against these optional characters if this is true go into aggregate chart and just pass the rest of the Json so one double tap there and otherwise if we didn't manage to bind anything just just return the final Json so maybe we want to return null here and as we've actually spoken in the beginning that is why I've put the immutable array over there we kind of want to say that we've returned nothing but it was still successful so for the result immutable empty let's say that that what I'm gonna return here with the final Json I think the type is a little bit wrong and this can just evaluate into aggregate Char pass the Json into there actually can just be aggregate chart and this is actually ignore character this almost looks like it can just be an expression but again we're capturing that thing from the outside into this function which is capable of being recursive all right so we have the ignore function we then want the final parser and the parser what what the parser should really do is it should try to execute ignore that should didn't return any values so we don't aggregate anything or we essentially merge two empty lists we then capture something we have a value and we merge it with nothing and then we have another ignore so if we go through this again we will just not append anything there should still be one final result from the parser that actually goes into there how do we Define a parser we essentially have a collection of parsers one thing I think I have as should do they have the same signature so capture is returning a parser of a string so we could hypothetically change that to a Json value we could probably do something like this if that happens then ignore and both of these are the same signature and if I scroll up here the result let's say Json value then that just makes things a little bit word here again this is mostly how you want to express yourself maybe what I want to say is capture a Json number a while the the this condition is true so that could be good one thing well we'll keep remap I don't mean to delete that much semicolon on the end here let's refactor capture we have the predicate that we want to specify and then we want the mapping function so let's grab this thing over here place it here and we'll say that actually we can just accept this string here no need to have any list or anything like that this is going to be a function that we return from here and actually no we still need our list but after we execute this so result and the closing brace here okay so this still works all we're doing now is that once we have the final string instead of kind of emitting that string along with the values we're just passing it down here and there is no longer an immutable array here we're just passing it and now we just need a parser that kind of can carry along this chain of other parsers okay so let's create this parser uh where do we want it let's do it besides the remap over here we will say this is a par sir let's remove the T over there and we will slap this on here we will say that for this notation again params is kind of nice that we can have an array of parsers and there we go that compiles hopefully the theme is now familiar here we want to create a function that kind of does something so we're gonna maybe slap something like this here we want to say aggregate bar serves so let's say aggregate parsers aggregate parsers here do we want to put things into a string Builder probably not so let's remove the result here we will remove this switch again generally when you are starting a new function that is going to build another function just clear your mind clear what's happening there and you can start from the beginning you're gonna get a piece of Json or a blob of Json right over here and what you're going to do is run parser's through it so you could do something like this grab a parser and then slap a Json into it but then you kind of want to chain them so we're going to do with the parsers what we're doing with Json when we're saying apply this parser then apply the next parser then apply the next parser so all the parsers are going to go into there we're going to have an array of remaining parsers so these are the parsers that should still be applied so while we still have remaining parsers so length more than zero we want to try to execute this remaining parser and I actually if we did manage to go through all of the parsers we should be outputting some kind of value for this value we should be aggregating it into somewhere at the array of parsers let's leave it at the end for the aggregator I'll say I'll once an immutable array of Json values this is going to be the final result from the beginning as this function is going to be executing it will have to start empty all right so we have something like this the final result should be that aggregate parsers we receive some Json we have somewhere where we aggregate things and then we have the remaining parsers as we kind of said here this is where we ran out of remaining parsers we should just output the result and wherever we have left off and thinking about this now we kind of always thread Json through the end so I'm actually going to put this on the end I don't know why it just makes more sense to me semicolon on the end here and for the remaining parts are here there should be a result here that we kind of want to get in between and process so if a parser ever fails we want to fail or everything else we should have a result I'll call it R and rest we are going to go into aggregate parser again a result we will add range which should be this R for the remaining parsers and let me spread these out onto new lines we should just grab the rest and then for the Json again we are grabbing the rest and then the comma over here uh is this better on one line is this better on multiple lines okay uh probably one of the more confusing functions over here however we still get a pretty good notation where we're saying bars where we're ignoring this we're capturing this as Json value and then we have this so let's go ahead give this a run and there we go so these are now passing all right and where parse true uh let's say something like this if we put a space here it doesn't work okay so actually another uh really cool thing with this parser combination reuse this to create something that is going to essentially do something like this because uh we want to say require null but have this on the sides right so I'll grab this parse number I'll place it over here as kind of like a helper function I'm going to call this white spaced as I'm drawing kind of a blank for surrounded with white space or ignore surrounding white space maybe a little bit better I think that's well you know I'll be honest there's a little bit too verbose for my liking so all we're doing is we are accepting some kind of parser and we're going to evaluate it to this enclosed parser which actually the notation we already have here so all we have to do is take this parser slap it in the middle here and there we have a quite a nice reusable parser where really where no the notation that we have here is kind of what we were well what we're aiming for uh I'm kind of you can see I'm running out of my English vocabulary but you get a notation that looks simple like this all right so white spaced uh so surrounded and white space we are trying to capture this I'm gonna put this on new line because I think it just reads better but now that we have this white spaced thing we can do the same thing here and let me do a little bit of ID magic if I come back to the tests the true that was failing is now passing right so the space we handle that there now as well and again if there is going to be a new line so for example for null and we're going to be able to ignore that as well all right so uh functional programming is pretty cool because you get something like this the downside of functional programming is a sharp is you get all of this okay so you can see how at this point hopefully our solution kind of splits into and this is what I was talking about with the declarative approach you get this really really readable thing on the surface and then under the hood is the stuff that you really have to work for and it's a lot harder to read but with a little bit of practice say you know you can get around because you can actually or you can rewrite the whole thing because the actual use case on the surface is so much easier to read uh so anyway uh the next thing that we were going to approach are strings let's take a look at some of these tests so the theory that we have here looks like so uh let's take a I didn't even look at that edge so like yep that's where it is and let's go ahead and write the next thing so pars Json string let's take this hopefully as we're gonna go through this more and more or kind of like once we have a lot of building blocks as the next parsers become a little bit easier to write so for the capture that we have over here we have a function that should kind of say that a character doesn't equal double quote right so that's going to be the containing thing and then with a string we also want to require let's grab it and hopefully we can just reuse it because at the end of the day more or less these are all just parsers we want to require this thing right here although we don't want to be capturing any value so perhaps we can say that look the value is optional if the value is null give me an empty list right here the T I can substitute for Json value cool that should work the parser that should go in there we can actually aggregate it with a parser right so this parser is kind of parsing this unit over here and it's busy with this work and we should require this thing on the end this should be a Json string that's where we're passing the value let's see how that works out for the tests non-exhaustive switch expression failed to match its input so oh this is a aggregate character right here for the capture whatever we were trying to capture at this point again I'm looking at this value over here the rest of the characters the string or sorry the double quote that we're looking at here I'm hoping there is really not a space over here so for two let's go to the tutus case I'll just remove that space over there just to make sure that it's not it and rerun it does that just pass so same error with or without a space which is fine let's put the space back there that means we are kind of not advancing correctly when we're parsing a string let's get rid of this white space here we have a white spaced parser which is surrounded with quotes and it is trying to capture a string and it's looking for something that is not a double quote ideally we should be advancing as the string when we see something like this or with white space yeah because as we're saying ignore spaces and new lines so how come we're not advancing let's see for the debugging experience so aggregate chart I'm going to place a breakpoint over here open up the tests and for the two test cases actually let's go with one why not I'm gonna press debug chances are what's going to happen is we will try to hit the aggregate character capture for the first number in the chain so interesting enough we are currently looking at what one so step in to aggregate character the debugger on M1 does that sometimes where it just says I don't feel like debugging today so again a step into aggregate character computer run we're going to succeed and there we go third third time is the charm you don't quite know which predicate you're going to end up in there but for the capture we are essentially just reading a double quote one which really shouldn't happen because we've actually slept a require there so perhaps I'm gonna say something is going wrong with the require put this on the site because I feel like that could be just placed inside of this so except an array one or many and just place in one in there so I'm just gonna put it like this so white space require and capture what is required doesn't capture its thing maybe we can get a little bit more clarity by placing a breakpoint over here debug this at this point the sequence is null which is not what I'm looking for nor true nor false and here is really the interesting bit so let's try to go inside capture require this should just build the function parser they should just build the function as well we're inside the real require where do we want to go here okay so we're hitting the require with null that shouldn't do anything and we shouldn't aggregate anything uh hitting the require with true again shouldn't do anything with false shouldn't do anything and then the next one we don't even reach so the error is actually happening in the number we're not reaching the string so what we need to do is account for the fact that perhaps the length that we're trying to capture is going to be no and if it is going to be null that doesn't mean we have failed uh let me remove the links here I think that will be all right what we want to say is that the array could be empty all right so we just haven't captured any result we've not failed parsing it's just not a thing that you know we recognize so Json value definitely not that Json value it didn't import anything okay let me run all the tests again the string is still failing however now it is returning no or why it is actually returning null let's double check the capture uh let's go inside here we are going to play place a break point inside aggregate chart here we will rerun one so for the first run we're going to enter here which should be for the number so we succeed here but we don't aggregate anything we then keep going out of our parser continue and forward so the next thing that is failing is our parser I don't want to step into there no no no no no no no shut down the operation that was a little bit hard but so uh what was the problem what's the problem the problem is with this parser for once we have captured something where was the number once we have attempted to capture the number the number has returned something it's not like it returned null and actually made us go further down the line to the next parser so maybe when I said that an empty string over here is well uh kosher maybe it indeed actually wasn't kosher so for all other operations let's actually just return null and say that the parser has failed actually that was the correct thing to do okay so now we have the what's it called the double or sorry the escaped character condition okay so Escape character is just another parser that we could potentially apply so what we're saying over here is we want to capture something that is not a double quote and then we want to capture something that is an escaped character how do you create rules for something like this so we have is digit and really here we're working on a character by character basis first of all let's say that we could have many predicates here we're gonna leave this error dangling and we'll specify a couple of additional operations first of all let's take this go all the way down because I think that's where this kind of stuff belongs we'll say funk Sharp all not double just assign this to this and now for the interesting part is with the Escape quote you actually want to Advance twice not once and what this will really do is instead of funk Char bold instead of these predicates you really want something that is going to keep advancing you something like a parser of characters and once we have these parsers of characters we no longer care about the length because these parsers of characters are going to try to do some work on this Jason and the first one that didn't return null is going to be the successful one and as soon as it's going to return no that's when it's not going to be successful or that's essentially when we pack our bags up and we say okay uh we're done here this has been fun while at last all right so semicolon on the end here remove the semicolon uh there's meant to be a coma here for the rest we no longer do the advancing that is done for us inside these parsers that is digit uh that's also another parser that we want to implement so here let's say that this is a par parser parser of character remember that we don't really get C we get our Jason and Json length while that is more than zero and Jason and yeah uh the recording is going into like a couple of hours now so you know I'm getting a little bit tired uh not double quote if this is indeed true we want to return a result of a Json span this character that we just managed to parse out and Json the rest all right and otherwise if we fail to parse this is a null so not a double quote uh let's say escaped character this is going to require it to be more than one length if the first length isn't all first zeroth is indeed equal to an escaped character we want to return the next character and then after one after one uh after a second one here I thought after zeroth after but that would have been just the whole array it doesn't matter right you get what I'm saying here this is an escaped character we then want another one is digit character do we even want character on the end let's just say is digit Escape we're adults here we know what's going on right Jason over one zero is this digit we can maybe a return or remove that is a digit by a bang but a boom again return the zeros here one here let's go ahead and apply some of these rules I will say is digit here or the capture uh I forgot to enable for Rams so now the notation is going to be succinct so not double quote and escaped the thing that it is screaming at me here is that these are initialized at the bottom so let's go ahead and drag him up to the top slap them over here above the parse I want these as read-only you know if you change up the signature you know you're going to be good but there you have it so power string with these rules uh let's just double check that at least something works so the immutable thing was not incorrect format again so what is this saying an immutable array of character was not in correct format and let's double check I think this has to do uh will it actually take me to the place where it crashed line 33 line hundred um right over here an aggregate character we're putting this into string Builder our immutable array character which overload is this using so append looks to want a character array so uh two array we'll just use the correct overload character array it does Rerun it looks good uh we have a little bit of a situation there uh let me double check the test uh what are we trying to find here uh where is this quote here nothing and then it seems like that we have just captured this backslash instead of the whole thing and we would exit if we would really encounter a double quote so I have a feeling I did think I Advanced the correct amount right over here so if it's escaped we had one Advance twice and we capture the next character so that looks alright to me let's double check from the strength perspective so where we're parsing string we're capturing we're supplying not double quote and escaped I'm wondering if we just want to go the other way around we'll checking here nope that completely doesn't like it so let's come back over here uh let's go into capture should really have only a character at a time let's just slap a zero on here again still with the backslash for the thing over here really curious what's going on again let's uh slap a break point over here uh let's debug this test so going inside aggregate I'll just step over once this is gonna put me straight into the Oblivion it's going to return and I think this is going to return out of the number parser this should now hopefully be the string parser the only one that has it so we managed to parse something the r is an empty character hopefully this is not too small we then have the rest all right now everything looks good let's go inside aggregate so We're looping another time now the r is a backslash and this is a problem this is a problem because we don't want to capture a backslash so I think I was right in saying that these should be the wrong way around so we should be looking for escaped characters first if it is an escaped character we want to just capture the next we don't want to capture it as a non-double quote okay but when I did this I got scared because all the tests started failing again it's like boom Oh man everything is broken again it's all right think it's all right because what this is going to do the first parser it's going to return null if it fails and sorry I was pointing to the wrong thing so escaped the First characters and strings are generally not escaped so we're just gonna be stepping over them so if it's going to be returned no let's just double check again how we're handling it so we have the okay so we're grabbing the first thing doesn't matter what it is or the default if it happens to be null uh well we don't care right so we want a little bit of a predicate over here so we're actually iterating through these parsers that we want to keep grabbing these until they're not no so we actually go through all of them okay and there we have it so that now works there I say easy peasy definitely not easy but you know you work for them and you work for them because you then have a notation that looks something like this so then we have an array an array parser let's say we have a string that looks like this or a string whether that sentence did not make sense uh we're slapping a parse array on the end of our chain you know you know what's going on over here what interesting thing is going to happen here is we have multiples of things right instead of backslash we want to require something that looks like this and instead of capturing a single value we want to capture multiple so let's just capture or capture multiple uh I would say multiple and then we want something like elements so a parser should go over here so let's say parser or maybe first value we already have a parser so we want to parse values that are separated by a white spaced and require a single comma right so that's their separators should look something like this we're missing the multiple function which is really the only function at this point I think that's about it one thing from this we could probably after reuse the remap function that we have abandoned so multiple let's see what this looks like we're gonna go to capture because I think well let's get rid of that because we found out that thing over there didn't do anything so multiple gotta adjust my setting position here we're going to have two parsers super simple parser Json value this is going or sorry uh parse Json value this is going to be the the array element that we're going to be parsing and this is going to be the separate tour separator separator I think that's spelled correctly so we can have an array that has zero values it has one value or it has multiple values if it has a single value I guess maybe this is a little bit easier to draw so no values we have a single value or if we have multiple values and let me draw this dotted optional thing right we can have some kind of value over here and it's going to look something like this so the main thing is just to say okay we want to parse for this or this and if we're parsing for this we want to keep parsing for the same thing so we really want an element and then a combination of the separate operator and the element so next element is going to be a par sir that is going to start with the separator and then the element so this assignment operator here again this is just to say that something like this actually exists here okay as I've said before delete the bodies because you're not going to be able to read them anyway let's say aggregate elements that's what we're going to be doing Nextel and element are going to return a collection of things hopefully that collection is going to contain a single value so from the beginning before we trigger anything let's attempt to extract a single element from the Json okay if what's happening is we have an object where the length is more than zero and we have the rest that means we have managed to capture a variable and then we want to go forward to aggregate elements and the VAR over here will need the rest the rest will go into Json over here and the captured elements will be an immutable array so the immutable array of adjacent value result can go over here the V can go here you can have a little comma the results can go on here if things are null things are bad we just pass it along if it so happens that the length is zero you're gonna have an empty collection here which you can just pass along and pass along the rest of the characters that you have parsed out okay that is going to be the result we're not going to go into aggregate elements uh is it worth spreading this out onto new lines I think it is let's take the next element we're going to try to parse it out a Json switch at some point we're going to fail parsing because we're not going to encounter the separator so we are going to return the result like so otherwise if we have our value and the rest we're just going to parse the result again so just pass on the V or actually a result result if I can spell it add range V hopefully that works right V is a collection it is yeah so we're merging to collections here and then we're passing along the rest a semicolon over here everything compiles bring the tests up let's see this a lot of null references uh whereabouts line 130 on multiple impossible let's see how we've defined multiple again we have parse value over here uh let's stick it after parse value here I like that uh let's run this again not null but this time at least we are getting a type failure so instead of Json array we just get Json true instead of Json array here we get Json null and instead of Json array here we get Json null again okay so we're just parsing the first value and that is because we didn't do the same aggregation that we did for the capture values over here we have a situation that looks something like this we did have a remap function that we briefly uh looked at so once we have gone through this whole thing we are gonna have our result which is an immutable array and new Json array I think this looks fairly simple okay let's run this again and this looks good except for the parse with Json actual Json null expected null yeah the is type we don't have nulls an hour we just have tokens that represent the train uh this was a bad translation of the test another type failure and here I'm not even starting the count from the correct character so again bad on my part we run this okay and this actually works cool so with this now let's take a look at the object implementation so slap this on here on here bars albeit we take parse object place this over here what does this look like we have white spaced parser of objects where is that really what it is I mean uh the white space can be around all these places right not just around the array itself so let's change this up a little bit we will and I mean once you have this notation doing stuff like this is like close to trivial right because that's how you've built it up to be so white space parser right so you're saying the quote can be surrounded by white space and this ending thing can be surrounded by white space internally that will handle white spaces as well but here you'll break you're just baking basically super mega extra careful right now let's move this here we're saying this can be surrounded by white space but we're saying this should be a double quote uh the one trailing brace brace uh parentheses over there don't need it we have this over here we have multiple uh what does the thing look like here uh it should be a parser or should it be a parser still a thing separated by commas a key value pair I mean a dictionary is a collection of key value pairs so the parser that we have is a parse string then a white-spaced colon and then a parse value so a parser of white spaces that has a multiples of our string and parse value that are separated by a comma and then we have sorry colon and then we have commas after each one of these separating them they are white spaced right so white spaces can be all over the place and then we ask for a closing brace what's happening with the aggregate over here this would be super simple with a simple for Loop but I mean it was functional programming we don't have for Loops right so you want to return a Json value or rather I want a Json object so to object I'll remove this I will add an immutable array Json object or sorry Json value values this is going to have a body I already kind of anticipating that I'm gonna need a little bit of recursion so get key value this is going to return an immutable dictionary of string and Json value then this should give me a body hopefully it's not too big perfect and let's actually change this to get dictionary I will change this to be multilined I will put the aggregate of the result into here as well and then we will say get dictionary from an empty mutable dictionary right over here this should be empty and then the values were going to pass them along these can be kind of to basically hide the values over here I'm going to Mark the function as static this can be a new Json object on the back of this semicolon here and are we using this function so to object let's come back where did we have this right over here let's slap this on right uh at the bottom we have the result and the values so values if I switch over here and I'll say that I want an array that looks something like VAR a VAR B and then the rest then this should be a get dictionary and we're gonna try to parse the rest right so add a and a doesn't work what I will actually receive here is a Json string this is going to be a Json value or maybe a VAR over here so A and B A should contain a value and then we should have the rest of the values so here I'm gonna skip two finally if we don't have anything like this let's just return the result and we're done here so that is the function uh if you don't know array pattern matching now you do let's run our tests uh one thing is failing if we have empty objects an empty object is failing the rest was good so again how do you debug this so parse object we have a parser so white space we see this we don't see any of this stuff and then we have a closing curly brace so maybe multiple is capable of emitting a no and it is emitting a null over here so with this right here maybe we're being a little bit strict and yeah for our I think for our Json parser it's actually going to be okay if uh we do something lines of this where yeah or both of these situations what you're gonna get is whatever Json you opted in for because that didn't get parsed and then you want an immutable array so maybe empty yeah well uh let's uh rerun this and there we go okay uh let's maybe torture the last object that we had over here well I mean it's already pretty tortured if we have a comma somewhere like this I mean if it looks uh you know a little bit bastardized does it still parse it does right so a pretty safe parser where my keybinds there we go a video on perhaps an even longer side I know how it's built in two and a half hours of recording hopefully you enjoyed it if you did don't forget to leave a like subscribe if you have any questions leave them in the comment section not much else to say but don't forget check out the description if you want the source code for this video as well as my other videos come support me on my patreon as you can imagine these videos not the easiest thing to make I would eternally appreciate your contributions and your support towards me a very big and special thank you to all my current patreon supporters your help is very much appreciated as always thank you for watching have a good day

Channel: Raw Coding

Views: 4,665

Keywords: c#, fp, functional programming, json parser, json, parser, example, from scratch, implementation, tutorial

Id: A49Xn1gsyPU

Length: 73min 10sec (4390 seconds)

Published: Sun Jul 02 2023