#circe - Part 1 - Introduction to working with #JSON in #Scala

hey you lot here dem insider.com welcome to another video in the previous couple of videos we talked about character encodings and the Jason spec and today we're going to finally talk about how to bring a semi structured format like Jason into a statically typed language like Scala was a very prominent library called Searcy which is powered by cats let's get right to it as in most of my videos I'm using an Ubuntu 1804 virtual machine and I'm running Windows has a host which gives me virtual desktops which has shortcuts that allow me to switch between Windows like this and back to a bundle like that alright now first and foremost I would like to mention that when I was preparing for this video I wasn't really sure what to talk about because Circe and to be fair many other Jason libraries in the Scala echo system they make it very trivial to take some string and convert it into something that the Scala compiler can understand which is typically an ADT or a co-product of some kind and also to go backwards from this ATT to a string but the devil is in the details and so there are many things to talk about in fact I'm afraid that we will have quite a few more videos about Jason and Circe alone so the plan for this video is for us to set up a playground SBT projects as we usually do then I'm gonna throw in a couple of Circe dependencies without explaining much and then I'm going to show you these couple of lines that allow you to convert from string to ADT and then backwards and after that we're gonna hit the brakes and talk about everything slowly and in the next couple of videos we're gonna talk about well you know what I'm not gonna tell you just yet because I'm talking too much let's get to business alright so I'm gonna open my terminal I'm gonna go to my death folder I'm gonna use the template that we're using pretty much in most of my videos and if you don't know this template I actually have a video where I created it so you can go and check it out alright so once it's loaded I'm gonna call the project Jason playground come on there we go Jason playground like this organization is gonna be comdev inside you and the package is going to be them inside you like this that's open it was Visual Studio code alright now as you might know no please not no I'm gonna go to build as p'tee over here not now I think you might know I'm using this template for my actual projects when I'm recording videos I'm just going to remove these couple of here okay because we don't need the because I want to avoid reloading bloop many times I'm just gonna copy paste the dependencies that we need so they're kind of grouped over here we have a couple of test dependencies and we're gonna throw the main dependences later over here but I'm also going to group the ones for Cersei so I'm just gonna paste them over here let me save the file for like it's gonna be formatted yes okay yeah we can know so the thing is that first of all is very modular and we're gonna go through these dependencies a bit later in the video but the second thing you need to know is that Cersei depends on cats in fact the original version the original name for Cersei was Jason for cats and it brings in the dependency on cats but I always advise you to specify the dependencies manually because then you're you're more in control of the version let me copy paste so that we don't waste our time all right whoops it did not copy basically I have no idea why top level and I'm gonna use these two I'm gonna Duke it score and the latest one is two dot one dot one which is a forest college as well but I think we have everything so it's finally imported and while imports I'm going to show you the build so we're using Scala two thirteen one we're using SBT one three eight nothing major I'm gonna go to remain which has our typical hello world once I save the files call it from T is probably going to wake up in a couple of seconds alright so it finally finished let me let's around SBT we're gonna run through SBT to them okay so while it loads we can start bringing our dependencies so we're gonna do iOS or C come on import Searcy and again I'm not explaining much as already mentioned we're just gonna throw in a couple of things I'm going to show you this you know back and forth example and I'm again I hate to hit the brakes and they explain everything slowly alright so we have three artifacts and therefore in fact maybe I can open the build real quick like this I can't maybe I can start writing it actually alright so again so we have three surface dependencies core generic parser and that we have therefore we have three imports Cersei and generic and part so what we're gonna do is we're gonna have we're gonna throw this out I'm going to say final kiss class person name string age and alright let's create an instance of person person which is going to be a person it's gonna be Alice Alice and she's gonna be 27 years old like this all right so the first thing that we're gonna do is we're going to encode this into Jason okay so we're gonna have a person encoded as Jason convert it to the string a very descriptive name and alright so what we're gonna do is we're gonna grab an encoder which is flying around an implicit scope and we do this by saying implicitly give me the encoder for person and you know most libraries that are based on on cats they they come with a with a some other object so the companion objects for the encoder it has a method called apply which looks exactly like implicitly so we don't even need implicit leave so we can just do this okay so this will grab the instance and we're gonna call apply on it and we're gonna pass in the person right which is our case class okay and then we're gonna do to string and this is already going to be our adjacent okay so if we scroll down a little bit I can do print out and we're gonna do a PA come on please outer completed why is it part be a maybe like this no come on there we go like this alright so we should already see this encoded into into Jason there we go this is typical chase alright now let's go back so we could actually take this this string and then decode it back to the case less but in fact I'm going to copy paste it just so that it's more more clear so I'm going to copy that and I'm gonna say that this is going to be some string string over here all right I'm gonna have these three quotes like this three quotes over here like this and please don't ruin the formatting for me of course you did there we go alright so this is going to be some string yeah let me scroll down a little bit so we're gonna do bow from string decoded person and it's going to return either an arrow an arrow or a person and arrow comes from I oh that's awesome alright so we're just going to call them as I call decode we're going to say please try to put that string to decode this the string into a person and this is the string there we go alright so now we can actually deal with something like that maybe and we can print out print line from straying decoded please I gotta complete it for me there we go alright so there we go we have case clouds go into Jason and our Jason going back to our case class it worked out which is why you know it's right and if it didn't work out let's say that we didn't have a come over here it should actually be an exception and there we go it's left now it's a partisan failure expected blah blah blah but got a coma whatever all right let's have a coma again and hit the works out bring it down we're going to play around with this a little bit later but let's do a couple of things already so first was the whizzing coating let's say that Alice was know how this was no let's see what's gonna happen well it's gonna throw an exception it's gonna say a nullpointerexception so what I want to show you is that if our name if it was an option of string option of string and we would say over here none then the to string method over here it would encode it as a as a null in Jason and what I want to show you is that there are many ways to produce this string so you can tap in spaces spaces and then press control space and you're going to see okay so the default version is space is two you can also do the space is to sort by key for example age come first let's actually try this one space space is to sort keys we're gonna see the age first what else did we have we had exactly the same thing was four spaces we also have no spaces and we also have something like something like drop null values and we also have deep drop null values right so if we have you know nested jason is going to go to all of them okay so we came to deep drop null null values and then we can say again spaces okay and let's see let's say for for fun okay spaces for sort keys and now it's not even going to display the null and it's gonna have the four spaces okay so i just wanted to play around with this a little bit let me actually revert this so let me actually say to string over here and let me say that this is a string and this is going to be Alice like this I just wanted to show you a couple of couple of ways before you know before we take everything slow again all right so let's talk about this there are many things happening and you know you could think about it as you know they're like many layers it's not super visible over here so let's go through it I guess so the first thing is that we're getting the encoder from the implicit scope we're going to talk more about this in the next video about where the encoders come from for now I'm just going to tell you that they come from this imports okay so if we remove this import then this exactly this line and the line somewhere was the decoder over there it's not going to compile okay so this is called automatic derivation and we're going to talk about this more in the next video okay so we're getting this encoder and then where we where we can you know by calling apply and giving it a person we can convert a case class into the Jason ADT right so the Jason ADT is defined somewhere in Searcy and it's just a representation of the typical Jason types you know like boolean you know number and and then the string and whatnot all right and after that we have a couple of ways how we can convert it into a string as we have just seen you know we can do the spaces two spaces for sort keys and so on now I want you to notice two things the first thing is that the only thing that we wanted to do during the encoding with our Jason was to decide how we converted into a string however we might want to be able to do something else right so this intermediate thing you know this Jason ADT that comes out of it we might want to traverse it and you know change it and then filter it and whatnot we're not gonna do this is actually and pretty much all of my videos because a typical example is that you're gonna have a web application where you know some Jason comes in it's gonna it's gonna get decoded into a case class and then when you're producing the response you're gonna encode it back into Jason okay so you usually you don't want to you're not interested in this intermediate layer okay but we're gonna talk about it a little bit more and the second thing is that the decoding doesn't look a symmetric to the encoding like notice over here what can I have like these three layers over here in the decoding the API is kind of bit different we're kind of like skipping the layers and this is happening because there's only one way to go from the JSON string to a case plus okay to a case class person even though we might be interested as well you know like like the same as during the encoding tape in fact maybe even more to sort of get in into this intermediate Jason ADT Larry and and transform it and so on but you know most of the time we can just skip this layer and go right through it and as I just mentioned it doesn't look very very symmetrical so I want to make it look a little bit more symmetrical so another thing that we can do is we can sort of introduce like these artificial layers well it's not as official just that we don't really need it okay so we can say you know parse actually you know let me let me just go ahead here okay so we could say parse some string first okay and then this is going to give us either a parsing failure or a person in parsing failure extends arrow and arrow by the way extends exception or or thurible okay so we can parse it first and then we can flatmap and now it's going to look a little bit more symmetric okay so the same ways we're grabbing the encoder over here we can grab the decoder and decoder for person over here and we can call decode JSON on it okay this is going to be exactly the same line let me do this it's getting very sunny over here so I had to close the blinds so my room is a bit dark anomaly alright so another thing is that I want to do is I want to I want you to really see these three layers so what we're going to do is we're going to go up and we're going to import scholar util chaining and we played around with this in DVD about about function juggling okay Cheney okay so what we can do now is we can go over here and instead of this we could say so let's get some string okay and let's pop it into parse okay and let's do flat map and let's do decoder person dot decode jason okay so this way you're gonna see more these like these like three layers and it doesn't compile because i don't know how to type there we go like this now there's also some syntax sugar both for decoding and for encoding so instead of doing this i can just say so whatever comes in over here i can just say as as person okay so it's exactly the same as grabbing the decoder and calling decode jason okay that's exactly the same thing and we can also import some syntax for the encoding so we can go over here and we can import i know that seriously that syntax let me actually sort the imports oh they're actually already sorted okay so over here where's our encoding okay so instead of grabbing the encoder calling apply and i'll go into this intermediate layer and stuff we can just go on and say person dot is geez i got an out to string or you know we can do like spaces spaces to okay so you have like a you know a dsl for encoding like on your case class you just say as jason and then you decide how to convert it into a string and during the decoding you just basically have if you have a case class you say that ass person if you don't have a case class if you're starting with a string and then you can you know go through the parse so again like this is some string okay and this is you know either a I believe it's called parsing or parse failure or Jason which is how the you know the da/dt is called and over here it's gonna be either an error or person like this okay and the same thing at the top over here if I were to press ENTER like this color fmt is going to spread this across three lines so over here we're gonna have our adt right over here we're gonna have the Jason ADT and over here we're gonna have the strength okay in fact I believe I should use the same things over here okay so I'm just gonna do hi Jason IDT and I'm gonna put it over here and so this is a Jason ADT and you know the person is our a beauty okay so maybe let me do like that alright we still don't see don't see both of them also notice that during the encoding before before we introduce the syntax we were getting the the encoder and they were calling apply on it however if you remember just a couple of seconds ago when we were grabbing the decoder we're not calling apply on it or calling decode Jason on it so there should be a way you know it should be even more symmetrical okay and let me actually show you this so before we were grabbing the decoder for decoder for person and we were saying decode Jason like this alright so in fact what we can do is we can also call apply but calling apply would require some other layer it's kind of its its layer where where you kind of get the cursor which allows you to to traverse adjacent so over here we can just map map to H cursor dot from Jason okay from Jason okay so over here we're kind of gonna get it the occurs and if we have the cursor then we can just call apply okay there's gonna be exactly the same thing it introduces like yet another layer there like there are many ways to do this okay but you know in in in most you know most of the time you just want to go like from your strength to to your case class and back so in fact we're gonna we're gonna remove all of that thing all of these things so I'm just gonna go from straying straight to decode decode and to decode into first okay so this is gonna give us well you kind of can I see the results okay so we have we just have the string and we're gonna go straight to decode person okay this is this is the same thing as we had before right this is one function called decode into person and just give it a string alright so this is pretty much the final result right so this is how you typically do it person dot as adjacent spaces to or the other way around just have some string you just pop it into the decode or just called decode and give it the string let me go on a tiny tangent here so as we have seen in the previous video Jason is a very straightforward format not I mean looking into the Java echo system in the sky Lipo system alone we have at least like JT liked like 10 Jason libraries and they all come with their own version of adjacent ADT and some of them called Jason 4's actually tries to unite all them sort of sadly it sort of says you know hey guys you know just take take this ADT okay and another thing that you need to know is that you know there's parsing layer the ones where you go from the strings to this jason adt you know also like every library comes with their own parser and this is one of the reasons why why a Searcy is actually some modular so you could actually go to the build and you could say you know don't include the pars are from from Cersei but include for example the Jackson parser let me actually go through these real quick okay so this is where the jason i did he comes from this is where the where you decide how are you going to generate the decode X right so the encoders and decoders we're going to talk about this more in the next video and over here you're choosing the parser and capsules not cats I'm sorry the Cersei motets as well but Cersei also works for Jason 4's and so if you just sing insert support over here if you're in on the JVM then it's going to use the so called John jwn parser and if you're in the JavaScript land and it's going to use a typical built in JavaScript you know Jason parser for educational purposes it would kind of make sense sort of to learn sources like layer by layer however as already mentioned a couple of times usually you didn't you don't need this you want to go from from strings to a case class and and then back so we're just going to skip that and it's gonna make sense so what I'm going to do is since we already look into the dependencies I'm actually gonna add one more and it's called a Searcy Searcy literal okay and we're gonna import the changes and we're gonna go to our main while it imports and we also going to import I also see a literal like this let me let me sort them sort them like this so what it allows you to do it allows you to use something like you know almost like a string interpolation well actually it is string interpolation you can ride this string over here and it's already going to be G ok so I'm just gonna go over here I'm gonna copy pop I'm gonna paste that and I'm gonna say that this is not just some string that this is actually already already Jason okay so the top of it is Jason right it's it comes from you know straight from Circe and all you need to do is you need to type in Jason over here alright so it's the same like a string interpolation but it's a different interpolator okay like this so this is this is already jason okay and now you can say basically similar to over here and we're just going to call it from jason right from jason decoded person over here we're gonna say that this is gonna be already or you Jason now instead of going through the decode we're just going to say underscore that as person okay let's go and print it out so over here it should look exactly the same okay there we go this is very very useful if you want to use it you know in your test for example now let's talk about it a little bit more so this is an interpolator which means you can use strings inside of it so let me actually copy that and paste it over here so we could just take this whole thing out okay and we could just say for example dollar person alright so the person is over here over here right so this is our case class so we're just sticking it in there and somehow it's still gonna work itself up okay let me actually print it out and so over there right so it looks exactly exactly the same let's throw this up all right so let's actually go and introduce an error over here one of the beauties of this interpolator is that it's gonna break it compile time right so as you can see it says not family it's an interpolator right okay another thing that we can do is we can use you know we can introduce some other errors so if i throw this out again we're going over here and i'm going to remove the comma for example okay and as you can see it doesn't really work or let's I don't know let's let's remove maybe this column okay as you can see it just it's doesn't work out of the case you have something you know you have a little bit of very checking over there another cool thing and this might not work in dotty once we finally arrived and dotty because of the way Makris works like this thing is this macros okay but at least for now you can use variables both for keys and for values right so the using it for using variables for keys might not be possible once we arrive at that dotty okay so let's do name key which is a string and it's going to be the name and we're gonna do age value and I'm just gonna give you you know giving you a couple more examples okay so we can do a name key over here note that we don't actually need to use the equals red so we just do you know typical interpolation age value over here change value whoops what happens age value there we go and it should behave exactly the exactly the same now let me revert this real quick and show you one thing that kind of got me and got me surprised all right so let's throw these out okay and let's where's the string so we have some string over here so let's take and copy paste it or came and we're gonna say did I copy just okay hold on let me copy this let me bring it down okay okay so I'm gonna say that this is you know still string or in fact let's call it not in Jason because I actually kind of want to break it okay so let's say that we're now gonna have this curly brace which makes it not a valid jason okay so over here in order to jason you can go over here okay and you can do $1 not jason and what's gonna happen is that it's still somehow gonna compile right you would expect the compilation error but somehow you know it's it's macros it's it's there's always dark magic in there okay so somehow it managed actually to you know to to go through the through the compiler and you know obviously it couldn't couldn't be decoded okay but it's you know I'm trying to avoid using like these you know as much as possible like these these magical things okay even in my tests I actually have my Jason in the file so I will show you it's actually not that hard to do and also because you know in the tippecanoe and typically editors alrighty ease if you have adjacent file and you also get you know you're ready or editor support you know you get syntax highlighting yet they get linting and and so on in fact since we're pretty much done was the overview let me actually throw everything out everything except for maybe no I hate it when it does them except for maybe the case class person like this and let me actually go and create a file so we're gonna have source main and we're gonna say new file I'm gonna say resources I'm gonna call it data dot Jason okay so I'm gonna open over here but you don't need so much space over there so let me bring it to the right so let me paste our Jason over there so you know we get like syntax highlighting you know we got we get the same like error handling over here we also have like Auto completion four four four keys or at some for example if I say name for press tab it's gonna insert the quotes for me so it's you know it's very very convenient all right so we're gonna have it like sitting over there all the time and now what we're gonna do is we're gonna go and create a helper that would you know would read out read it out from the file okay so I'm gonna call it I'm gonna call it a read and Jason from that's column alright so I prefer to have my packages like this alright so we're gonna have a bunch of imports so we're gonna do java.net URL I'm gonna have Scala I own we're gonna have cat simplicity and we're gonna have I also see and we're gonna have a Circe Circe dot Farzin I'm gonna have it you know just it's just gonna be an object that's called the read and jason from okay and now bloop should shut up because there's you know bloop and medals in there so I'm going to structure it very beautifully so that you can read it like top to bottom it's gonna be like I don't know six or seven methods but they're all gonna be like pretty much one-liners okay the first one is going to be cold we can be able to read adjacent from a URL into some eh okay so assuming that we have a decoder for that we can give it a URL which is this a string and it's gonna produce either a throwable right because you know maybe the file is not there whatever word you know do you overdo the resource is not there okay or it's kind of you know actually succeed and produce a name okay so it's gonna call get URL and it's gonna give it it's gonna give it the URL and it's gonna call flatmap URL into a okay so both of these methods don't exist get URL doesn't exist URL into also doesn't exist okay so get URL is going to be a private def I'm just kind of used - doing like this it's absolutely unnecessary I'm just gonna use to it they came so it's gonna be cold yet URL I gotta give you the URL which is a string it's gonna do either throwable or the URL okay all its gonna do is gonna use this message from cats catch none it'll okay we're just gonna do new URL in case it throws exceptions try this Java thing right so um now we also need to do URL into and URL into in fact it's going to look very much like this one okay so it's gonna be your lineage it's gonna be an overload you know an overload which already accepts a URL okay this is gonna produce a throwable or any other ways it's gonna look exactly the same okay so over here we're gonna do from URL URL okay from URL doesn't exist either okay so I'm going to do map get lines dot make string and I'm gonna get it a flat map flatmap and this is you know the coal to the decode from from sirs-e just decode a for us okay from URL doesn't exist yet so it should break over there okay so we're gonna go and we're gonna say prime with this I'm gonna say def from URL can I give it a URL okay now it's gonna produce either a throwable or a thing called buffered source which comes from a scholar I over here alright let's call them so this is going to be an equal sign okay I'm just gonna say either dot catch non-fatal source this is the scaleo source from URL okay I'm just gonna give it a de or like this and over here you can also specify the codec which also comes from scholar IO so you can say codec dot utf-8 for example right but you df8 is actually the default okay so this is the fella dot IO dot correct all right this one okay but let me start out this one out okay so you can pretty much read to top to bottom okay so we're getting the URL where we're converting the URL and you know converting the string URL into the actual URL then we're flat mapping it into URL into and URL into just gets this buffered source which is a stream of lines essentially writes looking to like get lines that make string and then you know we have the string and we flappin it into the third map and identity decode and we're also going to have the same thing for the resources okay so if we're gonna reuse a lot of them okay so I'm actually I'm let me actually copy this one bring this one over here and I'm gonna call it let's not do URL until is to resource into okay and it's gonna get a resource named resource name over here right but the it's gonna produce the same thing and it's gonna be the same pattern so over here the first thing we did was get URL the first thing that we're gonna do over here is we gonna say get resource okay I'm just gonna give it the resource name and then we're gonna do the same thing as over here flood map URL into okay that might be all into in this case it's already it already exists okay so all we need to do we need to do private this in fact let me actually copy paste this one so this is get URL over here I'm just gonna call it get resource and this is going to be a resource name and it's going to produce either a throwable or a URL the same as before and we do other cache non-fatal and instead of this we're just gonna do a get class get class loader get resource and we're gonna give it the resource name all right so now this thing is complete just a couple of lines and you know I'm gonna upload it into some gist or something so they can so that you can play around with this okay so now we can go back to our main we have our person and we can just say Val from resource decoded person which is gonna be either a throwable throwable or a person and then we're just going to call our method right so we're gonna say read jason jason from resource into into a person okay I'm gonna say that the resource name equals data dot jason like this alright so now that we're out we can go and do a print line from resource please autocomplete it there we go alright and as you can see it worked right says read this Jason int over here okay so now it's it was Ellis no it's Bob there we go nice and easy come on may get it back back to a string all right now I want to to make this a little bit more beautiful because we're gonna play around with it and I want you to see you know what clearly when there was an error and when there's not an error so the first thing that we're going to do is we're gonna go and you know use a tap over here so instead of just doing resource into we're also going to say tap tap and we're just going to print them right so this is just a little bit better this comes from comes from Scala util a scholarly chaining the same as pipe in fact we're gonna use pipe over here right so instead of this whole thing I think maybe I'm gonna do right over here right so we're going to do data that Jason writes are we taking a resource name are we doing it with a pipe in it to read Jason from that resource into into a person person person what's wrong and after that we're gonna do the tap right so this kind of reads like a nicer case we're gonna do I'm gonna do this I have no idea why it's so hard for me to type person today all right again taking a string popping it into there into the resource tapping into print line let me also introduce a couple of helpers in fact we're gonna play with this project in the upcoming medias as well so I'm just gonna go and create a package object and it's kind of cool because metals actually has a thing called new Scala file and I can just say please create a package object for me and that creates this how beautiful is this alright so over here I'm gonna have def print line good okay so it's gonna print out the stuff in green unit equals print line console dot green actually we don't need this again green plus n plus console dot reset like this and I'm gonna copy that and I'm gonna paste it over here and I'm gonna say a print line bad that is going to be in red like this and you know what since we're gonna play you know a lot was popping and tapping we're actually going to define it ourselves in this way we won't need to import it and also we can define you know are our own helpers okay so I'm just gonna say real quickly over here final implicit class this is gonna be syntax for pipe and tap in fact let's just call it pop and tap pipe and tap like this okay so it's gonna be over here it's gonna take a private private eval private aim which is aim stands and your vowel okay so I just gonna do inline final pipe it's gonna take some B it's gonna take a function that goes from A to B okay and it's going to produce it be and it's just going to produce it just by calling it and we're gonna do I have no idea why I contacted a pipe ant tap so for tap you shall let's say real quick because maybe metals can't really be to you for me okay maybe it can't maybe it can't you and you miss the function goes from a to you and so it's gonna call it over here man alright but then it's gonna throw out the result and you're just going to return a like this okay so this is the this is the pipe and this is the tap and I should return aim hey like this okay let's also create our own this one is not present in the standard library it's gonna be called tap that as nonetheless can't a path Tabitha okay so and that's just gonna get a you which is gonna be a by name parameter okay like this so all its gonna do is it's just gonna perform from a side effect right so this will allow us to do something like tap as and we can do a print line the hyphens for example right so we can ignore the ignore da and just you know just perform the from the side effect okay so now with these guys in place we can actually go back to our main we can throw a scholar yuto chaining and over here instead of just tapping print line what I'm gonna do is I'm gonna do tap and remember this is an either and if we have cats and bliss its imported which we do or in we have no we don't all right we actually need that import cats and bliss it's okay then on an either or you know or topple or something like this that has like two things you can actually call by map and then you can give it you know what we'll give it a function that will allow it to do something like was the left side which is going to be print line print line bad and something to do with the right side which is gonna be a print line brittle and good okay okay so basically what its gonna do is if there is an error it's gonna be displayed in red like over here it's gonna be rad and otherwise it's going to be green okay so this is very very very very cool and also because we're just gonna keep playing around it's not going to be just person it's actually gonna be called datum okay please help me data there we go nice another thing that I want to do is I want to improve the to string over here so I'm gonna paste something from script okay so now it's gonna render a render the data like this it's sort of it's sort of gonna look like Jason right so there's no no magic over here there was this one thing that was added in Scout 213 which allows you to get to get the the names of your products from from your case classes you can just get it get the names we're gonna zip it with the actual values and you know we're just gonna print them out you know with a couple of spaces it's not perfect but you know it's it's fine for for our use case all right so let's go and change this want for example to key one key one and this one is going to be key to all right yes please come on key to tap there we go all right so as you can see it's Canada I said we've seen the the arrows in red okay so let me change just these two there's gonna be key one and this is gonna be key too okay so basically what we're gonna do is we're going to to play around with different things that you can read it up for example the first thing over here so as you can see we're reading the key to into an integer but it turns out that we can actually use a string over here it's still gonna work okay notice that when we're gonna go back and you know encode it again which which we're not doing over here when we're going to encode it again it's gonna be encoded as and then so it's not gonna be you know and as a morphism okay in fact let me go back and actually use the 27 over here okay so the first thing that you need to know to know is that obviously you don't need to read in the entire thing okay so in this case for example we have more keys than where then we're parting which is totally fine okay we're gonna parse them all later okay so we're gonna have key three we're gonna have key 4 over here which is gonna be false we're gonna have key 5 which is going to be no we're gonna have key 6 6 which is going to be an array of 1 3 3 7 like this then what else in fact let me save it yeah there we go with this so that's what I thought what happened ok now let's also have key 7 which is going to be in the set object in case we're gonna have an object inside of an object and it's also going to have a key one and I'm running out of examples over here let me actually do that like this and on this one it's going to be key to okay and this is going to be a four or five six seven like this alright so now let's go and try to read them out we're gonna do ìiî ìiî which is going to be a boolean and as soon as I save it I can forget Mars call fmt in such a way that it will collapse these lines as soon as we have like three arguments alright so this is key three this is gonna be key for alright this is going to be key five now notice that if you say for example that it's gonna be an integer but key five is now so it's actually going to throw an exception and usually like this is the thing like when you're dealing with Jason that is not yours just wrap everything into option and you know Circe's gonna convert it into a nun alright so let's do key six key six the cool thing is that you know this is an array but you can read it into a sequence you can read it into into a list but you can also read it into a set and it's gonna remove this three for you right because that assets cannot have duplicates right so you have like one three and seven okay you can also do a sequence or you know you can also do you know lists whatever I probably also like something like vector right yeah so basically everything that is sort of like a secret let's actually believe this set okay so now let's do key seven and key seven is a nested object okay so we need a thing so we're gonna call it sub data or something like this so let's go let's go over here let's do a final case class sobbed ear and we're gonna have key one which is a string we're gonna have key to which is and okay notice that we don't have this fancy rendering for sub data but it's fine okay so this is you know this is how you do like mess of things and by the way the absent keys are treated as not as I showed you in the beginning so over here we're saying key five is an optional event and it's now and if we actually throw it out like this it's gonna be it's gonna behave the same way right so if T if the keys not there it's still going to be wrapped into Internet okay affectionately introduce it back another thing that you need to know is that if you have an object which has like only two fields then you can read it into a map and by default you know the encoders that are provided by by a sort of the decoders that are provided by D by default and they only work from maps which key which keys our strengths okay so if you want the key was a different was a different type you need to provide your own decoder all right so instead of doing a sub data over here looking into something like map string over here and it will be able to read this thing this thing in there actually it looks like both of them need to be strings that's interesting I kind of thought that the values could actually be P hints what what's wrong okay so both of them are strings that it works actually had different results when I was preparing for this video it's it's it's a bit weird anyway basically maps on the work for for all the strings if you remember from the previous video that Jason spider actually doesn't technically forbid duplicate values so if we had a duplicate duplicate value over here so let's say that key 3 was not true let's say that it was false over here right so we get a warning right but it's not an error so it reads it in and it looks like the bottom one actually overrides it right so key threes now isn't how false okay which kind of makes sense right why would you read a part of that reads jason backwards alright let's actually throw it out again right just so just so that you know also since we are reading it and with utf-8 over here remember our codec over here we could specify the code but the default codec is utf-8 anyway so we can do fancy things like Smiley's over here ok so there we go this is our smiley we can also copy it oh man alright let's go and copy it nice control shift C alright and bring it in and paste it over here like this let me save that and s Petite reload it soon ish come on there we go run and it still works the same as before we have our smiling over there alright cool another thing that I want to show you is that if you have default values it's still not gonna help you if the value is not present in Jason right ok so let's say we have a whatever here it's going to be an int right now obviously it's gonna explode but even if you had a default value like for example you know 1 3 3 1 3 3 7 over here it's still not gonna help why are you taking like so long to compile there we go okay whatever 1 3 3 7 it's still not gonna help it's still gonna explode there we go alright let me throw it up let me also show you the floating values okay so if we're gonna have floating floating 1 and we're gonna have zero dot one two three four five six seven eight and then we're gonna have this thing again three four five six seven eight okay not you know kind of forgot than that okay so we're gonna have floating one and we're gonna the same value for floating floating - and we're also going to have lung one which is going to be let me actually paste this okay so this is the this is one more than fits until along okay so we're gonna have come over here and we're gonna have a two over here okay now if we go and try to read this in so if we're gonna do floating one which is going to be a double okay then notice that it's going to lose a little bit of precision over here right so it ends with five six eight so this six seven eight nine they turned into an eight so if you're dealing with some other money values because you know remember in Jason there are no constraints or no there's no limit like this number could be as long as you want it okay but you know inside of an actual programming language like Scala you know which runs on the JVM Rachel limits because nation needs to fit into memory okay so if you read this in into big decimal you actually have your real value let's do a long one and it's going to be long and now it should throw an exception because this number as I already mentioned it's actually one one bigger than they are so long okay so this is this should fit as you can see 8:07 which is the max long but this one is not gonna fit okay but you know what you can do is you can do a long term and it can read it into a big integer okay so now it's kind of fit now this video is getting too long but I want it to be sort of like a little bit self-contained so I need to speak about automatic derivation a little bit and all of these videos in the data juggling place for example where you know we're introduced you know pure config and you know refined and then squats and all of these things most of them are based on shapeless and so the way they do automatic derivation of the implicit which we're going to talk more about in the next video is that you know it's it's it's really fancy but it but it comes at a cost so for example it's slower it's slow at compilation time it's also slow at runtime and also it doesn't always generate what you what do you need again we're going to talk about this more in the next video but I want to show you something that you might need from Jason and this is changing the keys okay especially this is a Jason that you don't control you know they could be you know could use a kebab case over here or a snake case over here however when you read it in most of the time you want to use camel case okay so we're gonna go through all the possibilities and in the next video but for now I want to show you what you can do you know I want to show you like at least one of them so that you know you might not be interested in watching the the upcoming videos okay so the first thing we're gonna do is we're going to rename this key for in two key for KML case okay so if I just copy that and go over here alright so if you do if you're doing exactly the same thing then it's gonna match and then it doesn't matter if you do you know key for key for underscore snake case okay so if you just do that over here same thing snake snake come on snake snake case like this if we just compile like this is not gonna oh actually that's compiled okay so it's gonna compile like this fortune 8 case but if this was a kebab case it won't compile okay like this it won't compile but you could use the the back quotes okay then what I should still work if I had the same thing over here write it down like this right so everything everything works okay let make sure convert this back to a snake case I want to have snake case okay now there is a way to to have a snake case or a kebab case or what you want in your Jason but when once you read it and use a camel case over here for this we're gonna have another module let's go over here and let's also have generic extras okay let me make sure that they're alphabetically sorted well almost okay let's import changes okay so now we can go back to our main and we can have so we can remove the automatic duration this is not the one that we're gonna use now instead what we're gonna do is we're gonna have import I oh that sir see the generic extras underscore okay so instead of using Auto we're just using these these extras and well what we can do now is we can go to our our data and again over here say at configured in Jason code okay and we need it as over here as well okay now the only problem is that this I'm gonna compile because it actually now needs a configuration I make sure to bring it bring the SBT back alright so what we're gonna do is we're going to go to our package object and we're going to create this this configuration okay so we're going to say implicit Val config configuration and we're just gonna do a configuration which is a package object in there and we're going to do default dot and now we can do some like wiz okay we don't have auto completion because I did not bring in the import okay maybe I can save it and maybe metals can do no okay alright so we can do import I own that circita generic dot extras that configuration like this and now we should be able to say with control space come on and now we can do something like with snake case with snake case members okay something like this let me actually also press ENTER over here okay it's not everything in Mein Kampf oz and now for example if we have a snake case over here we can still go over here and we can say key for snake is right so now it's called now it's called a snake case but it's but it is actually a a camel case over here there we go or a key for worked as a charm okay thank case falls there we go cool now we're pretty much done with the video it's way too long as always I want to give you just one more example we're actually gonna read in some some proper JSON API and that's some random example but some actual you know actual API so what we're gonna do is we're gonna use the github API so if I go to my browser and at Google for sample github API I'm gonna click on the very first link I'm gonna scroll down a little bit and the entire API is over here like let me zoom in a little bit okay like this so the API is over here so basically if I go and there I should see some Jason coming back let me zoom in again okay so this is a true REST API you rarely see an API that is that is designed as as good as this one so basically you have you a JSON object but every key actually points you to to the URL that you need to to get the next value so the one that we're gonna play around with is going to be user repositories URL so we can just copy this okay and I can go and copy paste the same and as a user over here which is a variable I'm going to do have inside you okay and the repository so we can specify the page and you know how you know how to sort and whatever so I'm just gonna do a repos just give me like all my reports and how did you get in there have inside your repos okay there we go these are all of my repositories so obviously we're not gonna read into the entire thing and by the way I look at that I use a snake case okay we're just gonna read in I don't know the name and maybe something nested the license for example okay so we're just gonna read this this one and so we're going to go back and we're going to let me copy paste that and paste it over here and I'm just gonna call it from URL decoded github responds okay so it's gonna give me an other a throwable or it's gonna give me a sequence of github response okay so by the way before I kind of forgot to move these guys out right now we don't really need them but still so I'm gonna I'm gonna cut them out so that they're not in the main I'm just gonna paste them over here okay and so what I'm gonna do is I'm gonna do a final case class github and get a response it's up response for now it's gonna be empty I just wanted to compile okay so we're gonna have a sequence of github responses over here and I'm gonna not read them into so I'm not gonna be the resource I'm gonna say URL into and it's kind of reading the sequence of github github a response and for this we we're going to need to use the same URL as we used over there which is this one right so just put it in over here and now it actually might succeed because we're not reading any values but everything else actually kind of works out let's see I finally push it up so city quarter oh yeah I forgot to to do that thing okay wait I need the decoder read your error messages folks all right so now I should actually succeed but we're not reading any any values in there okay so let's actually read them out so let's for example read end the name which is a string like let's just written the name okay there we go name string let's see should I not do the other one let's just let's just not do the tap okay there we go okay so these are all of my repositories install Google Chrome and so grab em and and all of these okay so let's read it in the license as well so let's say that we're gonna have the lessons over here which is going to be an option right so not not every not every project has a license okay so we're gonna do this and it's gonna be a final case class license and it's gonna have the key string name string s BDI X I have no idea what this is actually actually it's SPD X ID okay which is a string it's gonna have a URL which is an option of string and it's gonna have the note ID which is a string and by the way notice that all of these are camel case right so if I go back you're gonna see that the license actually has you know has done with a snake case okay all right let's go back and also let's improve the the print out a little bit so let's over here so on the right it's gonna be a list of case so we can just do for each ran over here okay that's already much better let's also come count them real quick so let me go to the package object and have and implicit final class sequence comps okay so we're just gonna add another add one more method to a sequence so it's gonna be self it's gonna be some sequence of a it extends and eval okay and we're just gonna have def was one based index and swap because usually the index would be on the right I want to have it on the left and I want it to be one base instead of zero base all right which means that it will start with a 1 instead of a zero okay so we're going to take our sequence and we're gonna do is e was index we're going to map that so this part is going to make sure that the index is actually one base okay and we're also gonna swap it over here like this yeah so over here what's happening is that I'm calling map on a tuple which you know and tuples didn't have that so when you Tim for cats implicit like this and now I should actually work alright so now we can go back to arm we can copy that and go back to our main and before we do in for each we can also do what also do that and now once my laptop cools down a little bit and we can actually actually finishes compiling it's gonna look more beautiful it's gonna have the the number of the repositories and it's gonna start with a 1 so the last number that we're going to see is actually going to be equal to the amount of the repositories that I have which is 30 over here right and by the way note is that if I scroll up a little bit in fact let me do this then we're going to see that not every repository for example has a license it's in done somewhere Iran so some some some some here this one for example the function juggling that I recently uploaded it actually doesn't contain a license okay and also the license itself it doesn't always contain all of the fields and basically when I was playing around with it this is what I meant in the beginning you know if you don't make these as options right so let's just make this one as a string maybe now that real license has a URL and then it's gonna explode because like one of them it's not gonna have it yeah there we go all right so usually when you're dealing with like foreign API is he would you would you know you would use options everywhere all right I'm sorry again that this video got way too long and please remember there's there's gonna be at least two more videos and for now as always system inside don't forget to like this video if you did subscribe if you want to improve the developer inside you and if you learn something today consider supporting me on patreon and thus watch all of my videos before everyone else and most importantly take care

Channel: DevInsideYou

Views: 5,847

Keywords: scala, software, development, programming, tutorial, dev, tutorials, code, learn, education, knowledge, diy, dev inside you

Id: 712WE9Ou7BE

Length: 52min 29sec (3149 seconds)

Published: Sat May 02 2020