Learn JSON path and YAML | Beginner's guide about JSON & YAML

hello this is moonshot mohamed and welcome to the yamo and JSON path course this is an introduction to the basics of llamó and JSON path made by code cloud Yamal in JSON path are a must-have career requisite for anyone entering into the modern software development cloud computing data analytics or devops world throughout this course we get you up and running with these concepts using lectures and hands-on labs for practice thank you for joining and I hope you enjoy the course hello and welcome to this lecture in this lecture we take a look at what llamo files are if you have worked with other data structure formats like XML or JSON you should be able to easily pick it up don't worry if you haven't worked on any of these you should still be able to easily pick it up going through the coding exercises that accompany this course a yellow file is used to represent data in this case configuration data here is a quick comparison of a sample data in three different formats the one on the left is XML where we display a list of servers and their information the same data is represented in JSON format in the middle and finally in yellow format to the right take a minute to compare the three formats let's take a close look at ya mo if you take the data in its simplest form such as key value pair this is how you would define it in ya mo key and value separated by a colon the keys are fruit vegetable liquid and meat and the values are apple carrot water and chicken remember you must have a space followed by : differentiating the key and the value let's take a look at how an array is represented we would like to list some fruits and vegetables we would say fruits followed by a colon on the next line enter each item with a dash in the front the dash indicates that it's an element of an array how about a dictionary a dictionary is a set of properties grouped together under an item here we try to represent nutrition information of two fruits the calories fat and carbs are different for each fruit notice the blank space before each item you must have equal number of blank spaces before the properties of a single item so they are all aligned together let's take a closer look at spaces in llamó here we have a dictionary representing the nutrition information of banana the total amount of calories fat and carbs are shown notice the number of spaces before each property that indicates these key value pairs fall within banana but what if we had extra spaces for fat and carbs then they will fall under calories and thus become properties of calories which doesn't make any sense this will result in a syntax error which will tell you that mapping values are not allowed here because calories already have a value set which is 105 you can either set a direct value or a hash map you cannot have both so the number of spaces before each property is key in the amel you must ensure they're in the right form to represent your data correctly let's take it to another level you can have less containing dictionaries containing lists in this case I have a list of fruits and the elements of the list are banana and grape but each of these element are further dictionaries containing nutrition information a lot of students new to Yama have reached out to me asking when to use a dictionary or a list so let me explain this a little bit better first of all it is important to understand that all of what we discussed so far such as XML JSON or llamó are used to represent data it could be data about an organization and all of its employees and their personal details or it could be data about a school and all of its students and their grades or it could be data about an automobile manufacturing company and all of its cars and its details it could be anything let's take an example of a car a car is a single object and it has properties such as color model transition and price to store different information or properties of a single object we use a dictionary in this simple dictionary I have properties of the car defined in a key value format this need not be as simple as this for example in case we need to split the model further into model name and make ear you could then represent this as a dictionary within another dictionary in this case the single value of model is now replaced by a small dictionary with two properties name and ear so this is dictionary within another dictionary let's say we would like to store the name of six cars the names are formed by the color and the model of the car to store this we would use a list or an array as it is multiple items of the same type of object since we are only storing the names we have a simple list of strings what if we would like to store all information about each car everything that we listed before such as the color model transition and price we will then modify the array from a list of strings to a list of dictionaries so we expand each item in the array and replace the name with the dictionary we built earlier this way we are able to represent all information about multiple cars in a single yellow file using a list of dictionaries so that's the difference between dictionary list and list of dictionaries I hope you understood the difference between the three and when to use each of these before we head over to exercises let's take a look at some key notes dictionary is an unordered collection whereas lists are ordered collection so what does that mean the two dictionaries that you see here have the same properties for banana however you can see that the order of properties fat and carbs do not match in the first dictionary fat is defined before carbs and in the second dictionary carbs comes first followed by fat but that doesn't really matter the properties can be defined in any order but the two dictionaries will still be the same as long as the values of each property match this is not the same for lists or arrays arrays are ordered collection so the order of items matter the two lists shown are not the same because apple and banana are at different positions this is something to keep in mind while working with data structures also remember any line beginning with a hash is automatically ignored and considered as a comment we are now ready to head over to the coding exercises and have fun playing with llamó files you hello and welcome to this demo my name is moon shot monomyth and in this video we will get introduced to JSON path we will start with a quick recap of llamo and the differences between llamo and Jason we will then look at Jason path where we see how we can get started with querying a JSON data set that consists of dictionaries les lists and dictionaries and complex data structure and how to apply a criteria to our query and finally we will look at how to access the practice exercises associated with this video and how to navigate through the tests note that this is for the absolute beginners so we're going to go slow and start with really simple examples and use cases and you will then get to practice what you learned and test your skills so let's get started and before we begin remember to subscribe to my channel to stay up to date with new videos like this in one of the earlier videos we got introduced to the basics of yellow we saw how EML was used to represent data in a structured format you gather information about an object in this case a car convert it into data and store it in a yamo format which is a format that can be easily read by humans and one that can be easily parsed through by machines or programs or code so what's the difference between yellow and JSON well they're one and the same thing you can represent the same data in almost the same way in either of these formats while yellow uses indentation to organize data into lists and dictionaries Jason uses braces or curly brackets a set of properties defined with the same indentation in yellow form a dictionary whereas it is everything within a pair of curly brackets in jason well in yellow we use a dash to denote an item in a list in jason we use square brackets to define a list and each item within the list is separated by a comma you can easily convert data in llamo to Jason and Jason to llamo using an online converter or a simple program the website Jason - yeah Malcolm helps you to easily convert a JSON data into a llamo format or vice versa all programming languages have support for reading and writing in either of Jason or a llamo format for the remainder of this video we're going to work with data in JSON format now that doesn't mean that this is not applicable to yellow as we just saw data in yellow can be easily converted to a JSON format with a converter so let's get into the topic of today's discussion JSON path JSON path is a query language that can help you parse data represented in a JSON or yema format just like query languages in popular database software's like SQL for example if you are given a table of data you could run a query against it to one extract only certain fields like color and price of cars to extract certain rows from it like all information about a blue car or three extract certain fields of certain rows like the price of a blue colored car so for any given data you apply a query and you get a result which is a subset of that data similarly in the JSON World JSON path is a query language that when applied to a given JSON data set gets you results that are subset of that data for example in this data that represents the color and price of a car use the query car to get the color and price of it say for example we have information about a bus as well to retrieve details about the bus use the query bus what if you only want specific fields from within each of this say for example the color of the car use the query car dot color in that case the dot notation in the query helps you select a particular field within a dictionary car is a dictionary and bus is another dictionary to get the price of the bus used the query bus dot price let us suppose the car and bus are encapsulated within a dictionary named vehicles now how would our queries need to change to get the same results as before we now have a parent dictionary vehicles then the car and bus which are child dictionaries and then color and price which are properties of the current bus we would now say y equals dark car to get the car details vehicles dot bus to get the bus details vehicles dot card or color to get the car's color and vehicles dot bus dot price to get the buses price so that's how the dot notation is used to extract properties of dictionaries and dictionaries of dictionaries in JSON data however there is something still missing if you try to use these queries now it won't work let's go back to our previous data the one without the vehicles in it our JSON document has car and bus in it as you can see they are encapsulated within a pair of curly brackets and as we said before anything within the pair of curly brackets is a dictionary so car and bus are two properties of a dictionary or two dictionaries within a dictionary but what dictionary what is the dictionaries name the top-level dictionary which has no name is known as the root element of a JSON document the root element is denoted by a dollar we had vehicles earlier but we don't have anything now so we will remove vehicles from our query and use dollar to denote the root element and that's the right way to form a JSON path query a query created for a JSON document with a dictionary at its root should start with a dollar like this dollar car dollar US dollar dot card or color dollar bastard price etc going back to the data with vehicles in it vehicles is now a dictionary within the route dictionary the queries should now be dollar dot vehicle store car dollar dot vehicle start bus door vehicles dot car dot color and dollar dot vehicles dot bus dot price great but there is one more thing that I haven't mentioned the results you see here on the right well that is what you would expect but that is not exactly how you get it all results of a JSON path query are encapsulated within an array so when you run a query that is what you get the same result that you expect but encapsulated within a pair of square brackets so just remember that any output of the JSON path query is available to you within a pair of square brackets let us now look at lists or arrays here I have a list of different types of vehicles like car bus truck and bike as you can see there are no curly brackets so there are no dictionaries this is a simple list of names of different vehicles the root element in this JSON document is an array denoted by the square brackets how do we get the first element in this list to get a particular element from a list use these square brackets in your query and specify the position of the item you want inside it the indexes start at 0 so remember that the first element is at 0 the second one is at 1 the third is at 2 etc and of course always remember to start with the dollar symbol for the root element so to get the first element in my list I say dollar of zero to get the fourth element I would say dollar of three if I want the first and the fourth element I could do zero comma three within the square brackets like this let us now look at dictionaries and lists here I have a data of a car its properties color price and wheels wheels is a list that has four items in it each one being in dictionary say for instance we have a goal to develop a query to retrieve the model of the second wheel of the car as always our query starts with a dollar symbol for the root element the root element of the object is a dictionary denoted by the curly braces so we know our query has to start with a dot following the dollar symbol the dot is for the dictionary within the root dictionary we have the car dictionary so that's next in our query within the car we have wheels in the current state the query would return an array of all the wheels but that's not what we want we just want the second wheel so how do we get the second element in the array we use the square brackets and specify the position of the item within the array the second element is at position one as the index starts at zero note that we did not use the dot here as wheels is not a dictionary it is an array we now have the second wheel details but that's still too much information for us and that's not what we want we just want the model of the second wheel the detail we have is a dictionary and you can get its model by adding dot model to our query finally let us look at applying some basic criteria or conditions to our query so why do you need criteria or conditions on the first place here I have a set of data which is basically a bunch of numbers what if we want to query numbers based on certain criteria such as list all numbers greater than 40 how do we do that well we start with the dollar symbol for root element and since the root element is an array we use square brackets within the square brackets earlier we were able to just give the position of items the list but in this case there could be thousands of numbers in the list we want to define a criteria where we say get me numbers that are greater than 40 so check if each item in the array is greater than 40 and if it is return that number to me and that has to go within these square brackets now of course it's not going to be this were both so the check if part can be replaced by a question mark followed by a pair of brackets this is used to specify a criteria or filter within the brackets we say check whether each item in the list is greater than 40 from this the phrase each item in the list can be replaced by the @ symbol the @ symbol in a criteria means each item in the list so our query is finally dollar followed by these square brackets within the square brackets we use the question mark to specify the criteria the criteria is always defined within the curved bracket and in this case the criteria is at greater than 40 meaning items greater than 40 similarly you can use other operators like at equals 40 at naught equals 40 or at in a set of numbers like 40 43 or 45 so that would return all the numbers that are either forty or forty three or 45 and add not in a set of numbers so that will return all those numbers that are not forty forty three or forty five so those are a few operators of the many available with JSON path so going back to this example where we have a car with four wheels you're now asked to find out the model of the rare right wheel of the car now looking at this data you could simply find the model of the third wheel because by looking at it you know the rare right is the third item in the list so you write a query to pull the model of the third wheel but this may not be true all the time maybe for the next car the person who was responsible for entering this information into the database followed a different order the rear right wheel is now the second in the list so our query now returns the model of the front left wheel which is not desired to our query no longer works for us for our query to work with data entered in any order we can use a criteria to identify the wheel which has the location property is set to rare right so instead of hard-coding the position of the wheel we replace it with a criteria we first add the question mark with brackets to specify a criteria and then inside that we say check if each item has the location property equal to rare right and there we have our criteria well that's a quick introduction to JSON path header words the practice exercises section and try to practice what you learned for this head over to the URL code cloud comm slash P slash chase on - path - quiz it will open up a quiz portal interface on left top side you have your questions below that you have the space to type in your answers in the lower half you have two sections the one on the left is your source data and the one on the right is the expected output your goal is to create a JSON path query that when applied to the source data gets you the expected output you can start typing in your answer in the space provided as you type it in your query is automatically applied to the source data and the result can be seen in the section in the top right corner of the screen this will help you compare the output of your query to the expected output as soon as you develop a query that gets the expected output the question is marked as complete successfully and you can move on to the next question by clicking on the next question button you can try different queries as many times as you want until you get the right answer you will be able to see the result of the query instantaneously in the section in the top right corner in case you're not able to figure it out click on the show solution button and it will show you the right answer well good luck with the practice test if you're interested watch me solve the test in my next video for more information about JSON path check out github.com slash JSON path for a full documentation if you'd like more advanced use cases to be covered please leave a comment below and I will try to get it done well thank you so much for watching and don't forget to subscribe to my channel for more videos like this goodbye [Music] you you hello and welcome to this demo my name is moonshot mohamed and this is a continuation of the earlier JSON path video where we got introduced to some of the basics of JSON path in this video we will get introduced to some of the advanced options available while working with JSON path queries let us use the same example that we used before in this dictionary we have two child dictionaries car and bus and each of them has two properties color and price to get the color of the car we use the query dollar dot car dot color and to get the color of the bus we say dollar dot bus dark color what if we want to get all different colors for example someone were to ask you to retrieve all the different colors of all the different vehicles available in your store you would say instead of saying car or bus you could replace it with a star or wild card meaning any so your query would be dollar dot star dot color similarly to get all prices you would say dollar dot star dot price so a star wild card within a dictionary means all or any property of a dictionary let us look at it in a list in this case we have a list of wheels each wheel is a dictionary that has a model and location so this is a list of dictionaries we know that to get the model of the first wheel we would use the query dollar of 0.2 get the fourth wheel we would say dollar of three dot model but what if we want the models of all the wheels instead of specifying a position within the list you could say star to say all wheels within an array or a list using the wildcard star means all items in the list let us try to mix the two now I have a dictionary of car and bus and each of them have information about two wheels I have removed the details about the other two wheels and their location to save space on the screen the JSON document at a top level is a dictionary with two child dictionaries car in bus within the car and bus dictionary we have wheels which is a list with information about two dictionaries each containing model to get the model of the first wheel of the car we say dollar car dot wheels of 0 dot model to get the model of all the wheels of the car we say dollar car dot wheels of star dot model we replace the wheels position in the array with the star to mean all items in the list similarly to get the model of all wheels of the bus we say dollar dot buzz dot wheels of star dot model to get the model of all the wheels in both car and bus we replace the car and bus with a star to mean any we say dollar dot star dot wheels of star dot model so if you're required to use a wild card and if you're not sure I would recommend following a step-by-step approach as you can see here if you were asked to find models of all the wheels in all the way calls you might have found it difficult to come up with this query directly the first time so try to follow a step-by-step approach where you solve one portion of it each time until you can finally put it all together as we did here we first tried to find the model of the wheels of the car than the bus and then we were able to put them together by using the wild card well that's it for now head over to the practice test and practice working with wild cards [Music] you you hello and welcome to this demo my name is Michelle monomyth and this is the continuation of the series of videos on JSON path in this video we will get introduced to some of the advanced options available while working with JSON path queries specifically lists we learned earlier about lists here is a list of top 10 brand names on Forbes to get the first element we say dollar of 0 to get the fourth element we say dollar of 3 and to get the first and the fourth element we say dollar of 0 comma 3 let us look through some additional options what if you want to get all names from the first to the fourth element for that you can use the semicolon to say 0 to 3 you will use the query 0 semicolon 3 but note that when you say first to the fourth element it does not include the fourth element itself it means from the first up to the fort not including the fourth element itself if you want to include the fourth element as well you should say 0 to 4 we now have a query that gets the first eight elements from the list what if we want every other item for example say we want Apple Microsoft Facebook and Samsung you can use the step option by adding another semicolon and by specifying the number of hops to take after each item a step value of 2 means increment the counter twice before fetching the next item in other words it means skip or hope over one item what if you want the last item on the list we know that the items in the list have indexes starting 0 in this case for 10 items the indexes are 0 to 9 to get the last item which happens to be my Donald's we could say dollar of nine but what if the list was smaller or larger then this query will not work so how do we develop a query that always returns the last item in a list even if the number of items in the array changes just like how you have indexes starting at zero from the first item in a list you also have indexes start at minus one from the last item in the list the last item is always at minus one no matter how many elements are in the list so you always get the last item you could say dollar of minus one you see that's how I'd expect it to work but that does not work in all of JSON path implementations for certain implementations of JSON path you must specify this in the start and end format with a semicolon so you should say dollar of minus 1 to 0 this means start from the last element and all the way to the end you can also simply leave out to 0 and it would assume it to be 0 similarly to get the last three elements you could say dollar of minus 3 to 0 or leave it blank well that's it for this lecture head over to the practice tests and practice working with lists in JSON path [Music] you hello and welcome to this demo my name is Michelle monomyth and in this video we will look at how to use JSON path in kubernetes with the cube control utility we will start by discussing why you might want to use JSON path in the first place we will then look at viewing and interpreting group control output in JSON format post which we look at the different steps involved in using JSON path with the cube control utility we then look at several JSON path examples and then go through loops before finally looking at custom columns and sort functionality of cube control once then you will go through a set of practice tests where you get to practice what you learn through some fun and challenging exercises so let's get started and before we get started remember to subscribe to my channel for more videos like this now before you begin you must know how to work with JSON path so that's a prerequisite if you have never worked with JSON path queries before check out the videos and practice tests on that first they are available for free on my youtube or on the code cloud website if you have worked with it already let's continue also it's good to first practice JSON path on kubernetes dataset so that you know how to navigate through it before actually working on JSON path with Cube control utility we also have a set of exercises on working with JSON path on kubernetes data set so make sure you go through that as well in this video our focus will be on how to use JSON path queries with the cube control utility so why it isn't path are on the first place when you're working with production environments for kubernetes you will need to view information about hundreds of nodes and thousands of objects like deployments pods and replica sets and services and secrets etc and you will be using cube control utility to view information about these objects you will often have requirements where you will need to print summary of different states about different resources you will want to view specific fields of all resources query data about the resources based on different criteria etc viewing such information by going through thousands of these resources is an overwhelming task which is why Cube control supports a JSON path option that makes filtering data across large datasets using complex criteria and easy task but let's take a step back and understand how the cube control utility works we know that the cube control utility is the kubernetes zeolite use it for reading and writing kubernetes objects every time you run a cube control command it interacts with the kubernetes api through the cube api server the cube api server speaks the json language so it returns the requested information in a JSON format the cube control utility on receiving the information in a JSON format converts it into a human readable format and printed out to our screen during that process a lot of information that came in in the JSON format is hidden in an effort to make the output readable by showing only the necessary details now if you'd like to see additional details you could use the - Oh wide option with the cube control get command this prints additional details but again this is not complete there are still a lot more details that are not part of this output for example the resource capacity available on on these nodes and the taint set on the nodes the conditions of the nodes the hardware architecture the images available in these nodes etc but you can see them if you run the cube control describe command but what if you want to see it like a report for example say I'd like to see the nodes and their CPU counts in a tabular format like this or the list of nodes and the taint set on them the architecture or print the list of pods and the images they use none of the built-in commands can give me these in this format that's where Jason path queries can help with JSON path queries you can filter and format the output of a command as you like and that's what we will see in this lecture and in the practice exercises that follow this video in order to get started with Jason path and keep control you must follow these four steps first you need to know the command that will give you the required information in the raw format for example if you need information regarding nodes then you must use d cube control get nodes command if you need information regarding pods you must use the cube control get pods command we have seen a lot of these commands throughout this course so that should be easy once you identify the command inspect its output in JSON format for this add the option - OH JSON to the command it will print the output in a JSON format this is the same format of documents you worked with during the JSON practice tests on kubernetes objects that I mentioned in the beginning of this video the next step is to look through the structure of the JSON document and form the JSON path query that will retrieve the required information for you for example to get the image you would use the query dot items of 0.8 the containers of 0 dot H and finally use the query you developed with the same cube control command to do that use the - Oh Jason path option and pass in the same decent path query that you just developed remember you must encapsulate the Jason path query within a pair of single coats and curly braces like this we now have our cube control command with the JSON path query so if you are a beginner to JSON path and to this kind of logic then I would recommend strictly following this approach where you first view the JSON version of the output copy the output to a JSON path query evaluator like JSON calm then play around with it until you come up with the right JSON path query and then move that query to the cube control command with that you should be able to come up with JSON path queries to pull useful information like the below a decent path query of table items of star dot metadata name with the cube control cat nodes command gives us the names of the notes in the cluster a query of items dot status dot node in for dot architecture returns me the hardware architecture of the notes and Status dot capacity dot cpu returns me the count of CPUs on these notes finally I can merge these queries together into a single command for example I could use the first query to get the node names and add the query I used to get the CPU count to it and that gets me both the results in a single command that's good and I can get any information from the JSON output as long as I can figure out the right JSON path queries for it but that's still not pretty is it that's not how I wanted to look so let's look at some of the predefined and formatting options in this I have two types of information in the same line I can insert a new line by adding the new line parameter in between the two queries like this the /n is for new line and slash t is for tab we will now look at loops using ranges this may be a bit advanced for some users so don't worry if you don't fully get it from a certification and exam standpoint we've covered what we need but I'd like to cover this as well before before we wind up with the queries we have built so far we were able to get different sets of information like names of nodes the CPU counts etc in this format but this is not what we really want we wanted to look like this one on the right the node names in one column and CPU counts in the other this is where we use loops to iterate through items in a list and print properties of each item we will focus on the JSON query part alone for now when I run the cube control get nodes command I get a list of items each item representing a node so I want to be able to say for each item or a node print the node name then inserted tab as a separator and then print the CPU count followed by a new line character this will get me the results in the format I want to specify the for each statement use the range keyword range items of star means for each item to print the node name use the same query we built earlier dot metadata dot name then to add a tab at the slash T keyword then to print CPU count use the query dot status capacity dot CPU and then to add a new line and the slash and keyword friendly end the loop using the end keyboard merge it all in into a single line and pass it as a parameter to the JSON path option of the cube control command you can also use JSON path for printing custom columns with cube control at times this is an easier approach when compared to using the loop method let's take the previous example where we use the JSON path option to print node names and CPU capacities what we really want is to print them as separate columns one column with the node name and another with the CPU count instead of using the JSON path option directly with the command you could use the custom columns option of the cube control command the custom columns option takes a set of column name and JSON path options for example for the first column the column name is known the JSON path corresponding to the node name is dot metadata dot name note that you must exclude the items section of the query as the custom columns assumes the query is for each item in the list this prints the first column with the given column name and its data similarly you can add additional columns by adding additional column and decent path pairs separated by a coma so we add the next column named CPU with this data at dot stated capacity dot CPU much easier and cleaner again for this too I would recommend first coming up with the JSON path queries for each column and then putting them together in the command finally JSON path can also be used while sorting objects by specifying the sort by option the queue control command comes with a sort by option where you can sort the output based on the value of a property from the json formatted properties of each item when you run the queue control get nodes command specify the JSON path query we developed earlier as these in the sort by option to sort based on name per CPU count well that's it for this lecture head over to the practice test and practice working with JSON path and some advanced cube control commands and before you go don't forget to subscribe to my channel for more videos like this [Music] you

Channel: Geek's Lesson

Views: 15,059

Rating: 4.989717 out of 5

Keywords: JSON tutorial, JSON tutorial for beginners, JSON, YAML, json, yaml, YAML tutorial, yaml tutorial for beginners, yaml tutorial

Id: XXZ6ne7-bQQ

Length: 47min 29sec (2849 seconds)

Published: Tue Aug 20 2019