OSM-MR#10 Hackfest - HD2.6 Intro to Juju Relations

uh here i'm going to to explain the basic and what the basics of youtube relations uh an introduction to to this topic uh we will follow a similar approach as in the previous session i will um remember that the performance monitor overview okay the scenario that we are building explain the basics of the youtube relations we will actually add a node exporter to a squid and we will [Music] relate that to prometheus then we will i will show the the descriptors uh onboard the packages to osm and deploy the network service instance execute actions and validate it but it will do a if we have time and you have time we'll do a couple of interesting things that i i think you'll like okay so let's get started the performance monitor that we are building is uh it consists of a script as a web proxy then in this session we are going to add the node exporter to it and that node exporter will expose metrics to prometheus and those metrics will be shown in grafana in some dashboards okay what we have deployed until now is a squid only switch but we will uh add the next of it in this session but especially what we will be focusing on is on adding the node exporter and the relation to prometheus but the prometheus and rafana terms i am providing those to you those you don't have to do them in fact there were the in the previous hackfest we were building those okay but we want because of a of a matter of time we cannot cover everything on this session but we will cover it two very very important things and also we will focus on showing some metrics in grafana to see that everything is working end-to-end okay so let's start with the basic of the basics of u relations so osm orchestrates a network functions both physical virtual and containerized as we previously said but we need to understand the the challenge of integration and it's not enough deploying a network functions and that are that work in standalone mode we need to be able to to integrate like in the example that we are proposing we need to integrate a prometheus a graph in in this case that relation is already done i did it so those are related and integrated but then we are adding a new piece of software that is the node exporter which will export metrics from the bot and we need to somehow integrate prometheus um on the node exporter and this is a challenge because we might be we might want to to integrate a different network functions of different of different natures we may want to integrate a pnf with the vnf and with the knf so having the same way of doing the integrations with charms makes it very very very useful and powerful so the integrations are done at operator level so what this means is that we have the operator part and we have the workload the operations so also the the integrations will have some scripts associated to those integrations and they will uh handle the integrations basically the operators and they will do the changes needed in the workload when executing the scripts so we will see that the more clearly in the in the example that we are going to implement so basically relations are integrations between operators so in this case we have uh two operators okay that are managing two different workloads and each one of the operators has installed scripts upgraded scripts scaling a configuration and things like that okay so just to put some some names there and let's think that the first operator is for mysql and then the other character is for wordpress okay so my sequel a charm or operator um provides a database and that is expressed with an endpoint so an endpoint says that it provides the database so in the other uh operator in the wordpress operator a we have a uh another endpoint and it says that it requires a database so as those two endpoints are matching endpoints can be related okay and this means that through that relation the operators can send data okay to configure themselves so uh yeah and basically these endpoints have a type or interface okay so uh if the interface is my sequel then both endpoints needs need to be my sequel in order to form the relation okay just a a quick um um a stop here and because i saw one one question uh well nd guys saying that i'm not a 100 or a 100 sure but i think they are deployed in the same pot yeah just to clarify that uh right now the current approach of a of the the charms the grenades charms you will get two different parts so one for the operator and another one for the workload okay and we are in progress of in juju in adding the support for a kubernetes sidecar charge which will have the operator as a sidecar in the in the workload and that will give us a many many advantages because we are writing the same so we'll have the code if running right inside of the application and that will be very very powerful um so so as soon as that is a released in in juju i think it will land into you uh two nine and now we are with two eight and so when uu29 is released i think that is going to be there and we'll have that enabled in innova okay okay so now let's go back to the to the presentation i'm sorry for for the interruption okay so as i was saying in the the relations happen at the operator level and to those in this to those relations there will be some integration scripts associated so in this particular case the the wordpress charm the wordpress chart needs the the mysql so in these integration scripts my sequel will send the data that through the relation that wordpress wordpress needs to to connect it or to configure itself to connect to the to my sequel okay so this integration scripts will write into the data in the relation wordpress will have some other integration scripts and it will get the data through that relation and execute those integration scripts we will affect in the workload itself okay in the wordpress and it will configure the the wordpress that the configuration files in the right way to connect to my sql okay so that's more or less how it works okay let's let's continue that okay the the end point is uh used to connect a one application with another okay and two endpoints connected to each other form a relation and there are three types of um of endpoints requires provides and peers okay requires is for saying let's say that a charm can make use of services represented or provided by another charm but by another charms end point so this will be in the in the previous example this is the wordpress that requires a database then we have provides that it this means that a charm can provide a service through that endpoint okay and only so only it requires so a required endpoint can be related to a provides endpoint and that's a common sense so you cannot relate that provides with the provides and it requires with that requires and then there's one special a type of endpoints that is fears so peers is well i'm going to to uh to explain this in the in a couple of slides uh after this one okay uh but it's used to integrate units of the same application okay uh i will get deeper into these a bit later okay so a problem okay so a relation provides requires this is a perfect example we have my sql that is one application okay and we have wordpress that is another application wordpress requires and my sql provides okay and then we have a peer relation so imagine we have a charm that is for mysql so we can as we said we can scale my sequence so in this case we have only one application my sequel but we can scale it to three units okay that will form a cluster okay so in this case we have my sql zero one and two these are the units but there's only one cluster one my sequence okay so these endpoints here are appear relation so basically some data can be sent between the different units of of one application in order to make a a a cluster uh so for example in my sequel um i don't know the the details of this but when you form a cluster is typical that in one of the nodes or all of them it depends on the application you might need some information of the rest notes of the cluster okay so that data can be sent through this relation here with with the scripts the same way that that the other example worked okay um so interfaces interfaces are the the communication protocol over a relation in between applications so for example a very common example and the example that we will be using is http so in its in in this case we have one charm in one side that provides http and the other one that requires so the relation has some data okay that is basically a key value data and a one operator can write into this relation data and it can write in the case of then um the http the interface will define that you need to provide the id and the board okay and this operator will right here and here and then the other one knows that that the one that requires the http will know that where to look this information in the relation so that's why it's very very important that the um to have interfaces because the interfaces define where to put the data into the relation and where to get the data from okay and in this way we can a create a complex let's say examples okay and in this case i show here in my sql chart wordpress those are related with my sql endpoints and then apache here and we relate a wordpress to apache and also to an aha proxy and we can do a complex builds like like this one okay and again as we said before reality is mixed and makes a messianic and we may have operators in managing different kind of workloads so what this mean is that we will have the ability of doing integrations between different um charms that are in different substrates and managing different network functions of different natures okay and this is very very powerful and obviously this can be achieved by using chunks that are a universal way let's say of writing operators and yeah okay so after the theory and now it's more practical okay so the next question is add node exporter to a squid okay so basically this is the slide that i showed before and when explaining the kubernetes spot so basically in this example what we will have is we will have this we will have a bot with squid with a node exporter and i we will add that the relation to probabilities so and we will show first the descriptors and how to avoid them to osm uh we'll go a bit quicker this time because uh basically it's the same thing as before so they're referencing the unit bundle this is exactly the same as before then the digi bundle change changes because we have added grafana prometheus here and we have added these relations there okay a relation between as you can see there between prometheus and grafana and between script and prometheus okay okay these are the day two operations exactly the same as before and so it's the same okay so now er i will do okay and that's one issue in this it's like i will fix it and upload it again but we will do a a similar approach as before okay so i will be so don't execute anything yet i will explain now the commands that we will be doing and then i will show my screen go and execute all the commands and i'm going to update the path so that you also can can go ahead and unfollow okay okay and this time important this time i will be a so i preferred some packages for you okay that are that have everything ready but i'm also going to um to explain and show the changes needed in the squid charm in order to and to let's say and at the relation and at the node exporter the changes are minimal so it will be very very easy for you to follow if you feel confident if no just use the packages that i that i put there okay so as before in the in this folder we have and then the youtube relations packages so we will copy that not here but in another folder and we will upload the packages okay then for the deployment exactly the same command nothing changes and let's create them the same parameters then we will wait until it finishes until everything is in various state then we will execute an action exactly the same as before okay and we will validate it exactly the same as before but additionally what we will be doing is we will go ahead and we will add a one url for the cloud images of ubuntu we will get a cloud image a big file into the system okay through the proxy and then we will log into grafana and we will see a spike on the network traffic thanks to the metrics that we are getting from the node exporter so that way we can have a an end-to-end a vision of what's going on okay okay so let's start by a okay i'm going to check that i don't have i don't have that folder so [Music] let's go ahead and copy this here oh hmm okay all right okay so copy that to your local okay no sorry i i am missing one thing yeah okay copy that to your local uh into your home directory then go change the directory okay and if you do ls you will see some packages those packages are already there okay so already we can go ahead and just upload them upload them yeah and um i'm going to do that but also i'm going to show okay okay so i'm uploading this and i'm uploading the network service also okay but here i will um make a a stop and i will show how to edit the the the one that the folder that is incomplete the cnf to uh to to show you how to implement the the the changes needed to add the node exporter and to other relation okay it will be very very very simple so you can go ahead and follow me but i would recommend you to use the ones the packages that that i have provided that way you will avoid some issues of while typing or maybe you can make a mistake and you won't be able to see everything that the whole picture so maybe it's not worth it but i will we will focus on that more on friday but anyway i'm going to go ahead and cd to the uh to the cnf okay so to the bnfd basically and now then we can see the juju bundles okay so in the youtube bundles if we edit them what we had before is only the squid one so if we remove all these comments okay we we will add the relations and we will add the graphana and prometheus charts okay so that's the first thing to do and then if you go to the charms directory so cd to the charms directory you will see okay that there are graphene operator prometheus operator and squid operator okay these are the charms that are built already okay but the source code is in ops okay so if i go to ops i will see the source code for the three of them okay i will focus only in the script so icd to a script and in the metadata i see that okay this should be commented out okay but basically this charm will provide a promiscuous target okay with interfaces http okay but basically it's a prometheus target okay so next step the source charm.pie this is the actual code that i showed yesterday okay for a charm okay let me show it so i'm going to make this really really really small okay so this that is like i don't know how many lines 100 lines of code more more or less this is what it takes to to to do a one kubernetes chart so it's really really simple and you get a lot from from from it okay so basically here i did some some functions to reload the squid configuration to apply the config to generate the allows the allowed urls config generate the conflict from the template so this is just python python code okay and a function to update the config which basically goes to the the functions accordingly and so on okay and then we have here the the constructor in which we observe events and events like start config change a url action and delete url action okay which are implemented here and so on okay so for the relation we only need to do this we we only need to remove four lines nothing else four lines the first one is to observe the prometheus target relation joint okay so when a charm joins the relation we will execute this function that is published prometheus target info which is defined here and what what this does is it will write in the host and part of the relation the host is the i will put the application name okay because usually in a service with the name of the application is created so the the prometheus will be able to access it through that name and then the port will be this one okay so we are writing into the event relation data in the unit relation in the unit data the a host okay just [Music] three lines four lines no no no more okay and then to add the the node exporter there's one i will explain this more deeply on friday so don't don't be afraid but basically we the the operator port will have one port spec that contains some information that will be sent to a to kubernetes in order to create the pots and so on okay so the first so basically the bot spec has one key that these containers and this is the container for the squid okay but we also can add more containers to this okay so if we uncomment this what we are adding is we are adding another container which name is exporter the image is promise slash node exporter that is a docker image provided by prometheus and is the node exporter okay and then the ports is basically this port and the name of the port and the protocol http which is the default that's it those are all the changes needed to to do the relation for the integration to prometheus and to add the node exporter to this chart not nothing else okay so i will save that and okay for those of you who want to build it i created a okay in the in the root of the folder i have created a build charms and that a script that you can execute and and it will build them for you and so on but but we're gonna skip that for now okay okay so let's go back okay and i have uploaded already the packages okay so i'm going to create a um a hd26 column okay in in the spreadsheet that says a load packages okay so i think i have uploaded them already yeah i have uploaded myself okay so just put the yes there once you have done that okay okay and now the next step same as before is instantiate okay so for instantiating we need to do execute this command okay this command that i will paste here and we need to wait until it gets deployed yes um if you get the conflict is because you you need to remove the old ones uh probably i should have a put different names to the to the descriptors and that way you didn't need to to remove them and sorry sorry about that and maybe for next hack fest i will i will fix that okay so okay let me copy this okay i'm creating columns in the spreadsheet okay uh that are basically the same as before but don't put guesses yet because in the ones that i haven't explained or haven't done myself but don't worry in fact i will be doing that on friday uh step by step with me oops okay i don't know if well my mic is stopped stopped and can you hear me now we can still hear you okay okay well i was just saying that we will do that step by step with no rush on friday so don't worry you you will learn a lot of of that um on friday okay so my network service is ready i see that seven of you have well have executed already the action uh okay so the action that we that i wanted to execute was different from from the one that is in in the um let's say in the slide so what i wanted to do let me see if i can [Music] copy and paste okay so in the url i want to um to change it to let me find it um okay i want to put this url okay oops i think i forgot something okay yeah so and let me go and paste the action here okay so copy that action but let me change the the domain in the path okay so we are going to enable cloud images.ubuntu.com because what we're going to do is we're going to to get one image from ubuntu okay let's hope that we don't have issues with the networking i don't think so but let's see okay so i have a i have executed the action i'm going to do this so so operations list of the web proxy metrics in network service instance and i see that the action is executed correctly okay to validate it what we are going to do is a similar thing as we did before so i'm going to go ahead copy so copy this um [Music] one sec so so i'm executing these two commands that you have already in the back okay and if i do an echo of the squid ip oh i got two [Music] i launch it twice i don't know i was just making that up oh sorry sorry sorry yeah yeah that command won't work don't worry yeah because here in this case we have graffana load balancer id and squid okay so the the command wasn't that generic okay so let me okay let me do one thing sorry sorry about that okay so the squid id we can get that doing a grab of load balancer and also a graph of squid okay and also we can find the graph ib doing a graph of raphael okay and now we can do echo of squid id and ib okay let me paste this here graphala and squid okay we can print them later okay i'm copying everything in the path for you to have it okay so once we have that we can go ahead and execute this and curl to http no one sec mark is it okay if we do a curve to uh to get one um senior limits from all the tenants at the same time the uh yeah you should be able to yeah okay and and we will point that to that new okay uh just we don't want to to save it to save the image and take a space of the of the machine okay so if i execute that okay i see that i'm getting the image so i'm seeing that the that the action it was a correctly uh so it ran properly okay i'm going to put that there okay so once you finish uh downloading the the image you can uh put the yes in the action validator okay the action completed you can put a yes there in that column when you did or when you do a ns of list and see that the action completed and validated when you actually run the the last command in the path okay this one okay okay i i will wait until at least some more people have um completed the action okay and now what we will do is say uh so what we are missing basically is apart from deleting everything okay that when it's a bit late and if you want we can do that for you automatically and and you don't need to bother doing that so forget about that right now and what we are only missing is going to grafana logging in and creating a dashboard to see the metrics okay we are only missing that and in those metrics we will see that the network a traffic has um a spike during the time that we were downloading the image okay if we get the the the id from from the command that i showed before the graph id then we can log in here okay so log into http i don't know if it is https or not i don't think so i think it's a http only okay then the graffana ip and the board is a 3000 okay okay so email or username and password is admin admin so admin user is i mean um password is admin and then you can skip this and you don't need to set a up another password and to to create a dashboard we will go to the plus sign here in the left and dashboard okay so i'm going to put [Music] this instructions in the back go to plus icon in the left and then click dashboard click on click on dashboard okay and we will create a new panel okay so then create a new panel okay so here what we need to do is in the metrics part here we need to add one a so the metrics that we want to show okay that are um um gathered by a prometheus and we need to place to to make a query there okay so i'm going to go ahead and i think is this should be enough let me test it first and then i will i will put that in there okay i'm changing the refresh rate to five seconds and updating this and i'm also putting a configuring this to show the last okay the last 30 minutes okay so if i show the last 30 minutes i can see a spike here in the number of um of packets that were retrieved and and this basically shows that that the metrics are actually working and it's one thing that is pretty interesting is that the node exporter lives in one container and the squid leaves in a different container but as i explained before if one pot can have multiple containers and those containers will share the same storage and the same networking and the same things okay resources so this means that the node exporter container can get metrics from the network of the bot and even if we are using the the squid the other container to go through the internet the the node exporter port can get metrics anyway and that's pretty pretty interesting and well we can make some queries here okay i'm going to put a query this one okay and we can put more queries here i'm not an expert honestly and so i think we there was something like we see okay we see total maybe let me look it up really quickly okay receive bytes total okay so [Music] yeah the order of magnitude is too different i think and we cannot see those really well but what i'm going to just remove that packet one and show that the other okay no oh sorry yeah there we have it there we have it okay and so you cannot update the spreadsheets and express it once you have a and check the the metrics in grafana i saw that i see that 10 and 60 tova has already finished so congrats and also the note for dean i think frantisco javier moreno so congrats you too for finishing the the first okay now i see more people

Channel: Open Source MANO

Views: 62

Rating: 5 out of 5

Keywords: ETSI, NFV, OSM, Open Source MANO, Open Source, Juju, automation, network, orchestration

Id: 8mdtDO4pQno

Length: 47min 50sec (2870 seconds)

Published: Thu Apr 22 2021