Building a Serverless Geolocation Search API using GraphQL, Gerard Sans, Developer Advocate @ AWS

can you hear me oh good awesome well I'm really happy to be here in Paris even with all the trouble that I had to just come here I mean there's always some some strike going on in this city I hope we don't we don't get that in graphic well we all like Yui how we should use graphic oil and today I'm gonna talk about how I use graphic well to do application with geolocation enable and I've done geolocation for two years at the security company tracking devices and I'm going to use that to show you the demo that I built for this conference so let's start first with the most important bit which is my name and besides what you can think I'm not French I was I mean this is a very French name somehow but I'm not French I was born in Barcelona and my name pronounced in Catalan is Jarrod sands Jarrod sands okay oh you can use er ah son something like that no very good okay I mean this is going to be hard so I'm a developer advocate at AWS and you can see that I have a lot of time of myself in selfies with my cat I'm a cat person for all the cat lovers I've been doing full-time talks for the last four years so I've been pretty busy as you can see from this from this map there's still some areas which I haven't visited yet so I can still keep keep on going so I'm going to talk about bike shirring and i also seen that you also have here a bike scheme sharing in in Paris so you will be familiar with it I'm going to talk about the scheme running in London and this is the Santander bikes and they look like this so this is a picture of one of the dock stations and of course after a few years of using one of these programs to share bikes it starts becoming part of the culture and you can see people taking pictures with the bikes I'm not sure how many of you use these bikes people going crazy this must be something I don't understand I mean this is some kind of British celebration of hats and obviously you need to use a bike just to get in trouble and of course yeah if you go to a park in London there will be some squirrels also using the bike the bike somehow they are not paying for sure so some benefits of these transfer flexibility I mean this is a modern solution that we can see in many cities reduce traffic emissions pollution there's also a chance to reduce traffic congestion so this is all a nice health benefits also for the users besides the accidents of course and the sand on their bike cycles they they use these logos so this is just a small intro how this looks it started a few years back so now it's quite mature I would say there's 12,000 bikes being monitored by by the system and 778 stations and this is actually the use case that I'm going to develop in this talk so this is quite a lot of data and I'm going to hook into the API that is going to give me that information one thing that I want you to think about is we cannot access the whole data set I mean if we want to get the information for all of the stations we have we have a problem because we cannot load that information in a meaningful way or at least in that time that is meaningful for the user to be responsive so we need to solve these data issues and I'm going to use graphic well to do that this this bike sharing scheme was also named as the as the mayor in London and I'm just going to show you how his welcome in the city yeah this is what happens I think he takes this really well so this is a this is the application that I'm going to build London bikes and I spent some time building their logo I hope you enjoy it it is just one slide so I'm going to be using full stack cervelas and this is a new paradigm that it's coming and probably you have seen several s alone but full stack surveillance is just having access to the full stack so you can build several as applications and also have all the tooling supporting the front-end and why is that important well usually if you want to get into surveillance it will take you quite an effort but now that's that's becoming easier and easier and I'm going to show you a tool that will help you to get there to get to a full stack several solution and okay several s and it's okay I've even meant to show a dollar there but yeah so one thing that Severus brings is that you don't have to manage these servers so for example when we were seeing the last talk you need to manage this server I mean who wants to manage a server that's going to bring you trouble when it's in under pressure there's a lot of usage so several is just removes that from the equation we don't need to think about the stack we don't need to think about if it's using a hyphen or note or any other stack we can just build our application and forget about what's happening in there in the server server side the next thing is that you don't have to pay for evil usage this is really interesting because when you set up a server you probably will get into some kind of contract that will have to be fixed on the on the usage but if you're using several is you just don't think about these issues the servers will only be running when you your users are using the application if no user is using your application it's for free then another interesting fact is that it's auto scaling as the demand on your application increases but not only when it increases but also when it's not so busy so that's that's what it's also interesting from serverless I'm not sure if you are familiar with several so I'm doing a little bit of intro and then probably the most interesting is that we don't need to take care of these full tolerance and high availability this is not an easy problem if you are building your graph coil API and you are building that from a node express server and you are facing some high availability issues you probably don't have the knowledge to face those and when you are deploying a new product and you face that after the release that's going to be very hard on on your team so if you don't want to face any of these issues you can move into a server less and just forget about it so what I'm going to be using is AWS amplify and of course this is from my team I mean if there's some conflict of interest here but this the tooling I'm going to be using and this is helping you to build these several as applications with no much effort so how it works is a platform that supports native and web and that's going to be iOS Android and react native and this is a all the pieces that will be used so there's a framework supporting these platforms developer tools which will make the job pretty easy and of course it will give you access to all of these cloud services we are talking a to the US but actually if you want to use the same approach with any other cloud provider that's also interesting we are talking about a new paradigm so in the future in the next year in the next two years people will be using several as more and more and you don't want to miss that on the center we will use CLI and this is this is pretty much standard solution these days so it will just help us to run some commands and just do the setup for us ok in AWS amplify we name different features by category and I'm going to show you some of them the way it works is you just run a CLI command amplify at and then you can use any of these categories and it will ask of all any components and any dependencies that you need on the client and also on the cloud so that will make the job pretty easy today I'm going to be developing using the API category so I would just ask of all my app in any of the frameworks that you may be using and just run amplify ad that will do the changes and also set up the services on the cloud so that's that's pretty and that's pretty cool for graph coil it's also able to create from the schema which is kind of a standard option it will create my API is on the client but also will create my server side which with supporting data tables which in AWS is dynamodb ok what what a small intro okay there's more things sorry about this and there's also machine learning of course we are talking about a lot of AWS services there's more than 150 services we are kind of integrating all of those in AWS amplifier and now we also have access just just a reason release we got like these five different services that you can use as well I'm not using those but I just wanted to show you okay so that's the reaction I get usually from people no not much reactions here oh my god you are completely destroyed bring some coffee for these people okay so let's talk about some details London unified API so London Transport for London created unified API to get access to all of the different transport options and one of them of course is the bike and descent on their bikes scheme so this is a Transport for London if you have been in London you are probably familiar with this with its logo and we are going to focus into these three different api's the first one is going to give us all of the information from all of the dock stations and we have seen that there are 778 which is accurate because I have been checking it and then the second one it's going to give us the information for just one station but the problem is that we need to know the ID for that station there's also this third API is to do a search using the name of the station but I don't think anybody knows the names of the station unless you are a neighbor of that dock station so it's not that useful so these are the three API sir I'm going to focus mostly on the first tool and the first one is going to give me the information of the coordinates the geolocation coordinates so I can put these stations in the map and then obviously when I interact with any these stations what I want to know is what's the life data for the number of facts that I can use so this looks more or less like this okay and this is a the response of course this is a response for the whole data set of the 778 stations so this is some API that we can manage I mean I use that API to get all of the coordinates but I cannot use it like on on a real on a real-time basis it's just too big of a data set but it looks like this we have some information identifying the talk station and then we have the number of bikes this is a in a kind of funky format which it comes with a key and value but then it's it's kind of the normalized and then we have the coordinates we are going to cover the coordinates if you are not familiar with these numbers so don't worry for the API that gets the information from one dock station it looks like this it just requires the name the ID of the station so that's something that you need to take into account and that's basically the same information but just for one and that's the one that I'm going to be using first okay so yeah let's do this let's move on so how I'm going to deal with the information of all the data all the daugher station is I'm going to gather the information just for the coordinates and then I'm going to put those coordinates in a layer of my of my map so then the user can see where these stations are with no real-time information and then when there's some interaction I can bring that number from the live API so that's the plan I need to do few data transformations and I already show you the read the result from that call backs point and that's not going to work when you are using geolocation data said you need to transform that information into some format that other tools using geolocation can can manage and this is geo Jason so geo Jason is going to provide me that that data said but in a in an open format and for that to happen I need to create features so every dock station it it will become a feature in a geo JSON format so let's see how that looks like so our feature looks like this so this is a kind of schema for that feature type and we are transforming the coordinates into a geometry and that geometry it's a point and we can facilitate the coordinates just by passing an array so that's a little bit of data manipulation that I need to do and any metadata I need to put into the properties entry so I just changed some of the fields I do this so I can then use that information on my map and that gives me something like this these are all of the stations and this is an online tool it just shows me that my joj zone is well-formed so I can I can check it out of course that's not something I can use in my app but it just it just shows me that it's working so let's let's introduce coordinates so when you are using coordinates we are just we just want to point geo location in the map so we can share that information and then show it displayed in our map so one thing that we are going to use is longitude and latitude coordinates and as you can see the center of this system is somehow south of Africa is like in the middle of nowhere but somehow is very is very close to Europe I don't know who decided because there's no reference in the wall but these this is quite Occidental view of the wall if you for some reason you go to Australia or Asia they ship the map so you go to a coffee place they would shift the map so the center will be Australia for example so this is a little bit funny but yeah this is the the coordinates that we are going to be using and for that is going to for London is going to give us something around this longitude and latitude so this is one of the stations that I'm going to be showing so it's a little bit on the negative side from from zero for the longitude and it's more or less between 50 and 60 on the latitude of course if you have this then you can translate more or less from a coordinates to a map and we are going to use this format which is from the Geo JSON which is Longy longitude and then latitude in that in that order so if you see a coordinate it will be longitude latitude very good on the client I decided to use map box and my main my main criteria was the easiest settup the other option is Google Maps I guess but Google Maps requires to enter your credit card details and the set up is a little bit more complex of course Google Maps have more features but I decided to use map box because it was easier so then anyone that wants to play with it it will be easier for them as well so this few things I need to do by using map box the first thing is map box requires a source and that will be all of these coordinates from the different dog stations and the other thing that map box uses these layers so every time that I show some feature on the map it will be on a layer and then the source will have this data from a Geo JSON file and this is the result so this is now in my client and I can I can show these these features on the map okay so this is the first stage we got all of the dog stations in in a map that I can show to the user and now we are going to build the firs interaction the first interaction is going to be point to a dog station and get the real-time data but so far we are good we add the captains of the geolocation whoa so how we are going to do this REST API integration we are going to use the schema that it's going to create all of the api's on the on the server side and to do that I'm going to use these type definition and if you see there's this ad model graph coil transform which is not part of the standard this comes from AWS app sync and that's going to create all of the api's and also create filters is also going to create the input types for the different crude operations is going to do all of that work which some of the tools that we can see in graphic well are also offering but it's also going to create the data tables supporting those which is pretty awesome here what I did to integrate with the REST API from Transport for London is I just added a field and then this field is going to do HTTP requests and getting that information live so that's the only change that I had to do and this on my schema on the way it works with AWS app Singh is that I can access my resolvers and I have two points where I can change both the request and the response for graph qual query so I'm going to use that resolver for that bike point bikes field and I'm going to change that to do the HTTP request that I require so that's going to do the translation that translate into a data source in AWS app saying and it's going to do that call that we saw on the api's okay the context is the context of the graph quad query so if you are familiar with that you will you will make you will join the dots the next thing that I want to use is the response so the response in this case is going to be the response from that HTTP request and that result is going to check if there is a 200 status code and then I'm going to just dig it into the data and bring back that value back to the aquarium of course this is using graph quayle resolvers infrastructure and then if there is any mistake is going to return no maybe you are not familiar this is a BTL template it's pretty common in Java and it's just a template engine that you can you can add your logic in there okay so that's that's going to return that value live not bad I also added some animations here for good measure let's enjoy this for a moment this is CSS I don't know if you use CSS these days very good it's just a small break so let's add the search this is more the most interesting bit and what happens is okay we have the coordinates and we can check let's do before I go into the search let's do let's see the feature life so we have the map with all the information of the different dog stations and what we are what we are doing now is okay I can point now to one of these dog stations I can interact with it and it will just make the call because now I know the idea and it will just get the number of bikes that are available this is life this is coming from the Transport for London API so this is the feature that I built so far if I query another station it will just do that for you that we saw I'm just interacting with the map that's that's the only thing and the next thing is the search so for the search we have a problem now and if you think about what will be the next thing to to do is probably you will say like ok so I want to I want to help the user but I don't want to get the 778 dog stations that's just too much and then I need to calculate because probably I just want to take like the 500 meters around the location of the user how are you going to do that and I want you to think about solving that problem so one thing is you need to know where the user is which ok I can do it I have this control this is from map box and it's going to find my location work of course the location is in Paris so this is this is not good but of course when you are using this in in London it will just try to find try to find the stations around you so this is good I mean is not bad the only thing is that it we are not we are not in London now so let me they activate that and go back I use this control just to get me back to London look at this this is probably the best feature whoa this animation is very smooth I mean we are using WebGL map box is really good because it's using WebGL and it's giving you that smooth animation but here I have this point like a fake location and I can find the bags around that point but the calculations are pretty intense if you have done that problem and try to solve it with JavaScript or any other language you know that it's going to take quite a bit of time and also if you are doing that for the 778 coordinates that's not going to work unfortunately not on the client at least so let's see how we can fix that this is just a technical problem so let's see how we can do it with AWS app saying what we can do in in our schema is we can add this at searchable and by adding that graphic well directive there I can do I can indicate to aw as absent to add elasticsearch support for this type and the next thing that I'm going to do is I also need to add the support for the location so this is going to be the beat here and this is because elasticsearch requires a field with this latitude and longitude so I'm going to add that to support elasticsearch and I'm going to add a new query which I'm going to use the location where the user is or any location that will serve as the center then I use meters so I will use like in my application I use 500 meters and then I also want to limit how much results I get back from that query I don't want to get like all the all the data set so that's going to help me to deal with that ok so we need to do a little bit more data transformation I need to run this query into my graph coil API this is the backs nearby and that's going to go into elasticsearch which will have to get all of these docker coordinates index it and then when I run a search it will be able to run these calculations for me so let's see how that works the first thing that I have to do is I need to create an index that is going to give me that calculations are pretty pretty fast and the way you do it is you create property in your index that will be of type Jo point that will allow me to do these searches so let's see how how that will work once once I have the index created and just by adding that at searchable I now have the option to create new bag points and every time that I create one new entry for that type is going to add that information into my elasticsearch index this is happening automatically so the moment I create by using a mutation is going to sync that with my index which is pretty awesome then the next thing that I want to show you is how a search will look like and this is a search on elastic search and here I do a query which should match all of the documents in my index and this is when it gets interesting I'm using a filter and then I'm translating my query into a search into a elastic search and I'm using also this distance I'm going to explain a little bit more but this arc is going to give me the distance as if the earth was a sphere which is an approximation of the actual situation here to calculate distances and then of course I also want to sort the results I mean if I get the results scramble I mean that one that wouldn't help and what I want is that the most closed to my location that will be first so this is how you do it and all of these calculations will be done on the elastic search index I mean if you have used an index on the on a database you know that this is going to be really really fast let me show you let me show you how fast that is so I have this environment this is Cubana and it shows me an environment where I can run these queries and I'm going to take a big query I'm going to just include everything okay it took six milliseconds let's see again maybe it's an error okay no it took ten it was a little bit of this time but it's really really fast of course I'm not going to get this type of performance because I also need to do the HTTP request to get the live data but at least the geolocation data which is what stations are closest to me that's going to be very far very fast look at this there's so many possibilities oh my god so how we do the distance calculations this is not easy I mean if you want to do this with Python or if you want to do it with JavaScript I mean try to do that for the 778 coordinates good luck of course this is what I'm using so my my solution with the elasticsearch is giving me all of that in 10 milliseconds I don't know if you have realized about the quality of my resolution and this is exactly how it looks in Java and this is the the actual implementation from elasticsearch I dig a little bit because this is open source I just dig in a little bit in the source code and this is actual the actual implementation you are seeing here good luck if you want to implement this in JavaScript for 778 coordinates and you also need to request that with with a call to the API I mean that's going to take forever very good we are in a good Jam here ok so I demonstrate that let's see in a very quick demonstration how that looks and how is the performance for this okay this is not ten milliseconds but of course it's because it has to do the HTTP request and there's no way to catch it so unfortunately I haven't found a solution for that one thing that I included to my solution we besides the find my location is that you can type of course imagine you are in London you want to find what's the situation for example in common garden how maybe you want to know what's in a certain location or this is too close okay let's maybe London Bridge so this is also very interesting you can use this geo coder this is from map box and I can introduce of course you don't know London you have your visiting London you don't know but you may know some some hotel or maybe a street or a famous place and this is a way to navigate the map and I can do the same I can run the query from here okay I hope I don't push it too much all right okay that's that's the end I guess very good if you want to know more somehow it is no work in the second try no worries here these are some people that developing on AWS amplify and AWS up seeing that you should know about and that's all thank you for listening [Applause] yeah thanks for that talk I've seen a few of the 80's demos and they're always just by I'm boiling and what's possible with server dysfunctions I think you got struck by the conference Wi-Fi gods yeah I'm a probably saw the time and they were like sorry mate yeah it's that okay yeah but definitely check out the demo it's open source right yes online yes you have any time I don't do hammer sign formulas on the client just be nice to your customers that's not nice so so thank you big hand of applause for drug thank you you have any questions sabotage him by the coffee

Channel: apidays

Views: 133

Rating: 5 out of 5

Keywords: apidays, api, apieconomy

Id: fpqCAgMSiSk

Length: 35min 24sec (2124 seconds)

Published: Tue Jan 14 2020