[Web Dev] Large-scale Geospatial Visualization with kepler.gl - Shan He

hi everyone my name is Shan I'm self engineer from the data visualization team at Wilbur today I'm going to talk a little bit about the open source library that the data visualization team built to to do large-scale geospatial visualizations in in the browser so I would be you know understanding geospatial data is crucial to our business we're doing millions of trips every day and it's we're doing trips in sevens 70 countries and five continents every day were collecting billions of geospatial data points and then we need to figure out a way to better understand them to better you know gain insights from this gazillion large amount of geospatial data so at the visualization team we have a big challenge and this is ization I'm a long time ago and actually first join Wooper which is four years ago I took one day of trips in London and then I'd made this animation was just this one day of trips every single line you're looking at here is one trip driving across a single streets in London City so there's no Road in this map everything is dark but just by looking at those single lines of trip you can see the whole city contacts in front of your eyes that's why I wager ization is crucial to us it's just by doing this kind of you know in this case just animation you can get an understanding of how large our business are where are we driving at and and then you know how much impact we're doing just to one city so this is like everyday life right we add a visualization team make hundreds of maps every day we try to use maps to help us better gain insights from all this geospatial everything looks great because you know we have large data set but we also have a large challenge you know at the end of the day what I don't want people would come to me is saying that I love the Maps you did can I use as my screensaver and we actually want people to you know look beyond the beautiful graphics to actually learn something from the data sets that's why we did the visualization team we develop this sets of tools for people to better build applications to visualize geospatial data sets all of this libraries are open sourced you can probably gain a little bit insight by the naming of it it's like kepler GL reactio tech GL uma GL and react with you know it's probably you know a lot of our front-end web is filled with react a lot of our web-based mapping visualization is built on top of WebGL which is the 3d high performance video rendering graphics are your browser already comes with nothing you have to install and we build as libraries just to make us eat much easier to build hi performance visualizations so today there's two two libraries I'm going to mainly focusing on one is the deck geo library the other is Kepler GL Takeo is layer based visualization library built specifically for high-performance geospatial visualizations it's a framework that you can use to build maps and then put visualization layer on top of it Kepler GL is actually a application but also a framework we use Kepler GL so that it is it is a framer so that allow you to visualize and analyze a large scale of data set without have to write one single line of code and Kepler gr is also built on top of that gia so starting with stack gia what is that GL [Music] so - gia is visualization focused is - is for rendering big array of data and it's focusing on geospatial all positions can be at your spatial data point and is also reactive it's coupled nicely with react so oh the phone time application will be always react and just import deck GL and render high-performance Maps Daksha had this layers ecosystem every single visualization is one layer that xgl managed to show on top of a base map and actually I'll come with a set of layers such as arc and hexagon and scatterplot already sets for visualizing geospatial datasets and Takeo uses concept-car incense rendering because if you think about an array of a thousand data points each of the data point a lot of common common attributes like latitude and longitude the only thing might be different is their metadata so with WebGL instancing rendering eggshell was able to render like a meaning of coins apparel in parallel in your browser was without too much of a choke up and then tech share is also handling a lot of geospatial operations inside your GPU you don't have to actually worry about it we do map projection inside the GPU we calculate we calculate camera view point instead of you that's why you were able to get this like rendering three hundred thousand points in this case actually a million particles in the browser you know smoothly this is a wind map that Jung here made is quite awesome and all this happens in your browser if you're interested we actually a deputy is open source we have a lot of examples hosting and tech gos website you can take any of the examples starting your own map application with tech geo pretty easy and go in Pyeongtaek GIS coupler tia this is a framer we just open source like a month ago and with tech GL your front-end developer you still write your own code you can build very flexible map applications but you know for someone who come with a data science background who don't want to write any code but just want to understand what is in their data says we've offered this another framework are kepler GL is hosted as a demo app on our website it can handle millions of data point and allow you to do explorations and visualizing it without have to write one single life code we build capillary ousts for fast visual exploration of medians of geospatial in the browser so cover geo string line this geospatial analysis process we start your starting by upload as Phi OCS NeoGeo JSON and then coupler G I'll give you all those options allow you to filter them and give you a set of nice very nice industry standard geo spatial visualization layers so you can just draw maps with them and those are the layers that come with the cobbler GL you don't have to write anything relying code to make this kind of visualization like the heat map or the scatter plot or hex meaning and it has a nice UI design and it's very easy to use like it's very intuitive you allow you to you know associate your values with like colors or size for you to you know joy insights from a draw nice-lookin geospatial Maps and just one of the demo images by jogging drag-and-drop this geo T's on fire god for I think this is San Francisco contour map I quickly assigned colors to the elevation of the contour and then I change between different color palettes and then I now moved the labels on top oh this just stunning like a one minute as as long as you have a gog song with some data with that you can interact and then make maps fairly quickly beyond that Kepler Gaea also has allow you to do geo spacial temporal visualizations if your data comes with any kind of time timestamp or dates completely allow you to add this animation to it so you can play back through the whole duration of your data side to see what happens the time he also does on the fly tube geospatial aggregations so for example in this case I'm drawing a hawk spinning map show you the density of other points in this case it's all the trips in LA and I was able to change the resolution of my hex pin I was able to add like a three dimension to it just so that all I hear is more visible in a three dimensional feature and in the end because a lot of goobers data is trip based in this trip base you have this concept of origin and destinations kepler jia also have this function that allow you to looking at like trips or arcs start or end in a specific region where your mouse hover over and this is it this is useful for us to understand which part of the city have trips goes to another part of city and all this interaction calculations down in the GPU that's why it is very smooth so you can just move around quickly you so tech that Pablo Gio's build was react and redux it's actually a react components that can be embed into other applications we package coupler GL state and inside a Redux reducer so that it's actually fairly easy for you to customize its behavior and then you can just import coupler GL into your react app and the import the capo gr reducer into your Redux restore and if you're interesting how to use it we also have a website covers the Academy around there we're listing over open source library tech geology out and then have a step-by-step tutorial tell you how to actually build your own app Azhar open open open source libraries

Channel: Uber Engineering

Views: 11,038

Keywords: Uber, Uber Engineering, Engineering, Software

Id: y-SA6bOv4Eo

Length: 10min 39sec (639 seconds)

Published: Tue Aug 21 2018