KML 101 -- How to Create Amazing Maps in Google Earth and Maps

good morning and welcome to the webinar this morning KML 101 we're going to show you guys how to create some awesome maps in Google Earth and and Google Google Maps so just off the top this is being recorded so don't you where you can see it later and review it at your own leisure I'm Erin coning product manager for FME server and I'm here with team hints senior analyst with pro services in the back we got a few people answering questions you can see the little red arrow is pointing to we've got mark Dan and iris who are all eagerly awaiting your questions so please do write in with any questions you might have and also in the back taking care of logistics is Stephanie so quite a quite a group on hand to help us out here today and a little bit of an agenda we're going to start with an introduction talk a little bit about safe software probably go a little faster through that then we're going to get into some kind of introductory information on KML a bunch of live demos wrap it up with QA and at the end you will get an email with a link to the all of the demos the files that we've used today as well as the recording so that's what we're doing today safe software just a little introduction we're a little Canadian company about 95 people founded in 1993 what we do is we do data transformation with how we help power the flow of data so if you work with us you understand what that means otherwise you're going to learn a little bit about how we power the flow of data to KML today from well any number of different formats right Dean yeah and how many customers do we have our I think around 10,000 something like that yeah in about what about 120 different countries yeah yeah yeah so just a quick overview of the product lines we have we have a FME desktop product which is allows you to author your different data transformation tasks then we've got FME server which allows you to kind of start to automate those tasks bring those tasks to the web allow you to push notifications and receive notification really a more of an enterprise product and really we're really excited we're not going to talk much about it but FME cloud is a new product offering that really is kind of commodity monetize is the FME server experience so you don't have to maintain any hardware or anything you simply pay as you go to use data transformation really cool products that so there's no locked in through your contract on that one that's it pay-as-you-go you can lock yourself in no don't worry we have a we have an option for that but go to FME cloud comm and sign up and you'll see more information there it's really exciting so we're on to our first poll this morning so we want to know go ahead Dean Dean will fire up that poll what is your FME level of expertise all right so we want to kind of know do you know a lot about FME do you know a little bit where are you in the in the chain here so yeah you want to select just one here not not all can you select all is that not general okay although you know if any is a big product so there's some areas where I'm a novice because I used heavens you know absolutely Lord of FME absolutely yeah 3d and point clouds a little new for me that sort of stuff but ok I think we'll probably close it in just a couple seconds here and got about 80% so I'm closing on 82 anytime you get over 80 I like that option aren't you sure yeah share the results there oh good so I've got quite a few new users here ok is that being shared on your screen or mine no that's on both isn't it ok and a few experts even a few experts so that's good to have some experts on board never use that for me before 20% well we're going to get a good overview today I think of what we mean when we say workspaces and transformers and all that kind of stuff ok back to you ok onward and upward if you are new to FME we don't want to spend a ton of time on that we will cover off a lot of the stuff you'll see in the weekly intro but I would encourage everybody who in that 20% to go to the weekly and Oh safecom / weekly intro and that's a live session with with couple people that you can ask questions you have mark and Craig there and it's and it's you know the best time to ask live questions and learn what FME is all about there's a great getting started page there as well yeah that has links to a lot a lot of different resources we've got tutorials and and and even things on the DFM a channel on youtube so you can work your way through some demos and just just to get your feet wet excellent yeah so good resources they're just you know a really brief overview of FME you can see all the different types of data that we handle there and of course behind that we handle over 300 different formats and it's not just about formats is game what else it's what's what's important about that well really it's the transforming of the data because you know going from one GIS format to another it might be a relatively easy conversion but we quickly found when we start to support things like CAD or databases that really it's the transformation where the hard work is yeah you have these different types of data because the data models are different they're different purposes yeah so as we said there are three hundred-plus formats and we continue to smash through new barrier each year really that just tells you whether you can use our product but then behind the scenes there's a lot of transformation that that takes place and you can see some of that today with KML so on to the Google related formats you can see we have see we've got some lesson formats here yeah so there's a number of we're going to spend most of our time today talking about KML but there are a number of related formats there's 3d formats such as Sketchup and colada so you can include models within kml mmm and do do 3d that way and then there are the online formats the Google Fusion Tables and Google spreadsheet and that's another handy way to actually put your data in the cloud and incorporate that into into your Google Maps yeah and actually an exciting new format for 2014 we just added was Google Maps engine which is it it's something I haven't even used yet and there's already talk of adding another version of Google Maps engine format to FME so we're all we're always continually pushing with the Google formats not just KML but this is about KML so yeah just a couple words about what is camel I mean we've all seen it but what is it sort of under the hood and it can be as simple as just a single file or it can be a whole directory of files or it can even be a container like a zip file cam jet or KMZ for our American friends which can contain supporting files like raster images models like like the Sketchup model I mentioned and so really that whole structure can be a KML data set so it doesn't really neatly fall into our directory versus file right it's this distinction it's such a powerful format that you can do a lot of different things with it yeah you can start simple and you can make it as complex as you want it to be so well so as you were saying there you know it sits on a virtual globe you know you you can add the imagery you talked about and of course your data really that's what we're talking about today yeah and I guess the other neat thing about KML is it's one of those few formats where you can make so many different kinds of data Union you can have both vector and raster you can have 2d data 3d data all all mixed together so it's really actually quite powerful excellent all right now we're back to to another poll we want to know what is your biggest challenge with KML data and let's let's see what the options are here and what are the options there gene reading styling yeah structuring so that would be any way that you're trying to structure the thought the folders of the data organize your data and then there's a whole range of advanced features that you get into network links to to link to external resources regions so that's basically controlling what's visible at different zoom levels or do you have challenges with data volumes so how to how to manage large numbers of users or large volumes of data yeah and it started off advanced features was way in the lead but it pretty much everything is catching up at this point although I'd have to say reading is too bad all my demos are about reading cuz I see okay I'm kidding you know he's kidding of course but still almost 20% reading I think we'll just give a few more seconds and then we'll close that poll about 70 percent voted 70 percent we like to get up to 80 but but we'll take 70 as well okay so we're closing that mm-hmm I'll share that so there's the results yeah so structuring and styling well yeah those top three are all pretty close yeah yeah so we're going to be covering uh well most of those today so that's good reading we won't so that's actually we put we find it that way okay although I will say that we do we have continued to improve our readings so it's not like it's yet important because you do one of it sometimes to be able to sometimes KML data may be the only thing you can get from somebody and absolutely get as much as you can out of it yeah it makes sense especially over the web being able to download that data directly from the web there's there's a lot KML out there for sure I'm speaking of we talked about formats already but of course Dean mentioned transformation is really key when you have those different data types and different data models so being able to go from something like in here we see a workspace converting lie to our data and shapefile data into well in this case is a de M but it could be came out you need to be able to change that data model yeah absolutely because there yeah why are being a point cloud it doesn't really relate very closely to anything we're used to using energy is so you actually have to do a fair bit of processing to get that into the right shape and that's where those four hundred-plus transformers will give you the power to do that yeah you don't you don't need to use them all and in every every workspace but have you used them all uh probably not no I mean you know you can see you can start simple but yeah it's kind of like that Lego size building blocks is where the way I like to describe it absolutely speaking of which I'll run through a couple of the kml supporting transformers there's actually quite a few other transformers that did make it on this list that do relate to kml these are the ones we would highlight there I think Google actually gets its own folder in our in our transformer categories unless yeah yeah so we'll go through a number of these today not all of them so the property setter that's for setting things like the name and description and you can also based on on things like the name control things like the the folder structure will see that enable as well through there yeah yeah and and yeah the description relates to the balloon under geometry that's where you can control whether something is extruded or not so getting into the 3d regions that are that's for regeneration so different levels of visibility based on scale styling everything from the the points to the lines and polygons and text and we'll see an example of time series as well in a few minutes and then the latter ones we don't have time to go through all of those yeah but building up fly throughs controlling the camera perspective and that Google chart API is pretty powerful as well very cool and there's two on here are not on here that one of them I use for sure what is it the geometry extractor can do KML that's right you can both extract and replace stream ml yeah geometry and I will give a nod to the XML transformers and some of the Intel fan the XML template er for example can be used to build up HTML for those for the KML descriptions uh and I'm a big I'm a big web web socket type guy so I I'm going to show up the Jason temp later Oh which I use later haha get the last word on that I get the last word all right we're back to another poll so we want to know where is that data coming from you got KML data it's going some it's going to KML uh your day is going to KML but where is it coming from where is it coming from so so I think with this one we we just said select one so what would be one of your primary sources for for your KML data yeah and that kind of gives us a feel for you know what kind of restructuring you might have to do yeah yeah it started out with a GIS way in the lead and the others are pulling up close now I guess in many cases you know maybe we could have distinguished between GIS database versus file a lot of people probably are going to be building this sauna on a spatial database of some kind yes they've got KML services yeah yeah but you'd be happy to see over ten percent from a sort of web Gmail XML crowd so okay so we're going to close that and share it there we go all right so over over half coming from GIS so yeah not a huge surprise there I wouldn't say two percent from other which we can only speculate on but it's a stereo source 14% from database excel I wonder if that is the non spatial crowd yeah that really excel or yeah that's a good point yeah I'm not sure well we'll assume that's a non spatial data primarily all right so we're back into it I think that's one of the last polls before we finish off the sort of a talking aspect here but now you're going to you're going to lead us on with the approach to KML here yeah just give you guys a high-level overview of a typical workflow it's not necessarily you're going to need to do all of these things all the time but these are some of the main tasks that you might need to hit in order to write KML and again that we can can read but the day of the workflow there really depends on what what outputs you want so it's probably this is an easier workflow to go through so we're going to read initially you have to read your source data doesn't matter what that is CAD GIS other databases and then you need to reproject everything into WGS lat/long 84 and that's pretty important because kml only supports that that coordinate system and and if you are going to do things like transfer attributes between features based on location you want to make sure everything is in the same spatial reference and then you need to build up the required attributes and geometries for your features so you know you might need to do is a feature merger to join you might have to build polygons from from from line work from a CAD file so just building up whatever features are going to display and then we start to get more KML specific so the names and IDs and descriptions for for your kml features and the ids and parents help you control the actual structure of the KML data so you can nest things in folders and we'll we'll see how that's important and then we get into how you package it so how you control the display so the display control can be by scale so KML regions it can be by time so as you'll see there's a time slider so we can see change over time or there may even be a real time aspect so maybe you have a time stamp on the data or data is streamed in and finally sort of recommending near the end to set your styles because you would go and set a style at the beginning it might get changed by by some feature manipulations that you're doing yeah overlays and things like that yeah and then lastly there are a number of format KML specific format attributes and parameters that allow you to control things specific as let's say the raster format that you want to use for imagery TIFF or JPEG and even even the file name extension cams that will automatically zip up that any support files into a hackage so yeah that's the kind of thing you set on the output so now in terms of the the overall approach mob yeah so here are just a little more detail on the property setter and styling so you can see there the the property setter lets you set the name and there's description and we'll see an example that in a minute and there's also the inter geometry their altitude mode and extrude and whether or not the object follows the terrain so so it's quite a few pretty much almost all my work spaces will have both of these transformers easily the main to KML transform is you're gonna use and the other thing I want to mention about these camel transformers is that we used to have initially when we first supported KML you had to set almost everything as a format attribute so you will see in a minute what that looks like again you have all of these KML underscore attributes that you can expose on your destination feature types and then you have to know what those are you have to comb through the docs and figure out what the correct value is to set and the nice thing about these gooeys that we've made specialized interface for supporting kml so you don't have to know any of that syntax you just interact with with the GUI set the colors that you want the styles that you want and in the background FME is setting those those tags absolutely so now in the in the approach there's also some special feature types that you can set and what these do is basically allow you to control you can set things like a style but you can define style that is available to throughout the word space otherwise you might have to have 10 10 s dollars in your in your in your work space if you keep using the same style so if it's something you want to create once and use often then then that's a good way to go and then also things like ground overlay for for raster support so again these are some special folders I guess you can think of them as reserved feature types that you can't use for you know you can't come up with your own future tech called you know ground overlay or folder and get an aspect to use that to display data it's really a unique format in FME in that the name of the feature type kind of dictates how that data the features will be interpreted yeah that's right and we'll see an example of folders in a minute so we can actually define a folder object and then insert features within that and these what you do and how you get these are you have to go to the the writer menu and select import feature type and you can import from any KML file and that's why on that slide you'll see a few things that are that are actually data like Metrorail those are not special reserved future types so any KML file if you go import future-tech you'll get this list and and then you can define these special format specific future types now I did mention format attributes and parameters so here is an example of what that looks like on the left side here we have KML a bunch of KML icon format attributes and most of these you can set in the styling transformer KML styler but you might find in the documentation something sort of esoteric some special setting that we don't have the ability to set within a transformer and you could set it this way and then there are settings like create attribute table in the description balloon create folder for feature types so there are some feature type and format specific attributes or Crowder's I should say right that you Rachel as well and finally you know here's a typical workspace and yeah so basically won't get into the detail of what this workspace does basically I just wanted to say on the left we've got the data sources that we're reading from in this case a SD f3 database and and then there are some transformation steps to build up the features that we want and then we do things like KML property settings to set names and descriptions and finally styling and then we write out to to KML so so that's a typical workflow in the top half the bottom half actually we use is a section where you could say that we write to these specialized feature types in this case a folder defining some folders yeah and we may be or to also need to define styles or ground overlays so these are sort of the supporting feature types that allow us to structure the document and then we get the output so we'll see lots of our we'll see that again yeah so why don't we pop back and I'm on it you got the folder on there so we want to know what you're interested in so this is some of the stuff we're going to cover today we want to know you know what do you prefer here what's most interesting time series which Dean is going to show WebSockets which isn't necessarily KML but I'm going to show that in the end network links which I'll show in just a minute web map tiling for so making those tiles for for Google Maps with your own data or you know things like 3d visualizations and fly throughs so the results are rolling in now what's your favorite there Dean would you say I guess time series like yeah doing my there they're pretty cool ah yeah the very cool time series I think somebody I noticed in the questions do we support video formats and we're we're not quite there yet in terms of video full on video but well we're going to see something pretty close to it it is it is actually yeah using images in thing so we do is support time-series so I guess you could say you could close the you could process slices or image images one by one and the Bentley another tool could assemble them into a movie yeah my favorite probably WebSockets it's the new thing people haven't necessarily heard a lot about it but it's pretty cool so how many people we're about 70% I think we'll just give it right about now I think we'll call it there so I have closed that poll I'm going to share it and a web map tiling actually a huge amount of people interested in that so we did that demo a lot in last year's version of this webinar but we won't be doing it this time however on fme PDFs will show you the link at the end you can go there and search web map tiling you get a bunch of demos show how to do it and then it's a web map Tylar transformer that allows you to do that really easy to tile up your data for for Google Maps and now that you mentioned it there are tons of really cool examples on FME pedia and of course many related to kml so that's that's one of the resources will remind you about absolutely yeah okay so I think we will now go on to should be hidden now okay okay oh yeah we got I think we've got a demo now demo time okay so I'm going to just steal back the presenter ship here right if that's even a word it is now no one can correct me so that's the one nice thing about one-way communication and I will get down to that real-time transit demo okay so we've got some I want to show basically a KML Network link and kind of show you an actual workspace that shows that through shows styling effectively and I'm going to start by bringing up Google Earth here we go we've got a transit network loaded into Google Earth but I want to see where the buses and things like that are located in real-time so the Google Earth handles at us through a network link which basically goes back to the server ask for the data at a regular occurrence basically refreshes that now with FME server I bring that up we have a service that allows you to well effectively hit run on this KML network link you get a camel network link which you can double click and then it'll pop up and there is authentication in this case so make sure I'm showing my screen I am excellent I can log in I can turn this indication off if I want but it's just a security feature and what happens now is it brings in bus the bus locations are these where all the buses are right now these are this is where they are right now absolutely and as i zoom in we'll see oh it went it went red that's never good let's try and refresh it refreshes once and let's see if we can get it to refresh one another time there we go ok so it's working you can actually see the buses moving around and these arrows actually show the direction of travel as well and yeah it's refreshed every I think five seconds is what it's set up here but you can refresh and go back to the server at any time you want so who's controlling I guess it's a pusher up full of the data this is actually uh this is actually a pull of the data from from FME server which looks at a sequel like database we'll go take a look at the workspace the actual data is provided by TransLink pulled down by a third party application or vendor named transit DV if you're in the Vancouver area you have heard of them they actually technically they interpolate the data and the bus locations are actually interpolated a little bit so it's it may not be exactly there but the main point is kind of showing that this cable network link can go back to FME server ask for the current data the real time data so then workspace is being run how often the workspace is right every five seconds and that's what you configure in the kamo network link here yet right so that really that Google Earth is calling that absolute space on the server absolutely and so you can see in terms of style there's the the arrow show the direction and also you can see the different colors for the lines if we zoom in but you're going to see show up is bus stops actually show up oh well so something that you don't see when you're zoomed out and the reason being is it's just too much data for your screen then right to is buddy would be too confusing so you make it use a region to define that once you've zoomed in enough the region becomes large enough you define the size it has to be it activates and then that layer appears and then you're there you can see the bus flying by there cool so region is really Google's term for that scale control scale egg dependency official facility exactly exactly and those buses are just a flying today so that's that's the kamo network link now I want to kind of dive in and look at a few things about how they were made how they were made so I would have to say those buses look like they're moving quicker than when I'm waiting for them okay yeah at least here in Surrey and sometimes they don't show up too which is which is not always the best okay they listen to a mobile app I want I want show first how the the transit network data was with style this is actually this data is pulled down from TransLink in a CSV format called general what is it called general transit data specification if um there you go gtfo GTFS are used to be called Google Transit feed feed uh Google Trends data specification but but it was changed to be more specific so it became so like an open standard in nature it did technically yeah um go to general that's right it's kind of handy yeah so we bring down CSV you turn it in since CSV is not spatial you turn into a spatial point with with KML you have to set the coordinate system because it needs to go to lat/long down here we're filtering out based on what type of lines so these are the transit routes themselves or routes we're filtering out based on the type and then we come over here and we style it so Expo Line is a type of train a sky train wrote we wanted that to be this is a KML styler we want that to be blue we wanted it to be a width of 5 and we wanted a label on it as well so if we look back there we would see a label on that if we look down here for the millennium line so just a different train line we've got colors yellow it's a wider color a wider line I should say still creating labels and so on and so forth one cool thing that we're doing in here which Dean alluded to is there are a bunch of KML format attributes underneath the hood and one of them is called draw order so in this case we wanted to have the Expo Line actually show up on top of the other lines so to do that we set the draw order KML a draw order that's a special format attribute it's all documented you can look it up but that's a special attribute that goes into that KML to make it to make that line show up now where where that might I find that documented it's good question I'm not sure no it's right here under help FME readers and writers you can go in there and find all of that information we try to expose via the Transformers the most common commonly used format attributes and functionality draw orders a little bit less commonly used okay we're going to keep going here here are the actual transit stops so those are those bus stops that only showed up once we zoomed in mmm how was that done you might ask which is a great question I wish I have a question I should have let you ask that one here we read all from GTFS again CSV make points then we use a region setter transformer so if I come in here I'll just drag it over what we're saying is we're defining a box on the earth that as you zoom in it gets larger and we're saying that once that box becomes 24,000 pixels which is way bigger than the screen obviously once that box becomes that large then that layer will appear and a lot of people ask well how do you haven't you come up with that number it's completely trial and error you put in a number yuzu min layer shows up you like that then you got it so which of those wins I mean you've got the min max XY yeah so that is just the size of what's being displayed like is that's the expense that's the that's this hidden invisible box that as you zoom in becomes larger and when it gets this size okay when that main message gets that big exactly okay so yeah related absolutely and then again we've got a test filter filtering things out and if we look at action going to look at the this one right here this is the C bus nope the West Coast Express one of the bus lines this is the old-school way of setting KML attributes yeah I was expecting a KML styler to be to open up when you clicked on that that's right label color icon that the location of the icon for the for that that I could that place mark can all be set in the KML styler this is the old school way of doing things it's harder because you need to know the names looks simpler because it's you know it looks simpler but then you got to know the name actually have to know and you also have to know the code for the particular color you can tell that this person who built this workspace was a bit old school in some ways in a bit new school in other ways right yeah kind of a hybrid hybrid so I just wanted to show that workspace and then I'm going to show the one where they're actually where we're actually reading that the current locations so how many workspaces are actually part of this slow demo this is just one here I mean there's there's a few but this is the main one that is doing the update so here it is reading from sequel light again filtering out now this is the actual vehicle location if you will the bus or the train coming over what are we doing in here again we're going old-school we could use a KML styler for for setting the bus placemark and setting the color and the label they chose the person who did this chose to use the format attributes themselves but again KML styler is the recommended approach here yeah and then one really cool thing here is based on an attribute coming in it's either you can tell the direction of travel effectively so they filter out based on the direction of travel and then if it's northbound they change the kml name to have this little kind of up up arrow Oh some of them be handy if you've got a couple of buses near the same intersection exactly which one's going where exactly so they filter that up based on an attribute then they set the label here and again this could be done with the KML property setter so that's probably the preferred method for doing this however this works as well cool and then out to KML so that is a that is kind of a quick whirlwind tour through some styling through some network links and there's an some property setting there's so much more you can do with KML and Dean is going to show us some and actually well that I think that works based just going away now but the default behavior I'll mention that I can see there and even in terms of the structuring yeah the default behavior is feature types become a one level of folder that's true we'll see in a minute how to control that that is true yeah so I guess I can I steal that a controlled on yeah absolutely try to make a little smoother transition this time I wanna let you guys see my our last slides before we get there so so people always people always write in to support and say how can I do folders and and and then and balloons as well actually how can I get fancy with balloons yeah I guess yeah folders and balloons both seem to be pretty popular they're not just done on birthdays but with came out they're popular year-round so I've got a demo that does a bit of both I actually have two demos so we'll just jump right into it I'll just click on the output and then we'll see what that looks like I've got some data I guess we're big fans of transit around here we're subconsciously trying to influence you all to to take transit to work or something that's right that's right yet or I don't think for they're not on the transit so love it payroll we're not getting any royalties for this I'm headed down to Oklahoma City in September so I don't know about this transit thing but okay well you're going to be taking public transit if your honor in a plane it's still public well okay very good okay so the trick here is I'm talking about folder structure I'll show you right off the top it's easy in KML if you as I said feature type by default becomes a separate folder in in the KML file now why don't you want everything at the top level like if you just download some freebie shape to KML tool off the web it might just spew out all your data at one level but then you'll double click on it and you'll see you know you'll have a nice list of about twenty thousand features here on the left side of your navigation pane which kind of can be a bit of a pain to look at that's almost elegant yeah so really folders are your friends learn how to use folders and that can it can help a lot it can allow users to do display control and find the data that they need so you can see what I have here is I've got two levels I've got bus versus rail and then within the bus I kind of have the the buses that are linked to the rail so the Metro buses are the ones that actually as you can see I turn off the non Metro ones and you can see those blue lines go away and so only the Metro ones are the ones that are actually connected in some way to the rail line now why would I care well I'm a big fan of trains so if I want to make sure my my rig to work includes a little ride on the train well that this is one way to do it I'm renting or something I don't want to be in the blue necessarily yeah yeah I mean there again you know sometimes they're like I know in the Vancouver area the sky train service is a bit more frequent than some of the bus service so yeah if you want to live somewhere where you can get some real quick you probably don't want to be in the blue all right and then the nifty thing about this is that I can actually click on one of these routes and this is where now we're into the balloon side of things so really if the balloons give you a lot of control because they accept HTML and so by definition when you've got HTML you can basically plug in anything from the web that you want and in this case we're actually integrating this with FME server so right off the top it gives me the name of the street it gives me the route number so that's dynamic if I click on a different route location I'm going to get a different code and then I can look at that data now if I want more information maybe I actually want the bus stops now it didn't include bus stops at this level because there's thousands of them and it's again it's a bit too busy for the global view but if I want to look at one route I can just click on PDF it's going to go away to server and render that as a an actual PDF file and then I can open that up natively in Adobe and now you can actually see all of the bus stops for that particular route and seeing how FME is semantically aware we're actually preserving all of the feature data all the attribution associated with those bus stops and additional information might you might get in there like like what time what times a day the bus comes to that stop that sort of thing so that's one way of seeing that data and another way is if I go back to one another one I can actually download this as a shape file so click on that and gives me a link to a shape file and this again this is from FME server and click on that download the file and I won't bother viewing it but you can see the shape date is all there and then the third way I've got an example of here is we can send this particular route to me as an email so lots of ways of basically offering data it essentially what we're doing here is using this as a data distribution platform not just write a viewing environment so I go to my email and it goes here we are at me job succeeded and here's your data so I can just pop in there and extract it and they'll open that up so if I look at for example just Lenny bus stops I can view that an inspector and neat thing of course an inspector is that it allows me to have a table view and so now you can see if you are actually going to use this as as input into it your GP GIS you'd have all that the rich data modeler so and now how did all that get put together just show you very quickly the workspace behind it so here we go and it's not too complicated I'll focus more on the the two aspects of configuring the actual the KML property setter here and if I open up the the editor you can you can see that really all I'm doing is typing in HTML and there are of course some functions here to pull in the values from FME features so the line ID is key because this HTML has embedded links to FME server which runs on a specific route number so whether or not you know one case it's routes to shape in the other case it's routes to DWG or PDF so really combining the ability of very some pretty simple HTML link to FME server you can do a lot with with your data really yeah yeah so it's quite powerful and you guys can get a hold of copy of isten when we email up the packages and then we have different styling per layer as Aaron already showed you and I just wanted to explain very briefly how the folders work so here at we I define two top level folders and so I have the special folder feature type and these are just folder objects you can just think of them as containers they don't have any geometry they're specifically just placeholders for folder names and so we have bus and we have rail and then I define both a man apparent for the subfolders so kml parent here is bus and its name is metro and then later on up here what I can do is I can assign every feature can be a child of metro or non Metro so based on what the spatial Association of whether or not it connects to the metro line define the parent name Wow so yeah basically start at the top level and you have to essentially define an ID and then all the children point to that ID with the kml parent makes sense and i think this is one you still need to do with the creator beyond attribute set or a specific we might look at making it easier in the future but yeah you'll get that workspace in the downloads so that's great and i want to open them up but there's a couple of other work each each of these workspaces basically just goes from the database to DWG or PDF but that's not kml so you guys can look at that when you download it all right so that's the and I think I have one more if that wasn't enough balloons for you I do have one other one and really quick I you know I'd look to see a slideshow right in in the balloon it's a possum yeah i funny should ask now I will before I actually run the slideshow will notice that where did we go we went off somewhere off Africa that's where you're going to go if your coordinate system maybe isn't quite right but anyway let's look at this slideshow and uh well to be honest it's not my kappa might this is what my cat looked like when I brought our our new pug home your cats not a pug fan well I think our public likes having fun and our cat doesn't so yeah we got a we got a question here what's your cat's name Oh my cat's name is Dory and I think she's also grumpy because her brother Nemo is still in Bulgaria but oh they're separated it's yeah like the maybe that'll be the Nemo - yeah they have cats trying to get together well they still stay in touch by a Skype now back and forth but yeah so still grumpy so just - I mean this is 2014 functionality technically this was added last week and and so you can get it in the beta but what let's show very very quickly what the workspace is absolutely I'll just pop open 2014 as you can see we can run 13 and 14 side by side and I'll just throw that workspace in there and you can see that we have a new section added to the KML property setter which allows you to attach additional files to that the keynote document package ah so in the past just to be clear in the past you had to reference these off some website so you had to maintain them separate from the KML file and then if they get deleted or there's a firewall in the way you couldn't get to those yeah you just see these broken image links so this is and also you can see I have HTML content but instead of actually having the the actual text of the HTML embedded here it is pointing to a file yet on disk you can see that there is this description HTML file here that's being point to that and the previous transformer just reads the contents of that file brings it in brings it in and then that's where you stick it in the balloon there stay it in so that's a little preview of 2014 you can get the beta with that functionality yeah that's working now so what else we got we've got a little little uh time series yes raster maybe yeah okay well I've got about three inches or four minutes interests of solving all the world's problems here or at least maybe simulating them we're just going to have a quick look at a time series demo and also looking at some raster output for change now you guys might be used to seeing Google Google Earth Google Earth and you look at Google Earth well Google's Earth is gone what happened to it well what we have here is essentially a data cube and it's got a raster image for every year and it is essentially a model of the emissions based on land use over the last 300 years and so you can see in in eastern North America that's where development started happening and so that's where the emissions started and you can kind of see it spreading to the West as we heard rest towards that this is real-time emissions or I did try real-time emissions first but it was kind of a boring demo because it just kind of sat there I guess you know we're not admitting oh you know I think I saw a little change because we are if the emissions are growing but no I think when we go over 300 years it's more pronounced so you've taken a bunch of different maps in a data what you call a data cube to me it's almost like hundreds of different raster maps that show the the the emissions over time and you've placed them all into Google Earth in a time series here yeah Wow yeah there was basically a big stack of data and what I did was it essentially explode that apart and and then you know the original source data I really you can't view it without having a time control it's one thing we don't have it you can see a big stack of values so there's many many bands to that raster data here is coming up just now yeah but there's all these different bands but there's really you're only looking at the last layer and so the only way to view that was using I'll show you the workspace really quick then I'll hand it back to you the only way to view that was with this workspace it's actually not that complicated we just have a raster band separator which blows those all takes that one big data set breaks out into individual features and then we have a list that we have to index to pull out the year number yeah and then we use that year number to define the the time step so the key transformer here is the KML time centers setter so that year that we get from the index we just call it a 101 because it's not there's no specific date right and so that gives us our time period start and end okay and that allows us essentially to make this KML movie Wow Wow so we've done it I mean we've done a lot today but we're gonna do more even okay what what more do we have we're on to WebSockets which starts to come away from KML but I will bring it absolutely back here I make myself to presenter and show my screen and let's see what happens there so what is WebSockets Oliver what is WebSockets is there anything to do with socks like it is it has to do the socks but actually your definition is the simplest I think what is a team what are they well I just thought of a pipe so it's a pipe that you can just funnel data through the apps for the absolutely our data pipes really uh when the web was created there is the idea of one-way communication where the web the client actually clicks a link goes to the server request something and it comes back so that was a really a one-way communication but you know things have evolved in the last you know ten five years for sure ten years where now servers are wanting to push data out to clients and so we've it David they've built technologies such as Ajax and and long polling to accommodate that but there are things like latency issues there are it's not standardized necessarily people are taking different approaches WebSockets is approach to to do low latency real-time high-speed full full duplex which means bi-directional communication from one node to the other node so the easiest way to think about this is a web browser going to a server so you can think about a Google Maps maybe a Google Maps application going to FME server to ask for data in real time without the overhead of HTTP so it hooks straight up to a socket so then so the big difference then between that and regular HTTP is that you don't have to re-establish the connection is it absolutely what do you say continuous or it's it's continuous absolutely and also they do have a solution called long polling which does that but it still is the overhead of HTTP WebSockets is pure TCP connection so I'm going to show a couple demos I'm going to show them really fast and this is again a preview of 2014 this just got added a month ago so you can start playing with it now let's take a look at some demos here so I guess there's both changes to desktop and server absolutely with us because there's new transformers and there's a new service absolutely we're going to see that here this is built with 2013 in FME there's a workspace always running that accepts WebSocket messages does transformation and then pass as a result back what is this this is a bunch of zip code centroids in the US as i zoom out and pan around the extent to this map is sent down to FME data is extracted and then pushed straight back up to the client so as I move you can see that very quickly very smooth all of those points are being loaded this is what we talk about when we say low latency look at how fast that data is coming through on to this web map so what's going through sockets it's the point data or is it the every all the maps or the basically adjacent message the point it is what's coming back to the browser the extent is being sent down as JSON JSON is being sent to this web browser to this client and it's being displayed and in the background is the background just straight up Google Maps is a forest fire that just started here did you see that as soon as I clicked on the zip code boundaries you went to server it asked for every what FME it asked for them and they string back in real time okay look how quick that data comes back these are full-on polygons coming from client to server okay so not just point data you can actually polygons absolutely really so this is this is this is how Gmail Facebook this is the kind of those are the kind of applications we're thinking of real time gaming real time web mapping like this this is where you're going to use this type of functionality now this is two-way directional there we're sending extents down we're getting data back here's a here's a more one directional example basically a dashboard again using web sockets so FME in this webpage is hooked up to FME FME is sending back the locations of vessels and also buses in real-time so as these vessels move we have an ID on them right that allows us to update the location same with the buses you see them kind of move around so again it's all about real-time communications rather than the server always asking and going hey hey or the client rather rather than the client always going to the servants and Hays or new data polling all the time which is heavy we have this push capability now pushing the data to the client we're not running a new workspace back in that first example the first transit example every time we wanted an update we actually had to fire off a similar process yes on server and that's not happening here is it the same workspace exactly it's a different workspace but the transformations would be similar yeah the Google Earth example came and I'll never plank is really the old-school polling method this is the new-school push method where it's not even client-server anymore it's node to node you could have many different nodes in a network it could be one sensor network pushing to another sensor network it's if things are going to get really cool the next few years in terms of pushing data around this will scale a lot better absolutely way better the latency is much lower now I want to show just very briefly in the last actually it's 859 so we're going to go over by a few minutes but we will send out an email with a link to this webinar and we will also well that's the intent you that the link to the data as well if you're those who have to run off I just wanted to thank you for your time and and but we're going to keep going with a few more things here yeah if you can't hang on yeah we'll give you a few more pointers yeah so we're actually going to look at the fme approach to this in 2014 so how do we do it in 2014 well actually before I jump into the FM easy specific demo the nuts and bolts of it I want to bring up the PowerPoint and talk very briefly we're going to do I'm going to do let's do the demo first actually yeah so this is this is a something I built with actual 2014 not 2013 which the last two were built with and here I've got a workspace I'm just going to run it and is there a KML angle to all this I was just wondering yeah there is it's coming up okay coming up if you're left already as well it's too late but I think they're still here what do I got here I've got a web page that hooks up to FME and and what it will do is it basically just buffers so if I click a point here it will buffer so that went all the way down to FME through FME and - me server did a transformation came back up so as I click around this is going full all the way through FME it's being buffered buffer is just an easy example to understand well so you've got this almost like some kind of web GIS going on here right they're very close it's very close right you could use it for that yeah or this this could be you know locations where we're gonna set up I don't know in FME store or something yeah could be could be um the really the point is that is you're sending something down with a server Jason message which we'll show in a second and then it's being processed transformed and pushed right back up and I guess it's not could anybody be viewing that output it wouldn't necessarily have to just be communicated between you and the server and back yeah those two are really not loosely coupled right those two processes that's right that's right it's it's really you can do everything you could do long polling and Ajax you can and and traditional HP you can do now with WebSockets so it's like a superset I would say yeah so if somebody if let's say people have mobile devices and they were just hitting a button saying there's an accident or there's a problem here on the road somebody else could be looking at a map that just shows all of the road problems absolutely yeah yeah and they would see that in real time just like we saw the vessels or very well so how does it work again this is 2014 a little a little a diagram of it the web browser uses a send function to send through a WebSocket down to FME and here's where we got those new Transformers the WebSocket receiver and the WebSocket sender so FME the workspace receives a message does a buffer in our case and then sends that message on to well can send it anywhere but in this case I'm using FME server as a relay effectively to relay it back to the web browser to that Java so that FME workspace is always running always running this is a new new sort of frontier for FME always running workspaces and and we've got a few people that are doing that already and when it works well so that particular workspace doesn't need server to run it just sits wherever you happen to have it as long as there's a you know port open in the firewall that allows add data to pass through absolutely that's and that's a good segue here is the workspace right here and I had this pulled up and running at the same time so you can see that as I click here you can see the coordinates change this message is going through this workspace oh and it's logging in and it's being logged all very cool so if we take a closer look at the workspace we've got the WebSocket receiver here which allows you to connect up send in a message the message comes in in a JSON format so we convert that to FME what's the Jason looked like it's very simple here it is right here yeah but I see KML you said it's Jason Allen there well your your what you're two steps ahead of me actually so the first thing we do is it converted into two to FME so we create a point on second coordinate system we do a buffer that's the transformation could be anything but yeah we're going to Google Maps so why don't we just give them KML oh very good give that webpage KML and so what do we do we extract KML so a KML is a put inside that Jason it's put inside and here's that here's the actual Jason message with KML as one of the attributes of that we then use the WebSocket sender to send that information over to FME server and then FME server takes care of it and here i was showing basically the message is coming through so if that for me server you can watch those messages come through let me pull up the web page and as i click you'll see this refresh you can see the message is flying through FME server and then ultimately back up to the web page now before I used to have to do a bunch of Python coding to get this done okay so how much scripting did you have to do for this I'm going to I'm going to show you how much scripting I had to do right now like just showing off your your scripting talents here my scripting counts right here I define the WebSockets here just to Mesilla I find the the Google map and that's it that is oil that is the entire scripting that I did to create that demo WebSockets and in the Google map and really that's only on there is that Hawaiian side of the server that is the concept and how much scripting the on the server side absolutely none no really no coding required to do to do the server side of this so if you're interested in WebSockets real-time communication whether it's with a web map or who knows what you're doing this is great great solution so download the beta you can go safecom slash beta and you can you can take a look this will be part of the stuff for Center this will be part of it absolutely you get a workspaces you get the web page so you can try this at home you absolutely can yeah so I know that went a little bit over time but we'll flip back here and it was worth it you think is worth it for me it was anyway all right that's what I like to hear here just the showing the FME server side you get the power point slide as well so no worries there that's the complete set up for server pretty easy no scripting involved if you want to brush up on the whole sort of publication topic subscription stuff there's lots of things on notification yeah in the training would be yeah and we'll show a link to the training coming up um save the dates so some somebody is going to do a webinar on on the 28th for CAD and GIS can't remember who that was it might have been me actually I think Dean's going to be kicking it old school August 28 showing off some CAD GIS and then September 10th a mystery webinar I have no idea about this one but somebody's going to do how to consume leverage and publish web services yeah sounds like a good one and again all the webinars are recorded interrupted safecom slash webinars uh FME training for kml topical i'd say yeah it's on about a month's time if you haven't haven't had enough for you getting hungry for more KML by the time September rolls around you can sign up for that how much is that going to cost I believe zero dollars unless you unless you want to get it from Aaron he'll charge you high you can definitely send me money I'll give you my PayPal account but yeah all of our training is free online so just go to safecom slash training you can get the server training which which which oh I see oh it just updated we're at one leg here but you can go to safe comm slash training sign up for the KML training or as Dean said the FME server training if you want to know more about notifications or anything about FME server that's a great training we talked about resources we would we would share links to any pedia so if any PD is a big one that's really our knowledge base and so it's continually evolving is always constantly new content being added and then there are things like the FME evangelist there's Twitter or the YouTube channel so lots of lots of resources there for you to get started yeah and there's a little community on mepd of people that that's that can you know it's like a forum basically - right it's really interactive aspect there yeah and just to take it home to summarize if you are still with us yeah it's still quite a few people there excellent yeah but basically FME allows you to transform virtually any data into KML and of course you can read KML you can fully master the style and the structure of the KML the soups how you want to present it and then leveraging real-time data and time series we had a couple of examples of that and really it's all about integrating KML into your workflow you can either start with KML and integrate it into other workflows like we've seen with the some of the download and emailing or you can start with the workflow and and feed it into the KML so really leveraging KML but not just having it as individual output but fully integrated and I'm super pumped about the last one as my friend iris would say WebSockets are truly very very sexy sexy technology as usual uh she challenged me to to talk about WebSockets and and I hopefully I did a good enough job iris I think they're sexy but it's really up to you to tell me if they're sexy iris all right that's enough say in Dorset just programming us to run out and get websocket that's right that's right so a Q&A do we have time for Q&A well throw us a few questions why not what we're here so let's see if there are any questions that we want to answer here live let's see any towards the end there when will you be doing a webinar on Google Maps engine I'm pretty new that will actually that there are talks about doing that but there's no date as of yet the audio came back which is good we will be sharing the workspaces some people have been asking about that and apparently the work can you use people somebody asked if they can use their data with our work spaces there should be no problem a lot of our work places aren't specific to I mean obviously you may you're gonna have to drop the source feature type and re-add but other than that KML data is set up to be global so it should work anywhere in the world so sitting here on earth that should be good yeah somebody's asking about showing a lot of different vehicles on Google Earth at the same time you can use what's called a network Google Earth network link update control which allow you to just it's kind of an in-between between web sockets and polling basically you ask for updates and there's an example of that on our web site so that was John asking about that yeah somebody asked if you can read from dgn certainly I will turn into my webinar at the end of the month on the 20th to find out about reading from dgn or going from katha GIS so that's certainly something we support somebody else to asked about reading from ECW yeah again KML will not store ECW but FME can translate from ECW to the TIFF or or JPEG or PNG that KML does support so and then it's up to you to structure the ground overlays as the it'd end and there's all sorts of I think I think if you just go new and generate a workspace ECW to KML it should do the right thing yeah it should so I mean that that'll be like Dean was saying that well it'll give you the basic KML file but as Dean was you were going to say it's up to you to structure it yeah you're going to if you have tons of data you'll quickly overload Google Earth as a client so then you have to do things like tile region eight maybe do web tiling that's that sort of thing yeah question do I need FME server 2014 to access sockets you don't you can use you will need either FME desktop or server 2014 both of them have the capability in them combined is probably the best approach but there they are 2014 so 2013 it's a little bit harder and so we focus on 2014 for WebSockets can i how can i name kml elements based on sequel fields probably using the property setter that's a good question and I would I would say probably the property setter would would do that and again don't forget about the reader writer doc because that's really a huge resource of information there's tons of stuff in there so do check that out though basically explained under the hood how FME is is modeling and generating KML somebody's asking about styling for speed as well as location I think maybe that what they meant is even as well as directions so yeah maybe those slow buses you could have a read if this bus is stopped and it's crawling along of the cup is 0 or 0 10 kilometers an hour you might want to avoid waiting for that bus so yeah there'd be no problem doing that as long as the now velocity wasn't part of the transmission we could always compute if you're running this every 5 seconds we could actually calculate the length of distance traveled and compute the speed so that's something FME can always do value-added on on your data yeah basically filtering and do any there's a question here about a boat clarify the point about the ll 84 projection KML data is always stored in lat/long so if you're reading in data and into FME and it doesn't have a coordinate system FME will will say I don't know how to get this into lat/long even if it already is in lat/long it wants to make sure so you basically have to tell in your workspace tell especially if you're coming from CSV or something like that that doesn't have a coordinate system you have to tag it with the coordinate system and then and then F and then the writer effectively goes oh okay it's lat/long or if it's not lat/long it can represent it yeah so you can have a coordinate system setter - ll 84 we just say Ola Forks that's the FME name for that coordinate system yeah good point but which case that would for the web web Mercator you only need to worry about that if you're building what tiled caches or something web map tile Tyler yeah yeah yeah so that's another coordinate system related to Google yeah which is was nine zero zero nine one three yeah was the epsg easiest you need you know that's you have Google yeah turn upside down it's like when you're spelling on a calculator yeah exactly I thought that was funny cool all right I think Carly that's it for questions seem to be slowing down so so we'll probably roll call it good there well thanks everybody for for tuning in looks like we had a lot of interest today and as they're in said this will be the recording of this will be up on on our website and will distribute the both the PowerPoint and the examples and certainly you can try this at home try to download an evaluation if you don't have FME you can try it out for free play with it and yeah have a go yeah don't and don't be shy about contacting us at support safecom slash support if you get stuck or you get into a work space you're wondering how to start I'd much rather you contacted us and to get started then didn't use the product so that'd be the last thing I'd leave you with yeah supports always there to lend a hand right thanks have a great day yeah thanks for turning in we'll talk to you next time laughs now

Channel: FME Channel

Views: 79,605

Rating: 4.4336281 out of 5

Keywords: FME, KML, Google Maps, Google Earth, Safe Software (Business Operation)

Id: MCbWq0v0jeY

Length: 74min 1sec (4441 seconds)

Published: Thu Aug 15 2013