Using GIS to Capture and Manage Structured Observations from Imagery

hello and welcome everyone my name is kelly here with ezri and i'd like to welcome you to our webinar today using gis to capture and manage structured observations from imagery before we get started i just want to go over a few items this webinar is being recorded and you will receive a follow-up email with a link to the recording location once it's made available you are encouraged to post your questions at any point during our demos and presentation today you can type them into the question or chat box in your go to webinar control panel and our presenters will be collecting these and addressing them at the end in a q a session and with that i'll go ahead and turn it over to our first presenter today ben conklin thank you for that kelly so thank you for joining us for the national government webinar series today we're going to talk about using gis to capture and manage structured observations from imagery we have a team here today to talk to you about this topic i'm ben conklin i'm our director for defense and intel solutions with the nazarene i'm joined by kurt chrisley and kyle talbot who are going to take us through the capabilities across arcgis for helping us manage structured observations from imagery some of you are probably familiar with this but you don't have to be to we're going to take you through what structure observation management is and how it supports sharing information it helps us normalize capture and record observations from sensors and sources in general when we're talking about these observations we're referring to activities events or locations at a point in time we obtain them from imagery intelligence reports or other means from an analyst capturing structure observations help support many different workflows like automated analysis and pattern detection using tools like artificial intelligence it helps us maintain situational awareness using structured reporting and it can support workflows like object-based production key takeaways from the session are that you're going to learn how to use arcgis as a management system it'll help show you how to establish a web-based workflow for locating identifying capturing observations and also how to configure your enterprise to create services that can support these workflows as well as how to share metrics with stakeholders provide some context just needed to look around our world as we see it today our security is increasingly challenged i don't have to tell you this many of you of course are probably attending this webinar from home because of the global pandemic but at the main or maybe you had to hurry up and get out of the way of the hurricanes that we're facing or the fires that we're facing with the world changing everywhere we see many threats and many challenges and this is continuing to cause us to need to work together to respond to these what we would also say about this is the pace of change itself is actually accelerating so it's creating many challenges which certains are security in society and we're in this new normal of constant change and we need to improve our understanding collaboration and action in order to respond to it i like to think of this in the context of how intelligence has changed over time as well so some of you in this call may be part of the intelligence community others may not but much of what we're going to talk about supports these changes and some of them might be to do with military intelligence and others might do with national strategic intelligence and some might be in corporations or companies as well in the early days of intelligence we really focused around humans in the field collecting data and around single threats and over time we've evolved to this technical collection capability and to respond to the threats on our homeland and using new capabilities like uavs and web-based tools and now let's say we're in this fourth age of intelligence and it's really we're facing this rise of peer or near peer threats um and continuing to focus and face threats from both domestic and global terrorism what's really necessitating is this multi-intelligence integration and also the need to address across multiple domains like land air sea space and cyber and bring all those together in our response and we're leveraging new technologies like ai and other tools to help us automate and what's clear to us is this increasing rate of change is continuing even in this community and what we're seeing is this need for this digital transformation of intelligence and i just say need we also see it happening in many organizations as they re-envision their workflows and in many cases what's happening is we're moving away from these very sequential and in stepwise workflows into this digital transformation which is allowing for simultaneous and integrated operations and it requires a transformation at an organizational level but also how we operate as individuals this massive transformation is helping us interconnect information processes and workflows and it's really all about things happening at the same time which is helping us create this smart and dynamic decision making and helping break down some of the barriers and help aid in collaboration and at the center of this is the way we see it as really location intelligence and that's because it helps us understand everything time and space are fundamental integrating capabilities all activity occurs at a time in a place and location provides us the framework for understanding communicating and providing context to our intelligence this demand for integration is putting increased stress on geospatial professionals and what we're seeing is organizations are quickly moving to provide tools training and development to make everybody in their organization be able to contribute to location intelligence as part of their daily work and in order to make this illustration even more clear we're going to go ahead and demonstrate to you this type of workflow of somebody capturing structured observations from imagery and contributing to a shared understanding of the battle space in order to do that i'm going to turn it over to chris lee who's going to take you through a song workflow uh hello good morning afternoon or evening depending on where you are in the world my name is chris lee and i'm a solution engineer on the defense team based out of the washington dc office and today i'm going to be introducing you all to excalibur and conducting a sample som workflow within it as well as explore one of our other technologies which is arcgis dashboards we can use to kind of view our analysis so here you have or here we have arcgis excalibur it's a project based application here you can see i have a essentially like a web map frame with a collection of 12 images over in this case over the san francisco and oakland bay area we also have some project layers which are feature layers representing uh the three observations that we're going to collect in this workflow and what we're going to do today is we're going to capture observations on moving targets within the oakland san francisco bay area uh using these images that were taken different times and from different sensors uh the first thing i like to show before we actually collect some observation is some of the [Music] exploratory capabilities that come with arcgis exp that you can do within arcgis excalibur and one of those is the use of this time slider so through this i can actually see which observations were taken at which time which allows me to you know understand my data from a temporal standpoint the next thing i like to do is actually explore again how we can keep track of the different observations that were taken and to do so i'm actually going to use the flicker tool and what this allows me to do is i can essentially turn on two different image layers which i've done in this case i have an image from september an image from april and what i can do is i can actually use this flicker tool to uh identify which observations were taken from which image now that i've explored some of the uh you know capabilities within arcgis excalibur i'd like to now actually collect an observation and to do that i'm going to turn on what is called a image space so currently we've been operating in the map space and this is where you know all the images have sort of been orthorectified and transformed so that they can be displayed on the map however we're actually able within arcgis excalibur to view this oblique imagery which as you can see here in the attribute table working with a rather oblique imagery we can actually view this imagery in its native format and viewing the imagery in its native format will allow us to collect accurate observations or collect accurate measurements while collecting observations in this case i'd like to uh capture an observation on this movement ship here and before i do so um one of the things that i'm capturing as part of my observation is the size of this these ships and so using the measurement tools within arcgis excalibur i can make accurate measurements of my observations collecting observations themselves is also very simple with an arcgis excalibur i can simply select the observation that i'm making place the point in this case we've collecting a variety of attribution with these moving ships we're tr capturing their class in this case commercial medium-sized boat i can hit submit you'll notice that you know i made the i captured the observation in the image focus panel however it's maintaining its geography when displaying into the map focus panel and that's an example of arcgis excalibur really doing a lot of heavy lifting on the back end from a mathematics standpoint to kind of make those transformations i'd like to now switch gears over to arcgis dashboards which this is an application that is commonly used uh to take your analysis and display it in a very interactive and uh intuitive way for the rest of your organization especially leaders and decision makers and i'm actually in edit mode of this dashboard and i kind of want to demonstrate some of the the functionality that we can do within arcgis dashboards and the first thing that i'd like to mention is that i'd like to demonstrate is that we have the option to filter our observations from the images that were taken we can also add these indicators as shown below the map i have an indicator for each of the three observations you'll notice as i zoom around the map uh the indicators change based on the extent so you know whenever i pan or zoom somewhere it's automatically filtering the number of features that is within the map extent as you can see here on the left we can also add various charts and graphs and i'd actually like to add another chart now earlier when i was capturing an observation of the moving ship we gave it a class and i'd like to now demonstrate how easy it is to display a pie chart that can accurately you know reflect the you know a breakdown of all the moving ships by their class so arcgis dashboards are not only simple to build interactive and intuitive they're actually also live feeds if i return to arcgis excalibur and i make another observation in this case i would like to capture an aircraft you'll notice that i was originally at 13 aircraft now as i move around that aircraft number just jumped from 13 to 14. so together between arcgis and excalibur it makes it easy for analysts to collect observations and then relay that those observations and that information to the rest of the organization and with that i'll hand it back over to you ben great thanks for that chris so what you saw there was a great demonstration of that that ability to capture these structured observations and then that information can be dynamically integrated into an operational view and shared out with decision makers or other people who are working on maybe managing the collection of the project or the data or other analysts who are working on the same problem or area so fundamentally data integration is a complex problem in intelligence we have a wide variety of different source data coming from sensors and people being able to ingest that data into a system that provides access to everybody in a variety of applications we will normalize that data in terms of data structure but also meaning and then be able to provide analytics on that data to provide that integrated view fundamentally say this is the power of gis js is all about integration it's about linking and combining information you can dynamically link to data together through management of features and records like you see in the structured observation use case it can be used to visually overlay information through mashups by bringing different data layers together to be visualized in a single view and analytics can be run across layers of data from a variety of different sources in other in order to detect patterns and trends when we talk about the intelligence support for gis we really talk about this ability to see patterns connections and relationships in data and intelligence is built through a cyclical process that cyclical process is repeated over and over again around specific areas of focus by planning and directing intelligence collection by doing processing and exploitation on data doing analysis and production of intelligence products and then disseminating that data and integrating it and what we're seeing in this new dynamic age is that these processes are rapidly increasing and there's more demand to share dynamically through every step in the process and in some ways the line between each step is blurring as we integrate tools like these and analysis and exploitation tools into a dynamic workflow where we can directly disseminate and integrate that data into products and workflows so then now we're going to get into the heart of this idea of what is structured observation management so fundamentally we define structure observation management as the ability to capture data in a structured form using a common data structure or ontology which which shares definition from one analyst to another or from one mission space to another so that way the data can be combined and used and analyzed in a variety of different formats and used by a variety of different consumers so analysts can capture this data using smart forms like we saw earlier on the excalibur for example where the there's drop downs and pick lists of existing structures that we've all agreed on as well as other formats editing tools in arcgis pro or even in forms that you run side by side with other analytic tools we can also train ai to classify and capture data and report it and label it in these using these same structures and in both cases that information can be fed into a common system speaking of a i want to talk a little bit about how artificial intelligence fits into some workflow but also in general how it can be part of many steps along the way so artificial intelligence of course in the computer vision case can act much like chris but i may be a super powered chris where he's looking for ships in the ocean well maybe ai can help relieve him of that burden of finding the ships and they can identify them for him and he can concentrate on understanding what the meaning is behind the locations of those ships or the activity so an ai can find and use computer vision to find those objects and it can record those as observations into a database in order to help improve our knowledge of the area you can also take observations recorded from analysts or from machines and it can use those to filter those in order to understand the importance of that data in order to understand larger patterns and trends in that data in order to do alerting on that data and then finally it can help us push out these key trends and identify patterns in the data in order to drive action so you can really think of ai as existing in many different steps along the way of the intelligence process both helping us at the computer vision stage but also artificial intelligence and other forms of automation can help us manage and respond to observations that are being recorded by humans and machines and also help us push out data to the right user of that data and this is really critical because as we look at the future of intelligence we see this real shift from this need to focus on known locations and known targets into this area of the unknown in intelligence so we'll define this chart for you a little bit so in general when you're thinking about intelligence and that you need to collect intelligence on your adversaries you might know things about your adversaries like what is their signature how do they look like um or how are they behaving or you might know their location so if you know their location and you know their signature you might just monitor the location so an example would be a missile silo that you know where it is in addition to that you might not necessarily know these locations or you may not know the signatures in which cases you might be searching for adversaries in a broad area or you might need a research to understand better their behaviors or signatures and then finally the cases where you have both of these two characteristics are unknown you need to do discovery and that is the realm of activity-based intelligence so this session is not on on activity-based intelligence we have a number of um recordings on activity-based intelligence on youtube you can also go to my website and read a lot more about how gis supports activity-based intelligence but i wanted to bring it up here because it's really very tightly interrelated with structured observation management so you can think of activity-based intelligence as being the method for discovering the unknown and then when the unknown is discovered we now have something that we can do structured collection against and so som is really about doing the structured collection that can feed back into identifying knowledge gaps or improving our knowledge of the known world and that's really the point of song we're going to come back to this topic a little bit more but the next thing we want to do is actually talk very specifically about the software and tools that are part of arcgis that help support structured observation management workflows in order to do that i'm going to turn this over to my colleague curt and let him take you through these capabilities of arcgis over to you great thank you ben let me go ahead and share my screen here okay well you know what ben said was exactly right with regards to the integration capabilities of gis and the science aware has been empowered by the ability to reach out and start exploiting all different sources of information coming from different sensors whether they be drones satellites aerial photography this is now a fundamental part of the arcgis software package and it's exciting about where we're going with this technology so if you look at some of the usage patterns that we're starting to support you know first of all we're dealing with a large variety of different imagery from different sources uh in different modalities and so we need a management capability and that's fundamentally at the heart of our platform what what chris was showing in his demo was kind of a very simple application but there was a lot of sophistication behind it to make it all work and it all started with that imagery management the fact is now we can bring in imagery from all these different sources many many terabytes petabytes of imagery and be able to make it instantly accessible to users and thin client applications so that's fundamentally the heart of imagery management but we can do more than that we can make that information that remotely sends content geospatially very precise using photogrammetry so taking advantage of sensor models other sources of perhaps some ground control that you have we can take that imagery bring it in make it geospatially precise and we can even use that to extract 3d information from that using photogrammetry so that's a fundamental capability that's now part of the platform and as ben was also showing and talking about our our new goi capabilities you know uh imagery has always been about making observations looking at the imagery but we can now leverage the computer uh to do that in in ways that are maybe much faster quicker and can handle and is scalable for the amount of imagery that's available if we can start leveraging geo ai and and that is now a fundamental part of our software offerings and then finally even more deeper than that we really want to start looking in those those very rich deep patterns that fall within the imagery and so we've added a variety of raster analytics that allows the users to define fundamental workflows potentially you could look at this as tradecraft and being able to do this advanced geo in type of work using all types of sophisticated algorithms and so that's really where we're going from a functional standpoint on our platform this is how it shapes up from a product standpoint and you can see the different components that that were used to actually give chris's demo you know from bringing the imagery into the left making it orthocorrect but then sharing it out in a in a server-based infrastructure whether that be cloud or on-premise and then being able to use the excalibur tools to exploit that something that's key though there in the middle is the work that we've done around pro and the image analyst extension for pro and these are the heavy lifting tools when it comes to imagery in the arcgis platform and we'll cover that more in a little bit of detail matter of fact that'll be part of the next presentation about about how we use pro to bring this all together to make it a very quick and easily solution for the vast majority of image analysts out there um but you know before we go into pro though i just want to make sure everyone is aware that we do support a large variety of different types of imagery arcgis has a notion that we call known raster data types and we're up to like 154 different known raster data types we also include point clouds in this particular data set as well so we can support you know imagery taken from lidar and those types of things so it's very exciting about all the different major sensors that we're supporting in our known raster data types and then within within the community nitf is a huge player in that and nitf is now a fundamental capability of our imagery management we can fully read all the nitf tags we can leverage that we maintain and persist all the metadata associated with those nitf images and we have now more more recently added in msp capability so with the mensuration services protocol we can leverage uh the nitf imagery to do all types of advanced mensuration ortho rectification from sources that usually get presented as nitf images at the heart as i talked about of what make this all works is our imagery management component and there's two elements inside of imagery management that make it all work uh the first of this is the mosaic data set and that is our data model for imagery and it's this data model that allows the image server which is the other component to rapidly go in and find the pixels that someone wants to exploit and work with the data and this is really important we we no longer need to get hung up by knowing which file which image file is located where we've set up the system so that the user just goes in draws their bounding box around their area of interest and between the mosaic data set in the image server it automatically finds the pixels that that analyst wants to work with and displays them on the screen so they can do their their analysis and so this is how you can very quickly go through you know massive archives of imagery find the pixels that you want to work with and be able to do your exploitation in a timely manner we can add something more into this particular workflow and that is uh the ability to add raster functions to do different types of processing and so let me explain how that works in the old days of working with imagery you would have an expert who was your image analyst and and they would know the different types of things that you can do for imagery they had the trade craft but you know it took an expert and they would go out and find the sources and they would actually create the information products as they were as they were requested by the consumers but what we're talking about now is the fact that you can take different functions and you can add that to your archive of information and the image server knows how to dynamically process that imagery on the fly and so what you end up is with a situation where the actual consumer can go in to find their area of interest and have the system automatically create the information products they need and have it brought back to the server and that's how we've set up ax caliber to work you know we put imagery in the image server we add raster functions to those archives of images and then when the excalibur brings that up to the analyst for exploitation it has things like the the look up curves it has the pan sharpening it has the ortho rectification all fundamentally uh in those it it's fundamentally done in the processing to bring those pixels across into the next caliber application to be able to get the work done so that's a key part of that image server and that really is the ability to embed this tradecraft in the form of these raster functions uh to be able to do this work so it's a very powerful solution and kyle as he goes through his demo he's actually going to show this and how this works and this is just an example and especially as it relates to saw so we have this area it's a target area over iran and you bring your image in but right now you're just using the geographic coordinates of the image and the sensor model to correct that well we can look at the metadata we can do a dynamic range adjustment to the brightness and contrast we can fix that but maybe we want to add some terrain in there to do an ortho rectification to make sure it's extremely accurate that's a raster function we can add and it's just a drag and drop function uh and now what we can do is because we've persisted the elevation data as part of that image service uh when the user goes in and does their som exploitation the points that they collect will have 3d coordinates and what that means is is that the next time when the ns goes out and finds the next image if that image was taken from a different look angle it doesn't matter because the the psalm exploitation was done with 3d coordinates that exploitation values will fall directly on top of that new image even though it was taken from a totally different look angle and so that's really how the arcgis imagery platform is supporting the som workflow and then to show this in a little bit more detail i'm going to turn it over to kyle and he'll walk you through some of the key things that uh allow this to happen so kyle let me turn it over to you all right so my point here i'm going to try and bring some of those uh those points home that kurt was talking about but i want to start off with very quickly in chris's excalibur project to set context excalibur really is a great tool for doing these som workflows because as you can see it's in a browser so it's very lightweight the interface is very easy to work with and it gives you all the tools that you need to capture capture these observations one of the reasons why excalibur is so lightweight is because it operates off of these image services or or wms layers so you don't have to download any of the imagery locally you can just work with what's available to you on the server and still get all of the pixel information all of the geographic information that you need to be able to quickly work with these images in map space and in image space so what we're going to do is i'm going to jump into our desktop software this is arcgis pro we have a couple of different ways that we can create image services making making that imagery accessible throughout throughout the arcgis platform there are a couple ways of doing it pro is honestly the best way of setting up these image services because it's the most powerful all-inclusive piece of software that we have so pro so unlike excalibur that works solely off of image services and wms layers pro can allow you to connect with your imagery in all sorts of locations so you can connect with your imagery locally what you have downloaded here and you can do that by simply adding layers you can also come to your catalog and connect your folders everything that you want to work with locally and by having it in the catalog you can simply drag and drop it into your into your frame you also have the ability to connect to other different storage containers so you can you can use pro to connect to different databases all different types of servers whether those be arcgis servers wms wmts servers you can even add cloud storage connections so for example we have a cloud storage connection set up with an amazon s3 bucket and so we can connect to our imagery directly in s3 and add it to our project without ever having to move it locally to work with it so there's all of our same san francisco images that we've had locally as well now once you connect to your imagery the best way for managing all this imagery especially the large collections of it is what kurt referred to earlier which is the mosaic data set now the mosaic data set um in in managing imagery and you can you can create mosaic datasets in any of your geodatabases in your project um a mosaic dataset does not store the imagery it's not it's not a it's not a container in that regard instead it's actually a pointer file and so what you can do is when you're adding imagery or adding rasters to your mosaic data set it isn't actually uh ingesting duplicating data rather it's just pointing to wherever those files reside and making them making the pixels from that imagery accessible when you need them by doing that on the fly processing and so when you're adding rasters to a mosaic data set you can you can pick your standard standard imagery type you can also identify different raster types that will help give the best performance to to the imagery that you're ingesting so if you're working with native imagery as you very likely will be doing for these for these types of workflows you can add native raster types you can add processing templates directly to the imagery as you're adding it to the mosaic data set and you don't even have to add individual files you can simply have you can simply have this tool crawl the imagery in a folder and essentially what it's going to do is it's only going the tool will crawl this imagery folder that i have and it will only add the knitted images uh within all of the folder directories into this mosaic data set now this tool takes about two to three minutes to run for for this data set that i have so i've already went ahead and created it and the mosaic data set not only lets you work with the images on the fly so if we open up the attribute table here you can see we have about 90 we have 94 images in this set so if we were to have 94 images open it would just take too long to process all the pixels the mosaic data set lets you get only the pixels that you need so not only does it do that but the mosaic data set allows you to view the attributes of each individual image so it's great for cataloging functions now one of the things that kurt talked about this is a huge advantage to working in pro to set these up is is working with these raster functions these raster functions again when we talk about the the calculations and analysis that you're running um unlike geoprocessing tools that are creating new data with every time you run an analysis on on your vector data raster analysis also works or raster functions also work on the fly so it's done very quickly and no data is duplicated so what we want to do with this is we want to add elevation so adding elevation to your layers will help increase the accuracy if you notice here um we're in san francisco san francisco has a lot of hills and so if we're just using a constant z when we make these when we make these collections um it's um we we want that increased accuracy with that the elevation gives us now there's a way that we can um we can edit the mosaic data set function as a whole and then you know i can go into my raster functions um drag in the geometric function connect that and and i could run this you know and i can add the dem here right however what this is going to do is it's only going to add it to this uh this function to the mosaic data set as a whole now if you notice in excalibur we pull out uh individual images and work with them as we're as we were capturing these observations so instead of applying it to the mosaic data set as a whole what i want to do is select all of the footprints in this mosaic data set and now what i'll do is i'll run a batch edit of raster functions so that this way it's the raster function will be applied to every single image within the mosaic data set so here i can select again that geometric function update it here and i can instead of using the constant c i'll use dem i'll use this srtm file which will greatly increase the accuracy and then it just asks me if i want to run it tells me all the object ids in the mosaic data set that it will run on and as you can see it goes through it very quickly on each of the different items now the mosaic data set is updated clear the layer now to give you an idea of what exactly that does we're going to zoom in to downtown san francisco and what we can do is we compare this we can compare this image here this is a non-corrected image and you can see the difference between the two of them how the elevation adjusts it and and makes it makes it much more accurate so now that the uh now that our image now that our mosaic data set is ready for um som analysis what will what we can do from pro we are connected to our arcgis enterprise this is the same enterprise that we were working with in excalibur this is where our image server is is located [Music] and so what we can do is we can share this mosaic data set as a web layer and so it's a simple matter of just giving it tags the summary you do need to either register the reference reference the registered data or copy the data up to the portal if you're working with imagery locally you'll need to manually move the imagery up there however if you were working with imagery in in s3 other cloud environments other different databases you can simply register the reference the registered data as long as those cloud stores are also connected to your arcgis enterprises as raster stores and so once that's shared as a web layer then it becomes accessible in your portal not only as you saw it there in excalibur but you can bring that layer into pro as well so we'll we can turn this s3 layer on and if i open up the properties you'll be able to see the source of this layer that it's connecting to the image server itself so so there you can see that it's got it's giving it's connecting to the rest endpoint rather than a local image and so and we can work with this image service just like we do with the local imagery one last thing that i want to show though just to highlight the interconnectivity you know pro is pros for great for all this heavy lifting but it also does the simple stuff too so that moving ship layer that we were working with in excalibur that is a feature that's a feature layer a hosted feature layer and we're able to bring it into pro and work with it here as well and so what i can do i can also work with these images and image service or the work with these work with this layer in image space in pro and while it's not as simple as an interface as an excalibur i can i do have the ability to create different features and update the attributes here directly in pro as well the advantage to working with this in pro is that you have the ability to [Music] work with different analysis tools so we could run different types of analysis on these on these points like finding hot spots calculating density etc of all of all of these features that we've captured we have access to the whole suite of analysis tools here in pro so with that i'm going to turn it back over to ben great thanks a lot for that kyle that was really great to dive in um deep into how we configure and provide out services as a way to make the imagery that we're using for a structured observation management accessible accurate precise we saw i'm going to have functions and processing on that imagery to make it easier to use and ready to use um for this type of analysis and to make services so the key about that is those services can be used anywhere in any of the clients in any of the applications across arcgis but also by others it's open platform that can be used by a variety of different systems as well anybody that has a standards-based approach to working with geographic information so and i'm going to dive into a little bit more about kind of principles around structured observation management as we started earlier so we talk about different kinds of intelligence structure observation management can actually support a wide variety of different types of intelligence it can help us with strategic intelligence by helping us understand and do better estimates on our adversary for example it can help us with current intelligence and can be in products like the president's daily brief with things like indications of warnings and day-to-day intelligence it can be used to produce data that can help us support basic intelligence like understanding factories and production output and other things like that of countries or nations or our specific capabilities and as i mentioned already it can help us with discovery intelligence by being that strong connection between discovery and known and the known universe when we talk about um this traditional intelligence discovery intelligence the kinds of things you're gonna be looking for are gonna be different in the different scenarios so in traditional intelligence you might be looking for these very durable or physical signatures of an adversary so missile silo at aircraft anti-aircraft and where the signatures in a discovery case might be changing constantly and so you actually need to adapt your model so an example you know in early days of operation um you know offer iraqi freedom whether then oh now we actually want to see where um white toyota pickup trucks are or something like that right so then the type of thing we're looking for is different the size of the unit we're looking for is often different in this new world we're often looking for individuals or specifically type observations where um in the traditional intel we're looking at larger things and i think some of the things that are really important is as we start to think about the frequency of collection and the types of things we're looking for it starts to become much more dynamic and much more real time so coming back to this idea of the relationship between abi and som as as we identify the kinds of things we want to collect using our abi workflows we want to go out and use a structured process to capture our observations related to that so for example we might determine back to the white toyota case that we decide that people driving vehicles around in the middle of nowhere are something we want to look for we don't know where they're going to be driving but we know they're out there somewhere and we need to look for them and when we look for them we need to capture what they're doing what their activities are and and in what ways they're operating and that's really how these two things relate together and this is really we're going to dive into our kind of next major area of psalm and that's around the areas of motion imagery and even still imagery can play into this and the idea here is what we're talking about is the idea of capturing activity so not just the existence of an object at a location but also the activity of what's happening at that location and often motion imagery is a really a key element here because it's the best way to observe short duration activity so imagery can be useful for long duration activity like ships in a port or cars in a parking lot things that stay in place for a long period of time but motion imagery is really about all the other activity that we all experience all the time in our lives which is that rapid activity and we can of course apply ai here but we can also do manual analysis or interpret analysis on imagery as well and to get us into that actually we're going to ask kyle to give us one last demo around motion imagery all right so yeah i'm going to show a very quick demo to end out here this is on um our full motion video capability this is something that is also available in in arcgis pro what it allows you to do is it allows you to connect with video data that is what we refer to as msb compliance so that that refers to a certain set of uh standard geographic standards in the metadata that allows us to uh project this um this video data uh onto a map and so as you can see i'm gonna go ahead and dock this uh video window right here as we have this video play in this 3d scene that we have here we can see the with the position of the sensor its flight path we can see its line of sight and we can see its field of view and so this tool the video player you know gives you many different uh standard video playing tools you know pausing fast forward you know being able to scroll ahead jump ahead to different parts of the video but it also gives us a lot of different tools for that are very helpful in some workflows so one of the tools that will show or one of the abilities that we'll show just to start here is the ability to use your feature services or your feature classes in in pro and use them to edit what you see in the video and have it appear in the map i'll give it just a second here if you'd prefer kyle i can come back to you if you want to try to troubleshoot that and we can come back yeah yeah let's try and troubleshoot this i apologize sure no no problem no problem we'll just keep keep on moving we're getting close to the end anyway so we'll close let me know at the end if you're able to do that and we'll come right back to it um so i think the the point we were going to show there we got to see the motion imagery which is the most exciting part um of course you've all seen us drop dots on imagery and that's exactly what kyle is going to do is drop a dot on the motion imagery just like we were doing on still imagery and it's really fundamentally the same concept from there which is which is pretty great and so really what we're going to then talk about is this idea of essentially it's getting information from data right so we've talked a lot about still imagery and motion imagery so you can get certain types of data from imagery so buildings roads features vehicles you can get from motion imagery you can get things like tracks and human activity other kinds of imagery can give us things like materials and and warm objects operating equipment etc but we can actually also get structured data out of other data sets like textual reports and financial transactions and in all these cases what we're thinking of in many cases is this observation this like discrete bit of data what we also want to do quite frequently is not just take that discrete bit of data we want to understand it in the context of what's happening around it so we might want the individual activity to be captured but we also want to compare that to context we understand um what's the typical patterns in that area for example we also might want to bring integrated biographical information around a specific entity or relational information describing relationships one way to do that is to link data through databases and identify identifying values which is one way we can do that the other way is to is to do what we call geo enrichment so based on the location of a piece of data we can compare it with and combine it with and enrich it with other data so for example you might want to know what is the demographic makeup or the tribal boundaries or the political boundaries for this observation what is that typical pattern or trend in the data and we can do that by just combining the location of the observation along with this reference and contextual data and that also applies as we start to think about taking in other data from other sources beyond imagery so as we think about integrating in observations and recordings from natural language processing so from textual data sources whether that's structured reporting that we might use using a tool called locate xt that takes location out of a structured report that's in a but it's not in a geospatial or a database format we can find locations in that data and make structured location aware data we can also use natural language processing tools that are found in ai to find entities and locations and create relationships that way so we can bring in observational data both from you know humans observing on imagery or from humans running processes on semi-structured data or even from natural language processes that run on unstructured data and ultimately we bring that all together into this common structure and that structure is stored inside of a data store in arcgis which can be a variety of different kinds of databases and then it's provided out as services which are standard open services called feature services it can be variety integrated into a variety of applications so you think of this as a standard way to record information but also it provides an interface that is standardized that we can use to push out these observations and records out to a broader audience so that's going to conclude our structured artificial measurement workshop i'm going to give kyle a second to see if he's able to show us the rest of that motion imagery um well he's doing that if he can um you guys can think about questions you have and ask him in the questions module kyle are you going to be able to i am ready to go awesome perfection 100 it's 20 20 man but we're good to go now so again we've got a car follow here and so you know i've got my uh feature layer here that i can drop dots on to the video and now you can see it's working very smoothly um but i can drop points breadcrumbs on the scarf as it follows and you can see in the map here that it has adjusted or that it has placed those features uh where they correspond i like any som workflow i have the ability here to update the attributes here so i can type out comments i can type suspicious vehicle and if we zoom in to the video here i can see that it's a it's a white pickup so we can add that and i can even uh get some measurements on the truck if i if i wanted to add that as well get the length of it change it from meters to feet we can add that as well 25 feet so very um very easy to integrate these types of uh traditional sum workflows uh directly into in this the other thing that i wanted to show actually [Music] was the other thing that i wanted to show was the ability to take a frame capture and add that directly to the map so what i can do with a single click i'll move it onto the bridge here this is a good spot when i move it onto the bridge here in a single click i can take the current video frame save it as a single image and it's going to automatically orthorectify that onto the map where it's supposed to go so you can work with these single imagery layers another thing that you can do instead of just clicking one image at a time and adding that to your current map you can save multiple frames to images as you're playing the video so this allows you to select the format that you want it saved in [Music] add the add the frequency in which [Music] you want this these frames to be captured so we can we can have it save a frame every four seconds or we can save it by the frame number themselves you now have the option actually to add these images directly to a mosaic data set and you can you can select here [Music] what mosaic data set you want them added to so we can add them to this one if we wanted to i already have the mosaic data set set up for for this portion of the demo but so as we have this selected and as we play as we play the video it's automatically saving these uh these images to a folder and so you can see the image is saving here as the video plays as i said you can [Music] you can have these images saved directly into a mosaic data set so this shows this is an area where we um where i collected the frames earlier and if we turn on the layer you'll be able to see all of those footprints that were captured and so now what we can do with this mosaic data set is we can take this workflow and take it back up to excalibur just like we showed earlier i can you know i can i can share this as a uh as an image service and then uh people can do the same workflows up in that lightweight environment as well so uh i think that's about all the time that we have so i'm going to turn it back over to ben to finish it out yeah great thanks a lot kyle that was perfect and why we were able to see the full um set of capabilities there so i'm going to i will go ahead and take over and close this out and just basically answer a couple questions as we're getting close to the end of the time here so again feel free to ask some more questions in the window if nothing else will help give us ideas of things we could answer in the future or maybe reach out to you directly so one of the questions that came in early in my presentation that i did want to address was actually from molly ross and it's and she asked can you discuss the back end is this a feature service into elastic or question mark and i guess i think it's a great question we did spend a lot of time on the back end of imagery but we didn't actually spend a lot of time on the back end of the observations that are being recorded so i just want to answer this question briefly this is actually a topic we cover in many other sessions as well so basically what i would just point you to is these observations are all recorded as what are called features and features are stored inside of a feature service a feature service behind it has a database and there's a lot of different databases we can use behind feature services if you want to look into arcgis enterprise is how you would do that on your own networks with your own databases or if you're using arcgis online all of that is basically hidden from you you just work with the feature service directly the main thing the main power of that is a feature service provides us a structured way to capture data and it provides us templates that we can use to make it really easy to quickly capture features so behind the scenes in excalibur or in arcgis pro attributes are being populated there's a lot of values that are getting captured like in excalibur for example it's automatically capturing information about the imagery that you're collecting the data from et cetera there's actually a lot of rich data being captured all the time in all these different applications and that's also getting stored in the database and that's pretty useful later on when you're trying to do analysis or share the data with stakeholders so i just wanted to address that specific um question okay one more thing i'm sure go ahead one more thing to add to that then too is if you don't have an existing template uh you know for your feature service excalibur allows you to actually create one and so you can actually add your own fields and whatever you want to collect right inside of excalibur which is kind of a a pretty powerful capability for a thin client application like like that yeah so then what's pretty great about that thanks for pointing that out is that you could basically start with nothing besides some imagery and get rocking and rolling with capturing observations with essentially no nobody else you could do it all by yourself and get get up and running real quick in a web browser with no no administrator or anybody has to step in for you so that's that's super useful great um i think some uh a couple other questions here really kind of related to um where can i do you that use this where can i have access to this so generally speaking um if you're running on you know some secure networks there are a lot of these capabilities available on on those networks if you want to know more about that just ask for us to contact you and when you close out of the webinar there's a little opportunity to say you want to be contacted by esri and we'll get you in contact with your account manager that services your specific agency and they can point you at resources for getting access to some of these tools um excalibur that you've seen a lot of today is a product that is something you add to your enterprise and that you use with uh other other to add to your enterprise so if you already have arcgis enterprise you already have image server and image services then you need to bring in excalibur and add that to your enterprise so again if you're more interested in that um please let us know and we'd love to get you connected with the right people to tell you what you need to do to get up and running if you've never used arcgis before we'd love to also help you with that yeah kurt you wanted to add yeah and you know the one question there from max uh you know one of the things that we're going to be releasing here very shortly is a image server capability for arcgis online so right now when you run excalibur it does require an image server enterprise and getting that all set up but in the future we're going to actually have an image server capability in arcgis online so users can actually just push their imagery and arcgis online it'll create the image service and then you can hang an excalibur application off of that so i just wanted to point that out then that'll be out before the end of the year great yeah thanks a lot yeah thanks for bringing that up that is pretty darn exciting actually as well so that'll be great when we have that that capability available all right i think with that we're going to go ahead and wrap it up i really appreciate a pretty lively set of questions from the audience and a great attendance overall if you enjoyed this session i recommend registering for additional sessions um we have some more imagery related sessions coming up even as soon as next week and then some others around things like infographics and actually geoai which we talked about quite a bit today and which of course will touch on imagery and facility management so just revisit the national government webinars webpage to sign up for those upcoming webinars and we're adding webinars all the time as well basically as long as we stay in this covid situation we'll keep adding webinars every two weeks same kind of that time same bat channel and we look forward to seeing you again in the future so thank you very much and have a good rest of your day you

Channel: Esri Industries

Views: 524

Rating: 5 out of 5

Keywords: Esri, ArcGIS, GIS, Geographic Information System, ArcGIS Pro, national government, arcgis, arcgis pro, imagery, SOM, structured observation managment

Id: cOInd3Z1J7s

Length: 61min 33sec (3693 seconds)

Published: Mon Oct 05 2020