Field surveys with QGIS, Mergin and Input

my name is hung from the cross from ihe Delft Institute for water education I'm going to show you in this video how to do a field survey using different open-source applications QGIS the merchant platform and the input app this video is based on a tutorial that is on our OpenCourseWare website you can find it at ocw.mit.edu we're for pre-processing GIS data for hydrological models then you can find it under the tab field service with qg s merjan and input the tutorial was developed together with Lutra consulting who has developed the tools the tutorial basically follows the workflow that you need first preparation of the project loading background layer setting up the forms apply the styling and set the map themes then transfer of the project data to the merchants cloud service and then it is explained how to work with the input mobile app when you are in the field using the forms using the GPS and making edits and then synchronizing it back to the cloud service we can download the data from this tutorial from this website let's first do that you'll use here Sentinel 2 data and we're going to make a two color image and a false color image as a background for in the app we'll use Open Street Map as a reference for topography using an online and offline version because we can be without network connection we're in the field demonstrate how to use a vector layer also in the background in this case we'll make a boundary of our study area and we'll make a survey layer so let's download the file I've been dipped it in a folder on my hard disk let's go to QGIS if you go to the browser panel I'll first add crop mapping folder that I made and where the Sentinel images as a favorite there are this I can simply drag it to the map canvas and the first thing what we need to do is to style this one this is a map stack and it has the layers 2 3 4 & 8 of sent an hour to is blue 3 is green for is red and 8 is near-infrared but it is of course stored in layers 1 2 3 & 4 in the layer styling panel we can now set the proper bands so we want 3 2 1 4 red green and blue to get a true color image this looks nice already but in other cases where you're not happy with the contrast you can change that by stretching changing the stretching options here and changing the minimum maximum values that it uses for the stretching and you can choose to stretch to the whole range in the raster or to use the area that you're zoomed in on the map canvas in this case we will leave it like this you cannot use this image as such that would not be very efficient on the phone so we have to change it to an XYZ tiles file and we're going to create a so-called MB tiles file therefore we need a plugin you go to a plugin manager and we look for the tiles XYZ plug-in which is also developed by Utrecht consulting which has developed the Merton Cloud Servers and the app that we are going to use the input app so this plug-in adds to the processing toolbox the tiles XY is that processing plug-in and we have their generate XYZ tiles MB tiles we click there in the dialog we choose for the extent a layer extent of the original sentinel image and we set the zoom level to 15 by 15 well it's an optimum between the resolution and the file size I've experimented a bit with it so you have to do a bit of trial and error but in this case 15 works well you can also see that we can put a range of zoom levels they all make the file bigger but then you can more efficiently zoom in and out so it's all a trade-off for our purpose just one zoom level 15 gives nice results we keep the other things at default and we save it to our folder as sentinel true-color mb tiles file and we run the algorithm there it is in the browser panel we refresh it and we see there two files edit and we add the roster with the MB tiles go back to our layers and we're going to compare the result and we see that indeed 15 is a good trade-off there's not much loss of the resolution while it's still efficient in tiles so we're going to repeat this whole procedure to create a false color image where we use a different band combination so we go back to the styling panel and there I choose been four three 2's combination where red is the near-infrared which results in bright red colors for parcels with crops that are growing so real green parcels so we choose again the layer extent from the original sentinel hook the zoom level back to 15 and we save this layer as sentinel false-color so we have now 2 MB tiles that can be used in our project we now no longer need the original sentinel image so we can remove that from the list of course you first gonna check the result and also here we see that there's not much loss with the original so we're going to now remove the sentinel original map stack and we're also going to remove it from the folder because the whole folder will be in the end synchronized with the web service and with the mobile phone so we don't need that file we need to remove all the files that we don't need to be synchronized ok the next step is to add OpenStreetMap if you go in your browser panels you'll find over street map under XYZ tiles which gives you access to the online web map and this is useful to include in our app to have some orientation with the topographical map now that's basically all you need to do for the online layer so I rename it to osm online so it's clear that this is the online one because we're going to create an offline version for if you don't have internet connectivity in the field and we do that in a similar way as for the sentinel because we're going to create the XYZ tiles the NB tiles file first there we have to set the projection of this project to the web Mercator that is because the ambit Isles use that projection so set project CRS MB tiles use it in this case to get the right extents so in the processing toolbox we choose generate and details take the extent from one of these sentinels because we want to have the same area covered by our offline of a street map and you see that it's projection three eight five seven web marketer and we put the zoom level also here on 15 which I tried with trial and error and gave good results so when we save that file to the MB tiles are call this on osm offline then the tool will just fetch all the information at the right zoom level from the server it's completed so go to the browser and we refresh again like we did before and you can now add the offline of a street map layer so we have now an online and an offline of a street map layer you can see that when we switch off the online one we see that the offline one is fitted to the extent of the Sentinel images and we switch back the projection of the project to the UTM as we'd like to use and project there are many other ways to add online maps we can use for example the quick web services plug-in or use other WMS services for example it is a good moment to save your file save your project and for our purpose we need to save this oqg s because that's the format supported by our workflow now the next thing that we are going to do is to digitize the boundary layer of our study area so we're going to create a new geo package layer to create the vector and we call that survey that will be also the database geo package that we're going to use for survey layer that we create later so table name is boundary that's basically the name of the layer set it to polygons keep the projection and you don't add attributes because you don't need it we just want a simple polygon it is of course a hypothetical case so I switched to editing and I'll add just quick and dirty polygons that we want to see in our app that's the boundary of our study area it's important to have boundary of the study area because then in the fields you will not do observations beyond you see that the feature ID is automatically generated it's difficult for a Geo package or database spatial database I'm gonna change the styling these stylings are also taken over by the app so I'll make it transparent make the polygon boundary red and make it a bit thicker so also be visual like this in the app the next step is to create the survey layer and that's basically the core of our app that's where we are going to collect the data so save the project and create layer and we're going to add it to the existing geo package so I choose that as a database survey and for table name make it a point and a stable name we choose their ground truth because this is our case here we want to collect ground truth data for remote sensing classification keep the projection and now we are going to create the attributes that we want for our crop mapping purpose here I use an example that comes from the FAO crop mapping trainings that we we gave so the first one that we need is the observation number and that's a whole number and we add it to the field list then the next field that we want is the observer name always important to take the name of the observer because people have different ways of estimating for example vegetation cover and always if you have questions you can go back to the observer you need an attribute date and time it's very important to lock the data and the time in the field and that's a date and time data type that's easy also capturing a picture in the field and we take that as text data because that's needed for the widget that you're going to create later and I put the amount of characters at 80 that should be sufficient you'll make an attribute land cover type that's also a text data same size then the next one is the crop type some properties then what kind of water source is used in the crops what is the irrigation method you want to lock that also in the attribute table what is the coverage of the vegetation there we want the decimal number the percentage it's a real value and we add one on a size limitation of three hundred three hundred meters and that will be a checkbox where we have one for true and 0 for false so there can be a whole number and then we are finally some notes where we can write extra observations we make that a bit bigger 200 characters so this is our attribute table and because we add it to an existing geo package we click add new layer so now both boundary and ground rules are part of our geo package so for ground truth we can define a styling so we add an extra simple marker and we are going to style the first simple marker make it a bit bigger so it's very important that it's very nicely visible on the app so don't make it too small and use some colors that you can distinguish probably needs a little bit of exercise to get the right styling there there are many options in QGIS for styling and for the second simple marker we use a cross so what I want is a crosshair with a circle so we really focus on a certain point make it a bit thicker also for the first simple marker can change also the colors to view for example and there we are styling that also be used in the app now the final step that we need to do here is to design the widgets so we use an attributes form and we can use their F ID we really don't use that as a widget so we're going to hide that one that's the one that's automatically added remember we didn't add that but that came from the Geo package so put that on hidden so we don't see that anymore servation number that's the observation number that we can fill in ourselves the alias field here is two if we want to give it another name when the names are okay we just leave it like it is and as widget type we choose here the text edit so people can fill in the number and we put a constraint there that it can't be no so it has to be filled in then for observer name we also want a text edit we also want it as a constraint not to be null then we go to date and time it's recognized that it needs to date time digits which is nice and we're going to change how to visit this place because we want a notation a little different one day month year and then hour minute second and we don't want a calendar pop-up and it should not be no and a default value we put dollar now so we can get the current date and time filled in automatically in the widget then for picture we choose widget type attachment it's important that we set it to relative paths because it's going to be used on the phone and we're going to change the integrated document viewer to image and we also want it not know then the land cover type going to be set to a value map and with the value map we can basically code values to different options so here we put different land uses and in the app you of course have nice drop-down menus for this so we also check here and that it's not know and we put the default value on one because our interest is crop mapping and the preview says already crop land then we go to the crop type similar way we also want to drop down menu in our app so we choose value map and we fill it in with different crops that we have in our study area now we need a special constrain here and we can put an expression there because we only want the scrubs to be chosen if the length cover type is on crops in and cover type equals and then we choose one which was the crops so we can only fill this in when the length cover type equals one that's what it says and that's also taken over by the input tab copy this because I need a similar constraint later let's go to water source also value map and similar constraint and go to crop irrigation method also value map and there we also have to constraint but we add an extra constraint to it so of course length cover type needs to be 1 but what a source needs to be to the pair's irrigation so only if both criteria are true then we can fill this one we will add coverage which is the vegetation cover we can use an alias for this to make it clearer in the table what it means so in the app we can use a widget here for a range and use a special one slider not all these widgets are supported by improve you'll see later that this one is not supported yet they're working on these widgets and put it from zero to hundred with step size of five then this size which is a minimum requirement to indicate for the parcel if the field size is a big enough which is larger than 300 by 300 meters and we use a checkbox and when it's true it's checked it's yes and when it's not checked it's no there it is and the last one the last widget is for the notes it needs to be a free text field where we can use multiple lines and that's it so this is our field form let's check in the attribute table how this looks like the final thing that we need to do is to set the boundary background layer to read-only so the background factor layers need to be set to read only and the ground truth of course needs to be edited in app so that's not read only based on this the app can identify the survey layers because they are not redone the other thing we need to do is to now define the map themes and map themes are basically presets that you can define in QGIS to fast visualize combination of layers so the first one its Sentinel to color where we have the vector boundary polygon layer and the true color sentinel activated and of course the survey layer and we make one now for the false colors and check the other things and we have the false color there with the boundary and the server layer checked so we call that one sent in our false color and then we create one for the OpenStreetMap offline and one for the OpenStreetMap online now you can check it by just switching between these different map themes and see if it works like we want so it is the offline then you have to send no false color and sent in our true color so in this way these map themes will also be used in the app we save the project now we need to install the merchant plug-in which links QGIS to the cloud service that is called merchant Merton is also developed by a Loutre consulting so that will add in the browser panel merchants folder and there you can configure your credentials and the link to the server and once you've done that you can check all the projects that are on the server or my projects if you already have some projects you'll see it there and like in my case and if you have a new project click right and you can choose create new project and type the name so we call this on crop mapping public one select a folder and then it starts synchronizing this folder with the cloud service of merging merging project created successfully so it's great so we find it under my projects and you see there are clouds so it's also in the cloud and we go to the clouds we sign in then you can find the project crop mapping and your personal projects and here are all the files so the next step is to synchronize this with the input app you can install the input app from the Google Play Store and I've already installed it and this is a previous project but after you're logged in you can see your own projects and there are my projects you see your projects and there's the crop mapping project and we downloaded from the server to synchronize it with the mobile phone and then under home we can tap it and it will load first time it takes a bit in the future it's cassius and we see that the GPS signal is green although we are not in this region in this demo we can tap polygon the background polygon of the boundary and it gives us the attributes not so interesting here and we can switch map themes switch to the of a street map offline or online so very useful when we are navigating in the area and here's the map theme where we have the Sentinel false-color it's useful to identify which parcels are vegetative desiring bright red because the near-infrared we can choose the true color center now when we want to add a point we tap record now it will move to our GPS location which is very far away because we're just simulating this case but when you find back your place you can tap on this move on the spot where you want to place the marker by moving the map around basically with your fingers once you've found the right parcel you tap at point and then you can fill in the observation form that we designed in QGIS if you click to date/time button and it will fill in the current date time you can add a picture and you can choose here what the land use is in red you see the fields that are depending on the answers of the other fields so if I choose your rain fat I cannot fill in the irrigation method type in the vegetation cover the slider widget is not yet implemented but for the field size we have the toggle and that is implemented and we can make some notes and then when we are satisfied we do safe and the point is placed on the map with a symbology that we've chosen if we tap the point it will give us a quick preview of some attributes that we chose and we can now still edit the form so let's change the irrigation method to sprinkler and then it's saved and then the final step is to synchronize this back to the surface of co2 my project you see this synchronization button and I can synchronize it so in this tutorial I've shown how to use QGIS merge in and input app to do field surveying if you want to know more about QGIS and visit my OpenCourseWare or come to ihe Delft for a short course now we can also organize daily mate trainings if you want to see more videos please subscribe to my youtube channel

Channel: Hans van der Kwast

Views: 13,674

Rating: 4.9775281 out of 5

Keywords: QGIS, Mergin, Input, cloud, XYZ tiles, mbtiles, widgets, fieldwork, crop, mapping

Id: 8AZ9gPAhL_4

Length: 29min 53sec (1793 seconds)

Published: Tue Jul 09 2019

Reddit Comments

[u/geo-special]

I came across this excellent tutorial and thought I would share it here.

It takes you through how to set up your phone or a GPS enabled tablet in order to carry out field surveys using a really great app.

[u/w_t]

Thanks, always looking for new field data collection stuff...

Do you know is Mergin free, or open source\self-hosted?