Merge pull request #403 from digitalearthafrica/io_cm

Add Indian Ocean crop mask to products and OWS

dev/products_dev.csv

@@ -7,6 +7,7 @@ crop_mask_sahel,https://raw.githubusercontent.com/digitalearthafrica/config/mast
 crop_mask_southern,https://raw.githubusercontent.com/digitalearthafrica/config/master/products/crop_mask_southern.odc-product.yaml
 crop_mask_southeast,https://raw.githubusercontent.com/digitalearthafrica/config/master/products/crop_mask_southeast.odc-product.yaml
 crop_mask_central,https://raw.githubusercontent.com/digitalearthafrica/config/master/products/crop_mask_central.odc-product.yaml
+crop_mask_indian_ocean,https://raw.githubusercontent.com/digitalearthafrica/config/master/products/crop_mask_indian_ocean.odc-product.yaml
 ndvi_climatology_ls,https://raw.githubusercontent.com/digitalearthafrica/config/master/products/ndvi_climatology_ls.odc-product.yaml
 ndvi_anomaly,https://raw.githubusercontent.com/digitalearthafrica/config/master/products/ndvi_anomaly.odc-product.yaml
 nasadem,https://raw.githubusercontent.com/opendatacube/datacube-dataset-config/main/products/nasadem.odc-product.yaml

prod/products_prod.csv

@@ -7,6 +7,7 @@ crop_mask_sahel,https://raw.githubusercontent.com/digitalearthafrica/config/mast
 crop_mask_southern,https://raw.githubusercontent.com/digitalearthafrica/config/master/products/crop_mask_southern.odc-product.yaml
 crop_mask_southeast,https://raw.githubusercontent.com/digitalearthafrica/config/master/products/crop_mask_southeast.odc-product.yaml
 crop_mask_central,https://raw.githubusercontent.com/digitalearthafrica/config/master/products/crop_mask_central.odc-product.yaml
+crop_mask_indian_ocean,https://raw.githubusercontent.com/digitalearthafrica/config/master/products/crop_mask_indian_ocean.odc-product.yaml
 ndvi_climatology_ls,https://raw.githubusercontent.com/digitalearthafrica/config/master/products/ndvi_climatology_ls.odc-product.yaml
 ndvi_anomaly,https://raw.githubusercontent.com/digitalearthafrica/config/master/products/ndvi_anomaly.odc-product.yaml
 dem_cop_30,https://raw.githubusercontent.com/opendatacube/datacube-dataset-config/main/products/dem_cop_30.odc-product.yaml

products/crop_mask_indian_ocean.odc-product.yaml

@@ -0,0 +1,36 @@
+---
+name: crop_mask_indian_ocean
+description: Annual cropland extent map for Indian Ocean Africa
+  produced by Digital Earth Africa.
+metadata_type: eo3
+
+license: CC-BY-4.0
+
+metadata:
+  product:
+    name: crop_mask_indian_ocean
+
+storage:
+  crs: EPSG:6933
+  resolution:
+    x: 10
+    y: -10
+
+measurements:
+  - name: mask
+    aliases: ['crop_mask', 'MASK']
+    dtype: uint8
+    nodata: 0
+    units: '1'
+
+  - name: prob
+    aliases: ['crop_prob', 'PROB']
+    dtype: uint8
+    nodata: 0
+    units: '1'
+
+  - name: filtered
+    aliases: ['mode']
+    dtype: uint8
+    nodata: 0
+    units: '1'

services/inventory/dev_af/inventory.json

@@ -1,5 +1,5 @@
 {
-  "total_layers_count": 40,
+  "total_layers_count": 41,
   "layers": [
       {
           "layer": "s2_l2a",
@@ -396,6 +396,18 @@
               "prob"
           ]
       },
+      {
+          "layer": "crop_mask_indian_ocean",
+          "product": [
+              "crop_mask_indian_ocean"
+          ],
+          "styles_count": 3,
+          "styles_list": [
+              "green",
+              "filtered_yellow",
+              "prob"
+          ]
+      },
       {
           "layer": "crop_mask_southern",
           "product": [

services/inventory/prod_af/inventory.json

@@ -1,5 +1,5 @@
 {
-  "total_layers_count": 38,
+  "total_layers_count": 39,
   "layers": [
       {
           "layer": "s2_l2a",
@@ -396,6 +396,18 @@
               "prob"
           ]
       },
+      {
+          "layer": "crop_mask_indian_ocean",
+          "product": [
+              "crop_mask_indian_ocean"
+          ],
+          "styles_count": 3,
+          "styles_list": [
+              "green",
+              "filtered_yellow",
+              "prob"
+          ]
+      },
       {
           "layer": "crop_mask_western",
           "product": [

services/ows_refactored/agriculture/ows_crop_mask_cfg.py

@@ -374,7 +374,7 @@
 
 This provisional cropland extent map has a resolution of 10m, and was built using Copernicus Sentinel-2 satellite images from 2019. The cropland extent map was produced using extensive training data from across Central Africa, coupled with a Random Forest machine learning model. For a detailed exploration of the methods used to produce the cropland extent map, read the Jupyter Notebooks in DE Africa’s crop-mask GitHub repository.
 
-An independent validation dataset suggests this service has an overall accuracy of 88.3 %. The algorithm tends to report more omission errors (labelling actual crops as non-crops) than commission errors (labelling non-crops as crops). Where commission errors occur they tend to be focussed around wetlands and seasonal grasslands which spectrally resemble some kinds of cropping.
+An independent validation dataset suggests this service has an overall accuracy of 88.3 %. The algorithm in this region tends to report more commission errors (labelling non-crop as crops) than ommission errors (labelling actual crops as non-crops). Where commission errors occur they tend to be focussed around wetlands and seasonal grasslands which spectrally resemble some kinds of cropping.
 
 The crop mask contains three measurements:
 - mask ("Cropped land"): This band displays cropped regions as a binary map. Values of 1 indicate the presence of crops, while a value of 0 indicates the absence of cropping. This band is a pixel-based cropland extent map, meaning the map displays the raw output of the pixel-based Random Forest classification.
@@ -407,6 +407,47 @@
                 ],
             },
         },
+        {
+            "title": "Cropland extent 2019 - Indian Ocean Africa",
+            "name": "crop_mask_indian_ocean",
+            "abstract": """
+Digital Earth Africa's cropland extent map for the Indian Ocean region of Africa (2019) shows the estimated location of croplands in the countries of Madagascar, Mauritius, Reunion, and Comoros for the period January to December 2019. Cropland is defined as: "a piece of land of minimum 0.01 ha (a single 10m x 10m pixel) that is sowed/planted and harvest-able at least once within the 12 months after the sowing/planting date." This definition will exclude non-planted grazing lands and perennial crops which can be difficult for satellite imagery to differentiate from natural vegetation.
+
+This provisional cropland extent map has a resolution of 10m, and was built using Copernicus Sentinel-2 satellite images from 2019. The cropland extent map was produced using extensive training data from across the islands of Indian Ocean Africa, coupled with a Random Forest machine learning model. For a detailed exploration of the methods used to produce the cropland extent map, read the Jupyter Notebooks in DE Africa’s crop-mask GitHub repository.
+
+An independent validation dataset suggests this service has an overall accuracy of 90 %. The algorithm tends to report more omission errors (labelling actual crops as non-crops) than commission errors (labelling non-crops as crops). Where commission errors occur they tend to be focussed around wetlands and seasonal grasslands which spectrally resemble some kinds of cropping.
+
+The crop mask contains three measurements:
+- mask ("Cropped land"): This band displays cropped regions as a binary map. Values of 1 indicate the presence of crops, while a value of 0 indicates the absence of cropping. This band is a pixel-based cropland extent map, meaning the map displays the raw output of the pixel-based Random Forest classification.
+- prob ("Probability of cropping"): This band displays the prediction probabilities for the 'crop' class during 2019. As this service used a random forest classifier, the prediction probabilities refer to the percentage of trees that voted for the random forest classification. For example, if the model had 200 decision trees in the random forest, and 150 of the trees voted 'crop', the prediction probability is 150 / 200 x 100 = 75 %. Thresholding this band at 50 % will produce a map identical to mask.
+- filtered ("Cropped land (object-filtered)"): This band displays cropped regions as a binary map. Values of 1 indicate the presence of crops, while a value of 0 indicates the absence of cropping. This band is an object-based cropland extent map where the mask band has filtered using an image segmentation algorithm. During this process, segments smaller than 1 Ha (100 10m x 10m pixels) are merged with neighbouring segments, resulting in a map where the smallest classified region is 1 Ha in size. The filtered dataset is provided as small commission errors are removed by this process, and the 'salt and pepper' effect typical of classifying pixels is diminished.
+
+More technical information about the cropland extent service can be found in the User Guide (https://docs.digitalearthafrica.org/en/latest/data_specs/Cropland_extent_specs.html)
+
+Cropland extent maps are a foundational, baseline layer in high-order crop health and crop productivity products which necessarily rely on knowing where cropping occurs before further analysis can take place.
+
+This service is accessible through OGC Web Service (https://ows.digitalearth.africa/), for analysis in DE Africa Sandbox JupyterLab (https://github.com/digitalearthafrica/deafrica-sandbox-notebooks/wiki) and for direct download click on a tile in the explorer page (https://explorer.digitalearth.africa/products/crop_mask_indian_ocean/extents)
+""",
+            "product_name": "crop_mask_indian_ocean",
+            "time_resolution": "year",
+            "bands": bands_crop_mask,
+            "resource_limits": reslim_alos_palsar,
+            "image_processing": {
+                "extent_mask_func": "datacube_ows.ogc_utils.mask_by_val",
+                "always_fetch_bands": [],
+                "manual_merge": False,
+            },
+            "native_crs": "EPSG:6933",
+            "native_resolution": [10, -10],
+            "styling": {
+                "default_style": "green",
+                "styles": [
+                    style_crop_mask_green,
+                    style_crop_mask_filtered_yellow,
+                    style_crop_mask_prob,
+                ],
+            },
+        },
         {
             "title": "Cropland extent 2019 - Northern Africa",
             "name": "crop_mask_northern",