USE CASE 2 - Big data earth observations
The use case on big data earth observations is processing on-demand for environmental monitoring focused on the land surface compartment. This is being led by IRD, together with the other French academic institutes INRAE and CNES (involved in the THEIA land data centre), the IT private companies Geomatys and Geolabs, coordinated by the CINES.
The objective of the developed demonstrator will be to show the feasibility of scaling up a processing chain prototype for production use, using the computing power of an HPC centre while allowing wide access to the products for the scientific community. That implies processing chain adaptation and building a whole environment to execute and access to the results of the processing chain.
Challenges
Optical and radar images observing the Earth's land surface have become an essential source of information to address and analyse environmental issues. The diversity of Earth observation sensors makes it possible to provide new insights into environmental monitoring. But as many recent diagnoses have shown, satellite imagery for environmental monitoring purposes is particularly underutilised by the land and natural resource management communities, the high level of technology and the need for digital resources required to use satellite imagery are the main reasons for this. On the other hand, THEIA scientific expertise centres have designed and implemented a proof of concept prototypes of new algorithms that meet the information needs of environmental monitoring stakeholders, such as land cover, soil moisture or natural vegetation Biomass.
"We use the supercomputer OCCIGEN of CINES to scale the execution of our processing chains. This is a great added value. In addition, the open dissemination of data that are produced by the processing chains are disseminated through their catalogue, which is consistent with the FAIR principles by offering a standardised discovery service."
Jean-Christophe Desconnets, IRD & PHIDIAS use case leader.
Solutions
The objective of this use case is to provide academic and land management community users with an interactive environment to ensure systematic or on-demand production of new knowledge useful for the environmental monitoring of territories. On the one hand, it is a matter of relying on the algorithmic developments carried out within the THEIA land data centre, machine and deep learning techniques adapted to spatial data, and on the other hand taking advantage of the complementarities (spatial and temporal resolution) of large spatial data sets from very high-resolution sensors (SPOT, PLEIADES) and SENTINEL 1 and 2-time series. With a view to the sharing and reuse of data and algorithms by the entire environmental community, the hardware and software environments deployed must allow:
-
interoperable access to the catalogues of SENTINEL 1 and 2 images provided by CNES with the PEPS platform and SPOT6 images provided by the GEOSUD infrastructure managed by IRD, INRAE and IGN,
-
access in an interoperable manner to the algorithms or processing chains encapsulating the algorithms,
-
mass production (on a portion of European territory) or on demand of maps,
-
presentation of the information produced via standardised discovery and access services in order to enable open and interoperable dissemination according to FAIR (Findable, Accessible, Interoperable, Reusable) principles.
The design and the implementation of the WP5 technical components will be based on three main data processing chains coming from the THEIA land surface academic community:
Sentinel-1/Sentinel-2-derived Soil Moisture product at Plot scale (S2MP) over agricultural areas providing the academic community with essential data to parameterise hydrological or agricultural models in a context of modelling meteorological processes, water balance cycle assessment or crop development models at the plot level. It will also be targeted to the agricultural sector (irrigation monitoring).
The engineering work in remote sensing, data and HPC systems have enabled the secure deployment of the processing chain Soil Moisture at Plot Scale (S2MP) in the supercomputer OCCIGEN at CINES. This deployment is achieved through the containerization of the source code of S2MP in a Singularity container. It allows us to take advantage of the great capabilities of the supercomputer. The tests conducted have confirmed the proper execution and obtaining mapping data in accordance with the reference executions. The next step will be to connect the WPS Zoo-project (http://zoo-project.org) server to allow an on-demand execution of the SM2P chain. Sending execution requests to the OCCIGEN supercomputer will be done through the SLURM task manager via a secure environment.
Production of radiometric indices over large territories of the European Union
This use case is based on the deployment of the OTB open source library on the OCCIGEN supercomputer. Orfeo ToolBox (OTB) is an open-source project for state-of-the-art remote sensing. Built on the shoulders of the open-source geospatial community, it can process high resolution optical, multispectral and radar images at the terabyte scale. To demonstrate the added value of the use of a supercomputer for remote sensing purposes, we take advantage of the MPI module which enables us to run massive parallel computation. We will define some data processing scenarios which use its capabilities. One of them will be to calculate radiometric vegetation index or water index in a large part of European Union territories.
Remote sensing images processed by artificial intelligence produce land cover mapping in super-resolution, exploring and taking advantage of the power of AI techniques.
The containerization of the two processing chains is in progress. The challenge of this use case is to evaluate the processing capacity of the new GPU processor architecture to produce very high-resolution images.
Impact
The different use cases addressed in the WP5 have already made it possible to deploy applications from the remote sensing community in a supercomputer in the form of a container, in compliance with the security recommendations of a computing centre. Moreover, the work on the on-demand execution of applications by a WPS (Web processing service) server makes it possible to consider the use of supercomputers in a way that is adapted to the needs of the environmental community: particularly for environmental monitoring needs that do not have the skills and access to high-performance computing resources.
Learn more about the PHIDIAS use case 2 "Big data earth observations"
Events:
- Bridging the gap to facilitate selection and image analysis activities for land surface monitoring
- PHIDIAS HPC – Building a prototype for Earth Science Data and HPC Services
News:
