Improving efficiency and genericity of the intelligent screening of environmental satellite data
Environmental satellite data are gradually becoming the key pieces of data that, put together, help build an understanding of the changing environment and a new engine of scientific discovery.
PHIDIAS has set up a data area testing case in "Intelligent screening of a large amount of satellite data for detection and identification of anomalous atmospheric composition events". The intelligent screening of a large amount of satellite data is associated with the Earth sciences, data-derived and integrated from Earth observation, as well as terrestrial, oceanic and atmospheric data, together with data on human activity, obtained from ground sensor networks and other sources.
The development so far
The overall objective of this data area testing case is to use the HPC and high-performance data management services established for the development of intelligent screening approaches for the exploitation of large amounts of satellite atmospheric data in an operational context. The corresponding activities are mainly led by SPASCIA, specified in the frame of PHIDIAS Work Package 4 (WP4), together with its partners:
- ICARE (Sentinel 5 Precursor, Data access, data provider, formatting ad architecture),
- HYGEOS (Level 1 data processing for extreme events),
- SRON (Expertise on S5P data and products, Scientific pilot user).
SPASCIA, ICARE and HYGEOS are coordinating the actions for selecting and getting Sentinel 5 Precursor (S5P) data and products, and for starting development work for preparing the processing algorithm of Level 1 (L1) S5P data, first L1 intelligent screening approach of S5P data, for the detection of the extreme events.
SRON has provided recommendations for selection and scientific issues of S5P Level 2 (L2) datasets, focusing on the atmospheric CO (Carbon Monoxide) products.
Work has been initiated by SPASCIA, in coordination with ICARE, to select and obtain a sample of the S5P L2 product, and to start the analysis of the corresponding images. This allows the preparation of necessary inputs or test data for the development of the processing algorithm of the L2 S5P products (2nd L2 intelligent screening approach of S5P data, for the detection and characterisation of pollutant plumes). ICARE has implemented and is processing the archiving of S5P data and products on the ICARE facilities for PHIDIAS.
Several exchanges between SPASCIA and ICARE have allowed the organisation of a complementary relationship between scientific and IT actions in WP4 and setting up the basis for data exchanges and computing resources.
Coordination between WP4 and CINES has been initiated to define the needs in terms of data, interface and storage in WP4. An important outcome of this meeting is that WP4 testing and developments will be performed on ICARE facilities for the first phase of the project (1.5 years), with regular exchanges and full visibility given by ICARE to CINES. Work and results will be transferred to CINES for operational implementation (coordination between WP4, WP2 and WP3).
SPASCIA and ICARE have written a first internal document for use-case specification in terms of data, architecture and software, which has been delivered to WP3 and CINES.
The Next Stage
The first phase of the project (about 1.5 years, up to next February 2021) shall implement, test and validate the processing for intelligent screening approaches for extreme event detection and pollutant plumes monitoring on 1 year of S5P data.
Then prototyping processing will be used to re-process 1 year of global data plus several specific on-demand periods/regions (March-June 2021) and will be exploited by pilot users in demonstrating added value, improving the processing, and consolidating the products and services of WP4 (March 2021 - June 2022).
The next 6-month period will be devoted to:
- HYGEOS/SPASCIA/ICARE technical development of the first L1 intelligent screening approach of S5P data,
- SPASCIA/ICARE technical development of the second L2 intelligent screening approach of the S5P product. Specification of processing based on artificial intelligence is ongoing.
Good news for researchers and specialists
The WP4 use case addresses a well-identified, key requirement for the exploitation of Earth Observations to provide the capacity of intelligent screening of large amounts of satellite data for targeting scenes of interest (e.g. extreme atmospheric events), given their dedicated processing or exploitation. The potential main users are:
- the scientific community (for the efficient specific/thematic analyses of the data);
- the operational services at national/European levels (for the monitoring and validation of their forecast systems);
- the service providers for implementing dedicated applications for end-users (e.g. alert or decision support services).
This use case aims to improve efficiency and genericity of the intelligent screening of environmental satellite data. Implementing these approached as a prototype service on the already available S5P European atmospheric sounding mission is proposed. The objective is to implement, demonstrate and make available to the users a service offering smart data filtering to:
- detect the relevant data within the huge volume of measurements,
- identify and assess the pertinent information,
- qualify the targeted data for dedicated exploitation by users
- most especially, provide the users with access to the filtered data and the corresponding metadata.
To find out more about PHIDIAS use cases updates, stay tuned to our regular insights and news.
www.phidias-hpc.eu I @PhidiasHpc I IN/company/phidias-hpc I info@phidias-hpc.eu
