The GHRSST-PP Data Processing Specification (GDS)
The GHRSST-PP Data Processing Specification (GDS) represents a consensus opinion of the GHRSST-PP community of how to pursue the optimal combination of satellite and in situ data streams within a globally distributed operational system to provide a new generation of global coverage SST data products. It is managed by a dedicated GDS Technical Advisory Group (the GDS-TAG) overseen by the GHRSST-PP Science Team. A copy of the latest GDS documentation can be found here.
The GDS is designed to produce SST data products that satisfy the requirements of operational ocean forecast and prediction systems and the GHRSST-PP re-analysis project. Based on a user requirements survey (see the proceedings of the 3rd GHRSST-PP workshop and the Medspiration User Requirements Document for example, both available here), the main requirements are:
- SST data are required operationally in near real time and should ideally be available within 6 hours of data acquisition at the satellite platform.
- An error estimate (bias and standard deviation) and confidence data (such as an indication of atmospheric aerosol presence at the time of measurement) for each SST measurement including a bias and SD. is required by users and by the GHRSST-PP as a necessary precursor to data analysis.
- A global ocean SST data product providing a best measure of the SSTfnd and SSTskin (including an estimate of diurnal magnitude and phase) is required at a minimum interval of 24 hours having a maximum spatial grid size of 1/12° (latitude x longitude).
- Coastal modelling groups require ultra-high spatial resolution SSTfnd and SSTskin data products on a grid cell size of 1-2km and a timeliness of ~6 hourly.
- A common file format based on the netCDF and Climate Forecast version 1.0 (CF-1.0) convension.
The figure shows a functional breakdown diagram of the GDS and identifies the main components of the GDS, the flow of data between GDS components, inputs and output data and data products. The GDS is split into work package (WP) sections to aid development and understanding of the system. The major work package components of the processing specification are shown in light blue together with their associated input and output data parameters that are shown in red and green respectively. The flow of data and information between each work package is indicated by the direction of connecting arrows.
The GDS data processing flow is designed to:
- Ingest and QC input SST data sets.
- Assign pixel confidence data, auxiliary dynamic flag data fields (wind speed, surface solar irradiance (SSI), aerosol optical depth (AOD), and sea ice concentration) to each SST measurement - used to interpret the state of the air sea interface and at mosphere at the time of SST measurement,.
- Assign error estimates (called Single Sensor Error Statistics, SSES) to each SST measurement
- Format assembled data as a GHRSST-PP L2P output data file. L2P data sets should be available to the RDAC user community and GDAC processor ideally within 6 hours of data reception at the satellite sensor.
- Co-locate L2P data records with in situ measurements according to defined time and space matchup criteria, construct a database record and submit these to the GHRSST-PP Match-up Database (MDB).
- Use MDB records to develop and monitor single sensor error statistics (SSES), monitor and validate L2P and L4 data products.
- Extract High Resolution Diagnostic Data Set (HR-DDS) data granules from L2P data files, reformat to HR-DDS data specifications and submit these data to the GHRSST-PP HR-DDS system.
- Generate diagnoistic maps at high, regional and global scales and at various time scales to monitor the quality of GHRSST-PP data products,
- Consolidate L2P SST data sets every 24 hours and analyse all data to provide a ‘best estimate' of the SSTfnd. Format analysed SST data sets as GHRSST-PP L4 data files and submit data to the GDAC archive. L4 data products should be available to the user community no later than 12 hours following the end of an APPW. The analysis procedures, than must be developed, should aim to capitalise on the synergy benefits of complementary infrared and microwave SST measurements with particular emphasis on data combinations that lead to better accuracy, minimise the impacts of cloud/rain contamination and atmospheric aerosol loading, optimise spatial and temporal sampling resolution and address diurnal variability amongst others.
- For each data processing operation, create unique metadata records and submit these to the GHRSST-PP Master Metadata Repository (MMR).
- Archive all data products and make available for use in the GHRSST-PP Reanalysis Project (RAN).
- Periodically produce GHRSST-PP re-analysis products.
- Monitor the progress of the GHRSST-PP using pre-defined performance metrics.
The GDS continues to evolve as new approaches and satellite data sets are made available to the GHRSST-PP.
(Last Updated: 17-04-2007)