Dheed v4 is an ERA5 based global database dry and hot extreme events from 1950 to 2023, developed in the context of ESA funded project DeepExtremes. The workflow runs in Julia 1.10.0, except for the consolidation of the data cubes, which is run in python. Most steps of the workflow were run on the MPI BGC-jena cluster. The input data are hourly ERA5 data retrieved from the Copernicus Climate Data Store and stored on a local server as netcdf files. Some derived variables were calculated prior to the processing presented here.
- Rechunk ERA5 data, agregating from hourly to daily: t2m (min, mean, max), tp (sum), ssrd (sum).
cd SlurmScripts
sbatch rechunk_data.slurm
Output: ERA5Cube.zarr
- Compute PET on hourly data (t2m, tp , u10, v10, sp, snr (calc), vpd_cf (calc)) and aggregate to daily.
cd SlurmScripts
sbatch compute_pet.slurm
Output: PET yearly cubes
- Rechunk PET and add to ER5cube
The daily PET is rechunked to match the chunk size of the ERA5cube and added to it.
cd SlurmScripts
sbatch rechunk_pet.slurm
Output: updated ERA5Cube.zarr
- Consolidate metadata of ERA5Cube
Metadata from the different variables in the ERA5cube are consolidated, so as to reduce the number of read operations on the backend store.
python -c import zarr; g = zarr.open_group(path2cube); zarr.consolidate_metadata(g.store)
Output: consolidated ERA5Cube.zarr
- Compute PEI
The precipitation evaporation index (PEI) is a moving average of the water balance between daily potential evapotranspiration and precipitation. The moving window is 30, 90 or 180 days.
cd SlurmScripts
sbatch compute_pei.slurm
Output: PEICube.zarr
The time series of the four indicators: t2mmax, PEI_30, PEI_90 and PEI_180 are ranked transformed between 0 and 1. No convolutional spatial filter is run on the results to smoothe the extent of the extreme events.
cd SlurmScripts
sbatch smooth_events.slurm
Output: tmax_ranked.zarr and pei_ranks.zarr
A pass over threshold is applied to the rescaled indicators and they are combined into a Byte integer (Int8), with one bit for each indicator and an extra bit encoding for non extremes. The first bit (little end) encodes the maximum temperature extremes.
cd SlurmScripts
sbatch compute_events.slurm
Output: EventCube.zarr
Unique labels are assigned to blobs of co-occurrent hot and dry extremes, i.e. where values are uneven (t2mmax extremes) and larger than one (PEI extremes), connected in space and time. A filter is applied before runnning the connected component analysis: temperature extremes must last at least three consecutive days.
Note: this will reduce the total number of tmax extremes in the cube...
Because the connected component analysis requires to load the full cube into memory and the algorithm is greedy, the analysis was split into seven tasks, covering each 16 years.
cd SlurmScripts
sbatch label_events.slurm
Output:
- labelcube_ranked_pot0.01_ne0.1_cmp_S1_T3_1950_1965.zarr
- labelcube_ranked_pot0.01_ne0.1_cmp_S1_T3_1966_1981.zarr
- labelcube_ranked_pot0.01_ne0.1_cmp_S1_T3_1982_1997.zarr
- labelcube_ranked_pot0.01_ne0.1_cmp_S1_T3_1998_2013.zarr
- labelcube_ranked_pot0.01_ne0.1_cmp_S1_T3_2014_2023.zarr
The labels are then merged into a single mergedlabels cube with scripts/merge_labels.jl.
Output:
- mergedlabels.zarr
Statistics for all labelled events are computed and gathered in a unique table.
cd SlurmScripts
sbatch stats_extremes_merged.slurm
Output:
- MergedEventStats_landonly.csv
Figures from the manuscript
Flowchart designed in ppt.
Example of dry and hot extreme event detection workflow over the 2003 summer heatwave in Europe.
See scripts/fig4dheed.jl
Extract timeseries at single locations with plot_city.jl
Trends in annual global/continental indicators.
Compute annual statistics of indicators, globally and by continent.
See scripts/plot_ind_annual.jl
Compute annual statistics of EventCube at global and continental scale, by summing all extremes by type.
See scripts/hist_EventType.jl
Plot the results of the EventCube analysis.
julia --project="ExtremeEvents.toml" plot_EventType.jl
Plot global spatial overview of trends of dry and hot extremes occurrences with scripts/plot_trendmap.jl. Plotting trend map based on indivdual grid cells and all years doesn't bring up significant trends, see plot_trendmap.jl. Decadal trend instead. Or, compare average number of extreme dry and hot days from two periods: 1970-1999 with 2000-2023 (scripts/plot_comparemap.jl).
Extract largest and longest events from scripts/MergedEventStats_landonly with scripts/largest_labels.jl and plot statistics with scripts/plot_stats.jl.
A map of the spatial footprint of the largest events is generated with scripts/plot_stats.jl.
Compare MergedEventStats_landonly with table of reported events compiled a priori with scripts/SanityCheck.jl.
Compare distributions and thresholds obtained from SPEI and PEI with scripts/compare_spei_pei.jl and visualize results with scripts/plot_spei_versus_pei.jl.
Plot Dheed time series at various locations with scripts/plot_city.jl.
Compare Dheed detected dry extremes with Pohl et al. 2023 and Liu et al. 2024 with scripts/compare_phl.jl.
For v4, scripts/Rechunk_data.jl was modified so that a new ERA5Cube.zarr with correct offset and scaling was produced. pet.zarr was already correct in v3. All processing and postprocessing scripts have been modified to use the corrected data. New figures have been added during the review process.
This work was funded by the European Space Agency (ESA) AI4Science projects "Multi-Hazards, Compounds and Cascade events: Deep Extremes," 2022--2024, and "Climate Adaptation, Extremes, Multi-Hazards and Geo-Hazards Science: ARCEME", 2024--2026, and the European Union's Horizon 2020 research and innovation program within the project "XAIDA: Extreme Events -- Artificial Intelligence for Detection and Attribution", (grant agreement 101003469). Recent developments in the Julia package YAXArrays.jl were funded by ESA AI4Science project "The DeepESDL AI-Ready Earth System Data Lab".