Implementation

High-level implementation plan

The EXCLAIM computing and data platform will be developed in five threads that are tightly interwoven:

Thread 1: Software framework

The software framework thread focuses on the development of the Python framework for our model system ICON. Following key steps will be undertaken: (i) Modularization of the ICON Software into system, model and interface components, (ii) Implementation of continuous integration/continuous deployment (CI/CD) pipelines for the ICON Software to run on different sites and systems, (iii) Development of the Python Framework and versions for the development of the model components, and (iv) Migration of interface and system components to Python to obtain the final EXCLAIM ICON implementation.  

We will also direct our attention to the software systems that permit the users to execute scientific simulations and workflows, and to the platform interfaces used by the scientist to deploy those simulations and manage the associated data. Here, the implementation of CI/CD pipelines allowing testing and validation is crucial. Further, we will focus on data interfaces to store, retrieve, compress, analyze and modify data to and from simulations, and to do data curation (in collaboration with external page SDSC and SPCL).

THREAD 2: Algorithms and parameterizations

The goal is to spearhead a strategy towards the future use of algorithms and parameterizations in climate and weather models. Regarding the latter, we will build and extend on ongoing work that uses Open-ACC directives to port some of the parameterizations to GPU. EXCLAIM will coordinate and spearhead some of these efforts, e.g., for the a two-moment microphysics scheme and the consideration of large-eddy turbulence (LES) models, which are available in conventional CPU implementations of ICON. As LES models entail horizontal coupling between neighboring grid dells, their implementation into the EXCLAIM framework has to be considered a part of the dynamical core. In addition, we will make an investment into the development of new and improved algorithms, especially with regard to time stepping.

THREAD 3: Data flow and data science

We will ensure that the user can establish an efficient workflow from the preparation of the simulation all the way to its analysis. This requires novel methods in data handling, since the currently deployed data flow models do not scale to the types of problems that EXCLAIM is aiming to tackle. Additional challenges arise e.g., for the use of limited area model configurations at high resolution, since they require the provision of lateral boundary conditions at high frequency as well. We will focus on the establishment of a new set of data services through converged cloud-HPC-based data services. The external page SimFS environment will continue to be developed, bringing it to the roll out phase. Finally, work will be initiated in the areas of machine learning tools for data compression, and online analyses.

THREAD 4: Scientific Use Cases

The four core scientific use cases identified form the scientific backbone of the project. EXCLAIM will ensure that these configurations are well tested and scientifically validated, while the scientists will take the lead in executing the simulations and analyzing the results.  

THREAD 5: EXCLAIM User Platform

EXCLAIM will assist external page CSCS and C2SM in building jointly an EXCLAIM user platform that will support domain scientists to fully benefit from the developments undertaken by EXCLAIM and other HPC oriented projects. This includes e.g., support in porting and optimizing codes, providing verification and testing suites, managing and analyzing data streams, and the containerization of codes.