Read More in Our Blog

Launching FAIR2Adapt Case Studies: Highlights from Hack4RiOMar

Event Overview 📑

The Hack4RiOMar workathon brought together six motivated participants in person, collaborating to advance the FAIR2Adapt RiOMar Case Study—the first workshop in our series of FAIR2Adapt case studies. We extend our gratitude 👏 to the external experts who participated at their own expense, bringing their expertise to tackle challenges and drive progress.

Participants 🙋🙋:

In the picture below, you can see the Hack4RiOMar team, listed from left to right: Tina, Even, Anne, Jean-Marc, Justus, and Ola.

Key Achievements 🏆

Aggregating Virtual Datasets

We successfully aggregated virtual datasets into a single dataset using VirtualiZarr with kerchunk. Despite challenges with icechunk, alternative approaches enabled us to:

• Write Virtual Zarr locally and upload to accessible locations (e.g., Pangeo-EOSC S3, Datarmor IFREMER HC cluster).
• Develop a Jupyter notebook showcasing this workflow.

Transforming Data into DGGS Grids

• A sample RiOMar dataset was transformed into a DGGS (Healpix) grid at various resolutions. Check the corresponding Jupyter Notebook here. In the figure below, you can see the DGGS-transformed RiOMar data (temperature in degrees Celsius).
• To validate the robustness of the procedure, other datasets were transformed to DGGS:
  • Jupyter notebook to transform GEBCO Bathymetry data into DGGS (Healpix) The figure below shows the GEBCO Bathymetry data transformed into DGGS:
  • Jupyter notebook to transform Argo Kinetic Energy to DGGS (Healpix)
• Visualizing transformed data with xdggs (based on lonboard).

Creating Conda Environments

To standardize workflows across platforms, we initiated Conda environments on Datarmor with:

• Support for Cubed and Dask libraries.
• Compatibility testing across platforms.

Testing Multi-Resolution Zarr Formats

Multiscale Zarr storage was tested for DGGS-transformed datasets, allowing us to:

• Explore hierarchical representations of RiOMar (see corresponding Jupyter Notebook).
• Identify the potential for improved data accessibility and interoperability.

RO-Crate for FAIR Metadata

A sample dataset was encapsulated into a RO-Crate, helping:

• Assess the existing metadata for FAIR compliance.
• Identify missing elements, such as unique creator identifiers and data provenance.
• Check the corresponding Jupyter Notebook here

Challenges & Insights 🧩

Icechunk Usability: Limited documentation hindered progress, but kerchunk provided a robust alternative for creating virtual datasets.

Interpolation Methods: Conservative interpolation is needed for future case studies (e.g., radionuclide distribution in Arctic scenarios) to ensure mass conservation during regridding.

Metadata Gaps: Significant metadata enhancements are required for RiOMar datasets to meet FAIR standards.

Jupyter Notebooks 📘

We developed 20 Jupyter notebooks during the event!

Where to Find Our Work 🔍

• GitHub Repository: Hack4RiOMar Notebooks
• Rendered Notebooks with Highlights: A Jupyter Book version of key notebooks showcases our successes, challenges, and next steps.

Next Steps 🚶

• Refine workflows for virtual dataset aggregation and DGGS transformations.
• Address metadata gaps to ensure comprehensive FAIR compliance.
• Expand testing on additional datasets to validate the robustness of our methodologies.

Hack4RiOMar demonstrated the power of collaboration 🤝 and gave a great head start to the FAIR2Adapt case studies. Thanks to everyone, and stay tuned for further updates as we build on these achievements!