Hydra App Restructuring

[jejjohnson]

This is a summary of our thoughts regarding how we can refactor the repo with the primary case being hydra.

Easy to Read Structure

It should be easy for the user to understand the underlying structure which corresponds to the types of stuff people would like to do. We will try to be categorical with the different pieces to give the users some structure so that they know where to find things and where to modify things

Demo Repo Structure

- data
   - NATL60
      - North Atlantic
      - GulfStream
      - Mediterranean
      - ...
   - Marine Data Store
     - AlongTrack
     - Reanalysis
     - In-Situ 
     - ...
- domain
   - region - lat | lon | depth
   - period - start | stop | step
- preprocess
  - validation
    - data
    - coordinate
    - attribute
    - units
- preprocess
  - coordinate transform  
  - variable transform
     - SSH | Velocity |
  - regridding
      - target grid | target alongtrack | ...
  - ...
- metrics
   - spatial statistics - mean | std | ...
   - temporal statistics - mean | std | ... 
   - isotropic PSD Score
   - spacetime PSD Score
- viz
   - Spatial Statistics
   - Temporal Statistics
   - Maps
   - Isotropic PowerSpectrum
   - SpaceTime PowerSpectrum
   - Isotropic Wavelet PowerSpectrum 
- task
   - interpolation
      -  stage - train | validation | test | estimation
      - domain - space, period
      - preprocessing - validation, regridding, variable transformation
      - postprocessing - 
      - metrics
      - viz
   - forecast
      - ...  
   - denoising
      - ...
   - calibration
      - ... 
   - estimation / Inference
       - ...
- leaderboard
   -  OSSE SSH Interpolation 2020
       - AlongTrack 
       - AlongTrack + SWOT
       - AlongTrack + SWOT + SSH
   - OSE SSH Interpolation 2021
       - AlongTrack GulfStream
       - AlongTrack North Atlantic
   - OSE Mediterranean 2023  
       - AlongTrack 

---

Simple Pipelines

It should be easy for the user to grab a few different recipes as key preprocessing strategies

• ML Callback for statistics/plotting
  • PyTorch Lightning/Keras Callback for plotting spatiotemporal statistics or power spectrum
• Evaluation Script
  • Example: Given a Field, pipe it through evaluation routine for data challenge
• Simple Regridding + Filtering Strategy
  • Example: pre-/post-processing for OI higher order gradients (KE, PV, Strain)

---

Dedicated Tasks

The developer should create specific tasks using the building blocks listed above.

• Tasks: Interpolation, Forecasting, Denoising, Calibration
• Pipelines: Data Download, Data Preprocessing ((Re-)Gridding), Postprocessing, Evaluation Script
• LeaderBoard: gather all available results and see how they perform on specific data challenges.