Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature/7-config add omegaconf and read config from a yaml file FW-354 #20

Merged
merged 3 commits into from
Oct 25, 2024

Conversation

dolegi
Copy link
Collaborator

@dolegi dolegi commented Oct 18, 2024

  • Move the GEAR files into /scripts/GEAR and G2G to scripts/G2G (this could be improved by having python modules)
  • Script now requires a config file, see the default for some examples
  • GEAR_config module includes omegaconf and dataclasses for validating

@mattjbr123 your config file should be config/matbro.yaml

start_year: 1990
end_year: 2016
start_month: 1
end_month: 13
input_dir: "/home/users/mattjbr/fdri/data/gear-hrly"
prefix: "CEH-GEAR-1hr-v2_"
suffix: ".nc"
target_root: "/work/scratch-pw2/mattjbr"
store_name: "gear_1hrly_fulloutput_yearly_100km_chunks.zarr"
target_chunks: 
  time: 8769.4
  y: 100 
  x: 100 
  bnds: 2
num_workers: 4
prune: 12 # no. of files to process, set to 0 to use all

Then run the script with python scripts/GEAR/convert_GEAR_beam.py config/matbro.yaml

@mattjbr123 mattjbr123 linked an issue Oct 22, 2024 that may be closed by this pull request
@mattjbr123
Copy link
Collaborator

Changed month_end config variable to be inclusive like end_year, didn't seem right having one inclusive and not the other (was probably me who had it that way though tbh)

@mattjbr123
Copy link
Collaborator

Otherwise very happy with this :)

@mattjbr123 mattjbr123 merged commit 1739ffa into main Oct 25, 2024
@mattjbr123 mattjbr123 deleted the feature/7-config-management branch October 25, 2024 15:06
@mattjbr123 mattjbr123 changed the title feature/7-config add omegaconf and read config from a yaml file feature/7-config add omegaconf and read config from a yaml file FW-354 Dec 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create a nice way of holding environment variables
2 participants