Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retrospective 2024-2025 Round 1 projections for SigSci-SWIFT #110

Merged
merged 7 commits into from
Feb 14, 2025

Conversation

vpnagraj
Copy link
Contributor

@vpnagraj vpnagraj commented Feb 5, 2025

We are opening this PR to contribute a new model for the hub.

We've included a model metadata file, a model abstract, and projections for 2024-25 Round 1. While we recognize this round was completed in the fall, preparing the projections retrospectively has helped us develop a model for future rounds. We hope the retrospective submission is useful for the hub.

Copy link

github-actions bot commented Feb 5, 2025

Run validation on files: 2024-08-11-SigSci-SWIFT.parquet

Required values:

❌ Error 006: The submission is missing some required values, please check:
origin_date: 2024-08-11
scenario_id: c("A-2024-08-01", "C-2024-08-01", "E-2024-08-01", "B-2024-08-01", "D-2024-08-01", "F-2024-08-01")
target: inc hosp
horizon: c("1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13", "14", "15", "16", "17", "18", "19", "20", "21", "22", "23", "24", "25", "26", "27", "28", "29", "30", "31", "32", "33", "34", "35", "36", "37", "38", "39", "40", "41", "42", "43")
age_group: 0-130
location: US

Columns:

No errors or warnings found on the column names and numbers

Scenarios:

❌ Error 204: At least 1 of the required 'scenario_id' is missing. The required scenarios ids for the group:
origin_date : 2024-08-11
target : inc hosp
horizon : 1, 2, 3, 4, 5, 6, 7, 8, 9, 10,
...
age_group : 0-130;
are: 'A-2024-08-01, C-2024-08-01, E-2024-08-01'. Please verify.
❌ Error 204: At least 1 of the required 'scenario_id' is missing. The required scenarios ids for the group:
origin_date : 2024-08-11
target : inc hosp
horizon : 1, 2, 3, 4, 5, 6, 7, 8, 9, 10,
...
age_group : 0-130;
are: 'B-2024-08-01, D-2024-08-01, F-2024-08-01'. Please verify.
❌ Error 204: At least 1 of the required 'scenario_id' is missing. The required scenarios ids for the group:
origin_date : 2024-08-11
location : US
target : inc hosp
horizon : 1, 2, 3, 4, 5, 6, 7, 8, 9, 10,
...
age_group : 0-130;
are: 'A-2024-08-01, C-2024-08-01, E-2024-08-01'. Please verify.
❌ Error 204: At least 1 of the required 'scenario_id' is missing. The required scenarios ids for the group:
origin_date : 2024-08-11
location : US
target : inc hosp
horizon : 1, 2, 3, 4, 5, 6, 7, 8, 9, 10,
...
age_group : 0-130;
are: 'B-2024-08-01, D-2024-08-01, F-2024-08-01'. Please verify.

Origin Date Column:

No errors or warnings found on the column 'origin_date'

Value and Type Columns:

No errors or warnings found on Value and Type columns

Target Columns:

❌ Error 601: At least one of the target_names is misspelled. Please verify, the target_names should be (optional target(s) inluded): 'inc hosp, inc death, cum death, cum hosp, peak size hosp, peak time hosp'. The data frame contains: 'inc_hosp, inc_death', as targets names.
❌ Error 602: No value found associated with the targets: inc hosp (required); output_type: sample. Please verify.
❌ Error 602: No value found associated with the targets: inc hosp (required), inc death (optional); output_type: sample. Please verify.
🟡 Warning 602: No value found associated with the targets: inc death (optional), cum death (optional); output_type: quantile.
🟡 Warning 602: No value found associated with the targets: inc hosp (optional), cum hosp (optional); output_type: quantile.
🟡 Warning 602: No value found associated with the targets: peak size hosp (optional); output_type: quantile.
🟡 Warning 602: No value found associated with the targets: peak time hosp (optional); output_type: cdf.

Locations:

No errors or warnings found on Location

Sample:

❌ Error 904: Samples are expected in the submission for the target(s): inc hosp. please verify.

Quantiles:

No errors or warnings found on quantiles values and format

Age Group:

No errors or warnings found on Age_group

Copy link

github-actions bot commented Feb 5, 2025

Run validation on files: 2024-08-11-SigSci-SWIFT.parquet

Required values:

No missing required value found

Columns:

No errors or warnings found on the column names and numbers

Scenarios:

No errors or warnings found on scenario name and scenario id columns

Origin Date Column:

No errors or warnings found on the column 'origin_date'

Value and Type Columns:

🟡 Warning 5043: All values associated with output type 'sample' should have a maximum of 1 decimal place

Target Columns:

❌ Error 607: At least one target week is missing in the time series. 8100 unique groups have been identified with this issue. For example:
group: 2024-08-11, A-2024-08-01, 30, inc hosp, 1, 0-4, sample, 159;
group: 2024-08-11, A-2024-08-01, 30, inc hosp, 1, 18-49, sample, 159;
group: 2024-08-11, A-2024-08-01, 30, inc hosp, 1, 5-17, sample, 159;
etc.
🟡 Warning 602: No value found associated with the targets: inc death (optional), cum death (optional); output_type: quantile.
🟡 Warning 602: No value found associated with the targets: inc hosp (optional), cum hosp (optional); output_type: quantile.
🟡 Warning 602: No value found associated with the targets: peak size hosp (optional); output_type: quantile.
🟡 Warning 602: No value found associated with the targets: peak time hosp (optional); output_type: cdf.

Locations:

No errors or warnings found on Location

Sample:

❌ Error 905: All the groups should contains the same number of trajectories per group. Please verify.

Column Pairing information:

Run grouping pairing:
No run grouping
Stochastic run pairing:
c("scenario_id (A-2024-08-01, B-2024-08-01, C-2024-08-01, D-2024-08-01, E-2024-08-01, F-2024-08-01)", "horizon", "location (01, 02, 04, 05, 06, 08, 09, 10, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 53, 54, 55, 56)", "age_group (0-130, 0-4, 18-49, 5-17, 50-64)")
Number of Samples: 300

Column Pairing information:

Run grouping pairing:
No run grouping
Stochastic run pairing:
c("scenario_id (A-2024-08-01, B-2024-08-01, C-2024-08-01, D-2024-08-01, E-2024-08-01, F-2024-08-01)", "horizon", "location (01, 02, 04, 05, 06, 08, 09, 10, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 53, 54, 55, 56)", "age_group (0-130, 0-4, 18-49, 5-17, 50-64)")
Number of Samples: 291

Column Pairing information:

Run grouping pairing:
No run grouping
Stochastic run pairing:
c("scenario_id (A-2024-08-01, B-2024-08-01, C-2024-08-01, D-2024-08-01, E-2024-08-01, F-2024-08-01)", "horizon", "location (01, 02, 04, 05, 06, 08, 09, 10, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 53, 54, 55, 56)", "age_group (0-130, 0-4, 18-49, 5-17, 50-64)")
Number of Samples: 279

Column Pairing information:

Run grouping pairing:
No run grouping
Stochastic run pairing:
c("scenario_id (A-2024-08-01, B-2024-08-01, C-2024-08-01, D-2024-08-01, E-2024-08-01, F-2024-08-01)", "horizon", "location (01, 02, 04, 05, 06, 08, 09, 10, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 53, 54, 55, 56)", "age_group (0-130, 0-4, 18-49, 5-17, 50-64)")
Number of Samples: 276

Column Pairing information:

Run grouping pairing:
No run grouping
Stochastic run pairing:
c("scenario_id (A-2024-08-01, B-2024-08-01, C-2024-08-01, D-2024-08-01, E-2024-08-01, F-2024-08-01)", "horizon", "location (01, 02, 04, 05, 06, 08, 09, 10, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 53, 54, 55, 56)", "age_group (0-130, 0-4, 18-49, 5-17, 50-64)")
Number of Samples: 288

Column Pairing information:

Run grouping pairing:
No run grouping
Stochastic run pairing:
c("scenario_id (A-2024-08-01, B-2024-08-01, C-2024-08-01, D-2024-08-01, E-2024-08-01, F-2024-08-01)", "horizon", "location (01, 02, 04, 05, 06, 08, 09, 10, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 53, 54, 55, 56)", "age_group (0-130, 0-4, 18-49, 5-17, 50-64)")
Number of Samples: 285

Column Pairing information:

Run grouping pairing:
No run grouping
Stochastic run pairing:
c("scenario_id (A-2024-08-01, B-2024-08-01, C-2024-08-01, D-2024-08-01, E-2024-08-01, F-2024-08-01)", "horizon", "location (01, 02, 04, 05, 06, 08, 09, 10, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 53, 54, 55, 56)", "age_group (0-130, 0-4, 18-49, 5-17, 50-64)")
Number of Samples: 282

Column Pairing information:

Run grouping pairing:
No run grouping
Stochastic run pairing:
c("scenario_id (A-2024-08-01, B-2024-08-01, C-2024-08-01, D-2024-08-01, E-2024-08-01, F-2024-08-01)", "horizon", "location (01, 02, 04, 05, 06, 08, 09, 10, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 53, 54, 55, 56)", "age_group (0-130, 0-4, 18-49, 5-17, 50-64)")
Number of Samples: 294

Column Pairing information:

Run grouping pairing:
No run grouping
Stochastic run pairing:
c("scenario_id (A-2024-08-01, B-2024-08-01, C-2024-08-01, D-2024-08-01, E-2024-08-01, F-2024-08-01)", "target", "horizon", "age_group (0-130, 0-4, 18-49, 5-17, 50-64)")
Number of Samples: 300

Quantiles:

No errors or warnings found on quantiles values and format

Age Group:

❌ Error 801: The age_group column should contain value written: 'AGEMIN-AGEMAX', one or more 'age_group' value(s) is not corresponding in the submission file, please verify: '65+'.
❌ Error 802: The age_group column contains unexpected value: '65+'. 'age_group' can only be: 0-130, 0-4, 5-17, 18-49, 50-64, 65-130, 0-17, 0-49, 0-64, 5-49, 5-64, 5-130, 18-64, 18-130, 50-130, for the target: inc hosp.
❌ Error 802: The age_group column contains unexpected value: '65+'. 'age_group' can only be: 0-130, 0-4, 5-17, 18-49, 50-64, 65-130, 0-17, 0-49, 0-64, 5-49, 5-64, 5-130, 18-64, 18-130, 50-130, for the target: inc hosp, inc death.

Copy link

github-actions bot commented Feb 5, 2025

Run validation on files: 2024-08-11-SigSci-SWIFT.parquet

Required values:

No missing required value found

Columns:

No errors or warnings found on the column names and numbers

Scenarios:

No errors or warnings found on scenario name and scenario id columns

Origin Date Column:

No errors or warnings found on the column 'origin_date'

Value and Type Columns:

🟡 Warning 5043: All values associated with output type 'sample' should have a maximum of 1 decimal place

Target Columns:

❌ Error 607: At least one target week is missing in the time series. 16254 unique groups have been identified with this issue. For example:
group: 2024-08-11, A-2024-08-01, 30, inc hosp, 1, 0-130, sample, 159;
group: 2024-08-11, A-2024-08-01, 30, inc hosp, 2, 0-130, sample, 159;
group: 2024-08-11, A-2024-08-01, 30, inc hosp, 3, 0-130, sample, 159;
etc.
🟡 Warning 602: No value found associated with the targets: inc death (optional), cum death (optional); output_type: quantile.
🟡 Warning 602: No value found associated with the targets: inc hosp (optional), cum hosp (optional); output_type: quantile.
🟡 Warning 602: No value found associated with the targets: peak size hosp (optional); output_type: quantile.
🟡 Warning 602: No value found associated with the targets: peak time hosp (optional); output_type: cdf.

Locations:

No errors or warnings found on Location

Sample:

❌ Error 902: The minimal accepted grouping includes the column(s): horizon, age_group, scenario_id, the column(s): horizon seem to be missing (or is missing a sub-group), please verify.

Column Pairing information:

Run grouping pairing:
c("scenario_id (A-2024-08-01, B-2024-08-01, C-2024-08-01, D-2024-08-01, E-2024-08-01, F-2024-08-01)", "horizon", "location (01, 02, 04, 05, 06, 08, 09, 10, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 53, 54, 55, 56)", "age_group (0-130, 0-4, 18-49, 5-17, 50-64, 65-130)"),
c("scenario_id (A-2024-08-01, C-2024-08-01, E-2024-08-01)", "horizon (22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 21)", "age_group (0-130, 0-4, 18-49, 5-17, 50-64, 65-130)")
Stochastic run pairing:
c("scenario_id (A-2024-08-01, B-2024-08-01, C-2024-08-01, D-2024-08-01, E-2024-08-01, F-2024-08-01)", "horizon", "location (01, 02, 04, 05, 06, 08, 09, 10, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 53, 54, 55, 56)", "age_group (0-130, 0-4, 18-49, 5-17, 50-64, 65-130)")
Number of Samples: 300

Column Pairing information:

Run grouping pairing:
No run grouping
Stochastic run pairing:
c("scenario_id (A-2024-08-01, B-2024-08-01, C-2024-08-01, D-2024-08-01, E-2024-08-01, F-2024-08-01)", "target", "horizon", "age_group (0-130, 0-4, 18-49, 5-17, 50-64, 65-130)")
Number of Samples: 300

Quantiles:

No errors or warnings found on quantiles values and format

Age Group:

No errors or warnings found on Age_group

Copy link

github-actions bot commented Feb 5, 2025

Run validation on files: 2024-08-11-SigSci-SWIFT.parquet

Required values:

No missing required value found

Columns:

No errors or warnings found on the column names and numbers

Scenarios:

No errors or warnings found on scenario name and scenario id columns

Origin Date Column:

No errors or warnings found on the column 'origin_date'

Value and Type Columns:

No errors or warnings found on Value and Type columns

Target Columns:

🟡 Warning 602: No value found associated with the targets: inc death (optional), cum death (optional); output_type: quantile.
🟡 Warning 602: No value found associated with the targets: inc hosp (optional), cum hosp (optional); output_type: quantile.
🟡 Warning 602: No value found associated with the targets: peak size hosp (optional); output_type: quantile.
🟡 Warning 602: No value found associated with the targets: peak time hosp (optional); output_type: cdf.

Locations:

No errors or warnings found on Location

Sample:

Column Pairing information:

Run grouping pairing:
No run grouping
Stochastic run pairing:
c("scenario_id (A-2024-08-01, B-2024-08-01, C-2024-08-01, D-2024-08-01, E-2024-08-01, F-2024-08-01)", "horizon", "location (01, 02, 04, 05, 06, 08, 09, 10, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 53, 54, 55, 56)", "age_group (0-130, 0-4, 18-49, 5-17, 50-64, 65-130)")
Number of Samples: 300

Column Pairing information:

Run grouping pairing:
No run grouping
Stochastic run pairing:
c("scenario_id (A-2024-08-01, B-2024-08-01, C-2024-08-01, D-2024-08-01, E-2024-08-01, F-2024-08-01)", "target", "horizon", "age_group (0-130, 0-4, 18-49, 5-17, 50-64, 65-130)")
Number of Samples: 300

Quantiles:

No errors or warnings found on quantiles values and format

Age Group:

No errors or warnings found on Age_group

@LucieContamin
Copy link
Collaborator

Hi @vpnagraj,

Thank you for the your submission and for providing the associated metadata and abstract files. I just want to verify I understand properly the pairing information in your projections: you provided 300 samples with the difference between each run is stochasticity (run_grouping all 1), and the runs are grouped/paired by scenario, horizon, location, target and age group. Is that correct?

Please let me know if any issues or questions,
Best,

Lucie

@vpnagraj
Copy link
Contributor Author

vpnagraj commented Feb 6, 2025

Thanks so much for reviewing Lucie. Yes that summary of the output pairing is accurate. If it would help to clarify anything else please let us know!

cc @abenefield1

@LucieContamin
Copy link
Collaborator

Thanks for validating! It's all good for now, I will merge the PR and work on integrating your projections.

Please let me know if any issues or questions,
Best,

Lucie

@LucieContamin LucieContamin merged commit 1a4206f into midas-network:main Feb 14, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants