Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update MAE metrics to limit potential forecast deviation by up to 48 hours #34

Open
aaTman opened this issue Jan 23, 2025 · 0 comments · May be fixed by #37
Open

Update MAE metrics to limit potential forecast deviation by up to 48 hours #34

aaTman opened this issue Jan 23, 2025 · 0 comments · May be fixed by #37
Assignees

Comments

@aaTman
Copy link
Collaborator

aaTman commented Jan 23, 2025

Currently, the method in which MaximumMAE is calculated takes the difference between the maximum temperature of the forecast at each unique instantiated forecast at each init_time (i.e. a 2023-01-01 00z forecast valid from 2023-01-01 00z to 2023-01-11 00z), and the maximum of the case.

There is a risk of edge cases in which a model captured - correctly, or incorrectly - an anomalous temperature event far enough in a forecast that it would be independent of the regime that we want to focus on for that case.

We approached filtering the events with a 48 hour buffer on the ends of each heat wave event with the intent of capturing onset, duration, and temperature deviations in case a model has a temporal displacement. With that in mind, we want to truncate the maximum temperatures in the forecast to +-48 hours of the occurrence of the actual maximum of the regionally averaged temperature based on ERA5 data.

I will submit a PR shortly with updated testing, but happy to invoke a discussion if anyone watching the repo has thoughts!

@aaTman aaTman self-assigned this Jan 23, 2025
@aaTman aaTman linked a pull request Jan 27, 2025 that will close this issue
11 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant