Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Model validation #9

Open
4 of 6 tasks
Aariq opened this issue Dec 12, 2024 · 4 comments
Open
4 of 6 tasks

Model validation #9

Aariq opened this issue Dec 12, 2024 · 4 comments
Assignees

Comments

@Aariq
Copy link
Member

Aariq commented Dec 12, 2024

Since switching to 10ºC for base temp, the new model outputs are more qualitatively different from pixel-wise linear regression and also seem to vary depending on either data resolution or k or both. (see also cct-datascience/organization#2354)

Might want to develop methods for this stuff, but not actually do it until we decide on thresholds (#5) since some model tuning will be specific to each dataset.

In no particular order (but I should order these before doing them):

  • Check that k is appropriately large for all models
  • Check residuals—is gaussian error family still appropriate?
  • Check (temporal and spatial) autocorrelation—itsadug package or spdep package (for Moran's I permutation test)
  • Explore gaussian process smoother #11
  • What about NCV? Is it worth trying again? (Explore/implement NCV #1)
  • Read papers I have about spatiotemporal analysis with GAMs—what, if anything, do they do differently?
@Aariq Aariq self-assigned this Dec 16, 2024
@Aariq
Copy link
Member Author

Aariq commented Dec 17, 2024

Temporal correlation is there, but values of rho are negative and relatively small (I think). ACF plots have really no pattern in them. I now know how to deal with temporal autocorrelation if I reviewer asks (on a branch, will push to GH soon)

@Aariq
Copy link
Member Author

Aariq commented Dec 18, 2024

Create a .qmd for model validation that can eventually become an appendix

@Aariq
Copy link
Member Author

Aariq commented Jan 23, 2025

I added the output from gratia::appraise() in PR #18, but need to evaluate this and k-check results on a per-model (i.e. per GDD threshold) basis.

@Aariq
Copy link
Member Author

Aariq commented Jan 27, 2025

Looks like at least some thresholds have leptokurtic residuals—a student T distribution may be more appropriate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant