Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ppfd and patm computation to CRU #70

Merged
merged 33 commits into from
Dec 20, 2024
Merged

Add ppfd and patm computation to CRU #70

merged 33 commits into from
Dec 20, 2024

Conversation

fabern
Copy link
Member

@fabern fabern commented Dec 19, 2024

This PR adds computation of ppfd and patm, when ingesting CRU data.

Since neither ppfd nor patm are measured variables in CRU they are derived in the following way:

  • patm: using the elevation of the site only. calc_patm(elv, patm0 = 101325), which assumes normal pressure at sea level
  • ppfd: using calc_daily_solar(), ppfd is thus computed based on geometric relationships (sunrise, sunset, etc...) given the (time-varying up to 1'000'000 BC) orbital parameters of earth. Note that this strongly assumes sunshine fraction of 1.0, i.e. it is not only "always sunny in philadelphia" but on the whole world.
    • TODO: the assumption of sunshine fraction of 1.0 could be improved by linking it to the number of wet days mwetd (since it is available from CRU) for which this could be set to 0.0 @stineb do you think would make a difference?
    • EDIT: actually we can use for each day of the month the monthly average cloud cover to compute sunshine fraction (sf = 1 - (ccov/100))
    • An alternative would be to use bias-correction with WorldClim data. However, currently bias-correction limits the extraction period to years 1970 to 2000 only. This could potentially be changed.

The code requires all four columns:
year_start, year_end, as well as date_start, date_end
Since it is possible to only provide two of them, the others need to be derived. The code previously was difficult to understand. This now simplifies the intent.

We also add a test in the unlikely case where all four are provided, but they are inconsistent.
Previously numerous if-statements lead to replication of code. This simplifies the logic and better documents intent with code comments.
Since VPD is recomputed anyway on daily resolution it is not taken care of by `expand_clim_cru_monthly_byyr()`, i.e. it silently drops that column. More robust is if the column is kept and daily values are initialized as NA. This is done by this commit. Moreover, the data.frame() after recomputing VPD is now ungrouped, since this is less errror-prone.
… the end of function

This reduces code duplication and is similar to how it is handled for ppfd or patm
This is not needed anymore, since vpd is now computed after downscaling in any case.
@fabern fabern marked this pull request as draft December 19, 2024 10:54
@stineb
Copy link
Collaborator

stineb commented Dec 19, 2024

@fabern , Thanks! On your points:

  • The CRU provides cloud cover fraction (cld), which can be interpreted as 1 - (sunshine fraction). Can you use that to calculate PPFD?
  • Bias correction is ideally done as an option, analogously to how it's done for WATCH-WFDEI.

May I add a point?

  • It would be great to have a vignette demonstrating how CRU data (interpolated to daily) look like when compared to locally measured data (met data from FLUXNET). I think we have such a vignette already for VPD.

@fabern
Copy link
Member Author

fabern commented Dec 19, 2024

Ok, I will have a look at cld.

Below is the comparison between ppfd between 1970 and 2000 for the PROFOUND sites when comparing WorldClim (ppfd_fine) with the the cru estimated (SPLASH method in column ppfd).

Note how WorldClim is only between 35 to 80% of the computed value (scale). So a cld would need to be on that order to successfully mitigate the difference.

# A tibble: 108 × 6
    sitename       month srad_fine  ppfd_fine      ppfd scale
    <chr>          <dbl>     <dbl>      <dbl>     <dbl> <dbl>
  1 bily_kriz          1     3204. 0.0000757  0.000168  0.449
  2 bily_kriz          2     5963. 0.000141   0.000264  0.532
  3 bily_kriz          3     9768. 0.000231   0.000408  0.565
  4 bily_kriz          4    14471. 0.000342   0.000560  0.610
  5 bily_kriz          5    17992. 0.000425   0.000676  0.628
  6 bily_kriz          6    19051. 0.000450   0.000728  0.618
  7 bily_kriz          7    18658. 0.000441   0.000701  0.628
  8 bily_kriz          8    16233. 0.000383   0.000604  0.635
  9 bily_kriz          9    11533. 0.000272   0.000464  0.587
 10 bily_kriz         10     7208. 0.000170   0.000314  0.542
 11 bily_kriz         11     3477. 0.0000821  0.000196  0.420
 12 bily_kriz         12     2453. 0.0000579  0.000142  0.407
 13 collelongo         1     6329. 0.000149   0.000253  0.591
 14 collelongo         2     8922. 0.000211   0.000348  0.606
 15 collelongo         3    12936. 0.000305   0.000481  0.636
 16 collelongo         4    16654. 0.000393   0.000611  0.643
 17 collelongo         5    20838. 0.000492   0.000705  0.698
 18 collelongo         6    23505. 0.000555   0.000744  0.746
 19 collelongo         7    24291. 0.000574   0.000723  0.793
 20 collelongo         8    21381. 0.000505   0.000646  0.782
 21 collelongo         9    16104. 0.000380   0.000528  0.720
 22 collelongo        10    11693. 0.000276   0.000394  0.702
 23 collelongo        11     6964. 0.000164   0.000280  0.588
 24 collelongo        12     5294. 0.000125   0.000226  0.554
 25 hyytiala           1      795. 0.0000188  0.0000449 0.418
 26 hyytiala           2     2923. 0.0000690  0.000129  0.535
 27 hyytiala           3     7175. 0.000169   0.000280  0.605
 28 hyytiala           4    12646. 0.000299   0.000465  0.642
 29 hyytiala           5    17952. 0.000424   0.000625  0.679
 30 hyytiala           6    19584. 0.000462   0.000703  0.657
 31 hyytiala           7    18388. 0.000434   0.000664  0.654
 32 hyytiala           8    13720. 0.000324   0.000526  0.615
 33 hyytiala           9     7910. 0.000187   0.000348  0.537
 34 hyytiala          10     3285. 0.0000776  0.000181  0.430
 35 hyytiala          11     1004. 0.0000237  0.0000676 0.351
 36 hyytiala          12      419. 0.00000989 0.0000260 0.380
 37 kroof              1     3751  0.0000886  0.000178  0.498
 38 kroof              2     6470. 0.000153   0.000273  0.560
 39 kroof              3    10261. 0.000242   0.000413  0.586
 40 kroof              4    14595. 0.000345   0.000561  0.615
 41 kroof              5    18265. 0.000431   0.000672  0.642
 42 kroof              6    19098  0.000451   0.000722  0.625
 43 kroof              7    19240. 0.000454   0.000696  0.653
 44 kroof              8    16762. 0.000396   0.000602  0.657
 45 kroof              9    12152. 0.000287   0.000467  0.614
 46 kroof             10     7507. 0.000177   0.000322  0.551
 47 kroof             11     4032. 0.0000952  0.000205  0.465
 48 kroof             12     2876. 0.0000679  0.000152  0.447
 49 le_bray            1     4588. 0.000108   0.000213  0.509
 50 le_bray            2     7608. 0.000180   0.000306  0.588
 51 le_bray            3    11956. 0.000282   0.000439  0.643
 52 le_bray            4    16222. 0.000383   0.000574  0.668
 53 le_bray            5    18799. 0.000444   0.000672  0.660
 54 le_bray            6    20714. 0.000489   0.000715  0.684
 55 le_bray            7    21438. 0.000506   0.000693  0.731
 56 le_bray            8    18526. 0.000437   0.000610  0.717
 57 le_bray            9    14687. 0.000347   0.000488  0.711
 58 le_bray           10     9210. 0.000217   0.000352  0.618
 59 le_bray           11     5538. 0.000131   0.000239  0.547
 60 le_bray           12     3907. 0.0000922  0.000187  0.494
 61 peitz              1     2497. 0.0000590  0.000138  0.428
 62 peitz              2     4807. 0.000114   0.000232  0.490
 63 peitz              3     8704. 0.000206   0.000375  0.547
 64 peitz              4    14284. 0.000337   0.000532  0.634
 65 peitz              5    18051. 0.000426   0.000654  0.651
 66 peitz              6    19317. 0.000456   0.000710  0.642
 67 peitz              7    18875. 0.000446   0.000682  0.654
 68 peitz              8    16313. 0.000385   0.000578  0.666
 69 peitz              9    10926. 0.000258   0.000433  0.596
 70 peitz             10     6353. 0.000150   0.000281  0.533
 71 peitz             11     2815. 0.0000665  0.000164  0.404
 72 peitz             12     1811. 0.0000428  0.000113  0.378
 73 solling_beech      1     2447. 0.0000578  0.000141  0.410
 74 solling_beech      2     4950. 0.000117   0.000236  0.495
 75 solling_beech      3     8311. 0.000196   0.000381  0.515
 76 solling_beech      4    13149. 0.000310   0.000539  0.576
 77 solling_beech      5    17244. 0.000407   0.000663  0.615
 78 solling_beech      6    17767. 0.000420   0.000719  0.584
 79 solling_beech      7    17740. 0.000419   0.000690  0.607
 80 solling_beech      8    15197. 0.000359   0.000586  0.612
 81 solling_beech      9    10181. 0.000240   0.000439  0.547
 82 solling_beech     10     6026. 0.000142   0.000286  0.497
 83 solling_beech     11     2917. 0.0000689  0.000168  0.410
 84 solling_beech     12     1818. 0.0000429  0.000116  0.370
 85 soro               1     1727. 0.0000408  0.000102  0.398
 86 soro               2     3757. 0.0000887  0.000194  0.457
 87 soro               3     7972. 0.000188   0.000342  0.551
 88 soro               4    13595. 0.000321   0.000508  0.632
 89 soro               5    18235. 0.000431   0.000643  0.670
 90 soro               6    19401. 0.000458   0.000706  0.649
 91 soro               7    18731. 0.000442   0.000674  0.656
 92 soro               8    15530. 0.000367   0.000560  0.655
 93 soro               9     9653. 0.000228   0.000403  0.565
 94 soro              10     5184. 0.000122   0.000245  0.499
 95 soro              11     2169. 0.0000512  0.000128  0.400
 96 soro              12     1200. 0.0000283  0.0000789 0.359
 97 solling_spruce     1     2451. 0.0000579  0.000141  0.410
 98 solling_spruce     2     4957. 0.000117   0.000236  0.496
 99 solling_spruce     3     8321. 0.000196   0.000381  0.515
100 solling_spruce     4    13191. 0.000311   0.000539  0.578
101 solling_spruce     5    17199. 0.000406   0.000663  0.613
102 solling_spruce     6    17803. 0.000420   0.000719  0.585
103 solling_spruce     7    17748. 0.000419   0.000690  0.607
104 solling_spruce     8    15195. 0.000359   0.000586  0.612
105 solling_spruce     9    10202. 0.000241   0.000439  0.548
106 solling_spruce    10     6014. 0.000142   0.000286  0.496
107 solling_spruce    11     2908. 0.0000687  0.000168  0.409
108 solling_spruce    12     1812. 0.0000428  0.000116  0.369

@fabern
Copy link
Member Author

fabern commented Dec 19, 2024

When using the daily downscaled cloud cover from CRU to compute, the bias with WorldClim disappears. (See below 'scale' of the monthly averages is around 1.0.)

  • I see to add an illustrative vignette based on the VPD.
# A tibble: 108 × 6
    sitename       month srad_fine  ppfd_fine      ppfd scale
    <chr>          <dbl>     <dbl>      <dbl>     <dbl> <dbl>
  1 bily_kriz          1     3204. 0.0000757  0.0000825 0.917
  2 bily_kriz          2     5963. 0.000141   0.000137  1.03 
  3 bily_kriz          3     9768. 0.000231   0.000218  1.06 
  4 bily_kriz          4    14471. 0.000342   0.000323  1.06 
  5 bily_kriz          5    17992. 0.000425   0.000375  1.13 
  6 bily_kriz          6    19051. 0.000450   0.000414  1.09 
  7 bily_kriz          7    18658. 0.000441   0.000434  1.01 
  8 bily_kriz          8    16233. 0.000383   0.000362  1.06 
  9 bily_kriz          9    11533. 0.000272   0.000294  0.927
 10 bily_kriz         10     7208. 0.000170   0.000184  0.923
 11 bily_kriz         11     3477. 0.0000821  0.0000956 0.859
 12 bily_kriz         12     2453. 0.0000579  0.0000704 0.823
 13 collelongo         1     6329. 0.000149   0.000151  0.988
 14 collelongo         2     8922. 0.000211   0.000200  1.05 
 15 collelongo         3    12936. 0.000305   0.000282  1.08 
 16 collelongo         4    16654. 0.000393   0.000360  1.09 
 17 collelongo         5    20838. 0.000492   0.000439  1.12 
 18 collelongo         6    23505. 0.000555   0.000495  1.12 
 19 collelongo         7    24291. 0.000574   0.000569  1.01 
 20 collelongo         8    21381. 0.000505   0.000495  1.02 
 21 collelongo         9    16104. 0.000380   0.000377  1.01 
 22 collelongo        10    11693. 0.000276   0.000252  1.10 
 23 collelongo        11     6964. 0.000164   0.000170  0.967
 24 collelongo        12     5294. 0.000125   0.000137  0.913
 25 hyytiala           1      795. 0.0000188  0.0000237 0.792
 26 hyytiala           2     2923. 0.0000690  0.0000779 0.886
 27 hyytiala           3     7175. 0.000169   0.000157  1.08 
 28 hyytiala           4    12646. 0.000299   0.000256  1.16 
 29 hyytiala           5    17952. 0.000424   0.000377  1.12 
 30 hyytiala           6    19584. 0.000462   0.000414  1.12 
 31 hyytiala           7    18388. 0.000434   0.000411  1.06 
 32 hyytiala           8    13720. 0.000324   0.000264  1.22 
 33 hyytiala           9     7910. 0.000187   0.000172  1.09 
 34 hyytiala          10     3285. 0.0000776  0.0000920 0.843
 35 hyytiala          11     1004. 0.0000237  0.0000320 0.742
 36 hyytiala          12      419. 0.00000989 0.0000130 0.763
 37 kroof              1     3751  0.0000886  0.0000861 1.03 
 38 kroof              2     6470. 0.000153   0.000154  0.989
 39 kroof              3    10261. 0.000242   0.000227  1.07 
 40 kroof              4    14595. 0.000345   0.000317  1.09 
 41 kroof              5    18265. 0.000431   0.000373  1.15 
 42 kroof              6    19098  0.000451   0.000399  1.13 
 43 kroof              7    19240. 0.000454   0.000443  1.03 
 44 kroof              8    16762. 0.000396   0.000387  1.02 
 45 kroof              9    12152. 0.000287   0.000304  0.944
 46 kroof             10     7507. 0.000177   0.000186  0.953
 47 kroof             11     4032. 0.0000952  0.000108  0.884
 48 kroof             12     2876. 0.0000679  0.0000760 0.894
 49 le_bray            1     4588. 0.000108   0.000106  1.02 
 50 le_bray            2     7608. 0.000180   0.000161  1.12 
 51 le_bray            3    11956. 0.000282   0.000251  1.12 
 52 le_bray            4    16222. 0.000383   0.000316  1.21 
 53 le_bray            5    18799. 0.000444   0.000389  1.14 
 54 le_bray            6    20714. 0.000489   0.000449  1.09 
 55 le_bray            7    21438. 0.000506   0.000476  1.06 
 56 le_bray            8    18526. 0.000437   0.000417  1.05 
 57 le_bray            9    14687. 0.000347   0.000305  1.14 
 58 le_bray           10     9210. 0.000217   0.000206  1.05 
 59 le_bray           11     5538. 0.000131   0.000123  1.06 
 60 le_bray           12     3907. 0.0000922  0.0000929 0.992
 61 peitz              1     2497. 0.0000590  0.0000624 0.944
 62 peitz              2     4807. 0.000114   0.000111  1.03 
 63 peitz              3     8704. 0.000206   0.000192  1.07 
 64 peitz              4    14284. 0.000337   0.000305  1.10 
 65 peitz              5    18051. 0.000426   0.000360  1.19 
 66 peitz              6    19317. 0.000456   0.000403  1.13 
 67 peitz              7    18875. 0.000446   0.000428  1.04 
 68 peitz              8    16313. 0.000385   0.000347  1.11 
 69 peitz              9    10926. 0.000258   0.000247  1.04 
 70 peitz             10     6353. 0.000150   0.000151  0.996
 71 peitz             11     2815. 0.0000665  0.0000771 0.862
 72 peitz             12     1811. 0.0000428  0.0000521 0.821
 73 solling_beech      1     2447. 0.0000578  0.0000623 0.927
 74 solling_beech      2     4950. 0.000117   0.000111  1.05 
 75 solling_beech      3     8311. 0.000196   0.000186  1.06 
 76 solling_beech      4    13149. 0.000310   0.000289  1.07 
 77 solling_beech      5    17244. 0.000407   0.000357  1.14 
 78 solling_beech      6    17767. 0.000420   0.000382  1.10 
 79 solling_beech      7    17740. 0.000419   0.000393  1.07 
 80 solling_beech      8    15197. 0.000359   0.000343  1.05 
 81 solling_beech      9    10181. 0.000240   0.000228  1.06 
 82 solling_beech     10     6026. 0.000142   0.000143  0.993
 83 solling_beech     11     2917. 0.0000689  0.0000759 0.908
 84 solling_beech     12     1818. 0.0000429  0.0000500 0.859
 85 soro               1     1727. 0.0000408  0.0000497 0.821
 86 soro               2     3757. 0.0000887  0.0000937 0.947
 87 soro               3     7972. 0.000188   0.000178  1.06 
 88 soro               4    13595. 0.000321   0.000309  1.04 
 89 soro               5    18235. 0.000431   0.000383  1.12 
 90 soro               6    19401. 0.000458   0.000415  1.10 
 91 soro               7    18731. 0.000442   0.000426  1.04 
 92 soro               8    15530. 0.000367   0.000350  1.05 
 93 soro               9     9653. 0.000228   0.000229  0.994
 94 soro              10     5184. 0.000122   0.000136  0.903
 95 soro              11     2169. 0.0000512  0.0000666 0.769
 96 soro              12     1200. 0.0000283  0.0000399 0.710
 97 solling_spruce     1     2451. 0.0000579  0.0000624 0.928
 98 solling_spruce     2     4957. 0.000117   0.000111  1.05 
 99 solling_spruce     3     8321. 0.000196   0.000186  1.06 
100 solling_spruce     4    13191. 0.000311   0.000289  1.08 
101 solling_spruce     5    17199. 0.000406   0.000357  1.14 
102 solling_spruce     6    17803. 0.000420   0.000382  1.10 
103 solling_spruce     7    17748. 0.000419   0.000393  1.07 
104 solling_spruce     8    15195. 0.000359   0.000343  1.05 
105 solling_spruce     9    10202. 0.000241   0.000228  1.06 
106 solling_spruce    10     6014. 0.000142   0.000143  0.990
107 solling_spruce    11     2908. 0.0000687  0.0000759 0.905
108 solling_spruce    12     1812. 0.0000428  0.0000500 0.856

@fabern
Copy link
Member Author

fabern commented Dec 19, 2024

I have now updated the vignette.

The computed PPFD look alright compared with fluxnet data from CH-Lae. (Albeit the monthly bias-correction introduces discontinuous steps between months. See the jump between June/July 2012 in the red line, not present in the blue line.)

image

@fabern fabern marked this pull request as ready for review December 20, 2024 13:11
@fabern fabern requested a review from stineb December 20, 2024 13:11
@fabern
Copy link
Member Author

fabern commented Dec 20, 2024

@stineb This is now ready for review. In a nutshell this PR:

  • added functionality to get patm and ppfd when ingesting CRU (in R/ingest_globalfields.R)
  • clarified some date checks and streamlined code in R/ingest.R
  • also cleaned up some warnings in R CMD Check
  • limited the CI checks to be performed on workstation 2 only (and deactivated some other checks see Reactivate deactivated CI tests #71)
  • currently CI still failing on macOS:
    We might need to update or remove installation of gdal in the workflow file here. Do we have another package with rgdal dependency we could copy?

@fabern fabern mentioned this pull request Dec 20, 2024
@stineb
Copy link
Collaborator

stineb commented Dec 20, 2024

Thanks @fabern . We'll eventually have to remove all use of {rgdal} as the library is discontinued. This means also that we'll have to replace all use of the {raster}, now using {terra}. There might be some places left where raster is used, these should eventually be changed.

@stineb
Copy link
Collaborator

stineb commented Dec 20, 2024

I'm going ahead with the merge anyways.

@stineb stineb merged commit 845dc95 into master Dec 20, 2024
4 of 5 checks passed
@fabern fabern deleted the add-ppfd-patm-to-cru branch December 20, 2024 17:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants