Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Algoa Bay Forecast] Model output web-visualizations #12

Closed
2 tasks done
zachsa opened this issue Jul 5, 2022 · 5 comments
Closed
2 tasks done

[Algoa Bay Forecast] Model output web-visualizations #12

zachsa opened this issue Jul 5, 2022 · 5 comments
Assignees
Labels
enhancement New feature or request

Comments

@zachsa
Copy link
Collaborator

zachsa commented Jul 5, 2022

The model output includes a NetCDF file with a 'normalized' grid (lat/lng coordinates in the center of the grid box) and with a subset of the variables that are of immediate use.

Variables:

  • temperature
  • salinity
  • depth (for translating grid depth to meters)

Dimensions:

  • time
  • lat
  • lng
  • depth (grid depth)

Some visualization ideas:

Temperature

  • time-based animation of temperature contours at surface level (to reinstate previous work)
    -- Can the depth at which the temperature contours are derived by configurable?
    -- Would it be useful to 'average' out the water column to create temperature contours that are an average of the whole/some of the water column (rather than just a specific level)
  • Depth based animation of temperature contours (see how temperature bands change across the water column)

Salinity
TODO

@zachsa zachsa added the enhancement New feature or request label Jul 5, 2022
@zachsa zachsa self-assigned this Jul 5, 2022
@zachsa
Copy link
Collaborator Author

zachsa commented Jul 5, 2022

@mattcarr03 - any ideas/other input is welcome!

@mattcarr03
Copy link
Collaborator

Those sound great!

I think what would be cool as well is the current speed and direction. The speed can be added as a variable in the post processing script (calculated u and v ). The it can be added on as a layer like the temperature.

What is really cool is if we could get animated velocity vectors. I used this package for the previous site (https://github.com/onaci/leaflet-velocity). Which is a leaflet package but I'm sure there are openmaps packages as well. There is also an ERSI package (https://github.com/Esri/wind-js). Not sure how difficult it is to implement, I was creating json files from the data to do this but there is probably a better way forward

Really looking to to seeing the visualisations!

@mattcarr03
Copy link
Collaborator

Just saw this from a quick google search openlayers/openlayers#9947 maybe it isn't straight forward in open layers but possible

@zachsa
Copy link
Collaborator Author

zachsa commented Jul 5, 2022

Okay - noted on the vector layer. Here is a nice vector layer example in openlayers on that thread. openlayers/openlayers#9947 (comment). could that work? Could always use both leaflet and openlayers for different visualizations

zachsa added a commit that referenced this issue Jul 5, 2022
…ng the netcdf data into postgis for visualization (#12). There are now two 'modes' of operation: "transforming" the model output and "loading" the transformed output somewhere else. The load will likely be a gdal call or two (TBD)
@zachsa
Copy link
Collaborator Author

zachsa commented Jul 6, 2022

This is the print of the current netcdf file that i will use as a the basis for all visualizations @mattcarr03

It's actually a fairly large file (2.4GB so far). so if it's possible to get the current speed (or is it velocity?) from the u / v variables, then that would be better than expanding the file size.

how can the current speed be calculated?

<xarray.Dataset>
Dimensions:      (time: 240, depth: 20, lat: 106, lon: 152)
Coordinates:
    lon_rho      (lat, lon) float32 ...
    lat_rho      (lat, lon) float32 ...
  * depth        (depth) float32 -0.975 -0.925 -0.875 ... -0.125 -0.075 -0.025
  * time         (time) datetime64[ns] 2022-06-30T01:00:00 ... 2022-07-10
Dimensions without coordinates: lat, lon
Data variables:
    temperature  (time, depth, lat, lon) float32 ...
    salt         (time, depth, lat, lon) float32 ...
    u            (time, depth, lat, lon) float64 ...
    v            (time, depth, lat, lon) float64 ...
    m_rho        (time, depth, lat, lon) float64 ...
Attributes:
    description:  CROCO output from algoa Bay model transformed lon/lat/depth...

zachsa added a commit that referenced this issue Jul 6, 2022
…and on all variables in the processed netcdf model output. The current approach is to register out-db rasters into a single postgres table, that i can then push/shove into a polygon view for geoserver (#12)
zachsa added a commit that referenced this issue Jul 6, 2022
…ng "load" step, though I'm not sure how this will work specifying an out-db raster... TBC (#12)
zachsa added a commit that referenced this issue Jul 7, 2022
  - Removed nodemon from dev tooling (was conflicting with chomp dep watching, and is likely not required any more with prettier --cache flag)
  - Updated site settings panel to load mui components lazily
  - Updated suspense loading to allow for either linear or circular loading animations
zachsa added a commit that referenced this issue Jul 7, 2022
 (1) #12 Combined the model output post processing job with the job for loading the output to postgres (since these scripts are now the same cli)
 (2) Added code to test that PG env is being loaded correctly into the runner workflow
zachsa added a commit that referenced this issue Jul 11, 2022
… misconfigured postgis (since this script doesn't use postgis). Updated the loading function to also insert coordinate datasets from netcdf into postgis (#12, #13)
zachsa added a commit that referenced this issue Jul 13, 2022
…teps, since its hours since x date. And updated the scheduled algoa bay forecast run to be at 6AM SAST to avoid any potential problems with UTC 00:00 time zones. Also, previously work included setting the boundary data job to restart on failure (#12 #13)
zachsa added a commit that referenced this issue Jul 18, 2022
… variable. And started experimenting with raster union functions to produce a vector contour. However.. the srid is not set for the grid (obviously). so TBC. #12
zachsa added a commit that referenced this issue Jul 27, 2022
…n nginx proxy. And added a vector tile service to proxy the postgres db so that vector-tile maps can be provided (#12)
zachsa added a commit that referenced this issue Jul 29, 2022
zachsa added a commit that referenced this issue Aug 1, 2022
…ations page - test to see that I can thread geometry through the server to the client (#12)
zachsa added a commit that referenced this issue Aug 8, 2022
zachsa added a commit that referenced this issue Aug 15, 2022
…phql api to resolve model ids to a specifid model (#12)
zachsa added a commit that referenced this issue Aug 15, 2022
…lorer view to model view, converted the explorer view to use ESRI (#12)
zachsa added a commit that referenced this issue Aug 16, 2022
zachsa added a commit that referenced this issue Aug 17, 2022
zachsa added a commit that referenced this issue Aug 18, 2022
zachsa added a commit that referenced this issue Aug 18, 2022
…ve 3d coordinates but are not showing in 3d (#12)
zachsa added a commit that referenced this issue Aug 19, 2022
…P on SQL loading many points. Got 3d points for single model layer to display on the web UI (#12)
zachsa added a commit that referenced this issue Aug 22, 2022
…ots actually). Updated toolkit cli to load all depth level values at all timesteps into the postgres database (#12)
zachsa added a commit that referenced this issue Aug 23, 2022
…o see if zarr data can be retrieved directly via javascript (#12)
zachsa added a commit that referenced this issue Aug 28, 2022
… executions rather using python multiprocessing module. This should make the posgis load work, whereas previously it took too long to run (#12)
zachsa added a commit that referenced this issue Aug 29, 2022
… working on getting time values through to db (#12)
zachsa added a commit that referenced this issue Sep 5, 2022
zachsa added a commit that referenced this issue Sep 15, 2022
zachsa added a commit that referenced this issue Sep 15, 2022
…n database. should support easily showing the false bay model (#12)
zachsa added a commit that referenced this issue Oct 3, 2022
…d depth is below surface level, and also improved performance slightly via reducing number of geo transforms (#12)
zachsa added a commit that referenced this issue Oct 14, 2022
@zachsa zachsa closed this as completed Jun 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants