Skip to content

Commit

Permalink
publish extras data files on BC object storage
Browse files Browse the repository at this point in the history
  • Loading branch information
smnorris committed Oct 3, 2024
1 parent 5bfd025 commit 88050e9
Show file tree
Hide file tree
Showing 2 changed files with 14 additions and 1 deletion.
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -257,6 +257,6 @@ data/WBD_National_GDB.zip:
for table in $(EXTRAS) ; do \
echo $$table ;\
set -e; $(PSQL) -c "truncate whse_basemapping.$$table" ; \
set -e; $(PSQL) -c "\copy whse_basemapping.$$table FROM PROGRAM 'curl -s https://www.hillcrestgeo.ca/outgoing/public/fwapg/$$table.csv.gz | gunzip' delimiter ',' csv header" ; \
set -e; $(PSQL) -c "\copy whse_basemapping.$$table FROM PROGRAM 'curl -s https://nrs.objectstore.gov.bc.ca/bchamp/$$table.csv.gz | gunzip' delimiter ',' csv header" ; \
done
touch $@
13 changes: 13 additions & 0 deletions extras/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,16 @@ Value added FWA data that generally requires several hours to generate.
Rather than re-generate the lookups each time the database is loaded, `fwapg` defaults to loading the tables from cached csvs.

If updates to the lookups are required, regenerate the csv files using these scripts.


# Publication

To upload data to BC object storage on completion of scripts:

aws s3 cp channel_width/fwa_stream_networks_channel_width.csv.gz s3://$BUCKET --acl public-read
aws s3 cp discharge/fwa_stream_networks_discharge.csv.gz s3://$BUCKET --acl public-read
aws s3 cp precipitation/fwa_stream_networks_mean_annual_precip.csv.gz s3://$BUCKET --acl public-read
aws s3 cp lookups/fwa_assessment_watersheds_lut.csv.gz s3://$BUCKET --acl public-read
aws s3 cp lookups/fwa_assessment_watersheds_streams_lut.csv.gz s3://$BUCKET --acl public-read
aws s3 cp lookups/fwa_waterbodies_upstream_area.csv.gz s3://$BUCKET --acl public-read
aws s3 cp lookups/fwa_watersheds_upstream_area.csv.gz s3://$BUCKET --acl public-read

0 comments on commit 88050e9

Please sign in to comment.