Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow country to be set using quickstart #5797

Merged
merged 5 commits into from
Feb 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

- Null values in first column of first row of ingested data no longer cause flowetl to skip ingestion [#5090](https://github.com/Flowminder/FlowKit/issues/5090)

### Added

- Quickstart script now supports arbitrary countries via `EXAMPLE_COUNTRY` env var. [#5796](https://github.com/Flowminder/FlowKit/issues/5796)

## [1.18.2]

## Fixed
Expand Down
2 changes: 1 addition & 1 deletion development_environment
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ P_RELOCATE=0.1
INTERACTIONS_MULTIPLIER=5
DISASTER_START=2016-02-11
DISASTER_END=2016-03-01
DISASTER_REGION_PCOD="524 4 12"
DISASTER_REGION_PCOD="NPL.1.1_1"
SUBSCRIBERS_SEED=12345
CALLS_SEED=22222
CELLS_SEED=99999
Expand Down
2 changes: 2 additions & 0 deletions docker-compose-syntheticdata.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ services:
image: flowminder/flowdb-synthetic-data:${CONTAINER_TAG:-latest}
environment:
SYNTHETIC_DATA_GENERATOR: ${SYNTHETIC_DATA_GENERATOR:?Must set SYNTHETIC_DATA_GENERATOR env var}
COUNTRY: ${COUNTRY:-"NPL"}
N_SITES: ${N_SITES:?Must set N_SITES env var}
N_CELLS: ${N_CELLS:?Must set N_CELLS env var}
N_SUBSCRIBERS: ${N_SUBSCRIBERS:?Must set N_SUBSCRIBERS env var}
Expand All @@ -27,3 +28,4 @@ services:
INTERACTIONS_MULTIPLIER: ${INTERACTIONS_MULTIPLIER:?Must set INTERACTIONS_MULTIPLIER env var}
DISASTER_START: ${DISASTER_START:?Must set DISASTER_START env var}
DISASTER_END: ${DISASTER_END:?Must set DISASTER_END env var}
DISASTER_REGION_PCOD: ${DISASTER_REGION_PCOD:-"NPL.1.1_1"}
8 changes: 8 additions & 0 deletions docs/source/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,14 @@ bash <(curl -s https://raw.githubusercontent.com/Flowminder/FlowKit/master/quick

to get the examples with the larger dataset (the one used when producing this documentation).

You can also explore FlowKit using synthetic data for other countries, by setting the `EXAMPLE_COUNTRY` environment variable to a three-level ISO country code, for example to simulate data for the UK with a crisis in Greater London:

```bash
EXAMPLE_COUNTRY=GBR EXAMPLE_DISASTER_REGION_PCOD=GBR.1.36_1 bash <(curl -s https://raw.githubusercontent.com/Flowminder/FlowKit/master/quick_start.sh) examples
```

Available countries are those with boundaries in the [GADM 3.6](https://gadm.org/download_country36.html) dataset. Possible values for `EXAMPLE_DISASTER_REGION_PCOD` are any admin2 region for the corresponding country, based on the `GID_2` field in the GADM 3.6 shapefile for that country.

!!! info
The small dataset is sufficient for most of the worked examples, but the larger dataset is required for the [Flows Above Normal](analyst/worked_examples/flows-above-normal.ipynb) example because this uses data for dates outside the range included in the small dataset.

Expand Down
2 changes: 1 addition & 1 deletion flowdb/testdata/bin/9900_ingest_synthetic_data.sh
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ elif [ -f /opt/synthetic_data/generate_synthetic_data_sql.py ] && [ "$SYNTHETIC
--out-of-area-probability ${P_OUT_OF_AREA:-0.05}\
--relocation-probability ${P_RELOCATE:-0.05}\
--interactions-multiplier ${INTERACTIONS_MULTIPLIER:-5}\
--disaster-zone ${DISASTER_REGION_PCOD:-"NPL.1.1_1"} \
--disaster-zone \"${DISASTER_REGION_PCOD:-"NPL.1.1_1"}\" \
--disaster-start-date ${DISASTER_START:-"2015-01-01"} \
--disaster-end-date ${DISASTER_END:-"2015-01-01"} \
--country ${COUNTRY} || (cat cat /var/lib/postgresql/data/pg_log/postgres-* && exit 1)
Expand Down
2 changes: 2 additions & 0 deletions quick_start.sh
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,8 @@ then
$DOCKER_COMPOSE down -v
else
source /dev/stdin <<< "$(curl -s https://raw.githubusercontent.com/Flowminder/FlowKit/${GIT_REVISION}/development_environment)"
export COUNTRY=${EXAMPLE_COUNTRY:-$COUNTRY}
export DISASTER_REGION_PCOD=${EXAMPLE_DISASTER_REGION_PCOD:-$DISASTER_REGION_PCOD}
echo "Starting containers (this may take a few minutes)"
RUNNING=`$DOCKER_COMPOSE ps -q flowdb flowapi flowmachine flowauth flowmachine_query_locker $WORKED_EXAMPLES`
if [[ "$RUNNING" != "" ]]; then
Expand Down