Skip to content
This repository was archived by the owner on Jan 15, 2020. It is now read-only.

Commit

Permalink
👀 Improvements based on review
Browse files Browse the repository at this point in the history
  • Loading branch information
st3v3nhunt committed Mar 28, 2018
1 parent 998f246 commit 93ca0f3
Show file tree
Hide file tree
Showing 3 changed files with 24 additions and 19 deletions.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,19 +12,19 @@

The data sources are listed in [config.js](./config/config.js). Each data
source has two properties, the `filename` and the `url` where a JSON file can
be downloaded from. The file must be a JSON array of objects. Each file will
be downloaded from. The file must be a JSON array of objects. Each file will
be concatenated (along with some additional processing to ensure integrity) to
produce a merged data set (`merged-data.json`) which will be uploaded into the
Azure Storage account specified in `AZURE_STORAGE_CONNECTION_STRING`. The files
will be uploaded into a container as specified in `AZURE_BLOB_CONTAINER_NAME`
or `etl-ouput` as the default. If the storage account is `primarycare` and the
or `etl-ouput` as the default. If the storage account is `primarycare` and the
defaults were used the merged data set will be available to download from
[https://primarycare.blob.core.windows.net/etl-output/merged-data.json](https://primarycare.blob.core.windows.net/etl-output/merged-data.json).

## Running the application and scheduling

The application will run at startup and then on a daily basis, while the
container continues to run. . The time of day defaults to 7:15am, and can be
container continues to run. The time of day defaults to 7:15am, and can be
changed via the `UPDATE_SCHEDULE` environment variable. Further details on the
time format are available at
[here](https://www.npmjs.com/package/node-schedule)
Expand Down
26 changes: 14 additions & 12 deletions app.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,18 @@ const log = require('./lib/utils/logger');
const mergeAndUpload = require('./lib/mergeAndUpload');
const scheduleConfig = require('./config/scheduleConfig');

async function combine() {
if (!scheduleConfig.schedulerDisabled()) {
// run on initial start, then on the schedule
await mergeAndUpload();
}

log.info(`Scheduling sexual health service data merge with rule '${scheduleConfig.getSchedule()}'`);
schedule.scheduleJob(scheduleConfig.getSchedule(), async () => {
await mergeAndUpload();
});
}
(async function combine() {
try {
if (!scheduleConfig.schedulerDisabled()) {
// run on initial start, then on the schedule
await mergeAndUpload();
}

combine();
log.info(`Scheduling sexual health service data merge with rule '${scheduleConfig.getSchedule()}'`);
schedule.scheduleJob(scheduleConfig.getSchedule(), async () => {
await mergeAndUpload();
});
} catch (ex) {
log.error({ error: ex }, 'Error performing merge and upload');
}
}());
11 changes: 7 additions & 4 deletions test/unit/mergeFiles.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,16 @@ describe('merge', () => {
expect(result.length).to.equal(0);
});

it('should return an array with all entries when supplied with 2', () => {
const inputOne = [];
const inputTwo = [];
it('should return an array with all entries when supplied with two non-empty arrays', () => {
const inputOne = [1];
const inputTwo = [2];
const expectedMergedArrayLength = inputOne.length + inputTwo.length;

const result = merge(inputOne, inputTwo);

expect(result).to.be.an('array');
expect(result.length).to.equal(0);
expect(result.length).to.equal(expectedMergedArrayLength);
expect(result[0]).to.equal(inputOne[0]);
expect(result[1]).to.equal(inputTwo[0]);
});
});

0 comments on commit 93ca0f3

Please sign in to comment.