Code

Feature/tet 812 ingest stova data#5866

Code

Merged

DeanElliott96 merged 12 commits intomainuktrade/data-hub-api:mainfrom feature/TET-812-ingest-stova-datauktrade/data-hub-api:feature/TET-812-ingest-stova-dataCopy head branch name to clipboardDec 23, 2024

+281-125

Commits

Commits on Dec 18, 2024

• Add ingestion tasks for Stova Events

  DeanElliott96

  committed
  0c7aad4

  Copy full SHA for 0c7aad4

• Move class variable to fixture

  DeanElliott96

  committed
  b627839

  Copy full SHA for b627839

• Remove s3_processor param as can't be pickled and add transport_params to connect to s3_processor

  Show description for 0f65e66
  DeanElliott96

  committed
  0f65e66

  Copy full SHA for 0f65e66

• Fix flake8 issue

  DeanElliott96

  committed
  9253b0c

  Copy full SHA for 9253b0c

Commits on Dec 19, 2024

• Improve error handling and add more tests

  DeanElliott96

  committed
  f770f7c

  Copy full SHA for f770f7c

• Schedule stova event ingestion every hour

  DeanElliott96

  committed
  700e9d4

  Copy full SHA for 700e9d4

Commits on Dec 20, 2024

• Add new service for stova events and set each new event from stova to use the service

  DeanElliott96

  committed
  e68cb96

  Copy full SHA for e68cb96

• Rename function and add docstring

  DeanElliott96

  committed
  b37461b

  Copy full SHA for b37461b

• Add test for saving Event from Stova with default service

  DeanElliott96

  committed
  47cef81

  Copy full SHA for 47cef81

• Update ingest function name for stova to be consistent with others

  DeanElliott96

  committed
  adb58f2

  Copy full SHA for adb58f2

• Change from list to set for faster lookups

  DeanElliott96

  committed
  26d96a9

  Copy full SHA for 26d96a9

• Override S3_LOCAL_ENDPOINT_URL for tests

  Show description for d470f54
  DeanElliott96

  committed
  d470f54

  Copy full SHA for d470f54