Skip to content
This repository has been archived by the owner on Sep 23, 2024. It is now read-only.

JSONschema version not compatible with target-snowflake #228

Open
kingalban opened this issue Sep 13, 2023 · 0 comments
Open

JSONschema version not compatible with target-snowflake #228

kingalban opened this issue Sep 13, 2023 · 0 comments
Labels
bug Something isn't working

Comments

@kingalban
Copy link

Describe the bug
This tap produces a schema with exclusiveMaximum and exclusiveMinimum according the JSONschema V5, but these are incompatible with pipelinewise-target-snowflake's JSONschema v7 validator.

In JSONschema v6, the fields exclusiveMaximum and exclusiveMinimum were changed from boolean to numeric (JSON Schema Draft-06 Release Notes).

pipelinewise-target-snowflake uses the the incompatible v7 validator: pipelinewise-target-snowflake source

The exclusiveMaximum is set in the schema here

In JSON schema v5 and below, a numeric field can be described as:

{
      "type": ["null", "number" ],
      "exclusiveMaximum": true,
      "maximum": 10000,
      "multipleOf": 1e-38,
      "exclusiveMinimum": true,
      "minimum": -10000
}

The equivalent in v6 and above is:

{
      "type": ["null", "number" ],
      "exclusiveMaximum": 10000,
      "multipleOf": 1e-38,
      "exclusiveMinimum": -10000
}

Steps to replicate

  1. create config.json for a database with a numeric type colum, and for a snowflake database.
  2. tap-postgresql -c tap-config.json | target-snowflake -c target-config.json
  3. See error from the jsonschema validator: jsonschema.exceptions.ValidationError: 85000 is greater than or equal to the maximum of True

Expected behavior
The tap and target should communicate with the same standard

Your environment

  • pipelinewise-tap-postgres version: 2.1.0

I can see that just moving to JSONschema v7 might be simple, it will probably break something else. Can you give some input on what the best allround solution to this would look like?

@kingalban kingalban added the bug Something isn't working label Sep 13, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant