Replace validation testing with JSON Schema v7 compatible libraries

[acmiyaguchi]

This will fix #332, and also introduces the jsonschema python package in case testing is done locally. There are 3 failing tests, 1 in python and 2 in java.

I've tested this on my local machine (without java configured, because pyjnius was not playing well with my macOS instance) which will skip the java tests, and in the docker image which can run either. When running in docker, I recommend the -n flags.

command	stats
local, pytest	1 failed, 166 passed, 167 skipped in 8.25s
scripts/mps-test -k python	1 failed, 166 passed, 167 deselected in 157.60s (0:02:37)
scripts/mps-test -k java	2 failed, 165 passed, 167 deselected in 141.69s (0:02:21)

The scripts/mps-build and scripts/mps-test are helpers for this workflow.

Current errors:

test_validation_pass_python[glean/glean.1.baseline.pass.json]
E           jsonschema.exceptions.ValidationError: 'examples.labeled_timing_distribution_example' does not match '^[a-z_][a-z0-9_]{0,29}(\\.[a-z_][a-z0-9_]{0,29})+$'
E           
E           Failed validating 'pattern' in schema['properties']['metrics']['properties']['labeled_timing_distribution']['propertyNames']:
E               {'maxLength': 61,
E                'pattern': '^[a-z_][a-z0-9_]{0,29}(\\.[a-z_][a-z0-9_]{0,29})+$',
E                'type': 'string'}
E           
E           On instance['metrics']['labeled_timing_distribution']:
E               'examples.labeled_timing_distribution_example'

test_validation_pass_java[glean/glean.1.baseline.pass.json]
E   jnius.JavaException: JVM exception occurred: #/metrics/labeled_timing_distribution/examples.labeled_timing_distribution_example: string [examples.labeled_timing_distribution_example] does not match pattern ^[a-z_][a-z0-9_]{0,29}(\.[a-z_][a-z0-9_]{0,29})+$

test_validation_pass_java[telemetry/untrusted-modules.4.unknownmodule.pass.json]
E   jnius.JavaException: JVM exception occurred: #/payload/combinedStacks/stacks/0/2/1: expected type: Number, found: String

There are still a few more things to do, but this should be in a reviewable state.

Checklist for reviewer:

• Commits should reference a bug or github issue, if relevant
• Scan the PR and verify that no changes (particularly to .circleci/config.yml) will cause environment variables (particularly credentials) to be exposed in test logs
• Trigger the integration CI test by pushing this revision as discussed in the README and review the report posted in the comments.

For glean changes:

• Update include/glean/CHANGELOG.md

[jklukas]

Note that the Java validator will default to json schema v4 unless the schema specifies a draft via the $schema keyword or the validator is built with specific options to control the draft in use. It looks like what we're doing here matches the configuration in ingestion-beam, so that's good. Just noting that this PR does not mean we are actually using draft 7 yet. We can address migrating versions later.

I'm really happy to see that it's possible to use the Java library from python in this way. Well done in finding the machinery to make that happen.

So, this approach looks great. I'd recommend tagging @relud for additional eyes on the pytest code.

[acmiyaguchi]

@relud r?, in particular with respect to the pytest configuration for generating tests.

[jklukas]

I love it.

[dataops-ci-bot]

Integration report for "Add missing objects in java tests"

Report for upstream
Report for latest commit

No changes detected.