Skip to content

Commit

Permalink
Merge pull request #1 from zillow/extend-extension-contexts
Browse files Browse the repository at this point in the history
Extend compiler and validation contexts
  • Loading branch information
vassilvk authored Mar 10, 2024
2 parents 460fe6f + 326ac40 commit 1a2ef10
Show file tree
Hide file tree
Showing 16 changed files with 1,288 additions and 246 deletions.
221 changes: 2 additions & 219 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,220 +1,3 @@
# jsonschema v5.3.1
# jsonschema

[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
[![GoDoc](https://godoc.org/github.com/santhosh-tekuri/jsonschema?status.svg)](https://pkg.go.dev/github.com/santhosh-tekuri/jsonschema/v5)
[![Go Report Card](https://goreportcard.com/badge/github.com/santhosh-tekuri/jsonschema/v5)](https://goreportcard.com/report/github.com/santhosh-tekuri/jsonschema/v5)
[![Build Status](https://github.com/santhosh-tekuri/jsonschema/actions/workflows/go.yaml/badge.svg?branch=master)](https://github.com/santhosh-tekuri/jsonschema/actions/workflows/go.yaml)
[![codecov](https://codecov.io/gh/santhosh-tekuri/jsonschema/branch/master/graph/badge.svg?token=JMVj1pFT2l)](https://codecov.io/gh/santhosh-tekuri/jsonschema)

Package jsonschema provides json-schema compilation and validation.

[Benchmarks](https://dev.to/vearutop/benchmarking-correctness-and-performance-of-go-json-schema-validators-3247)

### Features:
- implements
[draft 2020-12](https://json-schema.org/specification-links.html#2020-12),
[draft 2019-09](https://json-schema.org/specification-links.html#draft-2019-09-formerly-known-as-draft-8),
[draft-7](https://json-schema.org/specification-links.html#draft-7),
[draft-6](https://json-schema.org/specification-links.html#draft-6),
[draft-4](https://json-schema.org/specification-links.html#draft-4)
- fully compliant with [JSON-Schema-Test-Suite](https://github.com/json-schema-org/JSON-Schema-Test-Suite), (excluding some optional)
- list of optional tests that are excluded can be found in schema_test.go(variable [skipTests](https://github.com/santhosh-tekuri/jsonschema/blob/master/schema_test.go#L24))
- validates schemas against meta-schema
- full support of remote references
- support of recursive references between schemas
- detects infinite loop in schemas
- thread safe validation
- rich, intuitive hierarchial error messages with json-pointers to exact location
- supports output formats flag, basic and detailed
- supports enabling format and content Assertions in draft2019-09 or above
- change `Compiler.AssertFormat`, `Compiler.AssertContent` to `true`
- compiled schema can be introspected. easier to develop tools like generating go structs given schema
- supports user-defined keywords via [extensions](https://pkg.go.dev/github.com/santhosh-tekuri/jsonschema/v5/#example-package-Extension)
- implements following formats (supports [user-defined](https://pkg.go.dev/github.com/santhosh-tekuri/jsonschema/v5/#example-package-UserDefinedFormat))
- date-time, date, time, duration, period (supports leap-second)
- uuid, hostname, email
- ip-address, ipv4, ipv6
- uri, uriref, uri-template(limited validation)
- json-pointer, relative-json-pointer
- regex, format
- implements following contentEncoding (supports [user-defined](https://pkg.go.dev/github.com/santhosh-tekuri/jsonschema/v5/#example-package-UserDefinedContent))
- base64
- implements following contentMediaType (supports [user-defined](https://pkg.go.dev/github.com/santhosh-tekuri/jsonschema/v5/#example-package-UserDefinedContent))
- application/json
- can load from files/http/https/[string](https://pkg.go.dev/github.com/santhosh-tekuri/jsonschema/v5/#example-package-FromString)/[]byte/io.Reader (supports [user-defined](https://pkg.go.dev/github.com/santhosh-tekuri/jsonschema/v5/#example-package-UserDefinedLoader))


see examples in [godoc](https://pkg.go.dev/github.com/santhosh-tekuri/jsonschema/v5)

The schema is compiled against the version specified in `$schema` property.
If "$schema" property is missing, it uses latest draft which currently implemented
by this library.

You can force to use specific version, when `$schema` is missing, as follows:

```go
compiler := jsonschema.NewCompiler()
compiler.Draft = jsonschema.Draft4
```

This package supports loading json-schema from filePath and fileURL.

To load json-schema from HTTPURL, add following import:

```go
import _ "github.com/santhosh-tekuri/jsonschema/v5/httploader"
```

## Rich Errors

The ValidationError returned by Validate method contains detailed context to understand why and where the error is.

schema.json:
```json
{
"$ref": "t.json#/definitions/employee"
}
```

t.json:
```json
{
"definitions": {
"employee": {
"type": "string"
}
}
}
```

doc.json:
```json
1
```

assuming `err` is the ValidationError returned when `doc.json` validated with `schema.json`,
```go
fmt.Printf("%#v\n", err) // using %#v prints errors hierarchy
```
Prints:
```
[I#] [S#] doesn't validate with file:///Users/santhosh/jsonschema/schema.json#
[I#] [S#/$ref] doesn't validate with 'file:///Users/santhosh/jsonschema/t.json#/definitions/employee'
[I#] [S#/definitions/employee/type] expected string, but got number
```

Here `I` stands for instance document and `S` stands for schema document.
The json-fragments that caused error in instance and schema documents are represented using json-pointer notation.
Nested causes are printed with indent.

To output `err` in `flag` output format:
```go
b, _ := json.MarshalIndent(err.FlagOutput(), "", " ")
fmt.Println(string(b))
```
Prints:
```json
{
"valid": false
}
```
To output `err` in `basic` output format:
```go
b, _ := json.MarshalIndent(err.BasicOutput(), "", " ")
fmt.Println(string(b))
```
Prints:
```json
{
"valid": false,
"errors": [
{
"keywordLocation": "",
"absoluteKeywordLocation": "file:///Users/santhosh/jsonschema/schema.json#",
"instanceLocation": "",
"error": "doesn't validate with file:///Users/santhosh/jsonschema/schema.json#"
},
{
"keywordLocation": "/$ref",
"absoluteKeywordLocation": "file:///Users/santhosh/jsonschema/schema.json#/$ref",
"instanceLocation": "",
"error": "doesn't validate with 'file:///Users/santhosh/jsonschema/t.json#/definitions/employee'"
},
{
"keywordLocation": "/$ref/type",
"absoluteKeywordLocation": "file:///Users/santhosh/jsonschema/t.json#/definitions/employee/type",
"instanceLocation": "",
"error": "expected string, but got number"
}
]
}
```
To output `err` in `detailed` output format:
```go
b, _ := json.MarshalIndent(err.DetailedOutput(), "", " ")
fmt.Println(string(b))
```
Prints:
```json
{
"valid": false,
"keywordLocation": "",
"absoluteKeywordLocation": "file:///Users/santhosh/jsonschema/schema.json#",
"instanceLocation": "",
"errors": [
{
"valid": false,
"keywordLocation": "/$ref",
"absoluteKeywordLocation": "file:///Users/santhosh/jsonschema/schema.json#/$ref",
"instanceLocation": "",
"errors": [
{
"valid": false,
"keywordLocation": "/$ref/type",
"absoluteKeywordLocation": "file:///Users/santhosh/jsonschema/t.json#/definitions/employee/type",
"instanceLocation": "",
"error": "expected string, but got number"
}
]
}
]
}
```

## CLI

to install `go install github.com/santhosh-tekuri/jsonschema/cmd/jv@latest`

```bash
jv [-draft INT] [-output FORMAT] [-assertformat] [-assertcontent] <json-schema> [<json-or-yaml-doc>]...
-assertcontent
enable content assertions with draft >= 2019
-assertformat
enable format assertions with draft >= 2019
-draft int
draft used when '$schema' attribute is missing. valid values 4, 5, 7, 2019, 2020 (default 2020)
-output string
output format. valid values flag, basic, detailed
```

if no `<json-or-yaml-doc>` arguments are passed, it simply validates the `<json-schema>`.
if `$schema` attribute is missing in schema, it uses latest version. this can be overridden by passing `-draft` flag

exit-code is 1, if there are any validation errors

`jv` can also validate yaml files. It also accepts schema from yaml files.

## Validating YAML Documents

since yaml supports non-string keys, such yaml documents are rendered as invalid json documents.

most yaml parser use `map[interface{}]interface{}` for object,
whereas json parser uses `map[string]interface{}`.

so we need to manually convert them to `map[string]interface{}`.
below code shows such conversion by `toStringKeys` function.

https://play.golang.org/p/Hhax3MrtD8r

NOTE: if you are using `gopkg.in/yaml.v3`, then you do not need such conversion. since this library
returns `map[string]interface{}` if all keys are strings.
Forked from https://github.com/santhosh-tekuri/jsonschema
4 changes: 2 additions & 2 deletions cmd/jv/go.mod
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
module github.com/santhosh-tekuri/jsonschema/cmd/jv
module github.com/zillow/go-jsonschema/cmd/jv

go 1.15

require (
github.com/santhosh-tekuri/jsonschema/v5 v5.3.1
github.com/zillow/go-jsonschema/v5 v5.3.1
gopkg.in/yaml.v3 v3.0.1
)
8 changes: 4 additions & 4 deletions cmd/jv/go.sum
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
github.com/santhosh-tekuri/jsonschema/v5 v5.3.0 h1:uIkTLo0AGRc8l7h5l9r+GcYi9qfVPt6lD4/bhmzfiKo=
github.com/santhosh-tekuri/jsonschema/v5 v5.3.0/go.mod h1:FKdcjfQW6rpZSnxxUvEA5H/cDPdvJ/SZJQLWWXWGrZ0=
github.com/santhosh-tekuri/jsonschema/v5 v5.3.1 h1:lZUw3E0/J3roVtGQ+SCrUrg3ON6NgVqpn3+iol9aGu4=
github.com/santhosh-tekuri/jsonschema/v5 v5.3.1/go.mod h1:uToXkOrWAZ6/Oc07xWQrPOhJotwFIyu2bBVN41fcDUY=
github.com/zillow/go-jsonschema/v5 v5.3.0 h1:uIkTLo0AGRc8l7h5l9r+GcYi9qfVPt6lD4/bhmzfiKo=
github.com/zillow/go-jsonschema/v5 v5.3.0/go.mod h1:FKdcjfQW6rpZSnxxUvEA5H/cDPdvJ/SZJQLWWXWGrZ0=
github.com/zillow/go-jsonschema/v5 v5.3.1 h1:lZUw3E0/J3roVtGQ+SCrUrg3ON6NgVqpn3+iol9aGu4=
github.com/zillow/go-jsonschema/v5 v5.3.1/go.mod h1:uToXkOrWAZ6/Oc07xWQrPOhJotwFIyu2bBVN41fcDUY=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
Expand Down
4 changes: 2 additions & 2 deletions cmd/jv/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ import (
"path/filepath"
"strings"

"github.com/santhosh-tekuri/jsonschema/v5"
_ "github.com/santhosh-tekuri/jsonschema/v5/httploader"
"github.com/zillow/go-jsonschema/v5"
_ "github.com/zillow/go-jsonschema/v5/httploader"
"gopkg.in/yaml.v3"
)

Expand Down
62 changes: 61 additions & 1 deletion compiler.go
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,50 @@ func NewCompiler() *Compiler {
}
}

// Clone returns a new compiler with a copy of the current compiler's
// internal state.
func (c *Compiler) Clone() *Compiler {
if c == nil {
return nil
}

clone := &Compiler{
Draft: c.Draft,
ExtractAnnotations: c.ExtractAnnotations,
LoadURL: c.LoadURL,
CompileRegex: c.CompileRegex,
AssertFormat: c.AssertFormat,
AssertContent: c.AssertContent,
resources: make(map[string]*resource),
Formats: make(map[string]func(interface{}) bool),
Decoders: make(map[string]func(string) ([]byte, error)),
MediaTypes: make(map[string]func([]byte) error),
extensions: make(map[string]extension),
}

for k, v := range c.resources {
clone.resources[k] = v
}

for k, v := range c.Formats {
clone.Formats[k] = v
}

for k, v := range c.Decoders {
clone.Decoders[k] = v
}

for k, v := range c.MediaTypes {
clone.MediaTypes[k] = v
}

for k, v := range c.extensions {
clone.extensions[k] = v
}

return clone
}

// AddResource adds in-memory resource to the compiler.
//
// Note that url must not have fragment
Expand Down Expand Up @@ -150,6 +194,22 @@ func (c *Compiler) Compile(url string) (*Schema, error) {
return sch, err
}

// GetCompiledSchemas returns a map of all the schemas that have been compiled by the compiler so far.
// This includes all the schemas which have been compiled using method Compile, as well as all the
// schemas those compiled schemas had references to.
// The map keys are the schema urls.
func (c *Compiler) GetCompiledSchemas() map[string]*Schema {
compiledSchemas := map[string]*Schema{}

for _, resource := range c.resources {
if resource.schema != nil {
compiledSchemas[resource.url] = resource.schema
}
}

return compiledSchemas
}

func (c *Compiler) loadURL(url string) (io.ReadCloser, error) {
// check in metaschemas
u, meta := strings.CutPrefix(url, "http://json-schema.org/")
Expand Down Expand Up @@ -792,7 +852,7 @@ func (c *Compiler) validateSchema(r *resource, v interface{}, vloc string) error
}

validate := func(meta *Schema) error {
return meta.validateValue(v, vloc)
return meta.validateValue(v, v, vloc)
}

meta := r.draft.meta
Expand Down
Loading

0 comments on commit 1a2ef10

Please sign in to comment.