Base Schema using $merge and the loadSchema option

[davidjamesstone]

What version of Ajv are you using? Does the issue happen if you use the latest version?
4.9.0
Ajv options object (see https://github.com/epoberezkin/ajv#options):

{ allErrors: true, v5: true }

Hi there.

I have been spending the last week or so getting into json schema. I'm really impressed with it so far although the lack of examples is a little frustrating.

I find this lib the best I have evaluated, you also seem active in this community so I thought I'd post the question here, even though it's not entirely related to AJV ;)

My issue is in trying to find the best way separate and reuse schemas.

For example, I have a schema that represents a database entity and I am exposing that resource with a HTTP API defined with hyper-media. I want to reuse bits of schema for both the database validation and the API payload validation. I can see there are various ways of achieving this. I tried allOf, $patch and $merge

I have settled on using $merge. I use a base schema to represent the POST payload and the my db schema builds on this and adds an id property. This code works fine: This code doesn't work as I thought. The required arrays are not merged - the base schema gets overwritten.

const Ajv = require('ajv')
const validator = new Ajv({ allErrors: true, v5: true })
require('ajv-merge-patch')(validator)

var baseSchema = {
  type: 'object',
  properties: {
    name: { type: 'string' },
    sku: { type: 'string' },
    price: { type: 'number', minimum: 0 }
  },
  additionalProperties: false,
  required: ['name', 'sku', 'price']
}

var schema = {
  $merge: {
    source: { $ref: 'product' },
    with: {
      properties: {
        id: { type: 'string', format: 'uuid' }
      },
      required: ['id'],
      additionalProperties: false
    }
  }
}

validator.addSchema(baseSchema, 'product')
const validate = validator.compile(schema)

const result = validate({
  id: '6438850b-b33d-475f-95d7-8997606431a5',
  name: 'Apple iPhone',
  sku: 'ABC001',
  price: 1
})

console.log(result)
//=> true
console.log(validate.errors)
//=> null

My first question is do you agree with this approach?

Secondly, I'd also like these schema to live in separate files and, rather than using addSchema, I'd like to use the loadSchema option to resolve the dependant schemas. I can't seem to get this to work though.

base-product.json

{
  "id": "base-product.json",
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "sku": { "type": "string" },
    "price": { "type": "number", "minimum": 0 }
  },
  "additionalProperties": false,
  "required": [ "name", "sku", "price" ]
}

product.json

{
  "id": "product.json",
  "$merge": {
    "source": { "$ref": "base-product.json" },
    "with": {
      "properties": {
        "id": { "type": "string", "format": "uuid" }
      },
      "required": [ "id" ],
      "additionalProperties": false
    }
  }
}

const fs = require('fs')
const Ajv = require('ajv')

function loadSchema (uri, callback) {
  console.log('loadSchema', uri)
  var filePath = __dirname + '/' + uri
  fs.readFile(filePath, (err, data) => {
    if (err) callback(err)
    callback(null, data)
  })
}

const ajv = new Ajv({ loadSchema: loadSchema })
const schema = require('./product.json')

ajv.compileAsync(schema, function (err, validate) {
  if (err) return

  const result = validate({
    id: '6438850b-b33d-475f-95d7-8997606431a5',
    name: 'Apple iPhone',
    sku: 'ABC001',
    price: 1
  })

  console.log(result)
  console.log(validate.errors)
})

This doesn't seem to work though. loadSchema never gets called and, although no errors are present in the callback for compileAsync, the validate function is not correct and seems to allow anything.

Apologies for the lengthy issue - I hope it all makes sense.

[epoberezkin]

@davidjamesstone thank you

1. $ref is more preferable than $merge as it is standard, so if you can get away with it I'd recommend using it. But if you want to extend the list of allowed properties without repeating them all (even with empty schemas) $merge/$patch is the only approach (even though it's not standard and it will take some time before it makes it into JSON schema or into a separate spec, so don't expect compatibility with any other platform soon :).
   Also, if you expect $merge to concatenate arrays in required keyword, that's not how it works. Underlying JSON-merge-patch spec simply overwrites arrays, rather than concatenating them, so you either need to list all properties in with or to use $patch.

2. $merge/$patch do not support asynchronous loading yet it seems, it is quite easy to improve - I will do it.

[davidjamesstone]

Hey thanks for the quick reply.

I was trying to achieve it by extending the list of properties rather than with $ref based composition. I had originally thought allOf was the right approach but additionalProperties: false doesn't work as I initially expected.

In regards to require overwriting, I swear this works in my example above. I'll try later when I'm back home but I thought I ended up with a union of 4 required fields. It's a shame it's not supposed to work like that.

The approach I had with $patch was to start with all 4 fields that represented the db resource, then using 2 remove 'op's, removed the 'id' property and the first item in the required array.

{
  "type": "object",
  "title": "Product schema",
  "properties": {
    "id": {
      "type": "string"
    },
    "name": {
      "type": "string"
    },
    "sku": {
      "type": "string"
    },
    "price": {
      "type": "number",
      "minimum": 0
    }
  },
  "additionalProperties": false,
  "required": ["id", "name", "sku", "price"]
}

Then

"$patch": {
  "source": {
    "$ref": "../db/product.json"
  },
  "with": [
    { "op": "remove", "path": "/properties/id" },
    { "op": "remove", "path": "/required/0" }
  ]
}

Is this what you mean by using $patch?

Thanks for looking into async loading with patch/merge too.

[epoberezkin]

In regards to require overwriting, I swear this works in my example above.

It works in your case in a sense that it passes validation. It doesn't mean it would fail in the case when some required fields are missing - you need test(s) where validation fails.

Is this what you mean by using $patch?

Yes. You can either start from a bigger schema and remove as you did, or you can start from a smaller one and add, whatever makes more sense for you. I would probably add, but it's just my preference, no difference really.

Thanks for looking into async loading with patch/merge too.

You're welcome. Just published ajv-merge-patch that supports async loading, if you npm update ajv-merge-patch, it should work.

[davidjamesstone]

Thanks - you've been a great help.