FromSchema type instantiation limit

[xblurx]

Hi and thank you for this amazing library.
I have a question regarding the current limit of FromSchema where T is very wide. This is an example of schema for T that errors with Type instantiation is excessively deep and possibly infinite.ts(2589):

Example JsonSchema

{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "",
"description": "",
"definitions": {
  // in the definitions, simple schemas like code0 and complex schemas like code20 with refs are declared
  "code0": {
    "type": "object",
    "additionalProperties": false,
    "properties": {
      "code": { "const": 0 },
      "meta": {
        "type": "object",
        "additionalProperties": false,
        "properties": {
          "component": {},
          "error": { "type": "string" },
          "type": { "const": "ThirdPartyComponent" }
        },
        "required": ["component", "error", "type"]
      }
    },
    "required": ["code", "meta"]
  },
  "code20": {
    "type": "object",
    "additionalProperties": false,
    "properties": {
      "code": { "const": 20 },
      "meta": {
        "type": "object",
        "additionalProperties": false,
        "properties": {
          "entity": { "$ref": "#/definitions/Entity" }, // this references Entity which could be oneOf that are declared there
          "condition": { "type": "string" },
          "type": { "const": "UnreachableCondition" }
        },
        "required": ["entity", "condition", "type"]
      }
    },
    "required": ["code", "meta"]
  },
  "Entity": {
    "oneOf": [
      { "$ref": "#/definitions/Entity_AgPrefix" },
       // here 20 more $ref-s are declared
    ]
  },
  // here are about 150 codes declarations similar to ones above in size and structure
},
"oneOf": [
  { "$ref": "#/definitions/code0" }
  // here 50 additional refs for codes from definitions are declared
]
}
// total size of this schema is 2420 loc

i read the FAQ regarding this issue and I can say that there are none allOff, not, else, ifThenElse keywords used.
maybe you could suggest a temporary workaround, or is this simply a typescript + json-schema-to-ts limitation and it was not designed for dealing with huge schemas such as this one?

Appreciate it!

[xblurx]

i tried to separate the large one into smaller ones, specifically dividing the codes from its references, and hydrating them with {references: [typeof A]} and after some number of references FromSchema "crashed" again. @ThomasAribart I did not find it in the docs or faq, whats easier for FromSchema to eat - $refs or hardcoded object instead?

[xblurx]

after a bit of experiments I came to a kind of a solution for a really large schemas with lots of references like this one:


JSON

{
"$id": "http://example.com/schemas/Entity.json",
"type": "object",
"oneOf": [
  {
    "type": "object",
    "additionalProperties": false,
    "properties": {
      "type": { "const": "Zone" },
      "data": {
        "type": "object",
        "additionalProperties": false,
        "properties": {
          "id": { "type": "number" },
          "name": { "type": "string" },
          "typ": { "type": "string" },
          "direction": { "type": "string" },
          "default": { "type": "boolean" }
        },
        "required": ["id", "name", "typ", "direction", "default"]
      }
    },
    "required": ["type", "data"]
  }
]
}

{
"type": "object",
"title": "Error codes",
"description": "Error codes format",
"additionalProperties": false,
"properties": {
  "entries": {
    "oneOf": [ // essentially the array items are codes with different structure
      {
        "type": "object",
        "additionalProperties": false,
        "properties": {
          "code": { "const": 0 }, // this is the primitive code without references
          "meta": {
            "type": "object",
            "additionalProperties": false,
            "properties": {
              "component": {},
              "error": { "type": "string" },
              "type": { "const": "ThirdPartyComponent" }
            },
            "required": ["component", "error", "type"]
          }
        },
        "required": ["code", "meta"]
      },
      {
        "type": "object",
        "additionalProperties": false,
        "properties": {
          "code": { "const": 1204 }, // this is the code with references
          "meta": {
            "type": "object",
            "additionalProperties": false,
            "properties": {
              "entity": { "$ref": "http://example.com/schemas/Entity.json" }, // there are different types the codes could refer to
              "type": { "const": "InvalidEntity" }
            },
            "required": ["entity", "reason", "type"]
          }
        },
        "required": ["code", "meta"]
      }
    ]
  }
}
}


- split references to a different schemas, use FromSchema<typeof Schema, {references: [typeof entitySchema, …]}>

but for some reason, after N-th reference is passed to FromSchema, it cracks with Type instantiation is excessively deep and possibly infinite. In that case I separated Entity.json references from the schema and built it by hand with codes, that are using the Entity.json references:


type Entity = FromSchema<typeof Entity>
type EntityCodes = 1024 // some codes that reference Entity.json

// and construct helper that will augment FromSchema<Schema> with necessary types (entities in this case)
type Entities<T> = T extends { code: infer U extends EntityCodes } ? T & { code: U; meta: { entity: Entity } } : T

type Schema = FromSchema<typeof schema>
type EntitiesSchema = Entities<Schema>


and with that case it works. however this workaround is kind of ugly, and I'd like to say that FromSchema is having difficulties with large schemas (up to 950 LOC) that are in turn reference lots of other schemas (in my case its 11 schemas like entitySchema, each up to 250 LOC). If @ThomasAribart  could provide a more elegant solution to this, that would be awesome!

[rebelwolfson]

Good day!

@ThomasAribart thank you for your work, really pushes the limits of what could be done using TS types system and big thanks to @xblurx for research and sharing the knowledge.

In my project I also have schema of size about 3k lines and dozens of local definitions and references. And it seems like I also hit the limit of library capabilities or/and typescript limitations.

Schema is more or less the same as above, TS version is 5.5, tsconfig contains all the requirements from instructions, no recursive definitions. However, I do have oneof construct at the root of the schema and FromSchema does its work perfectly up to 13 entries in oneof list, and then it breaks with type being inferred incorrectly.

I was trying to expand memory limit for TS or find out some kind of other workarounds, but still there is a unstable (in terms of exact number) but frustrating limit for oneof construct.

@xblurx 's tweaks helped in some way, but not really...

So it seems, that we need some kind of systemic solution for inferring type from large schemas with local definitions...

Thank you again, this library is really helpful!