Retrieving external reference in a deeply nested subschema

[adamtrs]

Hi!

First of all, thank you for creating and maintaning this package. We are using it through Python bindings, and it's blazing fast even when validating very large files using complex schemas.

We use schemas with lots of external references, and we provide them in-memory (via custom retriever). But we have encountered a problem with some schemas.

In this example schema:

{
    "$id": "foo://schema_1.json",
    "$ref": "#/$defs/a/b",
    "$defs": {
        "a": {
            "b": {
                "description": "in reality some complex schema with an external ref inside",
                "$ref": "foo://schema_2.json"
            }
        }
    }
}

The referenced foo://schema_2.json will never be requested from retriever. Instead, during validation the following error crops up:
jsonschema_rs.ValidationError: Resource 'foo://schema_2.json' is not present in a registry and retrieving it failed: Retrieving external resources is not supported once the registry is populated.

I've looked a bit through the code, and my understanding is that $refs in a document during retrieval phase are only looked up if they are part of a subresource, and subschema $defs/a/b does not get recognized as one.

I understand that this is a bit of an edge case, but we use a lot of referenced subschemas and this deep nesting of definitions makes the $defs better organized and fragments/JSON pointers to referenced subschemas neater.

What do you think about this problem, is this something that could be solved?

Alternatively, something akin to with_resources() validator option exposed through Python interface (maybe a resources parameter?) would also help us. I can try to contribute a pull request for that feature if you think this is a good idea to add.

Cheers,
Adam

[Stranger6667]

Hello Adam,

I am happy to hear that the library is useful for you :) And thank you very much for putting so many details into the issue description, it helps a lot.

The idea behind the registry is that it should resolve all reachable schemas during initialization. Therefore, if foo://schema_2.json is not resolved, then it sounds like a bug to me. I.e. all $ref should be traversed during initialization recursively, but apparently, it does not happen.

Additionally, exposing with_resources is something on my todo list, but I didn't get to work on it yet.

In any event, I'll be happy to review a PR and provide any guidance on making it merged :) I am not 100% about my bandwidth right now, but I'll take a look at the resolving part right now.

[Stranger6667]

@adamtrs

The issue should be resolved in 0.28.2! Please, let me know if everything works fine on your side :) I am going to check with_resources later on, or feel free to take a look if you'd like :)