-
-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Introduce PgRegistry; codecs own relations; type overhaul #260
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…Codec` to `PgCodec`, `PgEnumTypeCodec` to `PgEnumCodec`, `PgTypeColumn` to `PgCodecAttribute` (and similar for related types/interfaces). `source` has been replaced by `resource` in various of the APIs where it relates to a `PgResource`. `PgSourceBuilder` is no more, instead being replaced with `PgResourceOptions` and being built into the final `PgResource` via the new `makeRegistryBuilder`/`makeRegistry` functions. `build.input` no longer contains the `pgSources` directly, instead `build.input.pgRegistry.pgResources` should be used. The new registry system also means that various of the hooks in the gather phase have been renamed/replaced, there's a new `PgRegistryPlugin` plugin in the default preset. The only plugin that uses the `main` method in the `gather` phase is now `PgRegistryPlugin` - if you are using the `main` function for Postgres-related behaviors you should consider moving your logic to hooks instead. Plugin ordering has changed and thus the shape of the final schema is likely to change (please use `lexicographicSortSchema` on your before/after schemas when comparing). Relationships are now from a codec to a resource, rather than from resource to resource, so all the relationship inflectors (`singleRelation`, `singleRelationBackwards`, `_manyRelation`, `manyRelationConnection`, `manyRelationList`) now accept different parameters (`{registry, codec, relationName}` instead of `{source, relationaName}`). Significant type overhaul, most generic types no longer require generics to be explicitly passed in many circumstances. `PgSelectStep`, `PgSelectSingleStep`, `PgInsertStep`, `PgUpdateStep` and `PgDeleteStep` now all accept the resource as their single type parameter rather than accepting the 4 generics they did previously. `PgClassExpressionStep` now accepts just a codec and a resource as generics. `PgResource` and `PgCodec` have gained a new `TName extends string` generic at the very front that is used by the registry system to massively improve continuity of the types through all the various APIs. Fixed various issues in schema exporting, and detect more potential issues/oversights automatically. Fixes an RBAC bug when using superuser role for introspection.
benjie
changed the title
Introduce PgRegistry - PgSourceBuilder is no more
Introduce PgRegistry; codecs own relations; type overhaul
Apr 4, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
First: the term "source" is overused, so we've renamed
PgSource
toPgResource
. I'll try and refer to "resources" rather than "sources" from here on out. We've also renamedPgTypeCodec
to simplyPgCodec
.We want the schema to be exportable, which means the underlying "resources" need to be exportable. The resources relate to other resources, and sometimes these relations form circular dependencies, e.g.
User.posts -> Post
andPost.author -> User
. When we try and export these we can't, because we can't defineconst User = ...
without first definingPost
, and we can't defineconst Post = ...
without first definingUser
. How to solve?Initially I solved this with a
PgSourceBuilder
. This was a precursor to aPgSource
(nowPgResource
) - everything except the relationships - and then relationships could be to either aPgSource
directly or to aPgSourceBuilder
that would later "become" a PgSource (viaPgSourceBuilder.get()
). Code throughout the codebase had to say "if it's a PgSourceBuilder, .get() it, otherwise use it directly" and it was ugly. I didn't like it, and concluded that "relations" should be a separate thing (see #117).From discussion with @jemgillam we realised that we could significantly simplify things by building first the codecs, then the resources, then the relations... But that accessing the relations would be painful, and more importantly the relations might not export along side everything else, which might cause the exported schema to be incomplete. Clearly not viable. We needed something that was interlinked with circular dependencies at runtime, but that was produced without circular dependencies at build time... We needed a registry!
The registry
The idea with the registry is simple: we take a list of codecConfigs, resourceConfigs, and relationshipConfigs (all without circular dependencies - a directed acyclic graph) and then we build from that a structure (the "registry") that contains the built codecs, resources and relationships which can link back to the central registry. So the "resources" don't actually "know" their relationships - they just know the "registry" which stores all the relationship data. And the registry itself is built via a factory function which we can export with the raw configs as input and no circular dependency issues.
This also meant that we could maintain the types much better:
(Early screenshot, API has changed slightly.)
And also, the mixture of
PgSource
/PgSourceBuilder
has been cleared up via the following rules:PgResourceOptions
is generated during the "gather" phase, you don't deal withPgResource
directly during gather (until the very end when thePgRegistry
is built)PgResource
is consumed during the "schema" phase, you don't deal withPgResourceOptions
during the "schema" phase at all.So no more hacky
resolveSource()
/sourceBuilder.get()
nonsense!Lots of breaking changes
satisfies
andconst
-genericsPgSourceBuilder
is no more (but it has effectively been replaced withPgResourceOptions
)PgSource*
->PgResource*
PgTypeCodec*
->PgCodec*
PgTypeColumn*
->PgCodecAttribute*
listOfType
->listOfCodec
PgResource
andPgCodec
have changed significantly - the first generic is now a name (string) which allows us to build up the registry nicely{resource, relationName}
, instead accepting{registry, codec, relationName}
. (Note:registry = resource.registry
,codec = resource.codec
.)singleRelation
singleRelationBackwards
_manyRelation
manyRelationConnection
manyRelationList
Fixes #117