Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

specgen: re-enable acceptance tests, fix validation issues, add more tests #212

Merged
merged 17 commits into from
Nov 27, 2024

Conversation

hariso
Copy link
Contributor

@hariso hariso commented Nov 19, 2024

Description

Closes #210.

This PR:

  • Re-organizes the tests into
    • specgen/specgen/tests/parse_specs/ (tests related to parsing the specs)
    • (new tests) specgen/specgen/tests/write_and_combine/ (tests related to combining an existing connector.yaml file and the generated specs)
  • Fixes the acceptance tests (by fixing the configuration)
  • Fixes an issue with validation in DefaultSourceMiddleware and DefaultDestinationMiddleware.
  • Fixes an issue with parsing the schema type in SourceWithSchemaExtraction.

Quick checks:

  • I have followed the Code Guidelines.
  • There is no other pull request for the same update/change.
  • I have written unit tests.
  • I have made sure that the PR is of reasonable size and can be easily reviewed.

@hariso hariso changed the base branch from main to specgen November 19, 2024 12:17
@hariso hariso changed the title specgen: re-enable acceptance tests, benchmark tests, etc. specgen: re-enable acceptance tests, add more tests Nov 20, 2024
@hariso hariso changed the title specgen: re-enable acceptance tests, add more tests specgen: re-enable acceptance tests, fix validation issues, add more tests Nov 20, 2024
@hariso hariso marked this pull request as ready for review November 20, 2024 12:02
@hariso hariso requested a review from a team as a code owner November 20, 2024 12:02
Copy link
Member

@raulb raulb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if the second specgen directory is really necessary. Left other comments. Let me know what you think @hariso

acceptance_testing.go Show resolved Hide resolved
acceptance_testing.go Show resolved Hide resolved
@@ -309,12 +326,12 @@ func (s *sourceWithSchemaExtraction) extractPayloadSchema(ctx context.Context, r
}

func (s *sourceWithSchemaExtraction) schemaForType(ctx context.Context, data any, subject string) (schema.Schema, error) {
srd, err := schema.KnownSerdeFactories[s.config.SchemaType].SerdeForType(data)
srd, err := schema.KnownSerdeFactories[s.config.SchemaType()].SerdeForType(data)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should check whether s.config.SchemaType() returns a valid known type.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We rely on that s.config.SchemaType() converts s.config.SchemaTypeStr to a schema.Type, and SchemaTypeStr has already been validated. But I think you have a point, I can add a test or two to make sure s.config.SchemaType() is doing the conversion successfully.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hariso Just checking in. You haven't added a test for this, did you? Just in case I'm not seeing it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, good catch, I forgot about this comment.:) I'll add a test or two now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added to another branch, that's specifically for unit tests: 6cf5795.

Copy link
Member

@raulb raulb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving subject to a comment I left regarding a test for SchemaType()

@hariso hariso merged commit b4bd386 into specgen Nov 27, 2024
3 checks passed
@hariso hariso deleted the haris/acceptance-tests-etc branch November 27, 2024 14:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

specgen: re-enable acceptance tests
2 participants