Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automation/script to smoke test existing and new validation logic #1478

Open
5 tasks
radeksimko opened this issue Oct 24, 2023 · 0 comments
Open
5 tasks

Automation/script to smoke test existing and new validation logic #1478

radeksimko opened this issue Oct 24, 2023 · 0 comments
Labels
diagnostics enhancement New feature or request

Comments

@radeksimko
Copy link
Member

Context

After introducing enhanced validation in #1368 we have received a number of bug reports about false diagnostics:

and some more continue to trickle in:

Proposal

Instead of letting our users discover and report such cases of false diagnostics we can be more proactive and run our validation logic against publicly available repositories of Terraform code on GitHub, to perform a form of smoke testing of the validation logic.

This can in turn provide us with higher confidence in the validation logic and the schema, and detect at least some, if not most, of the bugs before they even reach users.

  • Identify some popular repositories with Terraform code on GitHub and the Terraform Registry
  • Create a script (Go code) which can run all existing validation logic
    • Context-less validation (SchemaModuleValidation, SchemaVariablesValidation)
    • Context-dependent validation (ReferenceValidation)
  • Run script against those repositories
@radeksimko radeksimko added enhancement New feature or request diagnostics labels Oct 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
diagnostics enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant