TestStep SkipFunc should not error but print to stdout

[drewmullen]

It seems that SkipFunc would be more helpful in test steps if it printed a message to stdout instead of erroring the test. that is how t.Skip() works

type TestStep struct {
    ...

	// SkipFunc is called before applying config, but after PreConfig
	// This is useful for defining test steps with platform-dependent checks
	SkipFunc func() (bool, error)

work arounds are easy enough for now:

SkipFunc: func() (bool, error) {
	if myVar != "" {
		return false, nil
	}
	fmt.Println("Environment variable MYVAR must be set.")
	return true, nil
},

[bflad]

Hi @drewmullen! 👋 The SkipFunc handling has been around for quite some time (origination back in combined Terraform repository). The error return is explicitly for returning a logical error in the skip function itself, not for denoting whether the test step should be skipped. I believe that behavior is correct.

The testing package does however provide the T.Logf() functionality, which could be used in the acceptance testing framework's logic in addition to outputting a warning log (which itself requires enabling Terraform's logging):

terraform-plugin-sdk/helper/resource/testing_new.go

Lines 111 to 114 in 9cecd71

	if skip {
	log.Printf("[WARN] Skipping step %d/%d", i+1, len(c.Steps))
	continue
	}

So I think if you or anyone is interested in remediating this issue:

• Update the TestStep.SkipFunc Go documentation to explain the bool and error returns
• Add t.Logf() where the warning log is also emitted

Please reach out with any questions. 👍

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.