Record run when a bundle action generates outputs

[carolynvs]

Right now the logic for when porter should record a bundle run is:

• the bundle action is not stateless
• the bundle action modifies bundle resources

It is possible for a stateless action to create outputs though, and unless modifies is artificially set to true, the outputs would not be captured by Porter.

A couple things come to mind here:

1. modifies and stateless are CNAB concepts that Porter has exposed without any user experience improvements. I feel like these are concepts that maybe the runtime needs to know but perhaps Porter can help map that better to how a bundle author thinks about what the action does.
2. If we don't tweak how custom actions are defined, we should at least change the runtime to record when a bundle output is created.

Is there a better way to represent the concepts of stateless and modifies in a way that works for us?

• stateless - does this require the bundle to already be installed? Could be renamed to "allowBeforeInstall".
• modifies - does this change any resources managed by the bundle? We can use that eventually to prevent concurrent runs that modify resources. Users can use this to figure out if it's safe to run an action. Could be renamed to "modifiesBundleResources"

A couple examples of how these two can combine to make real-world actions:

• preflight (stateless: true, modifies: true) - Run a series of checks against an environment before installing. If there are configuration problems identified, correct them.
• dry-run (stateless: true, modifies: false) - Print the actions the bundle would take but don't change anything. Could be run either before or after the bundle is installed.
• status (stateless: false, modifies: false) - Print the status of the bundle resources.

[carolynvs]

After discussing further, we realized that the problem is in the lack of documentation around those two fields. The names work just fine.

So what we need to do is fix the bug: When a custom action generates an output, it should always persist an installation/run/result/outputs to the database.

Here is the function that needs to be updated:

porter/pkg/storage/run.go

Lines 88 to 99 in 730b039

	func (r Run) ShouldRecord() bool {
	// Assume all actions modify bundle resources, and should be recorded.
	stateful := true
	modifies := true
	if action, err := r.Bundle.GetAction(r.Action); err == nil {
	modifies = action.Modifies
	stateful = !action.Stateless
	}
	return modifies || stateful
	}

[github-actions]

Closed by #2291.