Pipeline parameters

[rmb938]

When triggering a pipeline it would be nice to be able to specify input parameters. This would allow users to have dynamic configuration for their pipelines.

An example would be a pipeline that runs database migrations and it needs an input of the database name.

[Skarlso]

Hi.

So, a pipeline is just Go code. What prevents you from adding in parameters to your pipeline in whatever means you wish, like a config file or environment property?

Or are you looking for a different solution?

[rmb938]

You are correct however there is no way from what I can tell for gaia to pass configuration information when triggering a pipeline.

The configuration I am looking for would only be useful while a single pipeline is ran. So each run of the pipeline can have slightly different inputs.

[fubarhouse]

This is really high on my want/need list.

If I'm ever to adopt this as more than an exploration task, it's right up there.

For example in my case, I manage a lot of websites which happen to share codebases, being able to build and perform database/file sync tasks are dependent on a few input parameters. I could have a pipeline for each of these as individual tasks, which could be at least 60 or more individually maintained pipelines without effort - this would bring it down to single digits with capacity for other pipelines.

[Skarlso]

Oh I didn't know the front-end can't pass in stuff, like environment property values. :)

So, you could, in theory, create a single pipeline and iterate over a config file which contains all the data you need which could fire off a multitude of routines that would show up in the pipeline as things that diverge then come together and create a single view. Fan-out fan-in.

I realize that's probably not what you are looking for but could very well be a plausible solution for now. Or not? :)

[fubarhouse]

@Skarlso Yeah, that is my intention for the time being.

I understand it's no small ask, but it's one of two really important requirements I would need if I'm to ever adopt Gaia as more than a shiny thing in my own time.

The other is writing tasks which can write to the local file system. Unfortunately the only other product which still allows that is Jenkins - so I'm tied down for now.

[rmb938]

Yes that could be a temporary solution however having the ability to have input parameters would still be nice.

@Skarlso I am more than willing to contribute and add this feature if that is a concern.

[Skarlso]

Oh for sure, don't get me wrong, I'm not against it. :) Was just trying to come up with a temporary solution so you guys aren't blocked. There is no concern here.

@rmb938 By all means. I'm sure @michelvocks would be delighted. :) In fact, he probably has it on his todo list. :)

[michelvocks]

Exactly, this is on my todo list. 😄
I actually already planned it in the gRPC protocol: https://github.com/gaia-pipeline/protobuf/blob/master/plugin.proto#L15

The idea is to define arguments in the pipelines and Gaia should dynamically render a view for those arguments. I think this is really close to jenkins job arguments which is nice and needed for sure.

I also thought about questions (questions sounds somehow weird, we have to find a better name I guess). https://github.com/gaia-pipeline/protobuf/blob/master/plugin.proto#L20
Imagine this scenario: You have a huge pipeline with several jobs included. At some point, you want to stop the pipeline and maybe wait for manual input from a real human. Or maybe you want to ask for an argument from a real human.
This is identical to jenkins input functionality.

Anyway, this is a huge task and includes frontend and backend changes. If you are ready for a challenge, feel free to start on this task! I would be more than happy about this! 🤗

[rmb938]

Maybe for the questions follow what spinnaker is doing and rename them to manual judgements? https://www.spinnaker.io/guides/tutorials/codelabs/safe-deployments/#adding-a-manual-judgment-to-deployment-pipelines