Add create-pipeline and job-regexp options

[Jellby]

This addresses #107, and replaces #109 (changing source branch)

create-pipeline: If for some reason no valid pipeline is found for the MR (none exists, outdated, etc.), Marge will create a new one using the api. She will also leave a message, so she can know that she already created a pipeline and will not do it again and again. Default is false.

job-regexp: The CI may be configured to run different jobs depending on the branch name, or pipeline source (push, trigger, api...). Marge will check the jobs in the current pipeline against this regexp. If any job matches the regexp, it's considered good and the status is checked as usual. If none matches, it's not a valid pipeline and either the merge will fail or a new pipeline will be created (see create-pipeline).

If using both create-pipeline and job-regexp, make sure the important jobs are configured to run on api sources (see the only and except options of GitLab CI), so that the pipeline created by Marge will be valid.

[aschmolck]

I'm a bit confused by this. If you have set up your pipelines as intended in Github, why would something like this be needed on the marge side? Can you explain a bit more?

[Jellby]

What is "as intended in Github" (I guess you meant GitLab)? If "as intended" means that every push should create a pipeline that runs all jobs needed for verification, then right, this would not be needed. The point of this PR is to allow more flexible pipeline configuration. For example:

• Not every push creates a pipeline, only pushes to some protected branches. Not all developers have permission to push to the protected branches.

• Not every pipeline has all necessary jobs. Most pipelines could have "compile only" jobs, while some action (or special branches) are needed to create "test everything" jobs, which can be very expensive in runner time.

My use case is the second. I don't want to run the expensive tests for every push, and I don't want to require special names for MR branches. With these options, one can push to any branch, have basic sanity tests run, create a MR and, once it's assigned to marge, she will make sure the expensive tests are run before accepting the MR. The developer can also run the expensive tests locally before pushing.

[aschmolck]

[Yeah, meant Gitlab, sorry]

OK, I think I understand now: you want branches that are PRs to run special pipelines (too expensive to run on all branches), correct?

I agree that's highly desirable functionality Gilab appears to agree in principle as well, but it doesn't look like they're gonna get around implementing it soon. For self-hosted gitlab instances there's a plugin snippet you could use in the issue linked above, but that won't help you if you're running on gitlab.com.

So I'm sympathetic to adding some kludge to marge-bot in the interim. One thing that I'm not super-happy about is that as far as I can tell, there is no good way to deal with cheap pipeline for normal branches, expensive ones for (final) MR check before merging. If you're job regex fails to find the required job in the existing pipeline (if any) you trigger a new pipeline run. But in what cases would this actually help if there already has been a pipeline running for the sha? If it didn't have the required jobs, presumably the new one wouldn't either?

Maybe I'm overlooking something that selects the desired jobs, but it seems the best you could do with this create pipeline functionality is have only one expensive pipeline which you trigger with marge, but only for MRs and never run any pipeline on normal branches. Is that right?

[aschmolck]

Thinking about it, the way forward with this would probably to do something similar like we're doing for --batch and generate a "synthetic" MR which we then merge manually. This approach can also solve your problems in #114, you just need to make it so that that "synthetic" MR branch (name) triggers all jobs you want triggered in a merge request. What do you think @JaimeLennox ?

(In particular this also will still still work if you have flags active which create trailers, but you can't push to the MR repo)

[Jellby]

The way to ensure pipelines created by marge contain the required jobs is with proper CI configuration, with the only/except job options. For instance, setting only: api for the expensive job. Since marge creates the pipelines via api, these will contain the jobs, but pipelines created with pushes won't. Of course, other pipelines created via api will also contain the expensive jobs, but that's fine with me, that allows manually running full pipelines if desired (for instance, prior to assigning the MR to marge, in which case she will just use the result of it).

If the configuration is broken and the new pipeline does not contain the required jobs, marge will detect she tried to run a pipeline already, and won't try again.

[Jellby]

As for "synthetic" MRs (I guess you meant to refer to #113), I guess it could be possible to use temporary branches as with the batch job, and configure the expensive jobs with them. There wouldn't be much difference, except for the creation of additional branches. I worked on this before the batch option was available, so I didn't really look at it... and I actually think simply creating a pipeline is cleaner.

[aschmolck]

Cool, I didn't know that you could configure api-created pipelines differently! Yup, that makes it much more attractive. I still suspect the way to go is the generalize our use of temporary branches because it solves several problems in one fell swoop:

• CI setup problems on forks
• push permission problems on forks
• (the fact that we rewrite the source-branch before final CI passes; not a big problem in practice, but a bit unclean)
• merge-request-final-check-only pipelines

In particular the last one would be completely free and this PR and the associated extra flags and code would become redundant.

Anyway, I think we'll need to think a bit how to best go about it, but this is something I'd like to support.

[Jellby]

Regardless of how it is implemented, I think something similar to the job-regexp option I propose is needed. Marge should not blindly accept any pipeline result she gets without checking that it has tested what it was supposed to test. While that is probably the responsibility of the repository owner/manager, it is too easy to miss something sooner or later.