yaml triggers doc #1383

ericsciple · 2018-01-25T04:43:57Z

No description provided.

chrispat · 2018-01-25T15:30:39Z

docs/preview/yamlgettingstarted-triggers.md

+    exclude: []
+
+schedules:
+  whenUnchanged: bool


Schedules are an array of individual schedules.

schedules: - schedule whenUnchanged: bool days: time: # we could probably take the time and time zone info as a singe string hh:mmTZD timeZone: branches:

oops i thought i had it that way

chrispat · 2018-01-25T16:00:52Z

docs/preview/yamlgettingstarted-triggers.md

+
+  builds:
+  - name: otherDefinition
+    ci: true # Simple build trigger


lets use the "trigger" keyword instead of CI consistently.

mmitche · 2018-01-29T16:01:18Z

docs/preview/yamlgettingstarted-definition.md

-of your repository. A definition `<REPO_NAME>/<REPO_NAME> CI` will be created with a
-continuous integration trigger for the default branch.
+Push a YAML file `.vsts-ci.yml` to the root directory of your repository. A definition
+`<REPO_NAME>/<REPO_NAME> CI` will be created, and a CI build will be triggered.

 Note, the definition will only be created if whoever pushes the branch update has


What happens if someone without permissions were to push? Is there a process by which authorization could be given after the push?

Yes, if someone with permission pushes an update to the file. I'll clarify in master.

Yes there is a way to authorize after the push. We need to figure out a way to make this better with GitHub as we do not have the VSTS user context.

@chrisrpatterson Can you use the github user info in that case?

Without some other mapping stored in VSTS we don't know that a given GitHub user actually has permissions in VSTS.

mmitche · 2018-01-29T16:05:43Z

docs/preview/yamlgettingstarted-triggers.md

+```yaml
+trigger:
+- master
+- releases/*


Okay, so clarify here...if I push this to master (release branches don't exist yet), then the master's version of the yaml file will be used upon pushes to master. If I create a release/1.0 branch, then the release branch's version of the yaml file would be used on push to release/1.0.

What happens if I decide that pushes should happen on..say....dev/foobar? Can I do this simply by adding an entry for dev/foobar in the dev/foobar branch, or does the 'default' branch need to specify all the tracked branches?

I think it makes most sense for the triggers to be the 'union' of all branch's trigger sets, minus the opt out set.

The CI triggers will actually be the set that is defined in the version of the file for the particular branch. So in the case of /refs/heads/dev/foobar a push there would get no build because it is not specified as included. I could then update the file in the dev/foobar branch to include dev/* and the next push to dev/foobar would start building. However, if I then took a branch from master before the change from dev/foobar was merged for dev/foobaz a push there would not build as that branch is not yet specified.

So the set of branches is not a union across all branches, it is only the set specified in the version of the file read for the particular commit.

mmitche · 2018-01-29T16:12:03Z

docs/preview/yamlgettingstarted-triggers.md

+
+For build this behavior makes sense. Especially when we start supporting on-push defintion creation from any branch. And on-push resource authorization from any branch.
+
+For release, CI trigger would likely be opt-in? However with YAML, release will be tightly coupled with a repository. So opt-out CI behavior might also make sense for release? Or opt-out, but the default for RM only includes master?


I'm not sure about whether it should be opt-in/out. One thing to note is that release branches are often merged into other branches (especially master) as servicing fixes are created. This means yaml files are merged around a lot. Mistakenly enabling CI for a branch because of a merge is probably not a big deal, but mistakenly enabling release might be more significant.

In this case we are talking about the VSTS Release project not specifically release branches.

We have been discussing if the absence of any specified trigger pattern should mean trigger on all branches by default.

mmitche · 2018-01-29T16:14:04Z

docs/preview/yamlgettingstarted-triggers.md

+    exclude: []
+  paths:
+    include: []
+    exclude: []


What are the rules of include/exclude across branches?

As mentioned above the include/exclude is whatever is defined in the version of the file from the commit that triggered the build.

mmitche · 2018-01-29T16:17:40Z

docs/preview/yamlgettingstarted-triggers.md

+  paths:
+    include: []
+    exclude: []
+```


I would allow for a custom expression here, one that could utilize the incoming CI metadata. Examples:

- customExpression: eq(Commiter, 'specialized-ci-bot') - customExpression: contains(CommitMessage, 'test everything') - customExpression: not(CommitMessage, 'skip ci')

@mmitche that is a good idea.

mmitche · 2018-01-29T16:22:18Z

docs/preview/yamlgettingstarted-triggers.md

+## Schedules (future; needs cleanup)
+
+```yaml
+schedules:


I think this is okay, but I would allow for a cron string too. This is pretty common in the OSS world and has a lot of useful shortcuts. They also allow for 'star syntax', which means that times will randomly get picked. This causes workloads to be spread evenly over the day or hour. This is very useful for triggers that run often. Examples:

@daily - Runs every day at some point (same point every day) @hourly - Runs every hour, at some point in the hour (same point every hour) 0 0 10 ? * 6L - Fires at 10:00 the last friday of every month.

We have discussed CRON at the moment the job scheduler in VSTS does not support that syntax so we would have to do something custom. We have talked about ideas and maybe we could support a CRON like syntax but not necessarily everything right away.

mmitche · 2018-01-29T16:23:51Z

docs/preview/yamlgettingstarted-triggers.md

+  - 03:00 am
+  - 03:00 pm
+  branches:
+    include: []


What are the branch rules here? What happens if the branches are empty? What happens if I specify 'master' in master and 'master, release/1.0' in the my release branch. What happens if I specify release/1.0 in my dev branch?

In this case we have been discussing if we should treat schedules a bit differently than CI. Our thought is that the schedules are read form the repos default branch instead of trying to track and merge across all branches in a repo. In that model you would need to specify all schedules for all branches in master.

@chrisrpatterson That is a bit odd though, since you have to alter process in one branch (in the target branch for the schedule), but policy (scheduling) in another.

My thought would be that the schedule would be read from the target branch, just like CI. Since updating the branch policy is a commit, VSTS would notice the commit, update the schedule for that branch if necessary, etc.

Yes, however, a branch can be a pattern like releases/* so if I have specified in master to run releases/* every day at 3 and then in one of the release branches I change releases/* to once a week who wins?

The branch wins in that case. What I see here is that by specifying as many branches as you want in a single definition, you're improving the mergeability of these files across branches. However, as a general rule, I think if the process (what needs to be done) comes from the branch itself, then the policy (when it needs to be done) should also be in that branch.

chcosta · 2018-01-29T18:00:56Z

docs/preview/yamlgettingstarted-triggers.md

+  - thurs
+  - fri
+  - sat
+  timeZone: EST # Defaults to account time zone? Do we know that info? We might need to maintain a mapping for this abbrev, it is not in system.timeZoneInfo.


Is there any way you could reference a default value for an entire collection so that you could easily ensure everyone is using the same time zone? ie, I don't care what the timezone is, I just want it to be consistent with the rest of devdiv.visualstudio.com

This sort of goes into a general templating thing...I could define a standard dotnet core template for periodic builds, supply a parameter or two for specific info, and have the template take care of time/zone. e.g.

- standardNetCorePeriodicBuild([monday, friday])

Which goes to:

- schedules: - 1:00am - whenUnchanged: true - timeZone: PST - days: - <parameter from input on standardNetCorePeriodicBuild>

Yeah, templating would be one way. something like -timeZone: $(ProjectCollection.TimeZone) is another, though that could be in a template as well.

by default not specifying a timeZone is going to use the timeZone configured for your account.

Yes templating is also something we need to look at here.

@chrisrpatterson We were talking about overall build tooling (e.g. msbuild tasks) today, and versioning these build tools comes up a lot. We version them today through the use of nuget packages. I would be interested to know what kind of thoughts you have on versioning templates. It would be great if templates and other product tooling could be treated similarly (e.g. nuget packages that projects pull updates to as needed)

@mmitche lets take this topic discussion to a separate thread.

chcosta · 2018-01-29T18:04:45Z

docs/preview/yamlgettingstarted-triggers.md

+  branches:
+    include: []
+    exclude: []
+  paths:


An example of paths trigger might be useful, I certainly imagine it will see a fair bit of use. Does it include standard globbing syntax?

They syntax is not as complete as the globbing syntax, it is a basic starts with and wild card at the end.

Can a path filter be used without a branch filter? According to current build UI, its not possible.
If same limitation applies to YAML, please document it :)

mmitche · 2018-01-29T18:44:52Z

Some other things that appear missing:

PR/Push specific jobs - How can I specify that a job is run on commit, vs. one run on PR?
PR contexts - How can I specify that a PR job will show up in the UI (Github or otherwise) as "Release Build and test" or "Code Coverage build and test"
Optional jobs - How can I specify triggers are that optional? e.g. a job that runs the outerloop tests, but isn't run automatically?

ericsciple · 2018-01-29T22:25:55Z

@mmitche this triggers doc does not cover PR triggers (I need to update the doc, although we are targeting rolling out CI support first).

You can setup PR on the definition in the web today. You can add conditions to the phases so they only run on a PR versus a CI. build.reason is a variable that you can use in your condition. Here is a doc about phase conditions using variables. And here is an example definition. Ping me if it's easier to discuss offline.

chcosta · 2018-01-29T22:28:51Z

build.reason == nice, I'm glad there is now support for this.

mmitche · 2018-01-29T22:42:30Z

@ericsciple Sounds good. Just to confirm, you are planning on supporting PR triggers in the files in the near term, it's just not yet in the design.

chrispat · 2018-01-30T02:53:07Z

@mmitche yes we plan to do that just have not designed yet.

mmitche · 2018-02-07T17:52:31Z

@chrisrpatterson One other thing came up. It would be really ideal if the triggers could launch the jobs with specified parameters. This allow more code sharing. For instance, recently we've had issues with our official product build testing. We launch full builds overnight and would like to schedule one with testing on, and one with testing off. But the current triggers only launch with the defaults. In both the yaml and web interface world, that really only leaves you with the option of cloning the file/definition, which is non-ideal.

mmitche · 2018-02-20T23:26:55Z

@chrisrpatterson Any comment on the triggers with non-default parameters?

ericsciple · 2018-02-23T21:56:58Z

I split this into two PRs. One that is just for CI trigger features that are in master of VSTS. And this one, will track trigger features that aren't implemented yet.

RoopeshNair · 2018-04-26T08:56:42Z

docs/preview/yamlgettingstarted-triggers.md

+
+For build this behavior makes sense. Especially when we start supporting on-push defintion creation from any branch. And on-push resource authorization from any branch.
+
+For release, CI trigger would likely be opt-in? However with YAML, release will be tightly coupled with a repository. So opt-out CI behavior might also make sense for release? Or opt-out, but the default for RM only includes master?


For release, it can be opt-out provided the resource authorization is per branch.

chcosta · 2018-10-10T23:29:59Z

docs/preview/yamlgettingstarted-triggers.md

+## Schedules (future; needs cleanup)
+
+```yaml
+schedules:


Would there be any support for specifying a variable value on a certain schedule? ie, if you wanted to provide a perf property to a build scheduled weekly or just build debug once a week or something like that?

jtpetty · 2019-10-09T13:41:38Z

@vtbassmatt - Can you take a look at this one?

yaml triggers doc

83e6f8e

chrispat reviewed Jan 25, 2018

View reviewed changes

ericsciple added 2 commits January 25, 2018 10:44

replace ci with trigger

e7e2560

schedules is an array

958e11b

chrispat reviewed Jan 25, 2018

View reviewed changes

ericsciple added 2 commits January 25, 2018 15:15

update getting started and auth docs

710c583

ci follows branches

de88c72

mmitche reviewed Jan 29, 2018

View reviewed changes

chcosta reviewed Jan 29, 2018

View reviewed changes

chrispat added the Design label Mar 6, 2018

RoopeshNair reviewed Apr 26, 2018

View reviewed changes

chcosta reviewed Oct 10, 2018

View reviewed changes

jcagme mentioned this pull request Feb 11, 2019

Need support for custom build triggers that can set properties dotnet/arcade#143

Open

jtpetty requested a review from vtbassmatt October 9, 2019 13:41

vtbassmatt closed this Oct 9, 2019

vtbassmatt deleted the users/ersciple/m130triggers branch October 9, 2019 13:48


		For build this behavior makes sense. Especially when we start supporting on-push defintion creation from any branch. And on-push resource authorization from any branch.

		For release, CI trigger would likely be opt-in? However with YAML, release will be tightly coupled with a repository. So opt-out CI behavior might also make sense for release? Or opt-out, but the default for RM only includes master?

yaml triggers doc #1383

yaml triggers doc #1383

Conversation

ericsciple commented Jan 25, 2018

chrispat Jan 25, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mmitche commented Jan 29, 2018

ericsciple commented Jan 29, 2018

chcosta commented Jan 29, 2018

mmitche commented Jan 29, 2018

chrispat commented Jan 30, 2018

mmitche commented Feb 7, 2018

mmitche commented Feb 20, 2018

ericsciple commented Feb 23, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jtpetty commented Oct 9, 2019

chrispat Jan 25, 2018 •

edited

Loading