feat(generate:typetests)!: Make type tests configurable per-branch #12849

tylerbutler · 2022-11-08T23:03:56Z

Type tests can now be configured to use different baseline versions on a given branch depending on the type of release that
the branch is designated for. For example, for the client release group, the next branch is the major version series
branch and main is the minor version series branch. This can be declared in the release group root package.json, in the
fluidBuild.branchReleaseTypes section. For example, the following configuration designates the main and lts branches as
minor version series branches, while the next branch is designated for major releases.

"fluidBuild": {
  "branchReleaseTypes": {
    "main": "minor",
    "lts": "minor",
    "release/**": "patch",
    "next": "major"
  }
}

The type test generator takes this information into account when calculating the baseline version to use when it's run
from a particular branch. Baseline versions are set as follows based on the branch release designation:

Branch release designation	Baseline version	Example: version 2.3.4
patch	`previousPatch`	2.3.3
minor	`^previousMinor`	^2.2.0
major	`^previousMajor`	^1.0.0

BREAKING CHANGE: fluid-type-validator is deprecated. Use flub generate typetests instead.

msfluid-bot · 2022-11-11T02:58:25Z

Baseline CI build failed, cannot generate bundle analysis at this time

Baseline commit: c5f89d0

Generated by 🚫 dangerJS against 5660bd2

build-tools/packages/build-cli/docs/typetestDetails.md

build-tools/packages/build-cli/.eslintrc.cjs

Josmithr

One high-level thing I think would be worth some docs on in the overview doc is what to do when resolving conflicts when merging branches. E.g. what do I do if I'm merging main into next and get type-test / type-test config conflicts?

Josmithr · 2022-11-15T01:04:00Z

build-tools/packages/build-cli/docs/typetestDetails.md

+Generating type tests has two parts: _preparing package.json_ and _generating test modules_. By default, both steps are
+run for each package. You can run only one part at a time using the `--prepare` and `--generate` flags.
+
+[api type validation]: https://github.com/microsoft/FluidFramework/wiki/API-Type-Validation


Curious on your thoughts WRT conventions for where to put link aliases. I find it useful to be able to look in a single place in a given document for the links so I know what things I can re-use and what I need to define as I go.

Would it make sense to move this to the bottom under a "links" comment like we generate for content on the website?

Feel free to disregard in the context of the PR - it just took me a few seconds to find where this was defined as I was reviewing, since I wanted to make sure it was defined somewhere.

I usually put them at the end of the section in which they're used unless they're used in several places. Reason being that sections get moved/reorganized, and links sometimes break when they're far away from the content. But I don't care too much.

build-tools/packages/build-cli/docs/typetestDetails.md

Josmithr · 2022-11-15T01:12:26Z

build-tools/packages/build-cli/src/commands/generate/typetests.ts

@@ -59,10 +64,11 @@ export default class GenerateTypeTestsCommand extends BaseCommand<
                "~previousMinor",


Can we generate this list from the PreviousVersionStyle type?

It's a union type, so no, not as far as I know.

build-tools/packages/build-tools/src/common/fluidRepo.ts

build-tools/packages/build-tools/src/typeValidator/packageJson.ts

Josmithr · 2022-11-15T01:20:04Z

package.json

@@ -177,6 +190,7 @@
        ]
      },
      "additionalLockfilePaths": [
+        "azure",
        "common/build/build-common",


(Not for this PR) do the packages under common/lib need to be added here? For my knowledge, what is this field used for?

See #12726.

The azure entry shouldn't be needed. I'll investigate.

Seems related to switching the azure release group from a "simple" config to a "complex" one (an object instead of a string). I'm going to ignore for this PR since there's a clear workaround (adding the entry here) and follow up with another change later.

build-tools/packages/build-tools/src/bumpVersion/context.ts

build-tools/packages/build-cli/docs/generate.md

build-tools/packages/build-tools/src/typeValidator/packageJson.ts

build-tools/packages/build-cli/docs/typetestDetails.md

ssimic2 · 2022-11-15T18:35:39Z

package.json

@@ -138,12 +138,25 @@
      "common-def": "common/lib/common-definitions",
      "common-utils": "common/lib/common-utils",
      "protocol-def": "common/lib/protocol-definitions",
-      "azure": "azure",
+      "azure": {


It seems somewhat odd that we would define this here, and not in "azure" directory since it is its own Monorepo. I understand that root package.json can manage config across all monorepos (not just client), but when developing around azure-client I do spend most of the time in azure folder, (likely wrongly) assuming that its fully self contained.

@Josmithr had the same feedback. I'll explore configuring release groups in their root this morning.

build-tools/packages/build-cli/docs/generate.md

ssimic2 · 2022-11-15T18:53:40Z

build-tools/packages/build-cli/docs/typetestDetails.md

+It may be useful to configure a branch for a specific baseline instead of the default based on the branch release
+designation. To do this, you can use any of the following strings instead of major/minor/patch.
+
+- `baseMajor`


Maybe some super-short description here on "base" would be helpful. When I think of "previous" I think about previous to existing branch version, but "base" I cannot fit into my mental model, or at least I assume we don't want to run compat tests against self :)

Clarified both here and in the tsdocs:

The "base" versions are calculated by zeroing out all version segments lower than the base. That is, for a version v,
the baseMajor version is ${v.major}.0.0 and the baseMinor version is ${v.major}.${v.minor}.0.

The "previous" versions work similarly, but the major/minor/patch segment is reduced by 1. That is, for a version v,
the previousMajor version is ${min(v.major - 1, 1)}.0.0, the previousMinor version is
${v.major}.${min(v.minor - 1, 0)}.0, and the previousPatch is ${v.major}.${v.minor}.${min(v.patch - 1, 0)}.0.

The "previous" versions never roll back below 1 for the major version and 0 for minor and patch. That is, the
previousMajor, previousMinor, and previousPatch versions for 1.0.0 are all 1.0.0.

ssimic2 · 2022-11-15T19:03:53Z

build-tools/packages/build-cli/docs/generate.md

@@ -114,18 +114,17 @@ Generates type tests based on the individual package settings in package.json.
 ```
 USAGE
  $ flub generate typetests [-d <value> | --packages | -g client|server|azure|build-tools] [--prepare | --generate]


Could you clarify --packages: "Run on all independent packages in the repo." ?

It's a shortcut to run generation on all independent packages, like build-common, common-utils, etc. The alternative is to use the --dir option and pass the path to the package, but this is more convenient when you're updating them all. Clarified in the flag docs.

build-tools/packages/build-cli/src/lib/package.ts

This dependency was missed when some code was moved in PR #12849. It wasn't found due to lerna's hoisting behavior.

I manually updated the npm scripts to call flub with the correct arguments etc. Relevant changes: - Uses the `--pin` argument to pin the version in package.json to the maximum matching released version. This should help with the lockfile issues we have discovered. - All generated files now include `.generated` in their name and old files were deleted. This was detected as a rename by git. See #12849 or `typetestDetails.md` for documentation on semantics of these changes. Then I did the following steps to configure the baselines and generate tests. 1. `npm run typetests:prepare -- -b main --reset` - this reset all type test overrides and set the baseline versions according to the main branch config. For the client release group this is `~previousMinor`, which corresponds to version 2.0.0-internal.2.1.0. 2. `npm i` - installed new versions of packages. 3. `npm run typetests:gen -- -b main` - this regenerated the typetests. 4. Ran a build and added back broken test overrides as needed. 5. Disabled tests in some packages like PropertyDDS. I also did roughly the same steps for the independent packages.

feat(generate:typetests): Make type tests configurable per-branch

f9f1d87

github-actions bot added public api change Changes to a public API base: main PRs targeted against main branch labels Nov 8, 2022

tylerbutler added 6 commits November 9, 2022 18:36

updates

33dbc02

Merge branch 'main' into cli/typetest-branch-config

27f79ac

formatting

15d724c

Merge branch 'main' into cli/typetest-branch-config

67db2ee

updates

f4d713f

updates

f600918

tylerbutler added 6 commits November 11, 2022 09:06

policy

55246d0

Merge branch 'main' into cli/typetest-branch-config

7ed8899

updates

63fb832

Merge branch 'main' into cli/typetest-branch-config

fdca17a

updates

e38061a

Merge branch 'main' into cli/typetest-branch-config

56b33ed

tylerbutler marked this pull request as ready for review November 11, 2022 23:14

tylerbutler requested review from msfluid-bot and a team as code owners November 11, 2022 23:14

tylerbutler commented Nov 11, 2022

View reviewed changes

build-tools/packages/build-cli/docs/typetestDetails.md Outdated Show resolved Hide resolved

tylerbutler commented Nov 11, 2022

View reviewed changes

build-tools/packages/build-cli/docs/typetestDetails.md Outdated Show resolved Hide resolved

tylerbutler added 2 commits November 11, 2022 15:25

Update build-tools/packages/build-cli/docs/typetestDetails.md

2c0b042

Update build-tools/packages/build-cli/docs/typetestDetails.md

3561aaa

tylerbutler commented Nov 11, 2022

View reviewed changes

build-tools/packages/build-cli/.eslintrc.cjs Outdated Show resolved Hide resolved

Update build-tools/packages/build-cli/.eslintrc.cjs

bb70fb0

Josmithr approved these changes Nov 15, 2022

View reviewed changes

tylerbutler commented Nov 15, 2022

View reviewed changes

build-tools/packages/build-cli/docs/typetestDetails.md Show resolved Hide resolved

tylerbutler added 2 commits November 14, 2022 18:01

Apply suggestions from code review

5f6756b

feedback

eb72d80

tylerbutler added 2 commits November 14, 2022 18:32

full build

8041644

cleanup

361c9e5

ssimic2 reviewed Nov 15, 2022

View reviewed changes

build-tools/packages/build-cli/docs/generate.md Outdated Show resolved Hide resolved

ssimic2 reviewed Nov 15, 2022

View reviewed changes

tylerbutler added 7 commits November 15, 2022 18:30

feedback

7025b2b

docs

4eda596

Merge branch 'main' into cli/typetest-branch-config

4a758e5

policy

3dbdff3

fixup

85df5c8

add docs on pinning

71ea4ae

Merge branch 'main' into cli/typetest-branch-config

b597817

tylerbutler commented Nov 16, 2022

View reviewed changes

build-tools/packages/build-cli/src/lib/package.ts Outdated Show resolved Hide resolved

tylerbutler added 3 commits November 16, 2022 10:44

Update build-tools/packages/build-cli/src/lib/package.ts

29cd509

Merge branch 'main' into cli/typetest-branch-config

8fba738

cleanup

5660bd2

tylerbutler changed the title ~~feat(generate:typetests): Make type tests configurable per-branch~~ feat!(generate:typetests): Make type tests configurable per-branch Nov 16, 2022

tylerbutler changed the title ~~feat!(generate:typetests): Make type tests configurable per-branch~~ feat(generate:typetests)!: Make type tests configurable per-branch Nov 16, 2022

tylerbutler merged commit 8c29adc into microsoft:main Nov 16, 2022

tylerbutler deleted the cli/typetest-branch-config branch November 16, 2022 21:24

tylerbutler mentioned this pull request Nov 16, 2022

fix(build-tools): Add missing dependency #12979

Merged

tylerbutler added a commit that referenced this pull request Nov 16, 2022

fix(build-tools): Add missing dependency (#12979)

b9d33cf

This dependency was missed when some code was moved in PR #12849. It wasn't found due to lerna's hoisting behavior.

Abe27342 mentioned this pull request Nov 21, 2022

Prepare and generate typetests on main #13029

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(generate:typetests)!: Make type tests configurable per-branch #12849

feat(generate:typetests)!: Make type tests configurable per-branch #12849

tylerbutler commented Nov 8, 2022 •

edited

Loading

msfluid-bot commented Nov 11, 2022 •

edited

Loading

Josmithr left a comment

Josmithr Nov 15, 2022

tylerbutler Nov 15, 2022

Josmithr Nov 15, 2022

tylerbutler Nov 15, 2022

Josmithr Nov 15, 2022

tylerbutler Nov 15, 2022

tylerbutler Nov 15, 2022

tylerbutler Nov 15, 2022

ssimic2 Nov 15, 2022

tylerbutler Nov 15, 2022 •

edited

Loading

ssimic2 Nov 15, 2022 •

edited

Loading

tylerbutler Nov 16, 2022

ssimic2 Nov 15, 2022

tylerbutler Nov 16, 2022

		@@ -59,10 +64,11 @@ export default class GenerateTypeTestsCommand extends BaseCommand<
		"~previousMinor",

feat(generate:typetests)!: Make type tests configurable per-branch #12849

feat(generate:typetests)!: Make type tests configurable per-branch #12849

Conversation

tylerbutler commented Nov 8, 2022 • edited Loading

msfluid-bot commented Nov 11, 2022 • edited Loading

Josmithr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tylerbutler Nov 15, 2022 • edited Loading

Choose a reason for hiding this comment

ssimic2 Nov 15, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tylerbutler commented Nov 8, 2022 •

edited

Loading

msfluid-bot commented Nov 11, 2022 •

edited

Loading

tylerbutler Nov 15, 2022 •

edited

Loading

ssimic2 Nov 15, 2022 •

edited

Loading