Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix 2.7.0 Migration #2444

Merged
merged 2 commits into from
Nov 29, 2023
Merged

Fix 2.7.0 Migration #2444

merged 2 commits into from
Nov 29, 2023

Conversation

ada-x64
Copy link
Contributor

@ada-x64 ada-x64 commented Nov 20, 2023

This PR ensures that the version created by the migration script matches the version in the package.json. It does so by inlining the package version and bumping the version as needed.

@ada-x64 ada-x64 changed the title Fix 2.7 Migration Fix 2.7.0 Migration Nov 20, 2023
Copy link
Member

@texodus texodus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR!

Please find another way to do this without copying package.json around.

@@ -0,0 +1 @@
src/ts/package.json
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't add arbitrary files to .gitignore, tasks like twine and npm publish are not aware of these directives which is quite dangerous. If you need to create files as part of the build process, put them in /build so they are cleaned and not published - if they need to be published put them in /dist.

@@ -80,12 +80,13 @@ async function build_all() {
execSync(`cargo bundle ${IS_DEBUG ? "" : "--release"}`, INHERIT);

// JavaScript
const { default: cpy } = await cpy_mod;
await cpy("package.json", "src/ts");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't copy files within the source tree (see above notes on /build and /dist), but also especially don't do this with the package.json, which breaks node's module resolution in this directory (or any package root metadata like Cargo.toml).

} else if (options.warn) {
console.warn("Migrating from 2.6.1");
}
old.version = parse_semver("2.7.0");
old.version = parse_semver("2.6.1");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line is inert (because it is overwritten by assure_latest()), but if it weren't it would be wrong. If old == 2.6.1, the migration is applied (conditional on line 22 is false) but then the version field is set back to 2.6.1 again (line 27).

This specific script functionally upgrades from 2.6.1 to 2.7.0 (and presumably beyond), ergo old.version should be some version after 2.6.1 (which we wouldn't know at the time of authoring the migration).

@ada-x64
Copy link
Contributor Author

ada-x64 commented Nov 21, 2023

@texodus I've updated this PR with some general cleanup of the migration scripts and I've replaced the copied file with an import from node_modules.

I think there is still some thinking to be done about how we're going to go about migrations in the future.
The way I understand it, there are two ways to do API versioning. I'll call them from x and to x.

To-x migration scripts migrate a configuration from version x-1 to x. For example, 2-6-1.ts would migrate from 0.0.0 to 2.6.1.

From-x migration scripts migrate a configuration from version x to x+1. For example, script 2-6-1.ts would migrate from 2.6.1 to 2.7.0.

We're currently implementing the from-x migrations. The main issue with this approach, as we've discussed, is that there is no way of knowing what the next version will be. I've worked around this by simply placing the latest package version on top. In addition to fixing the latest-package version, the troubles we were having with versioning I think were caused by the mere fact that version migrations did not take the package version into account, which has been fixed in this PR.

There are other problems. Because we'll be developing API changes on master, and those migrations will be happening on the same version of perspective as the previous API version, we're still going to have to deal with a form of idempotency in the migrations. That is, we'll need to make sure that the current migration script can upgrade from any version x-n to version x, and stay at version x without change. (i.e., convert(convert({version: x-n})) == convert({version: x-n}) This shouldn't be too hard to accomplish, but is just something we'll need to keep in mind.

Copy link
Member

@texodus texodus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! Looks good!

@texodus texodus merged commit e1069c0 into finos:master Nov 29, 2023
12 checks passed
@texodus texodus added the bug Concrete, reproducible bugs label Nov 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Concrete, reproducible bugs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants