Feature request: running migrations for dependencies

[cag]

It would be nice if one could specify dependencies for which migrations are run when the project's migrations are run. For example, suppose there is a framework written in Solidity which requires certain contracts to be deployed. A project using that framework might specify its use using NPM or EthPM.

In a different project, maybe there is a configuration key which specifies the dependency migrations for which this project's migration depends. Basically, Truffle will truffle migrate those dependencies automatically before running the project's dependencies, and in the case of the aforementioned example project, the framework's migration will be run first.

I dunno whether it would be better to require all dependencies including child deps to be specified in order, or whether the deps should be toposorted and migrated in that order.

[cag]

I've started a branch containing a test scenario where this might take shape: develop...cag:migrate-deps

I think that build artifacts for dependency migrations should exist in the package folder's build folder. This would help backwards compatibility and avoid name collisions between packages, as artifacts are not aware of the file they came from, so to speak.

Maybe it would be good if the migrations can just be inferred from the package.jsons of the projects? That would require a toposort and might break backwards compatibility, though I think this could be the easiest way to go, and I don't think any Truffle project in the wild relies on migrations in child dependencies not being run. We could introduce an opt-out configuration parameter in case a project's dependency migrations aren't supposed to be run.

Alternatively, we could make an opt-in configuration parameter for packages.

Having read the https://github.com/ethpm/ethpm-spec I think that that is maybe also something to strive for, though some differences about how ethpm is (a bunch of pointers to stuff on IPFS, mostly contracts and blockchain locations) vs how npm is (tarballs with build artifacts, migrations, scripts and tests in addition to the contracts) maybe should indicate that npm should be approached somewhat differently? It's reasonable to assume that the migration scripts are also shipped with the packages on npm.

Also there is a way to artifacts.require ethpm packages documented already. I admit I don't know terribly much about it though, but it uses a 'package/Contract' name format. I am using an '@org/package:Contract' name format in my fork. Dunno if that is a good or bad idea, but considering artifacts would be '@org/package/build/contracts/Contract' if migrations succeed, maybe that makes sense? I dunno, what do you think?

Also, I heard @cgewecke was working on this...

Also hi @gnidan

[cgewecke]

@cag No no no. Completely deferring to your good judgement here. We were talking about this this morning and thought that you're probably in the best position to know what's needed because you have a real use case.

I'm just eventing out the migrations so that they can have reporters and be made interactive, more informative, etc. And use async/await more intuitively.

This work sound quite close to artifact level stuff that @gnidan is working on, which inevitably touches migrations. . .

Anyway awesome! Thank you.

[cag]

Alright, I am almost there, but everything is usable already, so I am putting it out, but please wait a bit before you start reviewing those PRs. Things could still use a bit of polish, and the test case for running tests fails (it is run before the migration test so that it won't produce passing-like behavior, as it should).

I am keeping backwards compatibility with the slashes instead of colons, but I do think that colons should be allowed. However, I will leave that as a separate issue.

[cag]

Hey, it is done, and you can try it out with

npm install --save-dev cag/truffle#migrate-deps-build

[cgewecke]

@cag Ok awesome! Will check this out this week . . .

[cag]

Okay I've had to make a few edits because I overlooked a scenario. The build branch is also updated to reflect those edits I've made.

[cgewecke]

Hi @cag oh cool!

I'm lagging really bad here sorry - I wanted to leave you a note because there's a bit of chaos coming down the line this week:

• we're moving the whole truffle project back into one repo (using Lerna, on wednesday). As part of that - I'm going to merge all of your work into staging branches that will carry over into the new format.
• we're planning a beta release for Truffle V5 at the end of the month, and was thinking that this work would ultimately target that branch (next). Are you ok with that? V5 will have some breaking changes because it upgrades web3 - might be kind of annoying. But your changes also fit really well with the Migrations rewrite which is part of it.
• Finally, there are some files where there will be conflicts - as soon as the ongoing migrations work is merged into next here, I'm going to resolve those and get this stuff re-opened as a single PR and ping you.

Sorry about the mayhem and thanks so much this work - it's going to be really really cool to have this.

[cag]

Hey, no problem! Yeah SGTM.

…

On Mon, Jun 11, 2018, 10:07 PM c-g-e-w-e-k-e-> ***@***.***> wrote: Hi @cag <https://github.com/cag> oh cool! I'm lagging really bad here sorry - I wanted to leave you a note because there's a bit of chaos coming down the line this week: - we're moving the whole truffle project back into one repo (using Lerna, on wednesday). As part of that - I'm going to merge all of your work into staging branches that will carry over into the new format. - we're planning a beta release for Truffle V5 at the end of the month, and was thinking that this work would ultimately target that branch (next). Are you ok with that? V5 will have some breaking changes because it upgrades web3 - might be kind of annoying. But your changes also fit really well with the Migrations rewrite which is part of it. - Finally, there are some files where there will be conflicts - as soon as the ongoing migrations work is merged into next here, I'm going to resolve those and get this stuff re-opened as a single PR and ping you. Sorry about the mayhem and thanks so much this work - it's going to be really really cool to have this. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#554 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AASNz5TmUeE7czYZlWqXmne64JpAoIUuks5t7zCNgaJpZM4PKzzW> .

[stale]

Thank you for raising this issue! It has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. If you would like to keep this issue open, please respond with information about the current state of this problem.

[stale]

There has been no new activity on this issue since it was marked as stale 7 days ago, so it is being automatically closed. If you'd like help with this or a different problem, please open a new issue. Thanks!

[haltman-at]

Reopening this, as this was never completed.

[haltman-at]

Note that @cag currently has a PR open to address this, #1511, but it's badly out of date and progress on it seems to have stalled. Need to determine what to do here.

[cag]

I'm guessing that it'd be easier to just start from scratch with new branches at this point, though if somebody wants to rebase/merge the stuff I did, feel free to do so. Hopefully the work done in the PRs make sense and help, but if they don't, 🤷

…

On Tue, Jun 21, 2022, 2:34 PM Harry Altman ***@***.***> wrote: Note that @cag <https://github.com/cag> currently has a PR open to address this, #1511 <#1511>, but it's badly out of date and progress on it seems to have stalled. Need to determine what to do here. — Reply to this email directly, view it on GitHub <#554 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACI3T7QSJXYCMTUBVV2C5TVQIKLRANCNFSM4DZLHTLA> . You are receiving this because you were mentioned.Message ID: ***@***.***>