sort-as: force an order on multiple profiles

[fenollp]

Ensures that directories inside _build/ have a fixed nomenclature: makes it impossible for rebar3 as a,b to create sometimes _build/a+b and other times _build/b+a.

I got caught by it when developing offline. Plugins were pulled to a+b but then rebar3 would decide to use b+a and would fail to load plugins.

Next step would be to symlink plugins instead of re-fetching them when using a new as combination. That would probably be an issue provided that plugins are (generally?) un-version'ed.

[ferd]

This shouldn't be done. The order of applied order between a+b and b+a is not the same and that was a deliberate choice, since the semantics should be different.

I don't think that it's a good idea to start replacing these. If people currently have profile like prod+osx to override production profile values with osx-specific ones, then all of a sudden you get osx+prod and it's possible we break builds.

[fenollp]

Hm I think I saw both forms appear given only one as form... will try and replicate. Thanks for your input!

What do you think about symlinking plugins?

[ferd]

Symlinking plugins could work as long as the plugin versions don't change across profiles, but we haven't been able to properly track plugin versions since they are downloaded and compiled as part of the build process itself.

[fenollp]

So I can replicate the issue:

1. make sure that a first run of rebar3 as a+b ... created _build/a+b/
2. run that same command again
3. now _build/b+a/ exists

My actual command looks like rebar3 as test,cake003 proper -m my_module -p p -n 99.
Version: rebar 3.5.0 on Erlang/OTP 20 Erts 9.2

[ferd]

Yeah I think the test breakage is due to no longer respecting the profile order defined by tests there.

I could not reproduce the issue on the rebar3 build itself, and I thought the issue could be related to the test profile, but it wouldn't work. The issue, unless I'm wrong can only be reproduced through the rebar3_proper plugin instead.

Can you reproduce the issue with any other command than proper? I'm a bit dumbfounded as to what could cause it, but it does appear related.

[fenollp]

On latest master 2560aaf:

$ ./bootstrap && ./rebar3 as test,bla cover
...
$ ls _build
bla+test  bootstrap  default

To better show the issue I have added the test deduplication_stability which takes some examples from test/ some of which are the reason this patch fails CI. The added test pass.

Note, with this PR:

$ ./bootstrap && ./rebar3 as test,bla cover && ls _build
bootstrap  default  test+bla

[ferd]

Okay, I think the issue is rather related to something about the handling of default profiles on some providers. proper, cover and eunit all include the test profile by default in their providers. I suspect we'd have the same thing with doc-related profiles and edoc.

The issue is not the reversing in deduplication alone, it's the deduplication when matched with the default profile of an issue being repeated explicitly in the list of as.

The current patch still fails builds in all versions because it breaks the ordering of profiles in some of the other cases. I think it's just a partial fix.

[fenollp]

Hm i see. I think reverting this and just appending the test profile if it’s not there yet would solve my issue.

[fenollp]

If you want a better commit message I’ll set it and squash everything.

[fenollp]

I think a final way to fix this would be to only add the profiles in the ‘profiles’ key of the providers “module” call that are not current profiles yet. I’ll gladly look into this however I’m not sure providers is even a real module or if it’s built at runtime I have issues finding where.

[ferd]

Providers library is implemented at https://github.com/tsloughter/providers

[fenollp]

Last commit is the best I could come up with :)

[fenollp]

If you like this I can squash and we'd be done with this PR :)

[ferd]

Seems to make sense, sorry for the delay.

[fenollp]

Ah you should have warned me I'd have squashed