Add Elixir.mk - Allows for use of Elixir deps within an Erlang project.

[artman41]

This change introduces a new core plugin called elixir.mk.

This change accomplishes the following:

• Converts mix.exs projects to use erlang.mk
• Compiles .ex files, allowing for a project to use either Erlang, Elixir or a combination of both
• Properly configures paths so that Elixir dependencies are added to the $(PROJECT).app
• Treats Elixir as a dependency & fetches it as such, rather than relying on a system installation

This change makes use of hex.mk

[artman41]

Currently rebasing the change :)

[artman41]

@essen I've added a test under test/core_elixir.mk though I've been testing locally with a sample project.

[essen]

Hello! I've not had a good look yet but I doubt this works with some Elixir NIFs since they often have a Makefile and thus wouldn't enter the elixir autopatch.

[artman41]

Hello! I've not had a good look yet but I doubt this works with some Elixir NIFs since they often have a Makefile and thus wouldn't enter the elixir autopatch.

Damn, that's a good shout - any idea of an elixir lib which uses NIFs?

[essen]

There's a few in this search but gotta filter. Oftentimes if the page says mix + make as a build tool, that's an Elixir NIF. But not always.

https://hex.pm/packages?search=nif&sort=total_downloads

[artman41]

There's a few in this search but gotta filter. Oftentimes if the page says mix + make as a build tool, that's an Elixir NIF. But not always.

https://hex.pm/packages?search=nif&sort=total_downloads

I've just commited a change which should now make it support Elixir Nifs (assuming they use elixir_make - I'm not 100% familiar with the Elixir NIF ecosystem so not quite sure if I've accounted for the right things there).

I've added an additional test which will attempt to compile libsalty2, though this depends on libsodium.

If you've any more suggestions @essen, I'm more than happy to add them 👍

[artman41]

Any chance I could poke you to look this over @essen? 😄

[essen]

Does this work even if using local system installed Elixir?

[artman41]

Does this work even if using local system installed Elixir?

In what sense @essen?

I'm expecting elixir to always be loaded as a dep of the project rather than as an installed version on the system - I can try and account for this though if you'd prefer that functionality to be in there.

I'll edit this for confirmation but the only thing I think that could be problematic with an installed version of elixir is if code:get_path() returned paths to the installed elixir version, though I wouldn't ever expect that to be the case with a standard install.

[essen]

I think we should take the dep Elixir if one is available as a dep, and the system Elixir otherwise. Basically the dep can be used as an override.

Most projects don't require a specific version of Elixir, and for example in RabbitMQ we explicitly use the system Elixir (we even build packages for it).

[artman41]

@essen could you have a look over when you have chance?

Based on your feedback, I've updated the MR to default to using the System Install of Elixir if available. This can be manually overriden by a user-set var though (ELIXIR_USE_SYSTEM)

Let me know what you think :)

[artman41]

@essen could you have a look over when you have chance?

Based on your feedback, I've updated the MR to default to using the System Install of Elixir if available. This can be manually overriden by a user-set var though (ELIXIR_USE_SYSTEM)

Let me know what you think :)

just poking you 😄

[essen]

I'll get to it no need to remind so often.

[artman41]

@essen sorry to poke on this one again, any idea when you'll be able to get round to it? :)

[essen]

Is the thing that transforms paths like Elixir/MyApp/MyMod.ex into module name Elixir.MyApp.MyMod something the compiler does, or am I just blind and I missed it?

[artman41]

@essen had some time recently so will be picking this one back up 👍

[artman41]

Is the thing that transforms paths like Elixir/MyApp/MyMod.ex into module name Elixir.MyApp.MyMod something the compiler does, or am I just blind and I missed it?

Compiler magic 🪄

[essen]

I will try to make this work with RabbitMQ CLI which is (currently) in Elixir. Could you rebase to run CI now that I fixed it? Also you will need to edit the workflow to add/change the new suites here https://github.com/ninenines/erlang.mk/blob/master/.github/workflows/ci.yaml#L27

[artman41]

@essen rebased, hopefully ci should work as expected 👍

[essen]

I'm looking to merge this now. No need to rebase, I am probably going to cherry-pick parts of the code while I try to make things work.

[essen]

OK first step is obviously to merge the version resolver, this will also be good to have for non-Elixir projects.

Sorry I'm taking so long, I tried hard to refactor autopatch into something more pluggable (one step detect one step patch based on detection) but I'll have to leave it for later.

[essen]

I don't think we need this check, at least not anymore. Since we use hex_core for fetching, and hex_core fetches over TLS, it necessarily has crypto available.

Even if we didn't I don't think we want to encourage the practice of splitting Erlang/OTP into multiple "optional" packages like some distros do. You are expected to have the full Erlang/OTP when you use Erlang.mk.

[artman41]

My only suggestion to this would be to mention crypto as an explicit dependency / add a description of having a full Erlang/OTP installation to the user guide and github README 👍

[essen]

We can do that yes.

[essen]

I pretty much have to merge some of the Elixir stuff into erlc.mk (and some stuff in deps.mk). Otherwise it just ends up conflicting during execution leading to things not getting rebuilt and whatnot.

[essen]

I am also removing the automatic rebuild on Elixir changes. We don't do it for Erlang, so no point doing it for Elixir. If the Erlang or Elixir version changes then you are expected to recompile yourself.

Since I am getting close to a complete cleanup/merge of the PR (pending a still very large refactoring of Mix autopatch so will take a few days) I have opened a PR at #1020

[essen]

I've removed the escript related changes since they don't have tests but it can be reintroduced later.

What remains is cleaning up Mix autopatching and making sure things work with make -j.

[essen]

Do you know why this line is necessary? Mix magic?

[artman41]

don't remember the version but elixir only started supporting on_undefined_variable in a specific version of elixir - all others prior would raise an error:badarg due to the key being unsupported.

To try and make mix more backwards compatible, I added that line 👍

Asking ChatGPT, it seems that

• The on_undefined_variable option was introduced to Elixir's Code.compiler_options in version 1.15, released in June 2023
• Elixir 1.15 supports Erlang/OTP versions 24, 25, and 26

[essen]

Thanks, I get that. But I meant is using an undefined variable in Elixir a normal OK thing to do? I don't know Elixir so just wondering.

[artman41]

Honestly, I'm unsure

I only found the issue when testing with a dep - I'm not sure whether it was phoenix or another elixir library

I'll try and have a look in a bit to see if I can find the library again 👍

[essen]

It did break tests when I removed it.

[essen]

The PR at #1020 is almost ready, just a few details and more tests to add. It completely replaces Mix in RabbitMQ in rabbitmq/rabbitmq-server#12922 except for running tests.

[essen]

Closing in favor of the PR mentioned above.

Thank you very much for the hard work on this!!