New dune ocaml debug command

[richardlford]

Add a new command, dune ocaml debug to set a mapping environment variable and then invoke
the OCaml debugger, ocamldebug.

For the deployment phase, we need the inverse of the BUILD_PATH_PREFIX_MAP. I've decided that this should use a different environment variable, so I've chosen DEPLOY_PATH_PREFIX_MAP. The reason two variables are needed is that the new dune command, dune ocaml debug is similar to dune exec, except instead of invoking the executable bytecode program, it gives the executable to the OCaml debugger. But before doing that, it builds the executable, if necessary, to make sure it is up-to-date. During the building, BUILD_PATH_PREFIX_MAP maps build-time absolute paths to abstract paths, but the debugger needs the inverse mapping, so will use DEPLOY_PATH_PREFIX_MAP.

This PR assumes a simplified forward mapping, namely, the build root, source root and the install root (usually ~/.opam/switch) map to /workspace_root.

The inverse mapping used is (in priority order, highest first):

• /workspace_root -> source root
• /workspace_root -> build root
• /workspace_root -> install root

Because of dune ocaml debug, dune requires the location of ocamldebug. Some test cases which set up special paths had to be modified to make sure that ocamldebug could be found.

The new command takes command line options similar to those that ocamldebug takes, except that multi-character options need to be preceded by --. It passes these onto ocamldebug. It is also possible to specify a different ocamldebug for use when testing experimental versions.

Fixes #7414

[emillon]

Ah this is what #7743 was refering to. I assumed it had been merged already. I'll have to make sure that the docs are consistent with the state of the release. What about the ocamldebug support, is this merged in ocaml/ocaml yet?

[richardlford]

Ah this is what #7743 was refering to. I assumed it had been merged already. I'll have to make sure that the docs are consistent with the state of the release. What about the ocamldebug support, is this merged in ocaml/ocaml yet?

I just need to make a small change to it, to use the DEPLOY_PATH_PREFIX_MAP, rather than the BUILD_PATH_PREFIX_MAP for the debugger (the compiler still uses BUILD_PATH_PREFIX_MAP). I had a big PR with both compiler and debugger changes as well as enhancements to ocamltest to enable testing the features, and new tests. I was waiting for review of those items so went ahead and made the dune changes.

Originally I wanted to change dune so that it computed a mapping so all the abstract paths mirrored the install hierarchy (see #7413 and #7540). For large projects with a lot of packages the mapping could get quite complex. I think the #7540 is functionally correct but there were issues in the review, in that getting the mappings to the low level dune_engine was difficult and I had resorted to some mutation that was not acceptable to @rgrinberg. I'm starting a new job in less than two weeks and will no longer have as much time to work on this, so I've decided to stop work on the more complex approach. Instead, a simpler approach which depends on the developer's having a source layout that mirrors the installation layout is implemented in #7741 (which just extends existing mapping to also map paths based on the install_root).

If possible, I'd like to get the essential parts of the compiler, debugger, and dune changes completed before I start my new job.

So the pending dune work is this PRs:

• New dune ocaml debug command #7750, the new dune ocaml debug command
• Rewrite install root based paths #7741, to also map install roots

And the ocaml PRs are:

• Modify bytecomp for BUILD_PATH_PREFIX_MAP ocaml#12139, compiler changes
• Add deployment phase support for BUILD_PATH_PREFIX_MAP ocaml#12126, ocamldebug changes
• Additions to ocamltest ocaml#12127, ocamltest changes

I'll now concentrate on rebasing and updating the ocaml changes, and getting clean test runs.

Note that the combination of the compiler and dune changes are such that mapped programs can be debugged in their build environment, regardless of whether the source hierarchy mirrors the install hierarchy. The latter is only needed if you want to set into libraries that are already installed (and not in your source/build tree).

[richardlford]

Aside from taking a little longer to build Dune, I don't see how this change could be causing a failure in the ocaml-benchmarks.

[Alizter]

@richardlford That failure is unrelated. If you click on the link you will see that the benchmark actually succeeds. There is currently a problem with the GitHub check status for our benchmarks and sometimes it shows a false failure.

The issue is tracked here: ocurrent/current-bench#440

[richardlford]

I have rebased and pushed.