Add support for exporting statically compiled libraries from C

[dpapavas]

This PR trivially extends the existing mechanism for statically compiled libraries to allow the user to export custom libraries from C. I currently use it to export a relatively complex API from the C side as an importable library in scheme (mainly for the benefit of allowing organization into sublibraries, selective import of symbols, renaming symbols, etc.) Use goes something like this:

From the C side:

#include <chibi/sexp.h>
#include <chibi/install.h>

static sexp test(sexp ctx, sexp self, sexp_sint_t n, sexp arg)
{
    return arg;
}

sexp init_foo(
    sexp ctx, sexp self, sexp_sint_t n, sexp env, const char* version,
    const sexp_abi_identifier_t abi) {
    sexp_define_foreign(ctx, env, "test", 1, test);
    return SEXP_VOID;
}

static struct sexp_library_entry_t sexp_static_libraries_array[] = {
    {"foo/foo" sexp_so_extension, init_foo},
    {nullptr, nullptr}
};

extern struct sexp_library_entry_t *sexp_static_libraries;

Then somewhere in the initialization function for chibi-scheme, I add:

sexp_static_libraries = sexp_static_libraries_array;

Then from Scheme, somewhere in, say ./foo/bar.sld:

(define-library (foo bar)
  (export test)
  (include-shared "foo"))

Then in the main program one can:

(import (scheme write) (foo bar))

(display (test "foo"))
(newline)

As it is, the design is quite elementary and doesn't make any attempt to cover up its co-opting of the statically included shared library mechanism. As such, the user is forced to bother with mimicking shared library import paths and extensions. This is not a big bother really and it is usable just fine as is, but if you prefer, we could look into further streamlining the interface.

Edit: This started off leaving the the sexp_static_libraries symbol undefined and simply defining it in the main program. This turned out not to be supported on Windows though. I then tried initializing sexp_static_libraries to an empty table, but marking it __attribute__((weak))), but this doesn't work on MSVC. This third attempt also fails in some cases, although I'm not sure why. I'll look further into it, but if you have any comments, let me know.

[dpapavas]

As far as I can see, the failure in the integration tests is not due to my changes. I tried replacing them with a single whitespace change, so that the code was identical to master and it still fails in the same way.

Other than that, the changes work as expected, at least on my setup, but I think that we should probably add a function, e.g. sexp_set_static_user_libs to allow the user to install their own library table, instead of setting the extern pointer directly. This can be defined only when the SEXP_USE_STATIC_LIBS_USER feature macro is set. Let me know what you think.

[dpapavas]

Well, it seems I no longer need this. I realized that it'd probably be best to statically link Chibi-Scheme, so that I now have to bake in the system C libraries anyway. I then simply get the table of "static libraries" from my side and copy it into a new table, adding my own libraries.

This seems to work fine, but it is still something of hack, in that it is not explicitly supported. It would probably not be too hard to add a function to Chibi to "register" a set of user-supplied libraries. Since Chibi is meant to be ebeddable, this seems like a desirable feature. The biggest change that would be needed, as far as I can see, would be to change the sexp_static_libraries symbol from being extern in eval.c and defined in clibs.c, to the other way around.

Let me know if you'd like me to try and put together a PR along those lines.

[ashinn]

Thanks! That's entirely up to you. I'm afraid I don't have time to help right now, but patches are always welcome.

[dpapavas]

I've updated the patch with an implementation that, as far as I can see, is the simplest in terms of needed changes, while still covering all use cases. As before, the feature is based on the machinery already in place to support statically compiled C libraries. This is essentially unaltered and works as before. When it is enabled, the table of exported C libraries is initialized via clibs.c as before. The only change is that now, more functions can be added via the new sexp_add_static_libraries function. (To avoid reallocation and unnecessary code complexity, this addition is done by using the last sentry element in the table to create a singly linked list of library tables.)

Since one might possibly want to export C libraries, without having any statically baked into Chibi, an additional configuration macro has been defined, SEXP_USE_STATIC_LIBS_EMPTY, which simply initializes an empty table (or rather, it defines the pointer which would normally have been supplied through clibs.c).

To use the feature, one would do something like:

static sexp add(sexp ctx, sexp self, sexp_sint_t n, sexp s, sexp t)
{
    return sexp_add(ctx, s, t);
}

sexp init_foo(
    sexp ctx, sexp self, sexp_sint_t n, sexp env,
    const char *version, const sexp_abi_identifier_t abi) {
    sexp_define_foreign_proc_rest(ctx, env, "add", 2, add);

    return SEXP_VOID;
}

int main(int argc, char** argv) {
  sexp ctx;
  static struct sexp_library_entry_t entries[] = {
      {"foo", init_foo},

      {NULL, NULL}
  };

  sexp_scheme_init();
  sexp_add_static_libraries(entries);

  ...
}

One can then either (load "foo") or (include-shared "foo").

By the way, note that, contrary to the documentation, both in comments in features.h and on the web page, defining SEXP_USE_STATIC_LIBS does generally not cause clibs.c to be statically included. For some reason, it only does so on Plan 9. That's easy enough to "fix", but not knowing why it is that way, I've let it as is.

[ashinn]

Could you fix the indentation here, and either use braces for both for's or neither?

[dpapavas]

I assumed that by "indentation" you meant the 4 spaces inadvertently used inside the if block. If you'd like me to fix something else, let me know. As far as the for statements are concerned, note that I cannot use no braces for the first (as it contains two statements) and also that the second is empty; it merely advances the entry pointer to the end of the list. Do you want me to add an empty pair of braces to it?

[ashinn]

Oh, actually I misread this, I thought the if was inside the for block. It's better the be clear and put the semi-colon on a separate line:

    for (entry = &table[0]; entry->name; entry++)
       ;

[dpapavas]

Done.

[ashinn]

Can you add a comment that libraries must outlive all Chibi uses?

[dpapavas]

I also added a few words to explain its use, as it will be tricky for prospective users to figure it out by inspecting the code. Let me know if this is not desirable.