[Julep/WIP] Standalone AOT compilation mode #32273
Conversation
There was a problem hiding this comment.
Having a working wasm target would be amazing ❤️
Here's a few naive impressions. Hopefully they are more useful than distracting (I am not an expert on this part of the code).
About jl_apply_generic — what might you hope for this to do? Would it be acceptable to embed the julia IR for such functions and run it in the interpreter? I suppose I'm a bit confused about overall aim here, other than "make wasm work well". Is the goal to
- Avoid runtime codegen?
- Avoid paying for the size of the standard sysimage?
- Allow embedding a sysimage?
- Avoid paying for the size of libjulia?
| // } | ||
| Module *M = data->M.get(); | ||
| Function* init_lib_f = cast<Function>( | ||
| M->getOrInsertFunction("init_lib", Type::getVoidTy(Context), NULL)); |
There was a problem hiding this comment.
Should have a jl_ prefix? jl_init_lib()?
| return LLVMNativeCode(native_code) | ||
| catch e | ||
| ccall(:jl_clear_standalone_aot_mode, Nothing, ()) | ||
| throw(e) |
There was a problem hiding this comment.
(1) Use rethrow() rather than throw(e). The latter will duplicate the exception on the exception stack. (2) But instead, you could just put the call to jl_clear_standalone_aot_mode in a finally block (3) Instead of both of those... this global setting seems kind of icky anyway - is it possible to put it in CodegenParams?
There was a problem hiding this comment.
Putting it in CodegenParams is probably the way to go. Will work on it.
| @@ -832,6 +832,180 @@ void _julia_init(JL_IMAGE_SEARCH rel) | |||
| jl_install_sigint_handler(); | |||
| } | |||
|
|
|||
| void jl_init_types2(void) JL_GC_DISABLED | |||
| { | |||
| jl_module_t *core = NULL; // will need to be assigned later | |||
There was a problem hiding this comment.
Is this still TODO or is "later" elsewhere in the diff? What's special about this set of types and how does it relate to the init cycle? Could we handle them in a way which is more similar way to the usual system?
There was a problem hiding this comment.
I should remove the comment. It's confusing. Later is somewhere else. These types are created in base/boot.jl and then assigned in C in init.c during post_boot_hooks. Without a system image, none of that happens, so this extra init step fixes up a few more basic types.
| jl_perm_symsvec(1, "msg"), jl_svec(1, jl_string_type), 0, 0, 1); | ||
| } | ||
|
|
||
| // Basic initialization that doesn't load a system image |
There was a problem hiding this comment.
There seems to be some duplicate logic going on here. Could you generalize _julia_init to take the sysimage via a resource interface rather than expecting to find it in a file? Then various resource loaders could then be plugged in, for example:
- In the usual case, loading from file via some search paths
- An embedded binary blob like your mini sysimage
- Loading it over the network (maybe this could have benefits for wasm, or maybe it's just crazy talk.)
It might allow you to avoid duplicating the init logic so much?
There was a problem hiding this comment.
There is some duplicate logic, but I tried to minimize that by using the code in 'dump.c'. Your interface idea is interesting. I don't see how it minimizes duplication, though.
There was a problem hiding this comment.
Well I noticed that jl_init_basics shares some 100 lines of code with _julia_init. It seems like this could possibly be factored back together with some more flags or factored apart by extracting some of the shared code.
There was a problem hiding this comment.
Yeah, "factored apart" might be best. I'm worried about too many if (standalone_aot_mode) statements.
I'm still struggling with the resource interface. I understand what you mean, but I'm not sure how to code it, yet.
| void hello(); | ||
| ]] | ||
| lib.init_lib() | ||
| lib.hello() |
There was a problem hiding this comment.
This is cool. Ultimately better to do this test from C though?
There was a problem hiding this comment.
Yes. I'm not really sure how C tests fit in with Julia's testing infrastructure, though (not that Lua helps with that--Lua was just easy to try).
There was a problem hiding this comment.
There's the test/embedding directory which seems quite similar in concept.
| @@ -239,6 +331,113 @@ static void makeSafeName(GlobalObject &G) | |||
| G.setName(StringRef(SafeName.data(), SafeName.size())); | |||
| } | |||
|
|
|||
| bool isinlibjulia(std::string name) { | |||
There was a problem hiding this comment.
I'm not quite sure what's going on here, but this and jl_name_from_type look fishy :-)
There was a problem hiding this comment.
By which I mean - having these lists written out makes me think "there must be a better way" ;-)
Running in an interpreter might be an option. I'm not sure how complex that would be to handle or how much overhead it would add. For now, I'm planning to just not support it.
Yes.
Yes.
No. There's a mini image embedded that holds global variables, but it doesn't hold code, and it's very limited.
Maybe. |
|
I just wanted to say thank you for looking at this - this could immensely expand where I could use Julia (e.g. at work we were discussing the difficulty of using Julia in AWS Lambda functions; smallish precompiled binaries would make this feasible, same for responsive CLI tools). |
|
Looking more at this, I still feel like the concepts of sysimage and mini sysimage might not really be that different and could share more code.
What are the essential points of difference, and can we make things neater by closing the gap a bit? For example
|
|
You're right on about the overlap, @c42f. I'll spend some time looking for ways to bridge the gap. |
| dt = jl_uint32_type; | ||
| } | ||
| else if (dt->size == 8) { // change the type to a UInt64 | ||
| dt = jl_uint64_type; |
There was a problem hiding this comment.
I don't really understand this. Saving the wrong type tag for something doesn't seem useful?
There was a problem hiding this comment.
It was useful in the sense that it could make some code compile and run where it wouldn't compile before. The wrong type is often not a problem because the compiled code doesn't really use the type information, it just needs to get the size right (I know that this isn't always true, so it'd be nice not to do this). I was having problems where saving some types would cause the mini-image to explode in size as it tried to pull in more dependent modules and types. That is likely a sign of another problem, and maybe (hopefully) this is just a temporary band aid, but I haven't found the right approach.
| dt = jl_uint64_type; | ||
| } | ||
| else if (dt->size > 0) { // change the type to a primitive type with correct size | ||
| dt = jl_new_primitivetype(jl_symbol("BitsTypeX"), jl_core_module, jl_any_type, jl_emptysvec, dt->size * 8); |
There was a problem hiding this comment.
If this code doesn't have a fixed repertoire of types --- such that it can handle this new primitive type --- then why not just save the correct type to begin with?
|
I don't understand the notion that |
I think of it more as "can't easily work" (at least by me at my state of understanding). I don't understand how to generate or store the table and the functions it points to. I don't think the functions that are pointed to are compiled as part of |
We already do that in the system image, so it's possible... |
|
Let me address some of the goals of this:
There are three nascent mechanisms related to this that could be developed further:
https://www.youtube.com/watch?v=4NHJqGA6fTw The easiest way to do this currently is to remove the stdlibs from the sysimg build (base/sysimg.jl). To improve further, I suspect we need some kind of tree-shaking mechanism that tries to remove everything that won't be used at run time (e.g. global bindings that are never referenced). Of course that can't work in general (e.g. if a program calls |
|
On avoiding the runtime codegen, @Keno did all that in julia-wasm. Regarding size, the cut-down "PackageCompiler" approach is interesting. It's not clear to me how the tree shaking would work. Another issue with that is if/how it would support cross-compilation. With the |
|
Closing as core developers have suggested that a better approach is through a PackageCompiler / static compilation approach. A key to small code size will be the tree shaking. |
|
Thanks for meta-shaking the AOT compilation tree anyway! |
|
@tshort can that path also potentially support minimal runtime targets like WASM and embedded ? |
|
WASM is not necessarily a minimal runtime target. In any case, that'll be the right place to start. We can come back here and any missing features to julia as necessary. |
This mode of compilation aims to statically compile Julia code to libraries or executables that do not need a system image. This will allow Julia to support more use cases:
To support these modes, the following compilation targets could be supported:
libjuliashared library.libjuliashared library.libjuliashared library.In addition to these, we'd also like to support these same targets, but statically link to
libjulia.afor smaller standalone executables or libraries.My main interest is compilation to WebAssembly (see this issue). See here for a simple web app compiled using this branch of Julia.
Approach
This is based on @vtjnash's work on jn/codegen-norecursion. That capability will be great to have for codegen work. Hopefully, that can be merged soon.
This approach works by introducing a
standalone-aot-modeinto Julia's code generation process. This is similar to theimaging-mode. The main differences are:ccall--foreigncall's normally are converted to calls to function pointers. Instandalone-aot-mode, these are compiled to normal external function calls to be resolved at link time.cglobal-- As withccall's, these are compiled to normal external references.Global variables -- This is a tricky part. Global variables (symbols, strings,
and Julia global variables) are serialized to a "mini image" (a binary array). An
initialization function is provided to restore the global variables upon startup. The serialization code reuses the machinery in "src/dump.c". Some non-core structs and types are converted to tuples or other types that have the same memory layout.
Initialization -- This is another tricky part. Initialization includes a
simplified version of
jl_initthat does not load the standard library. Itinitializes many types, including some defined in
base/boot.jl.Miscellaneous notes
Generic code that uses
jl_invoke()orjl_apply_generic()isn't supported. A warning is currently issued for code that is compiled with either of these. This often includes error-handling code.cfunctionisn't supported. I'm not sure how to handle that.The tests target Linux. The tests currently use
julia-debug.There's a garbage-collection bug lurking somewhere. For at least the
rand()test, it crashes unless GC is disabled.Feedback / next Steps
I'm looking forward to guidance on steps needed to get this into Julia as an experimental feature. This includes tests and code cleanups. If anyone one sees any big gotcha's or problems with the approach, that discussion would help, too.