[WIP] Add Base.build_sysimg()

[staticfloat]

This is basically just a julia translation of the makefile targets for generating sys.$(dlext) from sys.ji. I'm sure there are border cases where this won't work, but I've tested it successfully on 64-bit Linux and OSX 10.9.

My thinking behind this is that we can then ship a sys.ji that has been created with .setMCPU("x86"), or .setMCPU("core2"), or whatever, but after we've started up, looked around, and decided that it is possible to regenerate the system image, we can build a native system image that should hopefully be faster.

@Keno (nice!), @vtjnash, @JeffBezanson what do you guys think of this? If this general approach is looked upon with favour, I will add on to this PR with commits to make the .setMCPU() calls a little more dynamic. As it stands now, if we distribute a .setMCPU("i386") binary, even if we rebuild the system image like this, we don't get any benefit, since the binary already has these restrictions baked in.

In the meantime, this branch can be tested out by installing Julia, (e.g. make install prefix=/my/temp/prefix), deleting sys.{so,dll,dylib}, and then opening Julia and running Base.build_sysimg(). This should "just work", or print out intelligent errors when something doesn't work. I think something like this would be much nicer for our "power users" than manually mucking around with system images and such, and even allows for users to play around with their userimg.jl to get package insta-loading, regardless of whether they built from source or not.

[crayxt]

What if system image files installed in read-only location (for user), say in Linux, and being managed by package managers? I think this question was asked already.
Put a new sysimg file to $(HOME)/.julia and prefer it over the system one?

[vtjnash]

why does this need a full-blown CXX if it is only using it as a wrapper for ld (link.exe)? it should probably also go into interactiveutils.jl. otherwise, this seems useful

this should probably be a future pull-request, but it would also be cool to have it make a .exe instead of a .dll (with the appropriate main function, and modifying dump.c as needed)

[tkelman]

Would be nice if building the .ji sysimg can be performed regardless of whether or not a compiler/linker can be found. Only a handful of Windows users can use this otherwise. There is the .bat file for this, but would be cool if as much as possible can be done through the same command on all platforms.

[JeffBezanson]

This is great but looks like it should be an external script. You don't really need to be able to call it as part of the standard library.

[vtjnash]

This is great but looks like it should be an external script. You don't really need to be able to call it as part of the standard library.

Maybe a package? I feel this could be easier to maintain in julia, since you don't have to worry about bash-isms and cmd-isms.

Would be nice if building the .ji sysimg can be performed regardless of whether or not a compiler/linker can be found.

The llvm lld project is still a WIP, however, shipping ld wouldn't be that bad (much, much smaller than shipping a compiler)

[tknopp]

@vtjnash: Whats the size of ld and does it come with any extra dependencies?

We cannot use the pure .o files to reload the system image right?

It seems that this is a very crutial thing to be solved in order to allow precompilation of arbitrary packages on the user side. So if shipping a linker is not too much of a problem I think it is absolutely justified. Or is there some other masterplan that I missed how to enable package precompilation?

[staticfloat]

What if system image files installed in read-only location (for user), say in Linux, and being managed by package managers? I think this question was asked already.
Put a new sysimg file to $(HOME)/.julia and prefer it over the system one?

You can build the system image wherever you want, so you could put it in ~/.julia, and then call julia via julia -J ~/.julia/sys.ji to load ~/.julia/sys.so at runtime. You'd have to pass the -J option to julia at every invocation however, and you'd need a way to ensure it gets called when other programs invoke julia as well, such as IJulia, parallel workers, etc...

why does this need a full-blown CXX if it is only using it as a wrapper for ld (link.exe)?

Good point; I've switched it over to ld/link, which seems to work just fine.

this should probably be a future pull-request, but it would also be cool to have it make a .exe instead of a .dll (with the appropriate main function, and modifying dump.c as needed)

You're thinking ahead a few steps and wanting to make stand-alone Julia programs that are simply linked against libjulia and have precompiled code cached in them? That would be truly sweet.

Would be nice if building the .ji sysimg can be performed regardless of whether or not a compiler/linker can be found.

Julia won't startup without a .ji system image. ;) This PR allows us to bootstrap from a non-optimized system image to an optimized system image. (Assuming that I get the .setMCPU() stuff sorted out). For windows, we either need to distribute link.exe or we just live without this. (Which is no worse than what we already have)

This is great but looks like it should be an external script. You don't really need to be able to call it as part of the standard library.

It certainly seems that way right now, but I have a couple of reasons for not wanting to make it an external command:

1. We'd have to maintain two separate copies, one for windows, one for unix, since we don't have any way other than Julia of distributing shell code like this that runs on both

2. Ideally this would be something that Julia checks for on startup or install and alerts the user that such an operation can be done to reduce startup times

3. I don't pretend to fully understand the import of --build mode, but it seems to me the way this is implemented right now could be simplified if it was directly callable from within Julia. It would be really neat if I could take modules and dump them straight into object files. Something like code_llvm, but taken all the way down to native code, organized in .o format and made independent enough from the state of the current runtime that it can be loaded by julia later. That's probably so far off that this PR doesn't even need to think about it, but it's on my wishlist regardless. ;)

[tkelman]

Julia won't startup without a .ji system image. ;) This PR allows us to bootstrap from a non-optimized system image to an optimized system image. (Assuming that I get the .setMCPU() stuff sorted out). For windows, we either need to distribute link.exe or we just live without this.

I was thinking this could also replace the use case of prepare-julia-env.bat, of changing some of the .jl code in base and needing to rebuild the .ji sysimg

[tknopp]

Is there some drawback to use the run command for creating the .o file? Alternatively one could make this via ccall. Not sure how much functionality in this regards is in repl.c that would have to be replicated.

[JeffBezanson]

1. We'd have to maintain two separate copies, one for windows, one for unix, since we don't have any way other than Julia of distributing shell code like this that runs on both

The implementation in julia is totally fine, I just want to run it as ./julia build_sysimg.jl. I don't want the code inside the standard library; this isn't something you need to call all the time during run time.

[staticfloat]

The implementation in julia is totally fine, I just want to run it as ./julia build_sysimg.jl. I don't want the code inside the standard library; this isn't something you need to call all the time during run time.

Ah, I see. I agree; I'll put it in contrib/ for now.

[staticfloat]

Alright, I've pushed a new version of the commit up. I'm going to copy-paste the commit message here:

Support building of system image in binary builds

This commit adds a few new pieces of functionality:

• The contrib/build_sysimg.jl script which builds a new system image. This method can save the system image wherever the user desires, e.g. it could be stored in ~/.julia, to allow for per-user system images each customized with packages in their own userimg.jl, for instance. Or on a restricted system, this allows for creation of a system image without root access.
• The removal of compile-time JULIA_CPU_TARGET, in favor of runtime --cpu-target/-C command-line flags which default to "native" but can be set to "native", "i386" or "core2". This allows the creation of a system image targeting user-supplied cpu features, e.g. cd base; ../julia -C i386 --build /tmp/sys_i386 sysimg.jl.
• I implemented runtime selection of the cpu target by adding a new member to the jl_compileropts_t structure called cpu_target.
• Because all julia executables are now created equal, (rather than before where a julia executable needed to have the same JULIA_CPU_TARGET set internally as the system image had when it was built) we need to know what CPU feature set the system image is targeting before we initialize code generation. So a new function jl_get_system_image_cpu_target() is exported, which does exactly what it sounds like.
• I added newlines to the end of a few error messages.
• I found an old parser option -T which hadn't been removed yet, so I took the opportunity to do so.

When testing this change out, I found this gist helpful to put into my ~/.juliarc.jl

[staticfloat]

If I had to give one criticism about this however, it's that I've been unable to find an instance where the difference between an i386 system image and a native image is measurable. None of the benchmarks seem to be particularly affected by this, which worries me a little bit. This includes master; I can't measure the difference between a julia system image that was built with "native" and one that was built with "i386". Sic'ing analyze-x86 on both system images, it seems that the native image doesn't use anything beyond sse2, which surprises me:

$ analyze-x86 usr/lib/julia/sys.dylib                                                                                                           
instructions:
 cpuid: 0        nop: 51300      call: 0         count: 583151
 mmx:    295090
 sse:    10202
 sse2:   832

$ analyze-x86 usr_native/lib/julia/sys.dylib 
instructions:
 cpuid: 0        nop: 51223      call: 0         count: 582979
 mmx:    295065
 sse:    10202
 sse2:   832

I even added in support for avx instructions (at least, I tried. I can't seem to find an easily digestible list anywhere outside of LLVM's testing code) to analyze-x86 and tested it on libopenblas.dylib, which seems to show more of what I expected:

$ analyze-x86 libopenblas.dylib 
instructions:
 cpuid: 3        nop: 19084      call: 0         count: 2474957
 i586:   3
 i686:   1435
 mmx:    637899
 sse:    190614
 sse2:   129730
 sse3:   612
 avx:    59051

It also surprises me that the i386 system image contains sse2 instructions, but I guess that could just be code that never gets called.

[nalimilan]

This sounds really great, but I don't understand what happens by default: is a native image built and loaded automatically on subsequent starts? Do you need to build the image manually?

As I see it, most people are going to use distribution packages, in which case it is great to be able to build specialized images in addition to the i386 one. Such images could be automatically built for the native architecture on the first run, and saved in ~/.julia/. But since one can use several computers, or change computer, I think it would make sense to give each image a name identifying the architecture it corresponds to, so that you can have several images in parallel, the best one being loaded automatically. Sysadmins and distributions could even choose to ship a few standard architectures in system directories.

(But indeed the fact that i386 uses SSE2 and native does not use more recent instruction sets makes the whole thing kind of pointless at the moment. ;-)

[Keno]

Do you have a Haswell CPU? If so, you're seeing #7155

[vchuravy]

It might be that an empty target string defaults to i386 as a lowest common denominator and not to native.

[staticfloat]

The empty string is what we were passing before, but I'm definitely not knowledgable about LLVM, so if you've got documentation you can share, I'd love to go over it. I can't find much about setMCPU(), but then again I don't really know where to look.

[vchuravy]

I tried to find some reliable documentation and the only thing I know is that clang defaults to i386 if no -march and -mcpu is given.

[Keno]

This isn't clang though. This is MCJIT, which defaults to cpuid (from TargetSelect.cpp):

  Triple TheTriple(TargetTriple);
  if (TheTriple.getTriple().empty())
    TheTriple.setTriple(sys::getProcessTriple());

[vchuravy]

But isn't TargetTriple -march?

In trunk MCPU is just passed through to createTargetMachine (compare https://github.com/llvm-mirror/llvm/blob/0b6cb7104b15504cd41f48cc2babcbcee70775f3/lib/ExecutionEngine/TargetSelect.cpp#L100)

And the question is what the backend is going to do with it. I haven't found the implementation for that yet.

And the TargetTriple just makes sure that the right bitcode is created for x86_64 vs x86 and linux vs mac vs windows. While MCPU sets the capabilities of the CPU.

[vchuravy]

But that's my point with MCPU we are setting the CPU and not the target triplet and also not MARCH.

I think we probably should set it like here https://github.com/llvm-mirror/llvm/blob/master/unittests/ExecutionEngine/MCJIT/MCJITTestBase.h#L328

[staticfloat]

@vchuravy if you can give me example lines to put in, I'm willing to experiment. We've already got some pretty good evidence that the .setMCPU("") call is causing AVX instructions to be emitted (whereas .setMCPU("i386") is stopping at sse2) but perhaps there are other considerations to be taken into account.

[ArchRobison]

As far as I can tell, .setMCPU(sys::getHostCPUName()) has the same effect as .setMCPU(""). Indeed, it's when I discovered that sys::getHostCPUName() was returning a generic x86 string for a "Haswell" processor that I worked on implementing #7155.

[Keno]

Yes @ArchRobison is correct. If you still don't believe me, I'd encourage you to step through the function you referenced in a debugger to see the control flow.

[vchuravy]

I do see an effect.

~/src/analyze-x86/analyze-x86 sys_i386.o
instructions:
 cpuid: 0        nop: 42077      call: 0         count: 490683
 i486:   1
 i686:   1104
 mmx:    7625
 sse:    8915
 sse2:   482
~/src/analyze-x86/analyze-x86 sys_native_pre.o
instructions:
 cpuid: 0        nop: 264        call: 0         count: 457087
 i486:   1
 i686:   1105
 mmx:    7625
 sse:    8915
 sse2:   482
~/src/analyze-x86/analyze-x86 sys_native_post.o
instructions:
 cpuid: 0        nop: 258        call: 0         count: 459942
 i486:   1
 i686:   1107
 mmx:    7376
 sse4.2:         3
 avx:    9720

Where I applied the following change to codegen.cpp in L4330

 if (strcmp(cpu_target, "native") == 0)
        cpu_target = sys::getHostCPUName().data();

The sysimages where generated by

../julia -C i386 --build /tmp/sys_i386 sysimg.jl
../julia -C native --build /tmp/sys_native_pre sysimg.jl

Applying change and rebuilding julia

../julia -C native --build /tmp/sys_native_post sysimg.jl

I am on a second generation i5 (Sandybridge???) and my architecture should be something like corei7-avx.
So for me there is a definite change when I specify my host architecture and I am also seeing llvm defaulting to i386.

Edit: Oh and I am on llvm trunk, if that matters.

[staticfloat]

@nalimilan What happens by default, (e.g. if you run build_sysimg() from julia with no arguments, or if you run julia build_sysimg.jl) is if there is no system image already loaded, and if the default location (right next to libjulia) is writable, and if we can find a linker to do our dirty work for us, then we start to do the system image generation process. The defaults ensure that when we're done, if you start up Julia she will automatically find the system image at startup, due to it being placed next to libjulia. (This is because of how Julia's RPATHs are setup) If you place it somewhere else, you have to pass the -J argument to tell Julia where to look.

That's the main problem with shipping multiple system images right now; because you will have to supply the -J argument every single time Julia starts up; there's no easy way to bake that string into the executable, and having the executable load up a configuration file before it's even initialized codegen sounds like too much intelligence too early on in the startup phase to me.

@Keno doh! I read that issue, and I was like "Haswell..... that seems pretty recent, so that can't be what I have!". Turns out it is! I applied ArchRobison's patch, rebuilt LLVM and I now get more interesting results from analyze-x86:

$ analyze-x86 usr_i386/lib/julia/sys.dylib 
instructions:
 cpuid: 0        nop: 51225      call: 0         count: 582981
 mmx:    295065
 sse:    10202
 sse2:   832
$ analyze-x86 usr_native/lib/julia/sys.dylib 
instructions:
 cpuid: 0        nop: 819        call: 0         count: 538671
 mmx:    294592
 avx:    11779

Unfortunately, I still don't see much of a difference in benchmarks. The only one that has any kind of difference is kernel, where i386 took a total of 44 seconds, and native took a total of 41 seconds, but I somehow thought that the difference would be greater.

[nalimilan]

@staticfloat Images could be called sysimg-$CPUTARGET.so or something like that. Julia could simply look for these files in RPATHs or in ~/.julia, without reading configuration files at all, so it could be done very early on startup.

[ViralBShah]

Or we could try and do what OpenBLAS does for dynamic arch. That would probably be a lot more work than having multiple libraries and loading the right one.

[staticfloat]

Until we know for certain that there is a measurable performance difference between cpu targets in the system image, this is all moot. If we can ship an i386 image and get the same performance as a native image, there's no reason to go through the work to create a dynamic arch system image, or even multiple system images that get chosen at boot time.

[nalimilan]

I'm not sure I understood you correctly, but from the discussion above I got the feeling that the CPU target of the image determines the instruction sets that are enabled when compiling all further code. This would mean that with an i386 image people would not be able to make use of AVX, which would be quite bad, even if for the system image it doesn't make a big difference.

[nalimilan]

BTW, regarding @staticfloat's comment #7124 (comment): the i386 image contains SSE2 instructions only when it is 64 bits, which is great, since it means LLVM understood that it can rely on the fact that all x86_64 CPUs support them. On the other hand, in 32 bits, they are not used (this file comes from my RPM package):

 ./analyze-x86 sys.so 
instructions:
 cpuid: 0    nop: 54218  call: 51772     count: 714504
 i486:   2

(Though strictly speaking i486 instructions should not be used if somebody wanted to run Julia on a true 80386 machine... designed when I wasn't even born. :-).

[staticfloat]

the i386 image contains SSE2 instructions only when it is 64 bits

Ah, that makes perfect sense. I currently have a shortage of 32-bit processors around me, so I didn't notice. ;) I also think that if someone wants to run Julia an an 80386, I will build a custom binary for them myself.

from the discussion above I got the feeling that the CPU target of the image determines the instruction sets that are enabled when compiling all further code. This would mean that with an i386 image people would not be able to make use of AVX, which would be quite bad, even if for the system image it doesn't make a big difference.

Yes, that is what I expect, and what is shown by analyze-x86. However, what I am surprised by is that none of the tests in test/perf/{micro,cat,lapack,shootout} seem to be affected by the lack of AVX instructions. Could it be that we're just not utilizing AVX instructions properly, or do I need to run specifically @simd-based tests or something?

[Keno]

I am surprised by that analysis. When I implemented that option I ran the perf test and noticed about a 20% performance drop on the micro benchmarks when using "core2" as opposed to "native".

[staticfloat]

Here are my results, from which I draw this analysis.

First; build this branch (I've rebased it on top of ArchRobison's LLVM patch, but you'll have to rebuild LLVM to reap the benefits of that) and create three separate system images:

include("contrib/build_sysimg.jl")
f(z) = build_sysimg("/tmp/sys_$z", cpu_target=z, force=true)
f("native"); f("core2"); f("i386")

Next, run the micro/perf.jl file with each of those system images created, recording the time for each:

$ for sysimg in native core2 i386; do time ../../../julia -J /tmp/sys_$sysimg.ji perf.jl; done

---------------------------------------------
cpu_target: native
julia,fib,0.068555,0.069229,0.068705,0.000293
julia,parse_int,0.213825,0.227219,0.221538,0.006827
julia,mandel,0.241870,0.243968,0.242555,0.000821
julia,quicksort,0.425397,0.486585,0.452723,0.026520
julia,pi_sum,45.035978,45.055818,45.045787,0.008299
julia,rand_mat_stat,19.936353,44.477638,26.736889,10.041047
julia,rand_mat_mul,62.308804,74.273467,70.009151,4.511674
julia,printfd,22.798280,25.881647,23.577526,1.297713

real    0m3.772s
user    0m4.428s
sys     0m0.532s

---------------------------------------------
cpu_target: core2
julia,fib,0.068232,0.068770,0.068350,0.000235
julia,parse_int,0.205918,0.222568,0.213497,0.007198
julia,mandel,0.238318,0.243711,0.239645,0.002290
julia,quicksort,0.417476,0.430051,0.423044,0.005748
julia,pi_sum,24.125680,24.345430,24.171405,0.097331
julia,rand_mat_stat,20.870514,39.848925,28.400367,7.962890
julia,rand_mat_mul,67.091589,70.950115,69.821133,1.552278
julia,printfd,23.383307,24.788063,23.895489,0.579523

real    0m3.588s
user    0m4.265s
sys     0m0.514s

---------------------------------------------
cpu_target: i386
julia,fib,0.068220,0.068903,0.068402,0.000286
julia,parse_int,0.205265,0.220214,0.212953,0.006341
julia,mandel,0.236900,0.238627,0.237630,0.000681
julia,quicksort,0.416860,0.428088,0.422285,0.005210
julia,pi_sum,24.136999,26.723711,24.794129,1.117481
julia,rand_mat_stat,22.421148,41.527072,27.763993,7.894704
julia,rand_mat_mul,64.142095,81.025020,73.140812,6.290072
julia,printfd,22.975660,28.273994,24.371769,2.195561

real    0m3.681s
user    0m4.362s
sys     0m0.522s

Note that I have the aforementioned gist as my ~/.juliarc.jl to ensure that the system images are actually the architecture they say they are. To drive this point home further, applying analyze-x86 to the three system images gives the result we would expect:

$ analyze-x86 sys_native.dylib 
instructions:
 cpuid: 0        nop: 822        call: 0         count: 539552
 mmx:    295004
 avx:    11827
$ analyze-x86 sys_core2.dylib 
instructions:
 cpuid: 0        nop: 909        call: 0         count: 539440
 mmx:    295412
 sse:    10247
 sse2:   805
 sse3:   12
 ssse3:  2
$ analyze-x86 sys_i386.dylib 
instructions:
 cpuid: 0        nop: 51220      call: 0         count: 583733
 mmx:    295384
 sse:    10249
 sse2:   832

[staticfloat]

@Keno when you get a free moment, I'd love to hear your input on reasons why I might not be seeing any differences between cpu architectures.

[staticfloat]

Rebased on top of recent tweaks to src/codegen.cpp