Update with experimental support for LLVM 6.0.0

[chalcolith]

This PR includes build system and code changes to support LLVM 6.0.0.

[SeanTAllen]

@kulibali does this work without the workarounds in llvm 4 and 5 support?

[SeanTAllen]

@kulibali do you know how to add support for CI testing of LLVM 6?

[chalcolith]

I haven't done anything on Linux yet, so I haven't tested the workaround or CI for Linux yet. The single AppVeyor failure seems spurious.

I'll look at Linux, including CI, today.

[chalcolith]

Ok, I have copied the CI configuration to make 6.0.0 happen; if I understand it correctly, someone will have to make a container for LLVM 6? Someone who knows Docker much better than me, ideally.

One caveat: LLVM seems to not have generated a binary tarball for Ubuntu 16.04: http://releases.llvm.org/6.0.0/

[SeanTAllen]

@kulibali i think either @jemc or i could create the container for you. and you could update the circleci config to test with 6 as well. Using Ubuntu 16.04 isnt a requirement for setting up the CI. If you check, you'll see we use different uses of Ubuntu for the various LLVM versions.

https://github.com/ponylang/ponyc/tree/master/.ci-dockerfiles

[SeanTAllen]

you can ditch these commented out lines

[chalcolith]

Hmm, I thought I had made and pushed a Docker image for LLVM 6; it seems I didn't do it right.

[jemc]

It looks like the docker image you built may be broken with an owner id issue, as noted by CircleCI.

I'm going to try building it myself to see if it resolves the issue.

[jemc]

The image also was broken when I built it.

I figured out last night that the LLVM archive seems to have been built with a "high" UID/GID, and when extracting tar archives as the root user, tar will keep the original UID/GID of the file. You can apparently override this with --no-same-owner, so I'm going to try that.

[jemc]

@kulibali - I was able to create (last night) and upload (this morning) a working image, using the following Dockerfile:

FROM ubuntu:14.04

ENV LLVM_VERSION 6.0.0

RUN apt-get update \
 && apt-get install -y \
  apt-transport-https \
  build-essential \
  g++ \
  git \
  libncurses5-dev \
  libssl-dev \
  make \
  wget \
  xz-utils \
  zlib1g-dev \
 && cd /tmp \
 && wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre2-10.21.tar.bz2 \
 && tar xjvf pcre2-10.21.tar.bz2 \
 && cd pcre2-10.21 \
 && ./configure --prefix=/usr \
 && make \
 && sudo make install \
 && cd / \
 && rm -rf /tmp/pcre* \
 && wget -O - http://releases.llvm.org/${LLVM_VERSION}/clang+llvm-${LLVM_VERSION}-x86_64-linux-gnu-ubuntu-14.04.tar.xz \
 | tar xJf - --no-same-owner --strip-components 1 -C /usr/local/ clang+llvm-${LLVM_VERSION}-x86_64-linux-gnu-ubuntu-14.04

So I think Linux CI is passing now.

The difference was using --no-same-owner in the tar extraction.

[jemc]

I think this is no longer true? https://github.com/Homebrew/homebrew-core/blob/cd015553ae9c25389f4e35f8a9a9aef0ac2cc733/Aliases/llvm%406

Is this still true?

[SeanTAllen]

yes

[jemc]

Have we figured out whether LLVM 6 also produces this bad code?

[chalcolith]

Yes, there are still problems with LLVM 6 on Linux, but different from 4 & 5. @Praetonus was going to look at it again in his copious free time.

[chalcolith]

I turned on the workaround again for the time being, which is why the CI tests are passing.

[jemc]

It looks like this just needs some OSX attention now.

[jemc]

Here's the error from the Travis OSX build:

src/libponyc/codegen/host.cc:42:45: error: no member named 'JITDefault' in namespace 'llvm::CodeModel'
  CodeModel::Model model = jit ? CodeModel::JITDefault : CodeModel::Default;
                                 ~~~~~~~~~~~^

src/libponyc/codegen/host.cc:42:69: error: no member named 'Default' in namespace 'llvm::CodeModel'
  CodeModel::Model model = jit ? CodeModel::JITDefault : CodeModel::Default;
                                                         ~~~~~~~~~~~^

[chalcolith]

I see a warning for LLVM 5 in the build log, so I think that LLVM 5 is getting installed instead of 6: Makefile:266: WARNING: LLVM 5 support is experimental and may result in decreased performance or crashes

[SeanTAllen]

Yes. We do all the builds on the same OSX instances. We install the different versions one at a time, testing each as we go.

…

On Wed, Apr 4, 2018, 4:44 PM Gordon Tisher ***@***.***> wrote: I see a warning for LLVM 5 in the build log, so I think that LLVM 5 is getting installed instead of 6: Makefile:266: WARNING: LLVM 5 support is experimental and may result in decreased performance or crashes — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2595 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAL0PH5lyOfgg5Nl5-itUFKQmZEVVjm_ks5tlTCTgaJpZM4Slh57> .

[winksaville]

I also just ran into the JITDefault problem that @jemc found. I looked at the llvm sources and found JITDefault was deleted last year with this change hopefully that helps.

[jemc]

Thanks, @winksaville!

So, based on that change, I think what's needed should be something like this:

#if PONY_LLVM < 600
  CodeModel::Model model = jit ? CodeModel::JITDefault : CodeModel::Default;
  TargetMachine* m = t->createTargetMachine(opt->triple, opt->cpu,
    opt->features, options, reloc, model, opt_level);
#else
  CodeModel::Model model = jit ? CodeModel::Large : CodeModel::Small;
  TargetMachine* m = t->createTargetMachine(opt->triple, opt->cpu,
    opt->features, options, reloc, model, opt_level, jit);
#end

[jemc]

\cc @Praetonus for confirmation on the small/large choice I made there with respect to the JIT...

[chalcolith]

That code is in master; my branch is a bit behind. I will rebase it, but why is travis building master and not my branch?

[chalcolith]

OK, I have:

• rebased this branch on master and fixed the compile per @jemc's comment
• updated the LLVM 6 dockerfile with @jemc's changes
• added build system warnings for LLVM 6

[Praetonus]

The change suggested by @jemc looks correct to me, but in case the default code models change in future LLVM versions, I think we should rather do something like this:

TargetMachine* m = t->createTargetMachine(opt->triple, opt->cpu,
  opt->features, options, reloc, llvm::None, opt_level, jit);

With a None argument to the Optional<CodeModel> parameter, the LLVM internals will determine the code model to use based on the value of jit.