Added OpenBSD support.

[ibara]

Hi --

As asked, here is a PR that adds OpenBSD support.
Must be built with default_pic=true.

Have not really done any runtime testing, but it passes the build test so that's a strong start.

[slfritchie]

Hi, Brian. Many thanks for the PR. I've a couple of questions.

1. Would you mind adding a subsection to the README.md file that specifies the minimum dependencies required for the OpenBSD release, compiler toolchain, libraries (like pcre2), and any other important bits? I'm a long-time FreeBSD user but only very occasional OpenBSD user ... and I've stumbled a bit trying to vet your PR.

2. I've encountered the following compilation error at commit 3c5e4fa.

ponyassert.c
src/libponyrt/platform/ponyassert.c:43:45: error: implicit conversion loses
      integer precision: 'stack_depth_t' (aka 'unsigned long') to 'int'
      [-Werror,-Wshorten-64-to-32]
  char** strings = backtrace_symbols(addrs, depth);
                   ~~~~~~~~~~~~~~~~~        ^~~~~
1 error generated.

I started with a Vagrant-managed VM using the SierraX/openbsd-6.3 box. Then I used the following commands to install stuff. I'm not sure yet if I've omitted any necessary stuff.

pkg_add git
pkg_add gmake
echo We hope (?) the below command installs LLVM version 5.0.something
echo The 'pkg_info -Q llvm' command for me says 'llvm-5.0.1p2'
pkg_add llvm pcre2 
echo We need libexecinfo for /usr/local/include/execinfo.h
pkg_add libexecinfo

... then cloned your repo, checked out the master branch and hit the error with both gmake default_pic=true and gmake default_pic=true config=debug.

[ibara]

Hi --

There's a type mismatch in ponyassert.c -- fixed now. I should add the amended build command that I use on OpenBSD/amd64:

gmake default_pic=true bits=64 verbose=true

OpenBSD getconf doesn't have support for LONG_BIT.

Please do not use OpenBSD 6.3 to test Pony. All development on OpenBSD is done on -current, and there will never be an OpenBSD-supported package of Pony for 6.3 (6.4 hopefully). We simply don't have the kinds of policies FreeBSD has.

As an extension of this, declaring library versions for us doesn't make a whole lot of sense. The only way I could offer support is if a user was running -current or a release with a pony package, and therefore be using the dependencies as obtained through the package manager. Which at the moment on -current is llvm-6.0.1, pcre2-10.31, and our version of libexecinfo. I expect these to be the same for the OpenBSD 6.4 release.

By the same token, there is only one compiler toolchain: at the moment, on amd64, armv7, and i386 it is clang-6.0.0 with our GNU ld (which will eventually move to LLVM lld); on arm64 it is clang-6.0.0 with LLVM lld.

I have also given the instruction (aimed at end-users) to use the package over building pony oneself. This is because the standard instruction to compile and run helloworld will not work on OpenBSD without manual tweaking (see src/libponyc/platform/paths.c) unless the package is used (or you manually install to /usr/local).

[slfritchie]

Hrm. I (begin edit) switched to the v2018.07.11 snapshot (end edit) of the Vagrant box/image/thingie 'SierraX/openbsd-snapshot' and was able to compile ponyc. However, installation & executing the compiler get stuck.

Using gmake default_pic=true bits=64 verbose=true install ponydir=/usr/local created a symlink that points to itself:

Makefile:314: WARNING: LLVM 6 support is experimental and may result in decreased performance or crashes
cp build/release/lib/native/libponyrt.a /usr/local/lib/native
cp build/release/lib/native/libponyc.a /usr/local/lib/native
cp build/release/ponyc /usr/local/bin
cp src/libponyrt/pony.h /usr/local/include
cp src/common/pony/detail/atomics.h /usr/local/include/pony/detail
cp -r packages /usr/local/
ln -sf /usr/local/bin/ponyc /usr/local/bin/ponyc             <--- Oops
ln -sf /usr/local/lib/native/libponyrt.a /usr/local/lib/libponyrt.a
ln -sf /usr/local/lib/native/libponyc.a /usr/local/lib/libponyc.a
ln -sf /usr/local/include/pony.h /usr/local/include/pony.h
ln -sf /usr/local/include/pony/detail/atomics.h /usr/local/include/pony/detail/atomics.h

If I remove that symlink and then use cp ./build/release/ponyc /usr/local/bin/ponyc, then I get a new error when trying to compile something:

openbsd-snapshot-63# ponyc ./examples/helloworld/                             
ponyc:/usr/local/lib/libLLVM-6.0.so: ponyc : WARNING: symbol(_ZTVNSt3__110__function6__funcIPFvRKN4llvm18PassManagerBuilderERNS2_6legacy15PassManagerBaseEENS_9allocatorISA_EES9_EE) size mismatch, relink your program
Error:
builtin: couldn't locate this path

[ibara]

I'm sorry, I don't know much of anything about Vagrant. But a quick Google search suggests that the v2018.7.11 thingy is -current. I thought you were on 6.3 because you said in your previous message that pkg_add llvm installed llvm-5.0.1p2.

The linker warning should be safe to ignore (famous last words...).

The not being able to locate the path is something I still have to work on; I haven't yet gotten around to making correct. I'm using a hacked version that just has the path for OpenBSD as ./build/release (I realize the issues with this). I also found an unstaged commit that I just pushed (make sure to link with -lc++abi on OpenBSD -- I'm unsure that the way I did it is the best way).

It seems like figuring out the paths issue will be the most difficult and the last push, as OpenBSD does not support getting a path at runtime. Unfortunately, I don't have much time the rest of the week to get to it :( I can confirm that with my frankenbuild, I can in fact build a working examples/helloworld program, which I'm attaching here as a demonstration of the compiler working (I guess it might not run on your hardware due to -march=native, but you can at least run it through objdump).

helloworld.gz

[slfritchie]

@ibara I think that this hack commit takes care of the path problem ... assuming I haven't done something insane with pointers.

slfritchie@379909f

[ibara]

This appears to do the trick.

[ibara]

Hi --

What can I do to move this along?

Thanks.

[SeanTAllen]

Sorry for delay @ibara. Most of core team and committers are off on vacation or work trips etc this month. There's been a lull in reviewing commits, issues etc.

I just restarted the failed build for this that looked like a transient error.

Question for you: we are going to be vendoring LLVM into Pony at some point as supporting multiple versions is problematic for us. One of your comments made me think that might be problematic with OpenBSD, can you comment? Would it be? Does OpenBSD maintain its own LLVM port?

[ibara]

No worries. I just believe in respectfully checking in once in a while to get what you want done :)

OpenBSD does maintain its own LLVM package. We are fairly aggressive at keeping our LLVM package up-to-date; usually a new upstream LLVM release ends up in our packages within a day. This is mostly due to the great work of our developers who are also involved with the LLVM project.

However, we do not keep around older versions of LLVM; once an update is made, that's it no going back. That means that today, our LLVM package is version 6.0.1. And whenever 7.0.0 comes out (September 5, according to the LLVM homepage), we'll move to that in short order.

I realize that might cause problems for you all. As I understand it, Pony is still dependent on LLVM 3.9, with only experimental support for later versions. As you are moving towards vendoring LLVM, there are I guess two options to consider pursuing:

1. Leave an option available for users to continue to compile against a separately packaged LLVM, at their own risk and reduced or no support from you all. And then using OpenBSD as an experimental side for Pony against newer LLVM releases. This would have the advantage of reducing the labor involved when you decide to upgrade your internal LLVM, but at the expense that things might break in either direction, preferring of course to allow breakage on the experimental side.

2. Working with downstream packagers to incorporate our local changes into your internal version of LLVM. I know we're far from alone in terms of traveling with modifications to LLVM. The upside here is that all downstreams are happy. The downside is that is likely will increase your labor every time you upgrade your internal LLVM, since the process will have to be repeated every time, but it might also give you an idea as to the platforms that are actively maintaining a Pony package, so it might balance out for you.

[winksaville]

Option 1 is what Rust supports and what my PR #2748 also supports.

[ibara]

Not pushing; just pinging. Otherwise it'll fall off of my recent activity list and I'll be less likely to remember it :)

[slfritchie]

Hi, Brian. The ping is ok & good. The last couple of Pony developer sync meetings have been cancelled due to summer holidays & biz travel. With luck, they will resume today.

[SeanTAllen]

This looks good to me.

[SeanTAllen]

@winksaville @ibara sounds like option 1 is the right approach.

[ibara]

I think that slfritchie@379909f should get incorporated too. But maybe it's a two-step commit.

[slfritchie]

@ibara Agreed, that patch should be added here. If you could append it to this PR, then @SeanTAllen is ready to merge it. The LLVM package management is a detail out of scope of this PR, unfortunately, but it's slowly chugging its way through Pony's process.

[ibara]

Merged in. We should be good to go once the tests check out.

[ibara]

Friendly weekly-ish ping. Btw, if I can help with the CI stuff mentioned on that other thread, lmk too.

[jemc]

Thanks, and sorry for the unresponsiveness - many of us have been swamped with other commitments recently.

[SeanTAllen]

@ibara I'm trying to draft release notes for this but don't feel particularly qualified. Is there anything in particular about this that should be communicated to the OpenBSD community aside from "support added"?

[ibara]

@SeanTAllen Not particularly. Soon they'll be able to pkg_add ponyc to get a pre-built package. But I haven't created one yet--it's on my todo list for this week.

Actually, now that I think about it, what do you all prefer as the name for a pre-built package? pony? ponyc?

@jemc No worries--it's summer! I get it :) I appreciate this getting in at all, so the wait didn't bother me.

[ibara]

@SeanTAllen I went ahead and made a package, calling it ponyc: https://marc.info/?l=openbsd-ports&m=153611679810213&w=2

I anticipate committing it to our package repository by the end of the week, so you could go ahead and mention that there is a package available in the OpenBSD package repository. The command to get the package will be pkg_add ponyc.

[SeanTAllen]

Thanks @ibara.

[ibara]

And committed! This will propagate as a package in a day or two, and it means that there will be a package of Pony for the OpenBSD 6.4 release.
https://marc.info/?l=openbsd-ports-cvs&m=153617109128366&w=2