Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dashing #18692

Merged
merged 27 commits into from
Nov 28, 2019
Merged

Dashing #18692

merged 27 commits into from
Nov 28, 2019

Conversation

SilasK
Copy link
Member

@SilasK SilasK commented Nov 17, 2019

Bioconda requires reviews prior to merging pull-requests (PRs). To facilitate this, once your PR is passing tests and ready to be merged, please add the please review & merge label so other members of the bioconda community can have a look at your PR and either make suggestions or merge it. Note that if you are not already a member of the bioconda project (meaning that you can't add this label), please ping @bioconda/core so that your PR can be reviewed and merged (please note if you'd like to be added to the bioconda project). Please see #15332 for more details.

  • I have read the guidelines for bioconda recipes.
  • This PR adds a new recipe.
  • AFAIK, this recipe is directly relevant to the biological sciences (otherwise, please submit to the more general purpose conda-forge channel).
  • This PR updates an existing recipe.
  • This PR does something else (explain below).

@SilasK
Copy link
Member Author

SilasK commented Nov 17, 2019

@dnbaker here we go, probably it won't pass the checks in the first time

@dnbaker
Copy link

dnbaker commented Nov 17, 2019

Thanks! I don't full understand how bioconda works, but I'll try installing via conda on my personal machine in the interim.

@dnbaker
Copy link

dnbaker commented Nov 18, 2019

https://github.com/dnbaker/bioconda-recipes/tree/dash

I have some changes, including requiring libomp. But it breaks when I try to install it, for what seems to be a fundamental compiler issue:

ld: warning: ignoring file /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib/libSystem.tbd, file was built for unsupported file format ( 0x2D 0x2D 0x2D 0x20 0x21 0x74 0x61 0x70 0x69 0x2D 0x74 0x62 0x64 0x2D 0x76 0x33 ) which is not the architecture being linked (x86_64): /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib/libSystem.tbd
Undefined symbols for architecture x86_64:
  "___bzero", referenced from:
      _deflateReset in libz.a(deflate.o)
      _deflateSetDictionary in libz.a(deflate.o)
      _fill_window in libz.a(deflate.o)
      _deflateParams in libz.a(deflate.o)
      _deflate in libz.a(deflate.o)
      _gz_write in libz.a(gzwrite.o)
      _gzputc in libz.a(gzwrite.o)
      ...
  "___sprintf_chk", referenced from:
      _gzdopen in libz.a(gzlib.o)

Maybe my libSystem is too new or too old?

@SilasK
Copy link
Member Author

SilasK commented Nov 18, 2019

@dnbaker Sorry I didn't use your fork. I'm working from https://github.com/bioconda/bioconda-recipes/tree/dash

I copied your recipe because I don't have access to it. I added the c++ compiler:
{{ compiler('cxx') }}

See if it passes the checks.

@SilasK
Copy link
Member Author

SilasK commented Nov 18, 2019

Do you have a list of requirements for the building process?

  • You were talking about omplib and libSystem?
  • They should have new compilers, as I understand it.

@SilasK
Copy link
Member Author

SilasK commented Nov 18, 2019

Relevant section from the docs: https://bioconda.github.io/contributor/guidelines.html#c-c

@dnbaker
Copy link

dnbaker commented Nov 18, 2019

The biggest issue here is that bioconda seems to want to use clang on OSX, but it doesn't come with omp baked in. (In the README, I recommend building with g++.) But I'm hoping that requiring libomp somehow helps.

There may have been an issue with clang compatibility in 0.3.6, so I've patched that and bumped to 0.4.0. I'll work on this today, but that's what I've been trying to figure out.

@dnbaker
Copy link

dnbaker commented Nov 21, 2019

dnbaker@696c9eb

I've made changes; a number of these dependencies only exist on github. I've used git as a package manager previously. I believe this will cause it to build, unless the omp problem remains. (I would definitely have preferred a recursive clone, as all modules would be loaded with correct versions already), but I've added releases to a variety of these. I'll test later, but I wanted to report on these changes.

@dnbaker
Copy link

dnbaker commented Nov 21, 2019

The current bug I'm experiencing is here: ContinuumIO/anaconda-issues#9096

But perhaps I should be testing on linux instead.

* Add nxtrim

* Add dep zlib

* Update build.sh

* Update meta.yaml
@SilasK
Copy link
Member Author

SilasK commented Nov 22, 2019

release instead of git

Ideally the submodules should be included in the release, but this is not yet the case on github.

build on linux

We can always skip the osx build for now and get it out on linux.

I merged your commits #18857

They are tested automatically on linux. see the checks

https://app.circleci.com/jobs/github/bioconda/bioconda-recipes/83605

@SilasK SilasK closed this Nov 22, 2019
@SilasK SilasK reopened this Nov 24, 2019
@SilasK SilasK mentioned this pull request Nov 24, 2019
5 tasks
@SilasK
Copy link
Member Author

SilasK commented Nov 24, 2019

circle ci fails on linux with gcc: Command not found

https://bioconda.github.io/contributor/troubleshooting.html#id8

@dnbaker
Copy link

dnbaker commented Nov 24, 2019

Maybe try adding {{ compiler('c') }}? gcc is needed for zlib, whereas the rest of the codebase needs g++.

@dpryan79
Copy link
Contributor

This build on Linux now.

@SilasK
Copy link
Member Author

SilasK commented Nov 28, 2019

Do you know a way to get the g++ 5 on macOS ?

Should we just skip the osx build ?

@dpryan79
Copy link
Contributor

GCC isn't used on OSX, so if you require that rather than clang you'll need to skip OSX.

@SilasK
Copy link
Member Author

SilasK commented Nov 28, 2019

If I see it correctly it passes the compilation macOS. But then the tests dashing --help doesn't work.

@dpryan79
Copy link
Contributor

@bioconda-bot please fetch artifacts

@BiocondaBot
Copy link
Collaborator

Package(s) built on CircleCI are ready for inspection:

Arch Package Repodata
linux-64 dashing-0.4.0-hfc8b89e_0.tar.bz2 repodata.json
osx-64 dashing-0.4.0-h2f51686_0.tar.bz2 repodata.json

You may also use conda to install these:

  • For packages on linux-64:
conda install -c https://84445-42372094-gh.circle-artifacts.com/0/tmp/artifacts/packages <package name>
  • For packages on osx-64:
conda install -c https://84452-42372094-gh.circle-artifacts.com/0/tmp/artifacts/packages <package name>

Docker image(s) built:

Package Tag Install with docker
dashing 0.4.0--hfc8b89e_0
showcurl "https://84445-42372094-gh.circle-artifacts.com/0/tmp/artifacts/images/dashing%3A0.4.0--hfc8b89e_0.tar.gz" | gzip -dc | docker load

@dpryan79
Copy link
Contributor

I think the problem is that it both compiles in libzstd.a but also links to it with -lzstd. GCC seems to ignore libzstd.so, but clang appears not to.

@dpryan79
Copy link
Contributor

That fixed it

@dpryan79
Copy link
Contributor

@bioconda-bot please merge

@BiocondaBot
Copy link
Collaborator

I will attempt to upload artifacts and merge this PR. This may take some time, please have patience.

@BiocondaBot BiocondaBot merged commit ad477ff into master Nov 28, 2019
@BiocondaBot BiocondaBot deleted the dash branch November 28, 2019 22:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants