Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DEVOPS-135 Fix capnproto object/code mismatch in manylinux build #1030

Merged
merged 11 commits into from
Aug 12, 2016

Conversation

vitaly-krugl
Copy link
Member

@vitaly-krugl vitaly-krugl commented Aug 5, 2016

Fixes #1013
Forced runtime linking of nupic.bindings extensions against capnproto symbols in the pycapnp extension

Fixes #1034
Reorganized build to link nupic_core static library tests against the "combined" nupic_core static library, which is considered an output artifact, instead of nupic_core_solo static lib, which is only an intermediate step.

Along the way, cleaned up dependencies and refactored merging of static libs.

Afterthought: it might have been better as separate PRs, but I was taking a deep dive in the understanding of the build files, and it was more efficient to make fixes along the way that supported upstream development, rather than coming back for another pass.

@numenta-ci
Copy link

By analyzing the blame information on this pull request, we identified @akhilaananthram, @scottpurdy and @rcrowder to be potential reviewers

@vitaly-krugl vitaly-krugl force-pushed the devops135-fix-futex-hang branch 5 times, most recently from 9576690 to de44526 Compare August 7, 2016 04:19
…er to avoid conflict

with capnproto methods compiled into pycapnp, possibly compiled with a different compiler/version
and with different compiler/linker flags. Instead, we rely on dynamic linking of the necessary
symbols from capnproto in pycapnp.
@vitaly-krugl vitaly-krugl force-pushed the devops135-fix-futex-hang branch 20 times, most recently from 447eb41 to 3e82745 Compare August 10, 2016 00:22

#
# tests_all just calls other targets
#
# TODO This doesn't seem to have any effect; it's probably because the DEPENDS
Copy link
Member Author

@vitaly-krugl vitaly-krugl Aug 10, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@scottpurdy: running of the tests from here is (and has been) dysfunctional. We actually run the tests from our higher-level build script after make completes. Do we really want to run tests during make? If not, we should remove this and the underlying custom targets, such as tests_cpp_region. What do you think? Then, I can file the appropriate issue and remove this TODO comment.

Vitaly Kruglikov added 8 commits August 10, 2016 22:19
…ions DLL in RTLD_GLOBAL mode,

which enables resultion of capnproto references when nupic.bidnings extensions are loaded.
…against the "combined" nupic_core static library, which is considered an output artifact, instead of nupic_core_solo static lib, which is only an intermediate step.
…termediate artifact and not intended to be an output product.
…ifying system libraries. It's a static library and shouldn't need additional linking information.
nupic_core_solo is the only consumer of the custom command's outputs.
…arget name

${CAPNP_STATIC_LIB_TARGET} and containing all capnproto library objects.
@vitaly-krugl vitaly-krugl force-pushed the devops135-fix-futex-hang branch 2 times, most recently from e18dba5 to 7e2f4c0 Compare August 11, 2016 05:43
@vitaly-krugl
Copy link
Member Author

@scottpurdy, the hick-up with cmake 2.8.7 (on travis's default linux environment) has been resolved; this PR's build should go back to green now. Fire away!

@vitaly-krugl
Copy link
Member Author

@scottpurdy: parallelized build also works now; I tested with make --jobs=20 install on Ubuntu 16.04.

…cLib depend directly on external project Apr1StaticLib instead of its library wrapper ${APR1_STATIC_LIB_TARGET}; the latter was incorrectly interperting the dependency as another external project instead of library; but worked correctly on cmake 2.8.12.

Try building windows AprUtil1StaticLib with empty string -DAPR_LIBRARIES value.
@vitaly-krugl vitaly-krugl force-pushed the devops135-fix-futex-hang branch from 7e2f4c0 to f800e70 Compare August 11, 2016 06:11
@@ -171,7 +149,7 @@ import_array();
x_min, x_max, 1, 65535);
return y.forPython();
}
*/
*/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this file missing the close for a multiline comment? I don't see an opening added.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@scottpurdy, regarding

Was this file missing the close for a multiline comment? I don't see an opening added

No, nothing changed in that regard. When I made other changes in this file, the editor removed trailing spaces, which is what you're seeing on this line.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh wow missed that, my bad. Make the old school diff is better 😉

@vitaly-krugl vitaly-krugl force-pushed the devops135-fix-futex-hang branch from c1ce71b to a1f6cda Compare August 11, 2016 20:17
@vitaly-krugl vitaly-krugl force-pushed the devops135-fix-futex-hang branch from a1f6cda to fd33070 Compare August 11, 2016 20:20
@scottpurdy
Copy link
Contributor

Do you think it would be cleaner to put the misc. .cmake helper files into a cmake directory (either in root of repo or in src)?

@scottpurdy
Copy link
Contributor

Other than .cmake file locations, 👍 . Merge when you're ready.

@vitaly-krugl
Copy link
Member Author

@scottpurdy, regarding

Do you think it would be cleaner to put the misc. .cmake helper files into a cmake directory (either in root of repo or in src)?

It seems alright to me as is, but I must admit I might not have a good feel for cmake project structure - this is the only one I've every worked on. What to do?

@vitaly-krugl
Copy link
Member Author

Offline, @scottpurdy agreed to merge as is.

@vitaly-krugl vitaly-krugl merged commit 7f8dd43 into numenta:master Aug 12, 2016
@vitaly-krugl vitaly-krugl deleted the devops135-fix-futex-hang branch August 25, 2016 20:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants