Stop building RDMA in "in place" mode #41
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jonhoo
force-pushed
the
dont-rely-on-build-dir
branch
3 times, most recently
from
e3467bd to
71953d5
Compare
|
I think this will also fix our broken docs.rs build 🎉 |
Closed
When using `IN_PLACE=1`, the build assumes that the build directory will _also_ be the run directory[1], which doesn't hold for our case. Furthermore, it's actively problematic as it means we'll end up fairly long paths, which can cause sadness (#21, [2]). Specifically, rdma-core has an assert[3] that triggers if the path to one of the build dependencies is longer than what's allowed for UNIX sockets[4]. The assert uses `IBACM_SERVER_PATH`, which is set[5] to `@CMAKE_INSTALL_FULL_RUNDIR@/ibacm.sock`. That is in turn set[6] to ${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_RUNDIR} (when `CMAKE_INSTALL_RUNDIR` isn't set to an absolute path, which is the case for the default of "var/run"). `CMAKE_INSTALL_PREFIX` is set to the project directory of rdma-core when `IN_PLACE=1`, which may be quite deep, and thus the whole thing is likely to exceed the 108 bytes allowed for `sun_path`. It's worth noting that since we only build in order to get the various header files so we can then generate bindings from them, the setting of the various `CMAKE_INSTALL_*` options that `IN_PLACE=1` set _shouldn't_ have anything to say for us. Fixes #21 and (hopefully) docs.rs builds. [1]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/CMakeLists.txt#L92-L100 [2]: https://docs.rs/crate/ibverbs-sys/0.2.1+52.0/builds/1297723 [3]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/librdmacm/acm.c#L105-L106 [4]: https://man7.org/linux/man-pages/man7/unix.7.html [5]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/buildlib/config.h.in#L35 [6]: https://github.com/linux-rdma/rdma-core/blob/fb965e2d0f670d1e3474f9333b332f6b4954201c/CMakeLists.txt#L138
jonhoo
force-pushed
the
dont-rely-on-build-dir
branch
from
71953d5 to
d4f0b81
Compare
jonhoo
added a commit
that referenced
this pull request
Dec 27, 2024
ibverbs-sys 0.3.0+55.0: 55.0 + bindgen + pub consts + non-in-place build ibverbs 0.9.0: -sys 0.3.0 + configurable work queue limits rdma-core bump is for linux-rdma/rdma-core#1485 bindgen bump is for harryfei/which-rs#104 Includes #38, #39, #41, and #42.
jonhoo
added a commit
that referenced
this pull request
Dec 27, 2024
It turns out that #41 was insufficient since `cmake` will forcibly _set_ `CMAKE_INSTALL_PREFIX` to the output directory (which is under the target directory) if it isn't already set[1]. So, we need to explicitly set a short dummy value. The version bump to ibverbs (non-sys) is just to force docs.rs to attempt re-generation of the docs. [1]: https://github.com/rust-lang/cmake-rs/blob/94da9de2ea79ab6cad572e908864a160cf4847a9/src/lib.rs#L699-L703
jonhoo
added a commit
that referenced
this pull request
Dec 27, 2024
It turns out that #41 was insufficient since `cmake` will forcibly _set_ `CMAKE_INSTALL_PREFIX` to the output directory (which is under the target directory) if it isn't already set[1]. So, we need to explicitly set a short dummy value. This should hopefully _actually_ fix docs.rs builds. The version bump to ibverbs (non-sys) is just to force docs.rs to attempt re-generation of the docs. [1]: https://github.com/rust-lang/cmake-rs/blob/94da9de2ea79ab6cad572e908864a160cf4847a9/src/lib.rs#L699-L703
jonhoo
added a commit
that referenced
this pull request
Dec 27, 2024
It turns out that #41 was insufficient since `cmake` will forcibly _set_ `CMAKE_INSTALL_PREFIX` to the output directory (which is under the target directory) if it isn't already set[1]. So, we need to explicitly set a short dummy value. This should hopefully _actually_ fix docs.rs builds. The version bump to ibverbs (non-sys) is just to force docs.rs to attempt re-generation of the docs. [1]: https://github.com/rust-lang/cmake-rs/blob/94da9de2ea79ab6cad572e908864a160cf4847a9/src/lib.rs#L699-L703
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When using
IN_PLACE=1, the build assumes that the build directory will also be the run directory1, which doesn't hold for our case. Furthermore, it's actively problematic as it means we'll end up fairly long paths, which can cause sadness (#21, 2).Specifically, rdma-core has an assert3 that triggers if the path to one of the build dependencies is longer than what's allowed for UNIX sockets4. The assert uses
IBACM_SERVER_PATH, which is set5 to@CMAKE_INSTALL_FULL_RUNDIR@/ibacm.sock. That is in turn set6 to(when
CMAKE_INSTALL_RUNDIRisn't set to an absolute path, which is the case for the default of "var/run").CMAKE_INSTALL_PREFIXis set to the project directory of rdma-core whenIN_PLACE=1, which may be quite deep, and thus the whole thing is likely to exceed the 108 bytes allowed forsun_path.It's worth noting that since we only build in order to get the various header files so we can then generate bindings from them, the setting of the various
CMAKE_INSTALL_*options thatIN_PLACE=1set shouldn't have anything to say for us.Fixes #21 and (hopefully) docs.rs builds.