Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Improve rocksdb build speed by link dynamic libs #1502

Merged
merged 7 commits into from
Mar 8, 2023
Merged

ci: Improve rocksdb build speed by link dynamic libs #1502

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
87ab422
ci: Improve rocksdb build speed by link dynamic libs
Xuanwo Mar 8, 2023
5e2c338
Fix typo
Xuanwo Mar 8, 2023
3eb1b21
Not for test
Xuanwo Mar 8, 2023
d69254b
Add note for rocksdb
Xuanwo Mar 8, 2023
7785366
Allow using system lib
Xuanwo Mar 8, 2023
472a373
cover docs
Xuanwo Mar 8, 2023
585e5b6
lower version
Xuanwo Mar 8, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,15 +51,22 @@ jobs:
with:
hdfs-version: "3.3.2"

- name: Install rocksdb
run: sudo apt install librocksdb-dev

- uses: Swatinem/rust-cache@v2

- name: Check license headers
uses: korandoru/hawkeye@v1
- name: Cargo format
run: cargo fmt --all -- --check
- name: Cargo clippy
env:
ROCKSDB_LIB_DIR: /usr/lib
run: cargo clippy --all-targets --all-features --workspace -- -D warnings
- name: Cargo doc
env:
ROCKSDB_LIB_DIR: /usr/lib
run: cargo doc --lib --no-deps --all-features

msrv_check:
Expand Down Expand Up @@ -126,9 +133,13 @@ jobs:
uses: beyondstorage/setup-hdfs@master
with:
hdfs-version: "3.3.2"
- name: Install rocksdb
run: sudo apt install librocksdb-dev

- uses: Swatinem/rust-cache@v2
- name: Build
env:
ROCKSDB_LIB_DIR: /usr/lib
run: cargo build --all-features

unit:
Expand Down
11 changes: 9 additions & 2 deletions src/services/rocksdb/backend.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,16 @@ use crate::*;
///
/// # Note
///
/// The storage format for this service is not **stable** yet.
/// OpenDAL will build rocksdb from source by default.
///
/// PLEASE DON'T USE THIS SERVICE FOR PERSIST DATA.
/// To link with existing rocksdb lib, please set one of the following:
Xuanwo marked this conversation as resolved.
Show resolved Hide resolved
///
/// - `ROCKSDB_LIB_DIR` to the dir that contains `librocksdb.so`
/// - `ROCKSDB_STATIC` to the dir that contains `librocksdb.a`
///
/// It is advisable to follow the steps provided in the [`INSTALL`](https://github.com/facebook/rocksdb/blob/main/INSTALL.md)
/// file to build rocksdb, rather than relying on system libraries that
/// may be outdated and incompatible.
///
/// # Configuration
///
Expand Down
6 changes: 0 additions & 6 deletions src/services/sled/backend.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,12 +39,6 @@ use crate::*;
/// - [ ] ~~presign~~
/// - [x] blocking
///
/// # Note
///
/// The storage format for this service is not **stable** yet.
///
/// PLEASE DON'T USE THIS SERVICE FOR PERSIST DATA.
///
/// # Configuration
///
/// - `datadir`: Set the path to the sled data directory
Expand Down