LDBC SNB Business Intelligence (BI) workload implementations

Implementations for the BI workload of the LDBC Social Network Benchmark.

To get started with the LDBC SNB benchmarks, check out our introductory presentation: The LDBC Social Network Benchmark (PDF).

📜 If you wish to cite the LDBC SNB, please refer to the documentation repository (bib snippet).

Implementations

The repository contains the following implementations:

• cypher: an implementation using the Neo4j graph database management system with queries expressed in the Cypher language
• umbra: an implementation using the Umbra JIT-compiled columnar relational database management system with expressed in SQL queries written in the PostgreSQL dialect
• tigergraph: an implementation using the TigerGraph graph database management system with queries expressed in the GSQL language

All implementations use Docker containers for ease of setup and execution. However, the setups can be adjusted to use a non-containerized DBMS.

Reproducing SNB BI experiments

Running an SNB BI experiment requires the following steps.

1. Pick a system, e.g. Umbra. Make sure you have the required binaries and licenses available.

2. Generate the data sets using the SNB Datagen according to the format described in the system's README.

3. Generate the substitution parameters using the paramgen tool.

4. Load the data set: set the required environment variables and run the tool's scripts/load-in-one-step.sh script.

5. Run the benchmark: set the required environment variables and run the tool's scripts/benchmark.sh script.

6. Collect the results in the output directory of the tool.

⚠️ Note that deriving official LDBC results requires commissioning an audited benchmark, which is a more complex process as it entails code review, cross-validation, etc. See the specification's Auditing chapter for details.

Cross-validation

To cross-validate the results of two implementations, use two systems. Load the data into both, then run the benchmark in validation mode, e.g. Cypher and Umbra results. Then, run:

export SF=10

cd cypher
scripts/benchmark.sh --validate
cd ..

cd umbra
scripts/benchmark.sh --validate
cd ..

scripts/cross-validate.sh cypher umbra

Note that the cross-validation uses the numdiff CLI tool.

Usage

See .circleci/config.yml for an up-to-date example on how to use the projects in this repository.

Data sets

Pre-generated data sets and parameters are now available for SF1-SF10,000 (projected FK variant) and SF1-SF1,000 (merged FK variant).

Data sets are also being uploaded to the SURF CWI repository and expected to be published there in 2023.