diff --git a/.github/img/bb_banner.png b/.github/img/bb_banner.png new file mode 100644 index 000000000000..43bb32acaa80 Binary files /dev/null and b/.github/img/bb_banner.png differ diff --git a/.gitignore b/.gitignore index 9a2dd48b7040..c58c92be4b75 100644 --- a/.gitignore +++ b/.gitignore @@ -26,4 +26,5 @@ terraform.tfstate* .bb_tmp # Terraform -*.tfvars \ No newline at end of file +*.tfvars +.supermavenignore diff --git a/.vscode/settings.json b/.vscode/settings.json index 2e0e5227291e..3646421e8ecd 100644 --- a/.vscode/settings.json +++ b/.vscode/settings.json @@ -171,5 +171,6 @@ }, "files.trimTrailingWhitespace": true, "cmake.sourceDirectory": "${workspaceFolder}/barretenberg/cpp", - "typescript.tsserver.maxTsServerMemory": 4096 + "typescript.tsserver.maxTsServerMemory": 4096, + "markdown.extension.toc.levels": "2..6" } diff --git a/avm-transpiler/Dockerfile.dockerignore b/avm-transpiler/Dockerfile.dockerignore deleted file mode 100644 index cc6b2b4c0d46..000000000000 --- a/avm-transpiler/Dockerfile.dockerignore +++ /dev/null @@ -1,5 +0,0 @@ -* -!noir -!avm-transpiler -**/target -**/node_modules diff --git a/aztec-nargo/Dockerfile.dockerignore b/aztec-nargo/Dockerfile.dockerignore deleted file mode 100644 index d3cc26d846db..000000000000 --- a/aztec-nargo/Dockerfile.dockerignore +++ /dev/null @@ -1,6 +0,0 @@ -* -!noir -!avm-transpiler -!aztec-nargo -**/target -**/node_modules diff --git a/barretenberg/README.md b/barretenberg/README.md index 1b3797dfd10e..69ec9c8361cc 100644 --- a/barretenberg/README.md +++ b/barretenberg/README.md @@ -1,160 +1,179 @@ -# :warning: This is not an actively developed repository, it is a mirror. See https://github.com/AztecProtocol/aztec-packages :warning: -# :warning: **https://github.com/AztecProtocol/barretenberg is a mirror-only repository, please only use https://github.com/AztecProtocol/aztec-packages. Do not use this for any purpose other than reference.** :warning: +![banner](../.github/img/bb_banner.png) -## Barretenberg, an optimized elliptic curve library for the bn128 curve, and PLONK SNARK prover -**This code is highly experimental, use at your own risk!** +# Barretenberg -### Benchmarks! +Barretenberg (or `bb` for short) is an optimized elliptic curve library for the bn128 curve, and a PLONK SNARK prover. -Table represents time in ms to build circuit and proof for each test on n threads. -Ignores proving key construction. +- [Installation](#installation) +- [Usage](#usage) + - [UltraHonk](#ultrahonk) + - [Proving](#proving) + - [Verifying](#verifying) + - [Solidity verifier](#solidity-verifier) + - [MegaHonk](#megahonk) +- [Development](#development) + - [Bootstrap](#bootstrap) + - [Build Options and Instructions](#build-options-and-instructions) + - [WASM build](#wasm-build) + - [Fuzzing build](#fuzzing-build) + - [Test coverage build](#test-coverage-build) + - [Formatting](#formatting) + - [Testing](#testing) + - [Integration tests with Aztec in Monorepo](#integration-tests-with-aztec-in-monorepo) + - [Integration tests with Aztec in Barretenberg Standalone Repo](#integration-tests-with-aztec-in-barretenberg-standalone-repo) + - [Testing locally in docker](#testing-locally-in-docker) + - [Docs Build](#docs-build) + - [Benchmarks](#benchmarks) + - [x86\_64](#x86_64) + - [WASM](#wasm) + - [How to run](#how-to-run) + - [Debugging](#debugging) + - [Debugging Verifification Failures](#debugging-verifification-failures) + - [Improving LLDB Debugging](#improving-lldb-debugging) + - [Using Tracy to Profile Memory/CPU](#using-tracy-to-profile-memorycpu) -#### x86_64 +> [!WARNING] +> :warning: This is not an actively developed repository, it is a mirror. See :warning: -``` -+--------------------------+------------+---------------+-----------+-----------+-----------+-----------+-----------+ -| Test | Gate Count | Subgroup Size | 1 | 4 | 16 | 32 | 64 | -+--------------------------+------------+---------------+-----------+-----------+-----------+-----------+-----------+ -| sha256 | 38799 | 65536 | 5947 | 1653 | 729 | 476 | 388 | -| ecdsa_secp256k1 | 41049 | 65536 | 6005 | 2060 | 963 | 693 | 583 | -| ecdsa_secp256r1 | 67331 | 131072 | 12186 | 3807 | 1612 | 1351 | 1137 | -| schnorr | 33740 | 65536 | 5817 | 1696 | 688 | 532 | 432 | -| double_verify_proof | 505513 | 524288 | 47841 | 15824 | 7970 | 6784 | 6082 | -+--------------------------+------------+---------------+-----------+-----------+-----------+-----------+-----------+ -``` +> [!WARNING] +> :warning: ** is a mirror-only repository, please only use . Do not use this for any purpose other than reference.** :warning: -#### WASM +> [!CAUTION] +> **This code is highly experimental, use at your own risk!** -``` -+--------------------------+------------+---------------+-----------+-----------+-----------+-----------+-----------+ -| Test | Gate Count | Subgroup Size | 1 | 4 | 16 | 32 | 64 | -+--------------------------+------------+---------------+-----------+-----------+-----------+-----------+-----------+ -| sha256 | 38799 | 65536 | 18764 | 5116 | 1854 | 1524 | 1635 | -| ecdsa_secp256k1 | 41049 | 65536 | 19129 | 5595 | 2255 | 2097 | 2166 | -| ecdsa_secp256r1 | 67331 | 131072 | 38815 | 11257 | 4744 | 3633 | 3702 | -| schnorr | 33740 | 65536 | 18649 | 5244 | 2019 | 1498 | 1702 | -| double_verify_proof | 505513 | 524288 | 149652 | 45702 | 20811 | 16979 | 15679 | -+--------------------------+------------+---------------+-----------+-----------+-----------+-----------+-----------+ -``` +## Installation -### Dependencies +For an easy installation, follow [bbup](https://github.com/AztecProtocol/aztec-packages/blob/master/barretenberg/bbup/README.md). -- cmake >= 3.24 -- Ninja (used by the presets as the default generator) -- clang >= 16 or gcc >= 10 -- clang-format -- libstdc++ >= 12 -- libomp (if multithreading is required. Multithreading can be disabled using the compiler flag `-DMULTITHREADING 0`) +## Usage -#### Ubuntu +TODO: -To install on Ubuntu, run: +All available `bb` commands: + -``` -sudo apt-get install cmake clang clang-format ninja-build libstdc++-12-dev -``` +> [!NOTE] +> Currently the binary downloads an SRS that can be used to prove the maximum circuit size. This maximum circuit size parameter is a constant in the code and has been set to $2^{23}$ as of writing. This maximum circuit size differs from the maximum circuit size that one can prove in the browser, due to WASM limits. -The default cmake version on 22.04 is 3.22.1, so it must be updated. You can get the latest version [here](https://cmake.org/download). +> [!NOTE] +> For commands which allow you to send the output to a file using `-o {filePath}`, there is also the option to send the output to stdout by using `-o -`. -#### MacOS +### UltraHonk -When running MacOS Sonoma 14.2.1 the following steps are necessary: +> [!TIP] +> Follow the [Noir Docs](https://noir-lang.org) for instructions on how to install Noir, write and compile programs, and generate witnesses. -- update bash with `brew install bash` -- update [cmake](https://cmake.org/download) +Prove the valid execution of your program: -### Installing openMP (Linux) +```bash +bb prove_ultra_honk -b ./target/hello_world.json -w ./target/witness-name.gz -o ./target/proof +``` -Install from source: +You can then compute the verification key for your Noir program and verify the proof: + +```bash +bb write_vk_ultra_honk -b ./target/hello_world.json -o ./target/vk +bb verify_ultra_honk -k ./target/vk -p ./target/proof -``` -git clone -b release/10.x --depth 1 https://github.com/llvm/llvm-project.git \ - && cd llvm-project && mkdir build-openmp && cd build-openmp \ - && cmake ../openmp -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DLIBOMP_ENABLE_SHARED=OFF \ - && cmake --build . --parallel \ - && cmake --build . --parallel --target install \ - && cd ../.. && rm -rf llvm-project ``` -Or install from a package manager, on Ubuntu: +If successful, the verification will complete in silence. -``` -sudo apt-get install libomp-dev -``` +### MegaHonk -> Note: on a fresh Ubuntu Kinetic installation, installing OpenMP from source yields a `Could NOT find OpenMP_C (missing: OpenMP_omp_LIBRARY) (found version "5.0")` error when trying to build Barretenberg. Installing from apt worked fine. +The usage with MegaHonk is similar to the above UltraHonk. Refer to all the available `bb` commands, using the `bb _mega_honk` syntax. -### Getting started +>[!WARNING] +> MegaHonk generates insecure recursion circuits when Goblin recursive verifiers are not present. -Run the bootstrap script. (The bootstrap script will build both the native and wasm versions of barretenberg) +### Solidity verifier -``` -cd cpp -./bootstrap.sh +Barretenberg can generate a smart contract that verifies proofs in Solidity (i.e. for usage in EVM chains). This feature is only available for UltraHonk, as the MegaHonk proving system is intended for use with apps deploying on Aztec only. + +First, prove the valid execution of your Noir program and export the verification key: + +```bash +bb prove_ultra_keccak_honk -b ./target/hello_world.json -w ./target/witness-name.gz -o ./target/proof +bb write_vk_ultra_honk -b ./target/hello_world.json -o ./target/vk ``` -### Installing +> [!IMPORTANT] +> `prove_ultra_keccak_honk` is used to generate UltraHonk proofs with Keccak hashes, making them gas-efficient. `prove_ultra_honk` in comparison generates proofs with Poseidon hashes, more efficient in recursions but not on-chain verifications. -After the project has been built, such as with `./bootstrap.sh`, you can install the library on your system: +You can now use the verification key to generate a Solidity verifier contract: -```sh -cmake --install build +```bash +bb contract_ultra_honk -k ./target/vk -c $CRS_PATH -b ./target/hello_world.json -o ./target/Verifier.sol ``` -### Formatting +>[!CAUTION] +> Solidity verifier contracts are work-in-progress. Expect significant optimizations and breaking changes, and *do NOT use it in production!* -Code is formatted using `clang-format` and the `./cpp/format.sh` script which is called via a git pre-commit hook. -If you've installed the C++ Vscode extension you should configure it to format on save. +## Development -### Testing +The following packages are required for building: -Each module has its own tests. e.g. To build and run `ecc` tests: +- cmake >= 3.24 +- Ninja (used by the presets as the default generator) +- clang >= 16 or gcc >= 10 +- clang-format +- libstdc++ >= 12 +- libomp (if multithreading is required. Multithreading can be disabled using the compiler flag `-DMULTITHREADING 0`) + +To install these on Ubuntu, run: ```bash -# Replace the `default` preset with whichever preset you want to use -cmake --build --preset default --target ecc_tests -cd build -./bin/ecc_tests +sudo apt-get install cmake clang clang-format ninja-build libstdc++-12-dev ``` -A shorthand for the above is: +The default cmake version on 22.04 is 3.22.1, so it must be updated. You can get the latest version [here](https://cmake.org/download). -```bash -# Replace the `default` preset with whichever preset you want to use -cmake --build --preset default --target run_ecc_tests -``` +When running MacOS Sonoma 14.2.1 the following steps are necessary: -Running the entire suite of tests using `ctest`: +- update bash with `brew install bash` +- update [cmake](https://cmake.org/download) + +
+

Installing openMP (Linux)

 + +You can get openMP from package managers. Ex. on Ubuntu: ```bash -cmake --build --preset default --target test +sudo apt-get install libomp-dev ``` -You can run specific tests, e.g. +Or you can install it from source: -``` -./bin/ecc_tests --gtest_filter=scalar_multiplication.* +```bash! +git clone -b release/10.x --depth 1 https://github.com/llvm/llvm-project.git \ + && cd llvm-project && mkdir build-openmp && cd build-openmp \ + && cmake ../openmp -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DLIBOMP_ENABLE_SHARED=OFF \ + && cmake --build . --parallel \ + && cmake --build . --parallel --target install \ + && cd ../.. && rm -rf llvm-project ``` -### Benchmarks +> [!Note] +> On a fresh Ubuntu Kinetic installation, installing OpenMP from source yields a `Could NOT find OpenMP_C (missing: OpenMP_omp_LIBRARY) (found version "5.0")` error when trying to build Barretenberg. Installing from apt worked fine. -Some modules have benchmarks. The build targets are named `_bench`. To build and run, for example `ecc` benchmarks. +
+ +### Bootstrap + +The bootstrap script will build both the native and wasm versions of barretenberg: ```bash -# Replace the `default` preset with whichever preset you want to use -cmake --build --preset default --target ecc_bench -cd build -./bin/ecc_bench +cd cpp +./bootstrap.sh ``` -A shorthand for the above is: +You can then install the library on your system: -```bash -# Replace the `default` preset with whichever preset you want to use -cmake --build --preset default --target run_ecc_bench +```sh +cmake --install build ``` -### CMake Build Options +### Build Options and Instructions CMake can be passed various build options on its command line: @@ -171,7 +190,7 @@ If you are cross-compiling, you can use a preconfigured toolchain file: - `-DCMAKE_TOOLCHAIN_FILE=`: Use one of the preconfigured toolchains. -### WASM build +#### WASM build To build: @@ -201,7 +220,7 @@ To add gtest filter parameters in a wasm context: wasmtime --dir=.. ./bin/ecc_tests run --gtest_filter=filtertext ``` -### Fuzzing build +#### Fuzzing build For detailed instructions look in cpp/docs/Fuzzing.md @@ -219,7 +238,7 @@ Sometimes you might have to specify the address of llvm-symbolizer. You have to For undefined behavior sanitizer `-DUNDEFINED_BEHAVIOUR_SANITIZER=ON`. Note that the fuzzer can be orders of magnitude slower with ASan (2-3x slower) or UBSan on, so it is best to run a non-sanitized build first, minimize the testcase and then run it for a bit of time with sanitizers. -### Test coverage build +#### Test coverage build To build: @@ -244,19 +263,53 @@ The report will land in the build directory in the all_test_coverage_report dire Alternatively you can build separate test binaries, e.g. honk_tests or numeric_tests and run **make test** just for them or even just for a single test. Then the report will just show coverage for those binaries. -### VS Code configuration +### Formatting + +Code is formatted using `clang-format` and the `./cpp/format.sh` script which is called via a git pre-commit hook. + +>[!TIP] +> A default configuration for VS Code is provided by the file [`barretenberg.code-workspace`](barretenberg.code-workspace). These settings can be overridden by placing configuration files in `.vscode/`. +> If you've installed the C++ Vscode extension, configure it to format on save! + +### Testing + +Each module has its own tests. e.g. To build and run `ecc` tests: + +```bash +# Replace the `default` preset with whichever preset you want to use +cmake --build --preset default --target ecc_tests +cd build +./bin/ecc_tests +``` + +A shorthand for the above is: + +```bash +# Replace the `default` preset with whichever preset you want to use +cmake --build --preset default --target run_ecc_tests +``` + +Running the entire suite of tests using `ctest`: + +```bash +cmake --build --preset default --target test +``` -A default configuration for VS Code is provided by the file [`barretenberg.code-workspace`](barretenberg.code-workspace). These settings can be overridden by placing configuration files in `.vscode/`. +You can run specific tests, e.g. -### Integration tests with Aztec in Monorepo +``` +./bin/ecc_tests --gtest_filter=scalar_multiplication.* +``` + +#### Integration tests with Aztec in Monorepo CI will automatically run integration tests against Aztec. It is located in the `barretenberg` folder. -### Integration tests with Aztec in Barretenberg Standalone Repo +##### Integration tests with Aztec in Barretenberg Standalone Repo When working on a PR, you may want to point this file to a different Aztec branch or commit, but then it should probably be pointed back to master before merging. -### Testing locally in docker +##### Testing locally in docker A common issue that arises is that our CI system has a different compiler version e.g. namely for GCC. If you need to mimic the CI operating system locally you can use bootstrap_docker.sh or run dockerfiles directly. However, there is a more efficient workflow for iterative development: @@ -269,7 +322,7 @@ cmake --preset gcc ; cmake --build build This will allow you to rebuild as efficiently as if you were running native code, and not have to see a full compile cycle. -### Building docs +### Docs Build If doxygen is installed on the system, you can use the **build_docs** target to build documentation, which can be configured in vscode CMake extension or using @@ -285,21 +338,77 @@ python3 -m http.server and tunnel the port through ssh. -### Debugging Verifification Failures +### Benchmarks + +Table represents time in ms to build circuit and proof for each test on n threads. +Ignores proving key construction. + +#### x86_64 + +``` ++--------------------------+------------+---------------+-----------+-----------+-----------+-----------+-----------+ +| Test | Gate Count | Subgroup Size | 1 | 4 | 16 | 32 | 64 | ++--------------------------+------------+---------------+-----------+-----------+-----------+-----------+-----------+ +| sha256 | 38799 | 65536 | 5947 | 1653 | 729 | 476 | 388 | +| ecdsa_secp256k1 | 41049 | 65536 | 6005 | 2060 | 963 | 693 | 583 | +| ecdsa_secp256r1 | 67331 | 131072 | 12186 | 3807 | 1612 | 1351 | 1137 | +| schnorr | 33740 | 65536 | 5817 | 1696 | 688 | 532 | 432 | +| double_verify_proof | 505513 | 524288 | 47841 | 15824 | 7970 | 6784 | 6082 | ++--------------------------+------------+---------------+-----------+-----------+-----------+-----------+-----------+ +``` + +#### WASM + +``` ++--------------------------+------------+---------------+-----------+-----------+-----------+-----------+-----------+ +| Test | Gate Count | Subgroup Size | 1 | 4 | 16 | 32 | 64 | ++--------------------------+------------+---------------+-----------+-----------+-----------+-----------+-----------+ +| sha256 | 38799 | 65536 | 18764 | 5116 | 1854 | 1524 | 1635 | +| ecdsa_secp256k1 | 41049 | 65536 | 19129 | 5595 | 2255 | 2097 | 2166 | +| ecdsa_secp256r1 | 67331 | 131072 | 38815 | 11257 | 4744 | 3633 | 3702 | +| schnorr | 33740 | 65536 | 18649 | 5244 | 2019 | 1498 | 1702 | +| double_verify_proof | 505513 | 524288 | 149652 | 45702 | 20811 | 16979 | 15679 | ++--------------------------+------------+---------------+-----------+-----------+-----------+-----------+-----------+ +``` + +#### How to run + +Some modules have benchmarks. The build targets are named `_bench`. To build and run, for example `ecc` benchmarks. + +```bash +# Replace the `default` preset with whichever preset you want to use +cmake --build --preset default --target ecc_bench +cd build +./bin/ecc_bench +``` + +A shorthand for the above is: + +```bash +# Replace the `default` preset with whichever preset you want to use +cmake --build --preset default --target run_ecc_bench +``` + +### Debugging + +#### Debugging Verifification Failures The CicuitChecker::check_circuit function is used to get the gate index and block information about a failing circuit constraint. If you are in a scenario where you have a failing call to check_circuit and wish to get more information out of it than just the gate index, you can use this feature to get a stack trace, see example below. Usage instructions: + - On ubuntu (or our mainframe accounts) use `sudo apt-get install libdw-dev` to support trace printing - Use `cmake --preset clang16-dbg-fast-circuit-check-traces` and `cmake --build --preset clang16-dbg-fast-circuit-check-traces` to enable the backward-cpp dependency through the CHECK_CIRCUIT_STACKTRACES CMake variable. - Run any case where you have a failing check_circuit call, you will now have a stack trace illuminating where this constraint was added in code. Caveats: + - This works best for code that is not overly generic, i.e. where just the sequence of function calls carries a lot of information. It is possible to tag extra data along with the stack trace, this can be done as a followup, please leave feedback if desired. -- There are certain functions like `assert_equals` that can cause gates that occur _before_ them to fail. If this would be useful to automatically report, please leave feedback. +- There are certain functions like `assert_equals` that can cause gates that occur *before* them to fail. If this would be useful to automatically report, please leave feedback. Example: + ``` [ RUN ] standard_circuit_constructor.test_check_circuit_broken Stack trace (most recent call last): @@ -350,8 +459,7 @@ Stack trace (most recent call last): gate number4 ``` - -### Improving LLDB Debugging +#### Improving LLDB Debugging It can be quite hard to make sense of field_t circuit values that indirectly reference their contents, and even plain field values that are typically in montgomery form. In command-line LLDB or VSCode debug console, run: @@ -362,12 +470,11 @@ command script import ~/aztec-packages/barretenberg/cpp/scripts/lldb_format.py Now when you `print` things with e.g. `print bigfield_t.get_value()` or inspect in VSCode (if you opened the debug console and put in these commands) then you will get pretty-printing of these types. This can be expanded fairly easily with more types if needed. +#### Using Tracy to Profile Memory/CPU -### Using Tracy to Profile Memory/CPU - -See Tracy manual linked here https://github.com/wolfpld/tracy for in-depth Tracy documentation. +See Tracy manual linked here for in-depth Tracy documentation. The basic use of Tracy is to run a benchmark with the `cmake --preset tracy` build type, create a capture file, then transfer it to a local machine for interactive UI introspection. -All the steps to do this effectively are included in cpp/scripts/benchmark_tracy.sh \ No newline at end of file +All the steps to do this effectively are included in cpp/scripts/benchmark_tracy.sh diff --git a/barretenberg/bbup/.gitignore b/barretenberg/bbup/.gitignore new file mode 100644 index 000000000000..9c418ce79fdd --- /dev/null +++ b/barretenberg/bbup/.gitignore @@ -0,0 +1,3 @@ +node_modules +yarn.lock +*.js diff --git a/barretenberg/bbup/.npmignore b/barretenberg/bbup/.npmignore new file mode 100644 index 000000000000..b3a74a69b824 --- /dev/null +++ b/barretenberg/bbup/.npmignore @@ -0,0 +1,3 @@ +node_modules +yarn.lock +*.ts diff --git a/barretenberg/bbup/.yarn/install-state.gz b/barretenberg/bbup/.yarn/install-state.gz new file mode 100644 index 000000000000..033148d68746 Binary files /dev/null and b/barretenberg/bbup/.yarn/install-state.gz differ diff --git a/barretenberg/bbup/README.md b/barretenberg/bbup/README.md new file mode 100644 index 000000000000..cd55c6295eb3 --- /dev/null +++ b/barretenberg/bbup/README.md @@ -0,0 +1,33 @@ +# BBup + +BBup is a CLI tool that makes it easy to install the [Barretenberg](https://github.com/AztecProtocol/aztec-packages/blob/master/barretenberg/README.md) proving backend. + +Although extensible, it assumes you are using [Noir](https://noir-lang.org) as the frontend language. + +## Installation + +BBup is an installer for whatever version of BB you may want. Install BBup with: + +```bash +curl -L bbup.dev | bash +``` + +> [!IMPORTANT] +> *Always* check what scripts do. The above one redirects to [the install script](https://github.com/AztecProtocol/aztec-packages/blob/master/barretenberg/bbup/install) which checks if you have `npm`, installing it with `nvm` otherwise. It then installs [bbup](https://github.com/AztecProtocol/aztec-packages/blob/master/barretenberg/bbup/bbup.ts) globally. + +## Usage + +To install the Barretenberg version compatible with the stable Noir version, run: + +```bash +bbup +``` + +You can specify the `-f` flag to match a different frontend language. At the moment only Noir is available, so `-f noir` is defaulted. + +You can pass [any Noir version](https://github.com/noir-lang/noir/tags) with the `-v` flag, or specify `nightly` for the nightly version. Examples: + +```bash +bbup -v 0.34.0 # installs the barretenberg version compatible with Noir 0.34.0 release +bbup -v nightly # installs the barretenberg version compatible with Noir nightly release +``` diff --git a/barretenberg/bbup/bbup.ts b/barretenberg/bbup/bbup.ts new file mode 100755 index 000000000000..754327ea39b0 --- /dev/null +++ b/barretenberg/bbup/bbup.ts @@ -0,0 +1,66 @@ +#!/usr/bin/env node +import { Command } from "commander"; +const program = new Command(); +import { installBB } from "./shell.js"; +import ora from "ora"; +import logSymbols from "log-symbols"; +import { getBbVersionForNoir } from "./versions.js"; +import { execSync } from "child_process"; + +const spinner = ora({ color: "blue", discardStdin: false }); + +const bbup = program + .command("install", { isDefault: true }) + .description("Installs Barretenberg.") + .option( + "-f, --frontend", + "Match the version of a specific frontend language", + "noir" + ); + +const options = bbup.opts(); + +if (options.frontend === "noir") { + bbup + .requiredOption( + "-v, --version ", + "The Noir version to match", + "current" + ) + .action(async ({ version }) => { + let resolvedVersion = version; + if (version === "current") { + spinner.start(`Querying noir version from nargo`); + try { + const output = execSync("nargo --version", { encoding: "utf-8" }); + resolvedVersion = output.match(/nargo version = (\d+\.\d+\.\d+)/)![1]; + spinner.stopAndPersist({ + text: `Resolved noir version ${resolvedVersion} from nargo`, + symbol: logSymbols.success, + }); + } catch (e) { + spinner.stopAndPersist({ + text: `Could not get noir version from nargo --version. Please specify a version.`, + symbol: logSymbols.error, + }); + process.exit(1); + } + } + + spinner.start( + `Getting compatible barretenberg version for noir version ${resolvedVersion}` + ); + const compatibleVersion = await getBbVersionForNoir( + resolvedVersion, + spinner + ); + spinner.stopAndPersist({ + text: `Resolved to barretenberg version ${compatibleVersion}`, + symbol: logSymbols.success, + }); + spinner.start(`Installing barretenberg`); + await installBB(compatibleVersion, spinner); + }); +} + +bbup.parse(); diff --git a/barretenberg/bbup/install b/barretenberg/bbup/install new file mode 100755 index 000000000000..848086087c41 --- /dev/null +++ b/barretenberg/bbup/install @@ -0,0 +1,42 @@ +#!/bin/bash + +set -e + +# Function to check if a command exists +command_exists() { + command -v "$1" >/dev/null 2>&1 +} + +# Function to install NVM and Node.js +install_nvm_and_node() { + echo "Installing NVM..." + wget -qO- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash + + # Load NVM + export NVM_DIR="$HOME/.nvm" + [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" + + # Install the latest LTS version of Node.js + echo "Installing the latest LTS version of Node.js..." + nvm install --lts + + # Use the installed version + nvm use --lts + + # Verify installation + node --version + npm --version +} + +# Check if NPM is installed +if ! command_exists npm; then + install_nvm_and_node +fi + + +# Install bbup globally +echo "Installing bbup..." +npm install -g bbup + +echo "Installation complete. You can now use the 'bbup' command." +echo "Please restart your terminal or run 'source ~/.bashrc' (or your shell's equivalent) to start using bbup." diff --git a/barretenberg/bbup/package.json b/barretenberg/bbup/package.json new file mode 100644 index 000000000000..c25aa876b811 --- /dev/null +++ b/barretenberg/bbup/package.json @@ -0,0 +1,24 @@ +{ + "name": "bbup", + "type": "module", + "description": "Barretenberg installation script", + "bin": "bbup.js", + "version": "0.0.5", + "license": "ISC", + "scripts": { + "start": "npx tsx bbup.ts", + "compile": "tsc bbup.ts --esModuleInterop true --module nodenext && chmod +x bbup.js", + "publish": "yarn compile && yarn npm publish --access public" + }, + "dependencies": { + "@inquirer/input": "^1.2.16", + "@inquirer/select": "^1.3.3", + "axios": "^1.7.7", + "commander": "^11.1.0", + "log-symbols": "^7.0.0", + "ora": "^8.1.0", + "tar-fs": "^3.0.6", + "tiged": "^2.12.6" + }, + "packageManager": "yarn@4.5.0" +} diff --git a/barretenberg/bbup/shell.ts b/barretenberg/bbup/shell.ts new file mode 100644 index 000000000000..81081ffc2f41 --- /dev/null +++ b/barretenberg/bbup/shell.ts @@ -0,0 +1,83 @@ +import { execSync } from "child_process"; +import logSymbols from "log-symbols"; +import { Ora } from "ora"; +import os from "os"; +import axios from "axios"; +import fs from "fs"; +import { createGunzip } from "zlib"; +import tar from "tar-fs"; +import { promisify } from "util"; + +import { pipeline } from "stream"; +import path from "path"; + +export function sourceShellConfig() { + const shell = execSync("echo $SHELL", { encoding: "utf-8" }).trim(); + + if (shell.includes("bash")) { + process.env.PATH = execSync("echo $PATH", { encoding: "utf-8" }).trim(); + } else if (shell.includes("zsh")) { + process.env.PATH = execSync('zsh -c "echo $PATH"', { + encoding: "utf-8", + }).trim(); + } else if (shell.includes("fish")) { + process.env.PATH = execSync('fish -c "echo $PATH"', { + encoding: "utf-8", + }).trim(); + } +} + +export function exec(cmd: string, options = {}) { + return execSync(cmd, { + encoding: "utf-8", + stdio: "pipe", + ...options, + }); +} +export async function installBB(version: string, spinner: Ora) { + let architecture = os.arch(); + if (architecture === "arm64") { + architecture = "aarch64"; + } else if (architecture === "x64") { + architecture = "x86_64"; + } + + let platform: string = os.platform(); + if (platform === "darwin") { + platform = "apple-darwin"; + } else if (platform === "linux") { + platform = "linux-gnu"; + } + + const home = os.homedir(); + const bbPath = path.join(home, ".bb"); + + spinner.start(`Installing to ${bbPath}`); + const tempTarPath = path.join(home, "temp.tar.gz"); + + if ( + !["x86_64", "aarch64"].includes(architecture) || + !["linux-gnu", "apple-darwin"].includes(platform) + ) { + throw new Error( + `Unsupported architecture ${architecture} and platform ${platform}` + ); + } + + const releaseUrl = `https://github.com/AztecProtocol/aztec-packages/releases/download/aztec-packages-v${version}`; + const binaryUrl = `${releaseUrl}/barretenberg-${architecture}-${platform}.tar.gz`; + + const response = await axios.get(binaryUrl, { responseType: "stream" }); + + const pipelineAsync = promisify(pipeline); + await pipelineAsync(response.data, fs.createWriteStream(tempTarPath)); + await pipelineAsync( + fs.createReadStream(tempTarPath), + createGunzip(), + tar.extract(bbPath) + ); + spinner.stopAndPersist({ + text: `Installed barretenberg to ${bbPath}`, + symbol: logSymbols.success, + }); +} diff --git a/barretenberg/bbup/versions.ts b/barretenberg/bbup/versions.ts new file mode 100644 index 000000000000..da9e77367c86 --- /dev/null +++ b/barretenberg/bbup/versions.ts @@ -0,0 +1,52 @@ +import axios from 'axios'; +import logSymbols from 'log-symbols'; +import { Ora } from 'ora'; + +async function getNamedVersions(githubToken?: string) { + const fetchOpts = { + // eslint-disable-next-line camelcase + params: { per_page: 100 }, + headers: {}, + }; + + if (githubToken) fetchOpts.headers = { Authorization: `token ${githubToken}` }; + + const { data } = await axios.get(`https://api.github.com/repos/noir-lang/noir/releases`, fetchOpts); + + const stable = data.filter( + (release: any) => + !release.tag_name.includes('aztec') && !release.tag_name.includes('nightly') && !release.prerelease, + )[0].tag_name; + const nightly = data.filter((release: any) => release.tag_name.startsWith('nightly'))[0].tag_name; + + return { + stable, + nightly, + }; +} + +export async function getBbVersionForNoir(noirVersion: string, spinner: Ora, githubToken?: string) { + let url = ''; + + if (noirVersion === 'stable' || noirVersion === 'nightly') { + spinner.start(`Resolving noir version ${noirVersion}...`); + const resolvedVersions = await getNamedVersions(githubToken); + spinner.stopAndPersist({ + text: `Resolved noir version ${noirVersion} to ${resolvedVersions[noirVersion]}`, + symbol: logSymbols.success, + }); + url = `https://raw.githubusercontent.com/noir-lang/noir/${resolvedVersions[noirVersion]}/scripts/install_bb.sh`; + } else { + url = `https://raw.githubusercontent.com/noir-lang/noir/v${noirVersion}/scripts/install_bb.sh`; + } + + try { + const { data } = await axios.get(url); + const versionMatch = data.match(/VERSION="([\d.]+)"/); + const version = versionMatch ? versionMatch[1] : null; + + return version; + } catch (e: any) { + throw new Error(e.message || e); + } +} diff --git a/barretenberg/cpp/installation/bbup b/barretenberg/cpp/installation/bbup deleted file mode 100755 index e26d38999f48..000000000000 --- a/barretenberg/cpp/installation/bbup +++ /dev/null @@ -1,117 +0,0 @@ -#!/usr/bin/env bash -set -e - -BB_HOME=${BB_HOME-"$HOME/.bb"} - -main() { - need_cmd curl - - while [[ $1 ]]; do - case $1 in - --) shift; break;; - - -v|--version) shift; BBUP_VERSION=$1;; - -h|--help) - usage - exit 0 - ;; - *) - err "internal error: unknown option "$1"\n";; - esac; shift - done - - if [ -z "$BBUP_VERSION" ]; then - err "--version option is required" - fi - - BBUP_REPO=AztecProtocol/aztec-packages - - PLATFORM="$(uname -s)" - case $PLATFORM in - Linux) - PLATFORM="linux-gnu" - ;; - Darwin) - PLATFORM="apple-darwin" - ;; - *) - err "unsupported platform: $PLATFORM" - ;; - esac - - # Fetch system's architecture. - ARCHITECTURE="$(uname -m)" - - # Align ARM naming for release fetching. - if [ "${ARCHITECTURE}" = "arm64" ]; then - ARCHITECTURE="aarch64" # Align release naming. - fi - - # Reject unsupported architectures. - if [ "${ARCHITECTURE}" != "x86_64" ] && [ "${ARCHITECTURE}" != "aarch64" ]; then - err "unsupported architecure: $ARCHITECTURE-$PLATFORM" - fi - - BBUP_TAG=$BBUP_VERSION - # Normalize versions (handle channels, versions without v prefix) - if [[ "$BBUP_VERSION" == [[:digit:]]* ]]; then - # Add v prefix - BBUP_VERSION="v${BBUP_VERSION}" - BBUP_TAG="aztec-packages-${BBUP_VERSION}" - fi - - say "installing bb (version ${BBUP_VERSION} with tag ${BBUP_TAG})" - - RELEASE_URL="https://github.com/${BBUP_REPO}/releases/download/${BBUP_TAG}" - BIN_TARBALL_URL="${RELEASE_URL}/barretenberg-${ARCHITECTURE}-${PLATFORM}.tar.gz" - - # Download the binary's tarball and unpack it into the .bb directory. - say "downloading bb to '$BB_HOME'" - ensure curl -# -L $BIN_TARBALL_URL | tar -xzC $BB_HOME - - say "done" -} - -usage() { - cat 1>&2 < -OPTIONS: - -h, --help Print help information - -v, --version Install a specific version (required) -EOF -} - -say() { - printf 'bbup: %s\n' "$1" -} - -warn() { - say "warning: ${1}" >&2 -} - -err() { - say "$1" >&2 - exit 1 -} - -need_cmd() { - if ! check_cmd "$1"; then - err "need '$1' (command not found)" - fi -} - -check_cmd() { - command -v "$1" >/dev/null 2>&1 -} - -# Run a command that should never fail. If the command fails execution -# will immediately terminate with an error showing the failing -# command. -ensure() { - if ! "$@"; then err "command failed: $*"; fi -} - -main "$@" || exit 1 \ No newline at end of file diff --git a/barretenberg/cpp/installation/install b/barretenberg/cpp/installation/install deleted file mode 100755 index f426d0415676..000000000000 --- a/barretenberg/cpp/installation/install +++ /dev/null @@ -1,48 +0,0 @@ -#!/usr/bin/env bash -set -e - -echo Installing bbup... - -BB_HOME=${BB_HOME-"$HOME/.bb"} - -BBUP_BIN_URL=${BBUP_BIN_URL-"https://raw.githubusercontent.com/AztecProtocol/aztec-packages/master/barretenberg/cpp/installation/bbup"} -BBUP_BIN_PATH="$BB_HOME/bbup" - -# Create the .bb directory and bbup binary if it doesn't exist. -mkdir -p $BB_HOME -curl -# -L $BBUP_BIN_URL -o $BBUP_BIN_PATH -chmod +x $BBUP_BIN_PATH - -# Store the correct profile file (i.e. .profile for bash or .zshrc for ZSH). -case $SHELL in -*/zsh) - PROFILE=$HOME/.zshrc - PREF_SHELL=zsh - ;; -*/bash) - PROFILE=$HOME/.bashrc - PREF_SHELL=bash - ;; -*/fish) - PROFILE=$HOME/.config/fish/config.fish - PREF_SHELL=fish - ;; -*/ash) - PROFILE=$HOME/.profile - PREF_SHELL=ash - ;; -*) - echo "bbup: could not detect shell, manually add ${BB_HOME} to your PATH." - exit 1 - ;; -esac - -# Only add bbup if it isn't already in PATH. -if [[ ":$PATH:" != *":${BB_HOME}:"* ]]; then - # Add the .bb directory to the path and ensure the old PATH variables remain. - echo >>$PROFILE && echo "export BB_HOME=\"$BB_HOME\"" >>$PROFILE - echo >>$PROFILE && echo "export PATH=\"\$PATH:\$BB_HOME\"" >>$PROFILE -fi - -echo && echo "Detected your preferred shell is ${PREF_SHELL} and added bbup to PATH. Run 'source ${PROFILE}' or start a new terminal session to use bbup." -echo "Then, simply run 'bbup' to install bb." \ No newline at end of file diff --git a/barretenberg/cpp/src/barretenberg/bb/readme.md b/barretenberg/cpp/src/barretenberg/bb/readme.md deleted file mode 100644 index eb1ff22b2c80..000000000000 --- a/barretenberg/cpp/src/barretenberg/bb/readme.md +++ /dev/null @@ -1,140 +0,0 @@ -# BB - -### Why is this needed? - -Barretenberg is a library that allows one to create and verify proofs. One way to specify the circuit that one will use to create and verify -proofs over is to use the Barretenberg standard library. Another way, which pertains to this module is to supply the circuit description using an IR called [ACIR](https://github.com/noir-lang/acvm). - -This binary will take as input ACIR and witness values described in the IR to create proofs. - -### Installation - -1. Install `bbup` the installation script by running this in your terminal: - - ```bash - curl -L https://raw.githubusercontent.com/AztecProtocol/aztec-packages/master/barretenberg/cpp/installation/install | bash - ``` - -2. Reload your terminal shell environment: - - macOS: - ```bash - source ~/.zshrc - ``` - - Linux: - ```bash - source ~/.bashrc - ``` - -3. Install the version of `bb` compatible with your Noir version; with **Noir v0.34.0** for example: - - ```bash - bbup -v 0.55.0 - ``` - - Check the version compatibility section below for how to identify matching versions. - -4. Check if the installation was successful: - - ```bash - bb --version - ``` - -If installation was successful, the command would print the version of `bb` installed. - -### Usage prerequisites - -Certain `bb` commands will expect the tool `jq` to already be installed. If `jq -V` doesn't return a version number, install it from [here](https://jqlang.github.io/jq/download/). - -### Version compatibility with Noir - -TODO: https://github.com/AztecProtocol/aztec-packages/issues/7511 - -For quick reference: -- Noir v0.34.0 <> BB v0.55.0 -- Noir v0.33.0 <> BB v0.47.1 -- Noir v0.32.0 <> BB v0.46.1 -- Noir v0.31.0 <> BB v0.41.0 - -### Usage - -TODO: https://github.com/AztecProtocol/aztec-packages/issues/7600 - -All available `bb` commands: -https://github.com/AztecProtocol/aztec-packages/blob/barretenberg-v0.55.0/barretenberg/cpp/src/barretenberg/bb/main.cpp#L1369-L1512 - -#### FilePath vs Stdout - -For commands which allow you to send the output to a file using `-o {filePath}`, there is also the option to send the output to stdout by using `-o -`. - -#### Usage with UltraHonk - -Documented with Noir v0.33.0 <> BB v0.47.1: - -##### Proving and verifying - -1. Follow [the Noir docs](https://noir-lang.org/docs/getting_started/hello_noir/) to compile and generate witness of your Noir program - -2. Prove the valid execution of your Noir program running: - - ```bash - bb prove_ultra_honk -b ./target/hello_world.json -w ./target/witness-name.gz -o ./target/proof - ``` - -3. Compute the verification key for your Noir program running: - - ```bash - bb write_vk_ultra_honk -b ./target/hello_world.json -o ./target/vk - ``` - -4. Verify your proof running: - - ```bash - bb verify_ultra_honk -k ./target/vk -p ./target/proof - ``` - - If successful, the verification will complete in silence; if unsuccessful, the command will trigger logging of the corresponding error. - -Refer to all available `bb` commands linked above for full list of functionality. - -##### Generating proofs for verifying in Solidity - -Barretenberg UltraHonk comes with the capability to verify proofs in Solidity, i.e. in smart contracts on EVM chains. - -1. Follow [the Noir docs](https://noir-lang.org/docs/getting_started/hello_noir/) to compile and generate witness of your Noir program - -2. Prove the valid execution of your Noir program running: - - ```bash - bb prove_ultra_keccak_honk -b ./target/hello_world.json -w ./target/witness-name.gz -o ./target/proof - ``` - - > **Note:** `prove_ultra_keccak_honk` is used to generate UltraHonk proofs with Keccak hashes, as it is what the Solidity verifier is designed to be compatible with given the better gas efficiency when verifying on-chain; `prove_ultra_honk` in comparison generates proofs with Poseidon hashes, more efficient in recursions but not on-chain verifications. - -3. Compute the verification key for your Noir program running: - - ```bash - bb write_vk_ultra_honk -b ./target/hello_world.json -o ./target/vk - ``` - -4. Generate Solidity verifier - **WARNING:** Contract incomplete, do not use in production! - - ```bash - bb contract_ultra_honk -k ./target/vk -c $CRS_PATH -b ./target/hello_world.json -o ./target/Verifier.sol - ``` - -#### Usage with MegaHonk - -Use `bb _mega_honk`. - -Refer to all available `bb` commands linked above for full list of functionality. - -Note that MegaHonk: -- Generates insecure recursion circuits when Goblin recursive verifiers are not present -- Will not have a Solidity verifier, as the proving system is intended for use with apps deploying on Aztec only - -### Maximum circuit size - -Currently the binary downloads an SRS that can be used to prove the maximum circuit size. This maximum circuit size parameter is a constant in the code and has been set to $2^{23}$ as of writing. This maximum circuit size differs from the maximum circuit size that one can prove in the browser, due to WASM limits.