feat(brillig): foreign call/oracle compilation (#1600)

* remove mac runner * chore: generate brillig opcode for simple identity unconstrained function (#1536) * feat(brillig): added arithmetic operations on brillig (#1565) Co-authored-by: kevaundray <[email protected]> * make ranges be polymorphic integers * chore(brillig): Clean up handling of Binary operations (#1571) * chore(ssa refactor): Rename Brillig example (#1563) * chore(brillig): added tests for all field binary operations (#1586) * chore(brillig): added tests for brillig integer operations (#1590) * feat: process blocks and jumps when compiling brillig (#1591) * process jumps between blocks * fix jumps * add doc comments * cargo fmt * code refactor * update code comment --------- Co-authored-by: kevaundray <[email protected]> * feat: process blocks and jumps when compiling brillig (#1591) * process jumps between blocks * fix jumps * add doc comments * cargo fmt * code refactor * update code comment --------- Co-authored-by: kevaundray <[email protected]> * feat(brillig): start of oracles/foreign calls * fix: broken tests * feat(brillig): parsing oracles/foreign calls (#1596) * feat(brillig): start of oracles/foreign calls * fix: broken tests * Update execute.rs * feat: more foreign call work * self.data -> self.vars * chore(brillig): Add handling of the not instruction (#1609) * test: brillig oracle * Reinstate option * make behavior consistent * remove closure * change index_type * Update crates/noirc_frontend/src/hir/type_check/expr.rs * feat(brillig): loops (#1610) * make ranges be polymorphic integers * feat: brillig loop support * fix: fixed brillig returns and stop * fix: do not apply constants folding to brillig fns * chore: update acvm pointer, cleanup * style: newline on cargo toml * make behavior consistent * remove closure * change index_type * Update crates/noirc_frontend/src/hir/type_check/expr.rs * better debug information for unsupported instruction * remove edge case for optimizations * clippy fix * patch infinite loop --------- Co-authored-by: kevaundray <[email protected]> Co-authored-by: jfecher <[email protected]> * feat: Foreign calls compiling and basic print executed in nargo (#1612) * get foreign calls compiling and basic print executed in nargo * cargo clipy and cargo fmt * missing * Update crates/noirc_evaluator/src/brillig/brillig_gen.rs * add issue num for logging --------- Co-authored-by: kevaundray <[email protected]> * chore: resolve immutable array merge differences (#1617) * chore(ssa refactor): Switch to immutable arrays (#1578) * Represent SSA arrays with im::Vector * Get tests passing * Implement assign with immutable arrays * Add constant folding pass * Update comments * Clippy * Update comment * Update type of array * Update crates/noirc_evaluator/src/ssa_refactor/ir/instruction.rs Co-authored-by: Tom French <[email protected]> * Undo formatting changes in instruction.rs * Massive acir_gen update * Refactor acir array operations into a shared function * Appease clippy * Update to_radix and to_bits in acir_gen to return arrays * Disable assert * Fix convert_type for arrays * Include AcirType in AcirValue::Var variant * Fix black box functions * Appease clippy * Fix simple_radix * Add doc comments --------- Co-authored-by: Tom French <[email protected]> * feat: Make for-loop range be a polymorphic integer instead of just Field in unconstrained functions (#1583) * make ranges be polymorphic integers * make behavior consistent * remove closure * change index_type * Update crates/noirc_frontend/src/hir/type_check/expr.rs --------- Co-authored-by: jfecher <[email protected]> * chore(ssa refactor): fix brillig post master merge * chore(ssa refactor): accidental merge undelete --------- Co-authored-by: jfecher <[email protected]> Co-authored-by: Tom French <[email protected]> Co-authored-by: kevaundray <[email protected]> * chore(ssa refactor): Add more documentation for truncation (#1607) * add more documentation * small change * Update crates/noirc_evaluator/src/ssa_refactor/acir_gen/acir_ir/generated_acir.rs * Update .github/workflows/test.yml * Update .github/workflows/test.yml * remove dbg * change printer for foreign call * Update crates/noirc_evaluator/src/ssa_refactor/ir/function.rs * Update crates/noirc_evaluator/src/ssa_refactor/ir/instruction.rs Co-authored-by: kevaundray <[email protected]> * Update crates/noirc_evaluator/src/ssa_refactor/acir_gen/acir_ir/acir_variable.rs --------- Co-authored-by: kevaundray <[email protected]> Co-authored-by: guipublic <[email protected]> Co-authored-by: Álvaro Rodríguez <[email protected]> Co-authored-by: jfecher <[email protected]> Co-authored-by: Maxim Vezenov <[email protected]> Co-authored-by: joss-aztec <[email protected]> Co-authored-by: Tom French <[email protected]>
noir-lang · Jun 9, 2023 · 6bd5a8d · 6bd5a8d
1 parent 66e7297
commit 6bd5a8d
Show file tree

Hide file tree

Showing 22 changed files with 200 additions and 37 deletions.
diff --git a/.gitignore b/.gitignore
@@ -18,3 +18,4 @@ result
 *.pk
 *.vk
 **/Verifier.toml
+**/target
diff --git a/crates/nargo/src/ops/execute.rs b/crates/nargo/src/ops/execute.rs
@@ -1,4 +1,6 @@
-use acvm::pwg::{solve, PartialWitnessGeneratorStatus};
+use acvm::acir::brillig_vm::ForeignCallResult;
+use acvm::acir::circuit::Opcode;
+use acvm::pwg::{solve, PartialWitnessGeneratorStatus, UnresolvedBrilligCall};
 use acvm::PartialWitnessGenerator;
 use acvm::{acir::circuit::Circuit, acir::native_types::WitnessMap, pwg::block::Blocks};
 
@@ -11,8 +13,39 @@ pub fn execute_circuit(
 ) -> Result<WitnessMap, NargoError> {
     let mut blocks = Blocks::default();
     let solver_status = solve(backend, &mut initial_witness, &mut blocks, circuit.opcodes)?;
-    if matches!(solver_status, PartialWitnessGeneratorStatus::RequiresOracleData { .. }) {
-        todo!("Add oracle support to nargo execute")
+
+    // TODO(#1615): Nargo only supports "oracle_print_impl" functions that print a singular value and nothing else
+    // expand this in a general logging refactor
+    if let PartialWitnessGeneratorStatus::RequiresOracleData {
+        unresolved_brillig_calls,
+        required_oracle_data,
+        unsolved_opcodes,
+    } = solver_status
+    {
+        if !required_oracle_data.is_empty() {
+            unreachable!("oracles are not supported by nargo execute")
+        }
+        for unresolved_brillig_call in unresolved_brillig_calls {
+            let UnresolvedBrilligCall { foreign_call_wait_info, mut brillig } =
+                unresolved_brillig_call;
+            let value = foreign_call_wait_info.inputs[0];
+
+            // Execute foreign call "oracle_print_impl"
+            println!("{:?}", value.to_field().to_hex());
+
+            // TODO(#1615): "oracle_print_impl" is just an identity func
+            brillig.foreign_call_results.push(ForeignCallResult { values: vec![value] });
+
+            let mut next_opcodes_for_solving = vec![Opcode::Brillig(brillig)];
+            next_opcodes_for_solving.extend_from_slice(&unsolved_opcodes[..]);
+
+            let solver_status =
+                solve(backend, &mut initial_witness, &mut blocks, next_opcodes_for_solving)?;
+            if matches!(solver_status, PartialWitnessGeneratorStatus::RequiresOracleData { .. }) {
+                todo!("Add multiple foreign call support to nargo execute")
+                // TODO 1557
+            }
+        }
     }
 
     Ok(initial_witness)

diff --git a/crates/nargo_cli/tests/test_data_ssa_refactor/brillig_oracle/Nargo.toml b/crates/nargo_cli/tests/test_data_ssa_refactor/brillig_oracle/Nargo.toml
@@ -0,0 +1,5 @@
+[package]
+authors = [""]
+compiler_version = "0.1"
+
+[dependencies]
diff --git a/crates/nargo_cli/tests/test_data_ssa_refactor/brillig_oracle/Prover.toml b/crates/nargo_cli/tests/test_data_ssa_refactor/brillig_oracle/Prover.toml
@@ -0,0 +1,2 @@
+x = "10"
+
diff --git a/crates/nargo_cli/tests/test_data_ssa_refactor/brillig_oracle/src/main.nr b/crates/nargo_cli/tests/test_data_ssa_refactor/brillig_oracle/src/main.nr
@@ -0,0 +1,15 @@
+// Tests oracle usage in brillig/unconstrained functions
+fn main(x: Field)  {
+    // call through a brillig wrapper
+    oracle_print_wrapper(x);
+}
+
+
+#[oracle(oracle_print_impl)]
+unconstrained fn oracle_print(_x : Field) {}
+
+unconstrained fn oracle_print_wrapper(x: Field) {
+    oracle_print(x);
+}
+
+
diff --git a/crates/noirc_evaluator/src/brillig/brillig_gen.rs b/crates/noirc_evaluator/src/brillig/brillig_gen.rs
@@ -14,10 +14,12 @@ use crate::ssa_refactor::ir::{
 };
 use acvm::{
     acir::brillig_vm::{
-        BinaryIntOp, Opcode as BrilligOpcode, RegisterIndex, Value as BrilligValue,
+        BinaryIntOp, Opcode as BrilligOpcode, RegisterIndex, RegisterValueOrArray,
+        Value as BrilligValue,
     },
     FieldElement,
 };
+use iter_extended::vecmap;
 use std::collections::HashMap;
 
 #[derive(Default)]
@@ -205,6 +207,21 @@ impl BrilligGen {
                 };
                 self.push_code(opcode);
             }
+            Instruction::ForeignCall { func, arguments } => {
+                let result_ids = dfg.instruction_results(instruction_id);
+
+                let input_registers =
+                    vecmap(arguments, |value_id| self.convert_ssa_value(*value_id, dfg));
+                let output_registers =
+                    vecmap(result_ids, |value_id| self.convert_ssa_value(*value_id, dfg));
+
+                let opcode = BrilligOpcode::ForeignCall {
+                    function: func.to_owned(),
+                    destination: RegisterValueOrArray::RegisterIndex(output_registers[0]),
+                    input: RegisterValueOrArray::RegisterIndex(input_registers[0]),
+                };
+                self.push_code(opcode);
+            }
             _ => todo!("ICE: Instruction not supported {instruction:?}"),
         };
     }
@@ -364,6 +381,7 @@ impl BrilligGen {
 
         brillig.convert_ssa_function(func);
 
+        brillig.push_code(BrilligOpcode::Stop);
         brillig.obj
     }
 

diff --git a/crates/noirc_evaluator/src/ssa/ssa_gen.rs b/crates/noirc_evaluator/src/ssa/ssa_gen.rs
@@ -189,9 +189,7 @@ impl IrGenerator {
                     let function_node_id = self.context.get_or_create_opcode_node_id(opcode);
                     Ok(Value::Node(function_node_id))
                 }
-                Definition::Oracle(_, _) => {
-                    unimplemented!("oracles not supported by deprecated SSA")
-                }
+                Definition::Oracle(_) => unimplemented!("oracles not supported by deprecated SSA"),
             }
         }
     }
@@ -499,8 +497,8 @@ impl IrGenerator {
         match expr {
             Expression::Ident(ident) => self.ssa_gen_identifier(ident),
             Expression::Binary(binary) => {
-                // Note: we disallows structs/tuples in infix expressions.
-                // The type checker currently disallows this as well but not if they come from generic type
+                // Note: we disallow structs/tuples in infix expressions.
+                // The type checker currently disallows this as well but not if they come from a generic type
                 // We could allow some in the future, e.g. struct == struct
                 let lhs = self.ssa_gen_expression(&binary.lhs)?.to_node_ids();
                 let rhs = self.ssa_gen_expression(&binary.rhs)?.to_node_ids();

diff --git a/crates/noirc_evaluator/src/ssa_refactor/acir_gen/mod.rs b/crates/noirc_evaluator/src/ssa_refactor/acir_gen/mod.rs
@@ -179,11 +179,19 @@ impl Context {
                                 // Generate the brillig code of the function
                                 let code = BrilligArtifact::default().link(&brillig[*id]);
                                 let outputs = self.acir_context.brillig(code, inputs, result_ids.len());
+
+                                if Self::is_return_type_unit(result_ids, dfg) {
+                                    return;
+                                }
+
                                 for (result, output) in result_ids.iter().zip(outputs) {
                                     let result_acir_type = dfg.type_of_value(*result).into();
                                     self.ssa_values.insert(*result, AcirValue::Var(output, result_acir_type));
                                 }
                             }
+                            RuntimeType::Oracle(_) => unimplemented!(
+                                "expected an intrinsic/brillig call, but found {func:?}. All Oracle methods should be wrapped in an unconstrained fn"
+                            ),
                         }
                     }
                     Value::Intrinsic(intrinsic) => {
@@ -236,6 +244,7 @@ impl Context {
             Instruction::Load { .. } => {
                 unreachable!("Expected all load instructions to be removed before acir_gen")
             }
+            _ => unreachable!("instruction cannot be converted to ACIR"),
         }
     }
 
@@ -298,11 +307,7 @@ impl Context {
             _ => unreachable!("ICE: Program must have a singular return"),
         };
 
-        // Check if the program returns the `Unit/None` type.
-        // This type signifies that the program returns nothing.
-        let is_return_unit_type =
-            return_values.len() == 1 && dfg.type_of_value(return_values[0]) == Type::Unit;
-        if is_return_unit_type {
+        if Self::is_return_type_unit(return_values, dfg) {
             return;
         }
 
@@ -344,6 +349,9 @@ impl Context {
             }
             Value::Intrinsic(..) => todo!(),
             Value::Function(..) => unreachable!("ICE: All functions should have been inlined"),
+            Value::ForeignFunction(_) => unimplemented!(
+                "Oracle calls directly in constrained functions are not yet available."
+            ),
             Value::Instruction { .. } | Value::Param { .. } => {
                 unreachable!("ICE: Should have been in cache {value:?}")
             }
@@ -606,6 +614,12 @@ impl Context {
             }
         }
     }
+
+    /// Check if the program returns the `Unit/None` type.
+    /// This type signifies that the program returns nothing.
+    fn is_return_type_unit(return_values: &[ValueId], dfg: &DataFlowGraph) -> bool {
+        return_values.len() == 1 && dfg.type_of_value(return_values[0]) == Type::Unit
+    }
 }
 
 #[cfg(test)]

diff --git a/crates/noirc_evaluator/src/ssa_refactor/ir/dfg.rs b/crates/noirc_evaluator/src/ssa_refactor/ir/dfg.rs
@@ -54,6 +54,11 @@ pub(crate) struct DataFlowGraph {
     /// represented by only 1 ValueId within this function.
     intrinsics: HashMap<Intrinsic, ValueId>,
 
+    /// Contains each foreign function that has been imported into the current function.
+    /// This map is used to ensure that the ValueId for any given foreign functôn is always
+    /// represented by only 1 ValueId within this function.
+    foreign_functions: HashMap<String, ValueId>,
+
     /// Function signatures of external methods
     signatures: DenseMap<Signature>,
 
@@ -189,6 +194,11 @@ impl DataFlowGraph {
         self.values.insert(Value::Function(function))
     }
 
+    /// Gets or creates a ValueId for the given FunctionId.
+    pub(crate) fn import_foreign_function(&mut self, function: &str) -> ValueId {
+        self.values.insert(Value::ForeignFunction(function.to_owned()))
+    }
+
     /// Gets or creates a ValueId for the given Intrinsic.
     pub(crate) fn import_intrinsic(&mut self, intrinsic: Intrinsic) -> ValueId {
         if let Some(existing) = self.intrinsics.get(&intrinsic) {

diff --git a/crates/noirc_evaluator/src/ssa_refactor/ir/function.rs b/crates/noirc_evaluator/src/ssa_refactor/ir/function.rs
@@ -6,12 +6,14 @@ use super::map::Id;
 use super::types::Type;
 use super::value::ValueId;
 
-#[derive(Clone, Copy, PartialEq, Eq, Debug)]
+#[derive(Clone, PartialEq, Eq, Debug)]
 pub(crate) enum RuntimeType {
     // A noir function, to be compiled in ACIR and executed by ACVM
     Acir,
     // Unconstrained function, to be compiled to brillig and executed by the Brillig VM
     Brillig,
+    // Oracle function, to be compiled to a Brillig external/foreign call
+    Oracle(String),
 }
 /// A function holds a list of instructions.
 /// These instructions are further grouped into Basic blocks
@@ -59,7 +61,7 @@ impl Function {
 
     /// Runtime type of the function.
     pub(crate) fn runtime(&self) -> RuntimeType {
-        self.runtime
+        self.runtime.clone()
     }
 
     /// Set runtime type of the function.

diff --git a/crates/noirc_evaluator/src/ssa_refactor/ir/instruction.rs b/crates/noirc_evaluator/src/ssa_refactor/ir/instruction.rs
@@ -92,6 +92,10 @@ pub(crate) enum Instruction {
     /// Performs a function call with a list of its arguments.
     Call { func: ValueId, arguments: Vec<ValueId> },
 
+    /// Executes an "oracle" call
+    /// These are unconstrained functions that may access external state.
+    ForeignCall { func: String, arguments: Vec<ValueId> },
+
     /// Allocates a region of memory. Note that this is not concerned with
     /// the type of memory, the type of element is determined when loading this memory.
     /// This is used for representing mutable variables and references.
@@ -128,9 +132,10 @@ impl Instruction {
             }
             Instruction::ArraySet { array, .. } => InstructionResultType::Operand(*array),
             Instruction::Constrain(_) | Instruction::Store { .. } => InstructionResultType::None,
-            Instruction::Load { .. } | Instruction::ArrayGet { .. } | Instruction::Call { .. } => {
-                InstructionResultType::Unknown
-            }
+            Instruction::Load { .. }
+            | Instruction::ArrayGet { .. }
+            | Instruction::Call { .. }
+            | Instruction::ForeignCall { .. } => InstructionResultType::Unknown,
         }
     }
 
@@ -158,6 +163,10 @@ impl Instruction {
                 max_bit_size: *max_bit_size,
             },
             Instruction::Constrain(value) => Instruction::Constrain(f(*value)),
+            Instruction::ForeignCall { func, arguments } => Instruction::ForeignCall {
+                func: func.to_owned(),
+                arguments: vecmap(arguments.iter().copied(), f),
+            },
             Instruction::Call { func, arguments } => Instruction::Call {
                 func: f(*func),
                 arguments: vecmap(arguments.iter().copied(), f),
@@ -245,6 +254,7 @@ impl Instruction {
             }
             Instruction::Truncate { .. } => None,
             Instruction::Call { .. } => None,
+            Instruction::ForeignCall { .. } => None,
             Instruction::Allocate { .. } => None,
             Instruction::Load { .. } => None,
             Instruction::Store { .. } => None,

diff --git a/crates/noirc_evaluator/src/ssa_refactor/ir/printer.rs b/crates/noirc_evaluator/src/ssa_refactor/ir/printer.rs
@@ -71,7 +71,9 @@ fn value(function: &Function, id: ValueId) -> String {
             let elements = vecmap(array, |element| value(function, *element));
             format!("[{}]", elements.join(", "))
         }
-        Value::Param { .. } | Value::Instruction { .. } => id.to_string(),
+        Value::Param { .. } | Value::Instruction { .. } | Value::ForeignFunction(_) => {
+            id.to_string()
+        }
     }
 }
 
@@ -148,6 +150,9 @@ pub(crate) fn display_instruction(
         Instruction::Call { func, arguments } => {
             writeln!(f, "call {}({})", show(*func), value_list(function, arguments))
         }
+        Instruction::ForeignCall { func, arguments } => {
+            writeln!(f, "foreign call {}({})", func, value_list(function, arguments))
+        }
         Instruction::Allocate => writeln!(f, "allocate"),
         Instruction::Load { address } => writeln!(f, "load {}", show(*address)),
         Instruction::Store { address, value } => {

diff --git a/crates/noirc_evaluator/src/ssa_refactor/ir/value.rs b/crates/noirc_evaluator/src/ssa_refactor/ir/value.rs
@@ -50,6 +50,11 @@ pub(crate) enum Value {
     /// An Intrinsic is a special kind of builtin function that may be handled internally
     /// or optimized into a special form.
     Intrinsic(Intrinsic),
+
+    /// This Value refers to an external function in the IR.
+    /// ForeignFunction's always have the type Type::Function and have simlar semantics to Function,
+    /// other than generating different backend operations and being only accessible through Brillig.
+    ForeignFunction(String),
 }
 
 impl Value {
@@ -62,6 +67,7 @@ impl Value {
             Value::Array { element_type, array } => Type::Array(element_type.clone(), array.len()),
             Value::Function { .. } => Type::Function,
             Value::Intrinsic { .. } => Type::Function,
+            Value::ForeignFunction { .. } => Type::Function,
         }
     }
 }
diff --git a/crates/noirc_evaluator/src/ssa_refactor/opt/inlining.rs b/crates/noirc_evaluator/src/ssa_refactor/opt/inlining.rs
@@ -207,6 +207,9 @@ impl<'function> PerFunctionContext<'function> {
             }
             Value::Function(function) => self.context.builder.import_function(*function),
             Value::Intrinsic(intrinsic) => self.context.builder.import_intrinsic_id(*intrinsic),
+            Value::ForeignFunction(function) => {
+                self.context.builder.import_foreign_function(function)
+            }
             Value::Array { array, element_type } => {
                 let elements = array.iter().map(|value| self.translate_value(*value)).collect();
                 self.context.builder.array_constant(elements, element_type.clone())
@@ -327,7 +330,7 @@ impl<'function> PerFunctionContext<'function> {
                 Instruction::Call { func, arguments } => match self.get_function(*func) {
                     Some(function) => match ssa.functions[&function].runtime() {
                         RuntimeType::Acir => self.inline_function(ssa, *id, function, arguments),
-                        RuntimeType::Brillig => {
+                        RuntimeType::Brillig | RuntimeType::Oracle(_) => {
                             self.context.failed_to_inline_a_call = true;
                             self.push_instruction(*id);
                         }
-Original file line number
+Diff line change
@@ Expand Up / @@ -18,3 +18,4 @@ result @@
     *.pk
     *.vk
     **/Verifier.toml
+    **/target