PoseidonChip compatibility

[mmagician]

Do you know whether the PoseidonChip is meant to be spec compatible with the native one? Perhaps I'm doing something wrong, but the following fails for me (Scroll's impl claims to be "in line with the reference and the test vectors"):

poseidon-native = { git = "https://github.com/scroll-tech/poseidon/", package = "poseidon"}
poseidon = { git = "https://github.com/axiom-crypto/halo2-lib", branch = "community-edition" }

let gate = GateChip::<F>::default();

let mut sponge = PoseidonChip::<F, T, RATE>::new(ctx, R_F, R_P).unwrap();
let assigned = sponge.squeeze(ctx, &gate).unwrap();

let mut native_sponge = poseidon_native::Poseidon::<F, T, RATE>::new(R_F, R_P);
let native = native_sponge.squeeze();

assert_eq!(assigned.value(), &native); // this fails

I know that you guys got inspiration for the PoseidonChip from the scroll sponge, so apologies if this is completely the wrong place for this issue, but at least I can run meaningful tests between the native & in-circuit Poseidon sponge using your repo 🙏🏼 (as the Scroll halo2-snark-aggregator is a little out of date).

Also @zhenfeizhang @noel2004

[jonathanpwang]

It is spec compatible. Poseidon has different domain separations and we use a specific one in PoseidonChip for variable length transcripts.

See discussion here: https://t.me/halo2lib/129

[jonathanpwang]

It is spec compatible. Poseidon has different domain separations and we use a specific one in PoseidonChip for variable length transcripts.

See discussion here: https://t.me/halo2lib/129

[mmagician]

Thanks @jonathanpwang.
Which native implementation are you using (to compare) with this in-circuit PoseidonChip (i.e. to satisfy the above assert)? It seems that there's a lot of implementations that are incompatible with one another...
E.g. the native implementation in axiom-crypto/halo2 modifies the state before each permutation: https://github.com/axiom-crypto/halo2/blob/98bc83be43af623f4f1a00c8c95db081844773c4/primitives/poseidon/src/poseidon.rs#L37

-> which IIUC is not found in PoseidonChip.

Is this what you mean by domain separation? Or rather that the Chip doesn't append F::one() to the final chunk:
https://github.com/axiom-crypto/halo2/blob/98bc83be43af623f4f1a00c8c95db081844773c4/primitives/poseidon/src/poseidon.rs#L56

I wonder what's the best way to go about circuit & non-circuit compatibility, maybe you've some suggestions? My use case is that I want to produce a non-interactive transcript for some protocol, and encode the protocol verifier as a halo2 circuit - which of course requires the sponges to agree.

[jonathanpwang]

I'm looking into this; will get back to you.

In the meantime, you can try https://github.com/axiom-crypto/snark-verifier/blob/main/snark-verifier/src/util/hash/poseidon.rs
This is the audited version and also designed for compatibility: use NativeLoader for native rust, Halo2Loader for halo2.