Feature: Orion PCS Integration #153

tonyfloatersu · 2024-11-21T06:46:32Z

Orion (base field / simd field) impls
Orion MPI functionality (proof aggregating from different parties)
MPI environment spin up for a real e2e testing
integrating PCS into GKR configs

…ening simd field

…eck impl again

…o ExtensionField

zhiyong1997 · 2025-01-17T01:52:35Z

arith/src/simd_field.rs

+    {
+        assert_eq!(simd_vec.len() * PackF::PACK_SIZE, Self::PACK_SIZE);
+        let mut temp: Vec<_> = simd_vec.to_vec();
+        temp.reverse();


Is this for gf2x128 and gf2x64 only? reverse here is a little confusing.

right - this was introduced to pack gf2x8s into gf2x64 or gf2x128... the reverse here is to fit into the bit ordering, that allows for unpacking in correct order, but truly, leaving without a note seems a bit confusing

Signed-off-by: Zhiyong Fang <[email protected]>

poly_commit/src/traits.rs

zhiyong1997 · 2025-01-20T03:15:12Z

I've went throught the base field impl, but haven't touched the simd-agg-related code. The protocol itself and the security parameters: proximity_repetition and query_complexity both look good to me. I'll leave an approve here first, and keep reading the simd-agg part to see whether there are any problems introduced by optimization.

* fix power gate (#128) * Recursive Verifier (#127) * tmp * transcript refactor * transcript refactor * tmp * tm[ * v1 * rm proof txt file * clippy * minor * fmt * tmp fix * fix * fix errors in merge * minor * file location update * clean up * clean up * fix grinding * lock/unlock proof in transcript * testing script * update script * hash test in rust * minor * script update * update script * update script * tmp * fix expander-exec * fmt * Lagrange Form Bi kzg update (#130) * tmp * clean up * fix clippy * fix clippy * add a bit more tests * minor --------- Co-authored-by: zhenfei <[email protected]> * [impl] adding sumcheck-cuda (#131) * stats code & release binary (#135) * stats code * release binary * fix-binary-release (#136) * Reuse eq_evals in prover, verifier, and recursive verifier (#138) * reduce randomness and reuse eq evals * add recursion in ci * add recursion in ci * pcs trait v1 (#140) * pcs trait v1 * &mut for pcs scheme * remove &mut self for verify * add cache for CI * Update nightly_e2e.yml * Features: Orion implementation related peripheral building blocks (#142) * Feature: Merkle Tree implementation as Orion pcs building block (#143) * init commit for merkle tree implementation * benchmarked and optimized, shit runs fast * allocate rather than concat * use move for intermediate results that can be dropped later * shorten bench time * additional benchmarks * Zf/config refactor (#146) * config refactor wip * config refactor wip * refactor config wip * refactor config wip * macros done * refactor done * fmt * clippy auto fix * fix grinding feature * fix verifier scratch pad * PCS Trait V2 & Integration (#150) Second version of PCS trait and PCS for Expander GKR interface, integrating Raw PCS to linear GKR prover * Minor: Merging `FieldSerde` into `Field`, and deprecating `try_deserialize_from_ecc_format` (#151) Co-authored-by: Zhiyong Fang <[email protected]> * Minor: Typo fixing for GF2 and tree bench shorten (#152) * Minor: MISC changes including rust fmt, CI caches, and MT serde (#154) * init commit for extension field from/to-limbs methods (#159) * Minor: Traits for multilinear polynomials, PCS minor changes (#158) * reference (read-only/write) multilinear polynomial prototype * full set of testing for both ref and mut_ref mle * minor, prototype (mut)-multilinear-extension traits for ref-mle-polys * continue with (mutable)multilinear-extension trait implementation * pcs interface change to box dyn multilinear-extension, collateral changes included * minor, add one more trait method of ref to hypercube basis * minor, use impl to get around lifetime specification in place * pcs crs from deterministic seed in testing across mpi (#162) * Minor: Fixing BN254 Recursion on RawPCS deserialization (#163) * Expose expander-exec APIs for ECC (#165) * expose exec apis * fix clippy * fix clippy * Minor: Poseidon M31 Transcript (#160) Signed-off-by: Tiancheng Xie <[email protected]> Co-authored-by: Tiancheng Xie <[email protected]> * Zf/virgo++ (#164) * e2e infrastructure * tmp * tmp * circuit deserialization * tmp * tmp * v1 * fmt * clippy auto fix * clippy * minor * minor * fmt * minor * fix mpi ci (#174) * remove rust cache * try... * clean up * profiling * the old fashion * Update install.py * remove apt update * Update benchmarks.yml * enable sudo for CI * [impl] update soft-lock based naive prepare impl (#173) * temporarily uses GitHub's CI (#175) * Minor: Recursion Script Typo Fix from CI Fix (#176) * Minor: Orion PCS only base field impls (#177) * Minor: Mersenne31 GKR recursive verifier (#168) * extracted changes for poseidon m31 transcript * minor - mimc tested, sponge set/get state trait methods * fkin hell - just wanna specify Transcript baseF challengeF ... didnt expect this comin * morphing field hash transcript, hash from base field output challenge field * squeeze, squeeze_state, and set_state in field hash sponge trait * changed interface for transcript - wondering why recursive mimc transcript not matching * so poseidon m31 works for gkr correctness, which is a relief * you squeaky lil bug there you go * Revert "you squeaky lil bug there you go" This reverts commit 1bc8381. * Revert "so poseidon m31 works for gkr correctness, which is a relief" This reverts commit 573b7d4. * Revert "changed interface for transcript - wondering why recursive mimc transcript not matching" This reverts commit a8ff4f2. * Revert "squeeze, squeeze_state, and set_state in field hash sponge trait" This reverts commit ce80aec. * Revert "morphing field hash transcript, hash from base field output challenge field" This reverts commit 90ebc09. * Revert "fkin hell - just wanna specify Transcript baseF challengeF ... didnt expect this comin" This reverts commit 559b565. * revert back to field hasher from previous interface * minor rewrite on field hash transcript - move FiatShamirFieldHash into arith * minor changes stashing * refactoring field hash transcript to using hash state (vec of base field elems) * minor rewrite - make transcript impl less lines * minor - move mimc to crate field_hashers under arith, remove hasher mimc impl * prototyped poseidon permutation and integrated into gkr correctness * export with name suffix fiat-shamir-hasher, rename to align names * minor - mpi sync up state * minor - typo fixing * stashing changes for sampling circuit elems * gathering stashed circuit changes * recursion CI setup * stashing CI related changes and notes on commenting out api println * minor - api for zeroes over extension field * stash changes from transcript side * raw commitment length w.r.t. base field needs input size * mpi size * simd size * take simd variables into consideration, and note places needs fixing * continue code read with gkr runtime and check against recursive verifier * minor - setup circuit for recursive test * update go mod version * prototyped recursive verifier with CI running * minor - follow up changes for merging dev changes * CI recursion steps update * minor - upload a small enough circuit for recursion test * update ecc branch - remove prev poseidon impl * i dont like it but it works for CI for now * clean up ci yml file --------- Signed-off-by: Tiancheng Xie <[email protected]> Co-authored-by: Tiancheng Xie <[email protected]> * Minor: Fix CI build (#178) * Minor: SIMD orion PCS impls (#179) * Feature: Orion PCS Integration (#153) --------- Signed-off-by: Zhiyong Fang <[email protected]> Co-authored-by: Zhiyong Fang <[email protected]> * CI: build ECC locally rather than downloading (#181) * Minor: Simpler CI from ECC script (#182) * init commit for the gf2 avx256 fix (#183) * fix fiat-shamir (#184) * hash the commitment for sufficient depth * clean up * clean up * clean up * use recursion gate to guard the grinding * Minor: Unified CI workflows (#185) * Minor: Bringing benchmark machine back (#186) * Minor: PCS trait changes (#188) * Minor: Fixing build workflow CI process (#187) * Update readme.md Signed-off-by: Tiancheng Xie <[email protected]> * Update readme.md Signed-off-by: Tiancheng Xie <[email protected]> * fixes broken link (#147) --------- Signed-off-by: Tiancheng Xie <[email protected]> Signed-off-by: Zhiyong Fang <[email protected]> Co-authored-by: Zhiyong Fang <[email protected]> Co-authored-by: Simon Lau <[email protected]> Co-authored-by: tonyfloatersu <[email protected]> Co-authored-by: mcfx <[email protected]> Co-authored-by: Tiancheng Xie <[email protected]> Co-authored-by: Yahya Hassanzadeh, Ph.D. <[email protected]>

tonyfloatersu added a commit that referenced this pull request Nov 22, 2024

init commit for misc changes from #153 development

1b185c0

Orion PCS implementation prorotyped

0fe9dc4

tonyfloatersu force-pushed the feature/orion branch from 6aba141 to 0fe9dc4 Compare November 22, 2024 04:44

rust compilation cache again?

0694446

tonyfloatersu force-pushed the feature/orion branch from 06038ff to 0694446 Compare November 22, 2024 05:07

tonyfloatersu added 25 commits November 22, 2024 20:18

orion interface changes, working on aligning with pcs trait

6ed985a

orion commit from multilinear poly over simd fields

4e81cd9

minor, refactoring orion opening base field, need to work on orion op…

533be26

…ening simd field

prototype simd version of orion opening, need to get some rest and ch…

a84e286

…eck impl again

minor, move original orion verify to base field impl

14f8532

minor, renaming from Circuit*F to Simd*F

a81d45d

minor, prototype for orion verify for simd field

4e08b59

minor, typo

ad2581d

move test to base field test, working on simd test

23ce12a

orion simd commitment consistency test

a065d8b

started working on test orion pcs simd full e2e

e664a4b

e2e test finalized, prototype first round of polishing starts

955d2b4

minor polishing and benchmarks for orion over simd polynomial

164ff4c

minor, remove old cloned iterator

ba2ce01

standard pcs trait implementation for both flavors of orion impls

7838aad

orion simd pcs aligning with PCSForExpanderGKR, restrict constraint t…

a885ed5

…o ExtensionField

simplified type constraints for orion, no need for SimdEvalF

0f42507

minor, fixing up a unattended impl for standard pcs for simd orion

441482b

minor, one liner change

73e8baa

working on mpi versioned commit/open/verify algorithm

a74cf76

change base pcs impl testing to test_pcs from common

3d9006e

tame the CI

f67463d

testing harness minor changes

0e85b7a

minor, renamings and using params in standard pcs trait

1673d3b

pcs trait impl file too large, separate into 2 files

e000787

tonyfloatersu added 10 commits January 9, 2025 22:53

base field orion pcs support for M31 prime

04d7f34

update on verifier side - sync up the randomness on verifier side

4fce220

orion m31 base pcs commit and open benchmarking

fcce135

M31 simd commit testing

cea7ebd

minor - updated orion implementation with m31 support prototype

0274b94

minor - each sumcheck IOP prove/verify sync up randomness beforehand

b1cfa70

benchmarking m31 case

075abe8

Merge branch 'dev' into feature/orion

0ba1d8b

push to 2^32 GF2 benchmark size

7fe83ca

Merge branch 'dev' into feature/orion

16a67f7

zhiyong1997 reviewed Jan 17, 2025

View reviewed changes

Add some notes to 'transcript_verifier_sync' and make clippy happy

0c35935

Signed-off-by: Zhiyong Fang <[email protected]>

tonyfloatersu force-pushed the feature/orion branch from fbbf6dc to 0c35935 Compare January 17, 2025 03:09

tonyfloatersu added 3 commits January 16, 2025 22:24

Merge branch 'dev' into feature/orion

511d8fb

trying to isolate potential problems

fdc0f01

smaller test size may work?

22abffa

tonyfloatersu force-pushed the feature/orion branch from 221f781 to 22abffa Compare January 17, 2025 07:03

still trying smaller test size...

09c6054

zhiyong1997 reviewed Jan 19, 2025

View reviewed changes

poly_commit/src/traits.rs Outdated Show resolved Hide resolved

zhiyong1997 approved these changes Jan 20, 2025

View reviewed changes

tonyfloatersu added 6 commits January 19, 2025 22:24

hamming weight naming suggestion

a2f62ae

Merge branch 'dev' into feature/orion

ebdb24f

Merge branch 'dev' into feature/orion

75530c9

Merge branch 'dev' into feature/orion

81fae91

Merge branch 'dev' into feature/orion

4f05d70

Minor: Orion related additional test (#180)

25d4461

tonyfloatersu merged commit 9a4248d into dev Jan 22, 2025
5 checks passed

tonyfloatersu deleted the feature/orion branch January 22, 2025 00:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature: Orion PCS Integration #153

Feature: Orion PCS Integration #153

tonyfloatersu commented Nov 21, 2024 •

edited

Loading

zhiyong1997 Jan 17, 2025

tonyfloatersu Jan 21, 2025

zhiyong1997 commented Jan 20, 2025

Feature: Orion PCS Integration #153

Feature: Orion PCS Integration #153

Conversation

tonyfloatersu commented Nov 21, 2024 • edited Loading

zhiyong1997 Jan 17, 2025

Choose a reason for hiding this comment

tonyfloatersu Jan 21, 2025

Choose a reason for hiding this comment

zhiyong1997 commented Jan 20, 2025

tonyfloatersu commented Nov 21, 2024 •

edited

Loading