Skip to content

babybear-labs/benchmark

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ZK Benchmark Project

Overview

This repository hosts the source code and resources for benchmarking various Zero-Knowledge (ZK) proofs. It builds upon the original repositories of Halo2, RISC0, Nexus, SP1, Jolt, Circom, and Powdr with our implementations of proofs including but not limited to SHA256, Fibonacci, Poseidon operations.

Our aim is to benchmark the performance of these proofs, comparing them in terms of time, space, and other metrics. This comprehensive comparison will not only highlight the most popular ZK proofs and their implementations but also deepen our understanding of their inner workings, developer friendliness and proving systems.

Goals

  • Benchmark Operations:

    • Fibonacci: Variants 1, 10, 100, 1000, 10000
    • SHA256: Variants 32b, 1k, 10k Bytes
    • Poseidon: Variants 32b, 100b
  • Additional Benchmarks:

    • Arithmetic Operations: Addition, Subtraction, Multiplication, Division
    • Loop: Variants 10, 100, 1000
    • Hash Functions: Pederson, RPO, Keccak (Variants 1k, 10k Bytes)
    • Merkle Tree: Generation, Membership/Inclusion, Merge
    • nth Prime: Variants 1, 10, 100, 1000, 10000
    • Cryptographic Verifications: ECDSA, Elliptic Curve, BLS Verification, BLS Aggregation

Proving Systems

General overview of features of proving schemes

Proof System Setup Complexity Features Post-Quantum Resistance Scalability Parallel Execution
Halo2 Transparent generally Recursive proofs No (ECC based) High Limited
Circom (Groth16) Trusted Setup Efficient proofs No (Pairing-based) Moderate High
Risc Zero Transparent General purpose Yes High High
Jolt Can support both Efficient proofs Yes Very High Very High
Nexus zkVM Transparent Privacy focused Partial Moderate High
SP1 Transparent rollup optimized Yes Very High Very High
Powdr Transparent Extensible Yes High High

The key differences between these proof systems include their setup complexity, proof features, post-quantum resistance, scalability, and parallel execution capabilities. Some use recursive proofs, while others focus on efficiency or general-purpose functionality. The systems also vary in their post-quantum resistance and ability to scale and parallelize.

Metrics

  • Prover Time
  • Verifier Time
  • Prover Space
  • Verifier Space
  • Proof Size
  • Parallel Execution
  • CPU Usage
  • Memory Usage
  • Verifier Gas Consumption
  • Cycles Count
  • Developer Friendliness

Reports

The detailed reports can be found in the reports directory.

Summary: Only selected operations are shown. The detailed reports have metrics for other variants and additional operations.

Fibonacci

10000 th number

Proof System Prover Time (s) Cycles Verifier Time (s) Prover Memory (KB) Proof Size
Halo2 0.196 - 0.004 9.8 1664
Circom 1.75 9999 0.81 466280 805
Risc Zero 6.37 65536 - - 206182
Jolt 21.73 280287 0.01 - 452398
Nexus (max 100) 35.2 - 2.4 - 47.9 MB
SP1 18.87 69101 0.174 - 2656912
Powdr 8.64 2990 - - -

SHA256

1 KB Input

Proof System Prover Time (s) Cycles Verifier Time (s) Prover Memory (KB) Proof Size (B)
Halo2 14.78 - 0.13 1134 4064
Circom 46.07 540736 1.14 3920848 805
Risc Zero 2.5 65536 - - 210157
Jolt 2.199 62231 0.052 - 401116
Nexus 30+ mins - - - -
SP1 17.6 71249 0.172 - 265691
Powdr 9.07 73731 - - -

Poseidon

32 Byte input

Proof System Prover Time (s) Cycles Verifier Time (s) Prover Memory (KB) Proof Size (B)
Halo2 8.74 - 0.086 25 2144
Circom 1.19 4184 0.72 373560 804
Risc Zero 5.47 524288 - - 256742
Jolt 91.38 554595 0.19 - 477746
SP1 112.5 39479 0.509 - 2876912
Powdr 21.54 286652 - - -

Benchmark Machine Specifications

  • Macbook M1 Pro: Core 8, Memory 8 GB
  • AlmaLinux 8.10: Core 16, Memory 32 GB, Disk 1 TB
  • Windows 11: Core 16, Memory 32 GB, Disk 1 TB
  • Intel(R): Core 4 - Memory 8GB
  • Macbook M2 Pro: Core 16, Memory 16 GB

Installation

Each project has its own README file with instructions on how to install the dependencies and run the benchmarks.

Roadmap

Please refer to the issues.

Contributors

This project is part of the ZK and Scaling Bootcamp organized by Encode club.

Other benchmarks

Acknowledgments

Special thanks to the ZK and Scaling Bootcamp team for organizing this event and providing us with the opportunity to learn and contribute to the ZK space.