Adds gen_substrs support

[ewynx]

Description

Note: this is added upon the original Noir support @olehmisar built.

Extraction of 1 or more substrings can be requested in both settings (raw, decomposed):

• raw: pass on a json file with -s <SUBSTRS_JSON_PATH> that contains the state transitions that should be revealed
• decomposed: mark the part as public

Note that the number of substrings that have to be extracted is not necessarily known beforehand. For example regex (substr=(a|b)+ )+ where the substring to be extracted is (a|b), should extract 2 substrings for substr=a substr=b and 1 substring for substr=b.

The information of where substrings should be extracted within the regex is added to substrings in the RegexAndDFA. In the case of raw only substring_ranges is filled so this is what this implementation uses.

How it works

The adjustments when gen_substrs = true are:

• return type of regex_match becomes a Vector of BoundedVec. We don't know the substring length beforehand, but it is bounded by N. And we don't know how many substrings are returned
• fill the substrings (BoundedVec) with the bytes when in the corresponding state. It checks when it starts and stops filling the substring

If the bool is false, nothing changes.

Note: For the raw setting the boolean was by default set to true, I changed it to false, just like in the decomposed setting. Asked in the TG group if this is ok (open question).

Testing

Decomposed

Create a file called substring_test.json containing:

{
  "parts":[
      {
          "is_public": false,
          "regex_def": "email was meant for @"
      },
      {
          "is_public": true,
          "regex_def": "[a-z]+"
      },
      {
          "is_public": false,
          "regex_def": "\\. And the next substring:"
      },
      {
          "is_public": true,
          "regex_def": "[a-z]+"
      }
  ]
}

Run

cargo run --bin zk-regex decomposed -d substring_test.json --noir-file-path testfile.nr -g true

Raw

This is the example from the README.md; create ./simple_regex_substrs.json containing:

    {
        "transitions": [
            [
                [
                    2,
                    3
                ]
            ],
            [
                [
                    6,
                    7
                ],
                [
                    7,
                    7
                ]
            ],
            [
                [
                    8,
                    9
                ]
            ]
        ]
    }

Run

cargo run --bin zk-regex raw -r "1=(a|b) (2=(b|c)+ )+d" -s ./simple_regex_substrs.json --noir-file-path simple_regex_substrs.nr -g true`

For example, the corresponding Noir code should return for input 1=b 2=bbcb 2=c 2=bb d the following 4 substrings: b, bbcb, c, bb, and d.

Note that the second substring is marked by 2 transitions [6,7] and [7,7], which is needed for the circom impl. However, in Noir we only need to know the substring is in state 7, so this is the information that is taken from substring_ranges.

Problem*

Resolves

Summary*

Additional Context

PR Checklist*

• I have tested the changes locally.
• I have formatted the changes with Prettier and/or cargo fmt on default settings.

[ewynx]

#8 contains this feature, closing this PR.