Full Multi-query support onchain #321

daveroga · 2024-12-03T10:10:47Z

Implement multi-query support similar to existing requests now in Universal Verifier.

Manage multi-query (Query) independently and in the same way we are managing Requests in ZKPVerifierBase now (setQuery, getQuery, …). Avoid ZKP and generalize methods in Requests. setRequest, getRequest, submitResponse, …
Change _proofs mapping from address user to uint256 userID to keep identifier of the user.
Create 2 mappings for managing link between userAddress and userID . In this first version 1 userID will have only 1 address? Review best approach or if we have to keep an array of addresses.

mapping(address userAddress => uint256 userID) _user_address_to_id;
mapping(unit256 userID => address userAddress) _id_to_user_address; // check address[] ?

Manage Auth in specific type of requests. Auth requests.

requestId. 32 bytes (in Big Endian): 31-0x00(not used), 30-0x01(requestType), 29..0 hash calculated Id,

For requestType:
   - 0x00 - regular request
   - 0x01 - auth request

Define challenge based on the ethereum address in this first version
Review to reuse same Auth between requests
Figure out linkID between the Requests and Auth of the same Query.
Consider that more than one type of auth may be available there for specific multiRequest. And each of the multiRequests should define it’s own set of valid auth along with requests.
RequestId change from uint64 to uint256.

Link to Tech Spec for changes: https://www.notion.so/privado-id/Multi-query-Tech-Spec-13d4f86a875180e68fc8e3fa5362805e

coveralls · 2024-12-03T11:12:16Z

Pull Request Test Coverage Report for Build 12543564150

Details

239 of 299 (79.93%) changed or added relevant lines in 12 files are covered.
2 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-1.8%) to 82.269%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
contracts/validators/CredentialAtomicQueryV3Validator.sol	12	13	92.31%
contracts/verifiers/EmbeddedVerifier.sol	7	8	87.5%
contracts/verifiers/ValidatorWhitelist.sol	12	13	92.31%
contracts/validators/AuthV2Validator.sol	0	7	0.0%
contracts/validators/EthIdentityValidator.sol	0	16	0.0%
contracts/verifiers/Verifier.sol	150	184	81.52%

Files with Coverage Reduction	New Missed Lines	%
contracts/validators/AuthV2Validator.sol	1	0.0%
contracts/validators/CredentialAtomicQueryValidatorBase.sol	1	76.39%

Totals
Change from base Build 12413543528:	-1.8%
Covered Lines:	1121
Relevant Lines:	1273

💛 - Coveralls

contracts/verifiers/UniversalVerifierMultiQuery.sol

AndriianChestnykh

Good enough to discuss interface finalisation.
A few comments but not the full code check which is still to early to do

contracts/verifiers/UniversalVerifierMultiQuery.sol

…cation timestamp

contracts/verifiers/UniversalVerifierMultiQuery.sol

…r history

test/verifier/universal-verifier-multi-query.test.ts

Co-authored-by: daveroga <[email protected]>

AndriianChestnykh

I haven't completed review of all the files. However, these feedback is what I have now.

AndriianChestnykh · 2024-12-20T14:55:21Z

contracts/interfaces/IRequestValidator.sol

+     */
+    function verify(
+        bytes calldata proof,
+        bytes calldata data,


Rename data to param maybe?

Renamed to params in both IRequestValidator and IAuthValidator

AndriianChestnykh · 2024-12-20T14:56:39Z

contracts/interfaces/IRequestValidator.sol

+    /**
+     * @dev Get the group ID of the request query data.
+     * @param params Request query data of the credential to verify.
+     * @return Group ID of the request query data.
+     */
+    function getGroupID(bytes calldata params) external view returns (uint256);
+
+    /**
+     * @dev Get the verifier ID of the request query data.
+     * @param params Request query data of the credential to verify.
+     * @return Verifier ID encoded in the request query data.
+     */
+    function getVerifierId(bytes calldata params) external view returns (uint256);


Can we introduce this method instead of those two?
function getGroupID(bytes calldata params) external view returns (uint256 groupID, uint256 verifierID);

Created

struct RequestParams { uint256 groupID; uint256 verifierID; } function getRequestParams(bytes calldata params) external view returns (RequestParams memory);

AndriianChestnykh · 2024-12-20T15:33:43Z

contracts/interfaces/IVerifier.sol

+     * @param queryId The ID of the query
+     * @param query The query data
+     */
+    function setQuery(uint256 queryId, Query calldata query) external;


There is already queryId inside the Query struct, so no reason to put it separatelly

AndriianChestnykh · 2024-12-20T15:36:02Z

contracts/lib/VerifierLib.sol

    struct Proof {
        bool isVerified;
        mapping(string key => uint256 inputValue) storageFields;
        string validatorVersion;
-        uint256 blockNumber;
+        // TODO: discuss if we need this field
+        // uint256 blockNumber;
        uint256 blockTimestamp;
        mapping(string key => bytes) metadata;


mapping(string key => bytes) metadata; is not used anywhere, so let's remove it for now and use uint256[46] __gap instead

AndriianChestnykh · 2024-12-20T15:56:40Z

contracts/lib/VerifierLib.sol

+        uint256 blockTimestamp;
+        // This empty reserved space is put in place to allow future versions
+        // (see https://docs.openzeppelin.com/upgrades-plugins/1.x/writing-upgradeable#storage-gaps)
+        uint256[45] __gap;


uint256[46] __gap is better for round number of slots (quals to 50). Not critical but good to have

AndriianChestnykh · 2024-12-20T18:05:12Z

contracts/verifiers/RequestOwnership.sol

+        super.setRequests(singleRequests, groupedRequests);
+        for (uint256 i = 0; i < singleRequests.length; i++) {
+            _setRequestOwner(singleRequests[i].requestId, _msgSender());
+        }
+
+        for (uint256 i = 0; i < groupedRequests.length; i++) {
+            for (uint256 j = 0; j < groupedRequests[i].requests.length; j++) {
+                _setRequestOwner(groupedRequests[i].requests[j].requestId, _msgSender());
+            }
        }
    }


Maybe overriding _setRequest and adding _setRequestOwner(singleRequests[i].requestId, _msgSender()); to its logic may help avoid these two cycles

AndriianChestnykh · 2024-12-20T18:07:07Z

contracts/verifiers/UniversalVerifier.sol

+    ) public override(RequestOwnership, ValidatorWhitelist, Verifier) {
+        super.setRequests(singleRequests, groupedRequests);
+
+        for (uint256 i = 0; i < singleRequests.length; i++) {
+            emit RequestSet(
+                singleRequests[i].requestId,
+                _msgSender(),
+                singleRequests[i].metadata,
+                address(singleRequests[i].validator),
+                singleRequests[i].params
+            );
+        }
+
+        for (uint256 i = 0; i < groupedRequests.length; i++) {
+            for (uint256 j = 0; j < groupedRequests[i].requests.length; j++) {
+                emit RequestSet(
+                    groupedRequests[i].requests[j].requestId,
+                    _msgSender(),
+                    groupedRequests[i].requests[j].metadata,
+                    address(groupedRequests[i].requests[j].validator),
+                    groupedRequests[i].requests[j].params
+                );
+            }
        }


The same, try to override the _setRequest method itself with additional functionality

AndriianChestnykh · 2024-12-20T18:07:57Z

contracts/verifiers/UniversalVerifier.sol

+        super.submitResponse(authResponses, singleResponses, groupedResponses, crossChainProofs);
+        for (uint256 i = 0; i < authResponses.length; i++) {
+            emit AuthResponseSubmitted(authResponses[i].authType, _msgSender());
        }
-    }

-    /// @dev Verifies a ZKP response without updating any proof status
-    /// @param requestId The ID of the ZKP request
-    /// @param inputs The public inputs for the proof
-    /// @param a The first component of the proof
-    /// @param b The second component of the proof
-    /// @param c The third component of the proof
-    /// @param sender The sender on behalf of which the proof is done
-    function verifyZKPResponse(
-        uint64 requestId,
-        uint256[] memory inputs,
-        uint256[2] memory a,
-        uint256[2][2] memory b,
-        uint256[2] memory c,
-        address sender
-    )
-        public
-        override(RequestDisableable, ValidatorWhitelist, ZKPVerifierBase)
-        returns (ICircuitValidator.KeyToInputIndex[] memory)
-    {
-        return super.verifyZKPResponse(requestId, inputs, a, b, c, sender);
+        for (uint256 i = 0; i < singleResponses.length; i++) {
+            emit ResponseSubmitted(singleResponses[i].requestId, _msgSender());
+        }
+
+        for (uint256 i = 0; i < groupedResponses.length; i++) {
+            for (uint256 j = 0; j < groupedResponses[i].responses.length; j++) {
+                emit ResponseSubmitted(groupedResponses[i].responses[j].requestId, _msgSender());
+            }
+        }


The same note regarding overriding. Try to avoid the loops

AndriianChestnykh · 2024-12-20T18:08:32Z

contracts/verifiers/ValidatorWhitelist.sol

+        for (uint256 i = 0; i < singleRequests.length; i++) {
+            IRequestValidator validator = singleRequests[i].validator;
+            if (!isWhitelistedValidator(validator)) {
+                revert ValidatorIsNotWhitelisted(address(validator));
+            }
        }
-    }

-    /// @dev Verifies a ZKP response without updating any proof status
-    /// @param requestId The ID of the ZKP request
-    /// @param inputs The public inputs for the proof
-    /// @param a The first component of the proof
-    /// @param b The second component of the proof
-    /// @param c The third component of the proof
-    /// @param sender The sender on behalf of which the proof is done
-    function verifyZKPResponse(
-        uint64 requestId,
-        uint256[] memory inputs,
-        uint256[2] memory a,
-        uint256[2][2] memory b,
-        uint256[2] memory c,
-        address sender
-    ) public virtual override returns (ICircuitValidator.KeyToInputIndex[] memory) {
-        ICircuitValidator validator = getZKPRequest(requestId).validator;
-        require(isWhitelistedValidator(validator), "Validator is not whitelisted");
-        return super.verifyZKPResponse(requestId, inputs, a, b, c, sender);
+        for (uint256 i = 0; i < groupedRequests.length; i++) {
+            for (uint256 j = 0; j < groupedRequests[i].requests.length; j++) {
+                IRequestValidator validator = groupedRequests[i].requests[j].validator;
+                if (!isWhitelistedValidator(validator)) {
+                    revert ValidatorIsNotWhitelisted(address(validator));
+                }
+            }
+        }


The same. Try to avoid the loops

AndriianChestnykh · 2024-12-20T18:10:27Z

contracts/verifiers/ZKPVerifier.sol

As we introduce breaking change. You may just remove this file

daveroga and others added 22 commits November 26, 2024 11:18

new universal verifier multi-query

bf71699

Discussion results

74b24a1

Discussion results 2

16a8b27

update universal verifier multi query

f05104d

remove ZKP

779513f

add auth validators

8da12c7

Broader discussion results

ba75a5d

Add agreed universal verifier solution

ae9cec1

update universal verifier multi query with agreed design

790a0e3

query status and submit response

1858dce

get request type from 30 byte of the requestId

4527834

add EthIdentityValidator

bb82ff3

updates from review

e66a160

Discussion result

d416b9a

updates from discussion

a26861c

check response field values from linked response fields

0c2e792

fix some errors in compilation

16f8b5a

test universal verifier multi-query

4afb814

different validators for sig and authv2

8f55467

check specific storage values

657fa6c

overrides solidity compiler optimizations for specific contract

46d17ab

remove some methods

1e7b877

daveroga added 4 commits December 3, 2024 12:16

fix solhint

2942cdf

update from discussion

a91c1e7

set group of requests and check linkID

849e790

fix solhint

a02592c

daveroga force-pushed the PID-2709-full-multi-query-support-on-chain branch from b70f050 to a02592c Compare December 5, 2024 11:33

AndriianChestnykh requested changes Dec 5, 2024

View reviewed changes

updates from review

6b82ff0

fix cyclomatic-complexity

d35e4b4

AndriianChestnykh requested changes Dec 6, 2024

View reviewed changes

daveroga added 3 commits December 9, 2024 09:56

some fixes from review

3811b9c

updates from tech spec

4030067

add auth validator stub and fix test

6135f84

daveroga requested a review from AndriianChestnykh December 10, 2024 00:05

daveroga added 2 commits December 10, 2024 10:01

add changes for user authentication structs

b860ab0

fix solhint

fc7bc91

AndriianChestnykh requested changes Dec 10, 2024

View reviewed changes

contracts/verifiers/UniversalVerifierMultiQuery.sol Outdated Show resolved Hide resolved

daveroga added 3 commits December 11, 2024 08:40

reestructure link between userID and address and mapping for authenti…

c3bb7e3

…cation timestamp

auth proofs different structure

14d08fb

add custom errors to UniversalVerifierMultiQuery

3734eb5

AndriianChestnykh requested changes Dec 11, 2024

View reviewed changes

contracts/verifiers/UniversalVerifierMultiQuery.sol Outdated Show resolved Hide resolved

add __gap in proofs storage for future updates and array of proofs fo…

4c8e01d

…r history

AndriianChestnykh reviewed Dec 12, 2024

View reviewed changes

test/verifier/universal-verifier-multi-query.test.ts Outdated Show resolved Hide resolved

param names in returned structs from solidity

24888ad

daveroga requested a review from AndriianChestnykh December 16, 2024 16:40

AndriianChestnykh and others added 6 commits December 17, 2024 17:45

Initial verifier authentication (#323)

14ca44b

Co-authored-by: daveroga <[email protected]>

updates for iteration 2

636f590

fix test multi-query

88ff859

update verifiers tests

fb2a5b4

fix validators tests and solhint

faa2d0b

updates merge from master

a8c8653

AndriianChestnykh requested changes Dec 20, 2024

View reviewed changes

partial changes review

6336ad5

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Full Multi-query support onchain #321

Full Multi-query support onchain #321

daveroga commented Dec 3, 2024 •

edited

Loading

coveralls commented Dec 3, 2024 •

edited

Loading

AndriianChestnykh left a comment

AndriianChestnykh left a comment

AndriianChestnykh Dec 20, 2024

daveroga Dec 23, 2024

AndriianChestnykh Dec 20, 2024

daveroga Dec 30, 2024

AndriianChestnykh Dec 20, 2024

AndriianChestnykh Dec 20, 2024

AndriianChestnykh Dec 20, 2024

AndriianChestnykh Dec 20, 2024 •

edited

Loading

AndriianChestnykh Dec 20, 2024

AndriianChestnykh Dec 20, 2024

AndriianChestnykh Dec 20, 2024

AndriianChestnykh Dec 20, 2024

Full Multi-query support onchain #321

Are you sure you want to change the base?

Full Multi-query support onchain #321

Conversation

daveroga commented Dec 3, 2024 • edited Loading

coveralls commented Dec 3, 2024 • edited Loading

Pull Request Test Coverage Report for Build 12543564150

Details

💛 - Coveralls

AndriianChestnykh left a comment

Choose a reason for hiding this comment

AndriianChestnykh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AndriianChestnykh Dec 20, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

daveroga commented Dec 3, 2024 •

edited

Loading

coveralls commented Dec 3, 2024 •

edited

Loading

AndriianChestnykh Dec 20, 2024 •

edited

Loading