Adding NISTDSA API to support ML-DSA-44 and ML-DSA-87

[jakemas]

Issues:

Resolves #CryptoAlg-2725

Description of changes:

This is a sizable PR, I've worked extremely hard to keep the size down in a single PR, but with ML-DSA touching X.509, PKEY, ASN1, there are a lot of changes that need to happen simultaneously to preserve library functionality, so I will document this PR well.

1. PKEY structure changes

This PR adds ML-DSA-44 and ML-DSA-87 to AWS-LC. Before this PR AWS-LC only supported ML-DSA-65, as such, we were utilizing the void *ptr of evp_pkey_st, rather than a distinct structure for ML-DSA.

This PR introduces the new structure NISTDSA_KEY * nistdsa_key; inside evp_pkey_st to support ML-DSA and any additional FIPS digital signature algorithms provided by NIST, should AWS-LC wish to include them in the library.

  union {
    void *ptr;
    RSA *rsa;
    DSA *dsa;
    DH *dh;
    EC_KEY *ec;
    KEM_KEY *kem_key;
    NISTDSA_KEY * nistdsa_key;
  } pkey;

While the structure DSA already exists it does not provide support for FIPS-like signature APIs (see more at NIST api conventions. Rather than create a PKEY struct of the form MLDSA_KEY that can only support ML-DSA, this is an opportunity to build support for future signature algorithms that have similar. calling structures, de-randomized testing modes, API converntions, etc. by making a more generic structure type -- much like we did for KEM_KEY. Under the design in this PR, adding SLH-DSA to the PKEY struct would be very simple, as it conforms to a NISTDSA_KEY in design. So too will be true of any additional signature algorithms produced by NISTs call for additional signature algorithms.

As, such, NISTDSA_KEY utilizes a new structure to hold public/secret keys, as well as a NISTDSA struct which defines signature algorithm specific information:

struct nistdsa_st {
  const NISTDSA *nistdsa;
  uint8_t *public_key;
  uint8_t *secret_key;
};

typedef struct {
  int nid;
  const uint8_t *oid;
  uint8_t oid_len;
  const char *comment;
  size_t public_key_len;
  size_t secret_key_len;
  size_t signature_len;
  size_t keygen_seed_len;
  size_t sign_seed_len;
  const NISTDSA_METHOD *method;
  const EVP_PKEY_ASN1_METHOD *asn1_method;
} NISTDSA;

This allows us to use a single PKEY structure for all NIST FIPS signature algorithms, much like the existing PKEY struct KEM_KEY *kem_key for KEMS. As a consequence, we are now able to define signature methods such as:

DEFINE_LOCAL_DATA(NISTDSA_METHOD, sig_ml_dsa_65_method) {
  out->keygen = ml_dsa_65_keypair;
  out->sign = ml_dsa_65_sign;
  out->verify = ml_dsa_65_verify;
}

DEFINE_LOCAL_DATA(NISTDSA, sig_ml_dsa_65) {
  out->nid = NID_MLDSA65;
  out->oid = kOIDMLDSA65;
  out->oid_len = sizeof(kOIDMLDSA65);
  out->comment = "MLDSA65 ";
  out->public_key_len = MLDSA65_PUBLIC_KEY_BYTES;
  out->secret_key_len = MLDSA65_PRIVATE_KEY_BYTES;
  out->signature_len = MLDSA65_SIGNATURE_BYTES;
  out->keygen_seed_len = MLDSA65_KEYGEN_SEED_BYTES;
  out->sign_seed_len = MLDSA65_SIGNATURE_SEED_BYTES;
  out->method = sig_ml_dsa_65_method();
  out->asn1_method = &nistdsa_asn1_meth;
}

much like the existing kem.c file.

These structures will be incredibly useful for subsequent PRs, in which we will be adding de-randomized APIs to support FIPS validation and KATs from seeds. In effect, we will be extending the method to include:

DEFINE_LOCAL_DATA(NISTDSA_METHOD, sig_ml_dsa_65_method) {
  out->keygen = ml_dsa_65_keypair;
  out->keygen_internal = ml_dsa_65_keypair_internal;
  out->sign = ml_dsa_65_sign;
  out->sign_internal = ml_dsa_65_sign_internal;
  out->verify = ml_dsa_65_verify;
}

The directory dilithium/p_dilithium3.c is used to house generic PKEY operations, as well as NISTDSA specific EVP functions such as EVP_PKEY_nistdsa_set_params, EVP_PKEY_nistdsa_set_params, which work in a similar way to p_kem.c functions EVP_PKEY_CTX_kem_set_params and EVP_PKEY_kem_set_params.

The directory dilithium/p_dilithium3_asn1.c is used to house generic asn.1 operations for NIST FIPS DSA algorithms.

Rather than rename/relocate these files in this PR (which will convolute the CR), I will move these files in a subsequent PR.

2. Internal Design

We introduce new structures: NISTDSA_METHOD, NISTDSA, NISTDSA_KEY which are very similar to how both EC Crypto, and the KEM API is implemented. The figure above shows the newly proposed structures and their integration into the existing EVP structures.

• NISTDSA_METHOD is a table of function pointers with 5 functions defined for a NISTDSA: key generation, key generation internal (for testing/FIPS), sign, sign internal (for testing/FIPS), and verify. Every NISTDSA implementation has to implement this API, for example sig_ml_dsa_44_method implements the three functions for ML-DSA-44 (analogous to EC_METHOD and for example, EC_GFp_nistz256_method).
• NISTDSA is a structure that holds basic information about the DSA: the id, size of parameters, and the pointer to the implementation of the corresponding NISTDSA_METHOD.
• NISTDSA_KEY structure is a helper structure that holds pointers to public and secret key and the pointer to the corresponding NISTDSA.
• NISTDSA_PKEY_CTX is a helper structure used to store DSA parameters in an EVP_PKEY_CTX object (the same as EC_PKEY_CTX). Since NISTDSA has everything we need, that’s what we store in NISTDSA_PKEY_CTX.

3. OID changes

We now have real OIDs available from https://csrc.nist.gov/projects/computer-security-objects-register/algorithm-registration. These have been added to the obj files and automatically populated by go run objects.go. We include the following NIDs for ML-DSA:

//2.16.840.1.101.3.4.3.17
static const uint8_t kOIDMLDSA44[]  = {0x60, 0x86, 0x48, 0x01, 0x65, 0x03, 0x04, 0x03, 0x11};
//2.16.840.1.101.3.4.3.18
static const uint8_t kOIDMLDSA65[]  = {0x60, 0x86, 0x48, 0x01, 0x65, 0x03, 0x04, 0x03, 0x12};
//2.16.840.1.101.3.4.3.19
static const uint8_t kOIDMLDSA87[]  = {0x60, 0x86, 0x48, 0x01, 0x65, 0x03, 0x04, 0x03, 0x13};

and a "top-level" NID EVP_PKEY_NISTDSA (similar to EVP_PKEY_KEM to subset all NIST FIPS DSA algorithms. Much like for the KEM API, we include functions SIG_find_dsa_by_nid(int nid) to return actual algorithm specific methods.

4. Testing structure changes

Prior to this PR all testing for ML-DSA-65 was done as a series of g-tests. As we now have the ability to get algorithm/security level specific parameters from the NISTDSA struct, I have overhauled the testing suite to be parameterized over the currently supported signature algorithms:

static const struct ML_DSA parameterSet[] = {
  {"MLDSA44", NID_MLDSA44, 1312, 2560, 2420,  "dilithium/kat/mldsa44.txt", mldsa44kPublicKey, mldsa44kPublicKeySPKI, 1333},
  {"MLDSA65", NID_MLDSA65, 1952, 4032, 3309,  "dilithium/kat/mldsa65.txt", mldsa65kPublicKey, mldsa65kPublicKeySPKI, 1973},
  {"MLDSA87", NID_MLDSA87, 2592, 4896, 4627,  "dilithium/kat/mldsa87.txt", mldsa87kPublicKey, mldsa87kPublicKeySPKI, 2613},
};

For each of the above parameter sets we run:

TEST_P(MLDSAParameterTest, KAT)
TEST_P(MLDSAParameterTest, KeyGen)
TEST_P(MLDSAParameterTest, KeyCmp) 
TEST_P(MLDSAParameterTest, KeySize) 
TEST_P(MLDSAParameterTest, NewKeyFromBytes) 
TEST_P(MLDSAParameterTest, RawFunctions) 
TEST_P(MLDSAParameterTest, SIGOperations) 
TEST_P(MLDSAParameterTest, MarshalParse) 

5. X.509 changes

Changes to X.509 code have been minimized to only the required changes to support this PR. This is already a big PR and I want to avoid expansion wherever possible. For X.509 the changes predominantly are regarding the change of NIDs from the old Dilithium NIDs to the new NIST DSA NIDs/OIDs.

As the OIDs changed, so too much the test certificates kDilithium3Cert, kDilithium3CertNull, and kDilithium3CertParam. These have been regenerated using the following:

Tool used https://github.com/google/der-ascii

1. First generate a valid test certificate in PEM encoding, say cert.pem
2. Convert PEM to DER: openssl x509 -in cert.pem -out cert.der -outform DER
3. Convert DER to ASCII: der2ascii -i cert.der >> cert.txt
4. Make edits to cert.txt as desired, say; certnew.txt
5. Convert ASCII to DER: ascii2der -i certnew.txt >> certnew.der
6. Convert DER to PEM: openssl x509 -in certnew.der -inform DER -out certnew.pem -outform PEM

6. Speed Tool

ML-DSA-44 and ML-DSA-87 have been added to the speed tool. To facilitate the new APIs used in this PR, the API version has been incremented by 1. The speed tool now produces benchmarks for all three parameter levels, example output:

Did 4870 MLDSA44 keygen operations in 1048820us (4643.3 ops/sec)
Did 1230 MLDSA44 signing operations in 1012232us (1215.1 ops/sec)
Did 4840 MLDSA44 verify operations in 1093069us (4427.9 ops/sec)
Did 2871 MLDSA65 keygen operations in 1051869us (2729.4 ops/sec)
Did 774 MLDSA65 signing operations in 1039468us (744.6 ops/sec)
Did 2893 MLDSA65 verify operations in 1089059us (2656.4 ops/sec)
Did 1600 MLDSA87 keygen operations in 1008766us (1586.1 ops/sec)
Did 654 MLDSA87 signing operations in 1179044us (554.7 ops/sec)
Did 1650 MLDSA87 verify operations in 1049198us (1572.6 ops/sec)

Callouts:

• dilithium/p_dilithium3.call dilithium3 pkey specific functionality is now generic
• dilithium/p_dilithium3_asn1.c all dilithium3 asn1 specific functionality is now generic
• dilithium/p_dilithium_test.cc all dilithium3 testing specific functionality is now generic
• dilithium/internal.h is a new file, added to maintain consistency with fipsmodule/kem/internal.h

Testing:

See above for description of changes to testing framework.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and the ISC license.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 78.65%. Comparing base (b77a698) to head (cf5ae3f).
Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1949      +/-   ##
==========================================
- Coverage   78.68%   78.65%   -0.04%     
==========================================
  Files         585      585              
  Lines      100854   100931      +77     
  Branches    14298    14315      +17     
==========================================
+ Hits        79357    79384      +27     
- Misses      20863    20911      +48     
- Partials      634      636       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.