Cross-platform Always Encrypted proposal

[edwardneal]

I've noticed that #3014 is marked as needing partner approval, so this discussion is to show my end goal and to add some context to make that review easier.

Background context

At present, SqlClient supports Always Encrypted on Linux, but the in-box SqlColumnEncryptionKeyStoreProvider implementations are Windows-only. The only way to achieve cross-platform AE support is by using a custom implementation (such as the Azure Key Vault one.) Porting these capabilities is relatively low-hanging fruit.

My end goal is for SqlClient to have in-box cross-platform support for Always Encrypted with similar semantics on both Windows and Linux. Both Windows and Linux should be able to encrypt/decrypt data using AE based on a certificate or based on a key stored in a hardware security module such as a TPM.

Always Encrypted provider changes

We currently have three in-box providers:

• SqlColumnEncryptionCertificateStoreProvider
• SqlColumnEncryptionCngProvider
• SqlColumnEncryptionCspProvider

All three have Windows support. #3014 adds Linux and macOS support to CertificateStoreProvider, with the caveat that it's limited by the underlying OS - it can only refer to the per-user personal certificate store on Linux. I'm personally content with that caveat - it's easy to understand why (Linux doesn't have a per-machine personal certificate store.)

This just leaves the notion of supporting TPMs on Linux. CngProvider and CspProvider are very tightly bound with Windows - CNG and CSP are Windows-only concepts and naturally can't be ported as-is to Linux. To provide a way for Linux clients to get the RSA key used for Always Encrypted, I'm thinking about a "SqlColumnEncryptionOpenSslProvider" implementation in SqlClient. This would naturally be an API change, and I've not yet prototyped it... but if the prototyping went well, I'd request that.

This would throw on Windows (and thus, on .NET Framework.) On Linux under .NET 9.0, it'd be nearly identical to CngProvider and CspProvider, but the RSA class would actually be an RSAOpenSsl instance. The new SqlColumnEncryptionOpenSslProvider would parse the master key path it receives from SQL Server, translating it into an OpenSSL provider name and key URI in the same way that CngProvider does. We can then pass those to SafeEvpPKeyHandle.OpenKeyFromProvider, and pass that to the RSAOpenSsl constructor. The existing AE implementation would be able to work with the RSA functionality from there as normal.

Between the two pieces of work, Windows and Linux would have similar capabilities: cross-platform Always Encrypted support which allows us to encrypt column values via a certificate, via an RSA key in a TPM, or via the existing Azure Key Vault provider.

[David-Engel]

I love this. I will loop in the AE feature team for input. Given the time of year, though, it might take a while to see any movement.

[saurabh500]

@edwardneal thanks for starting this.
The key store providers enable interaction with various stores to untimately be able to call unwrap/decrypt on the encrypted symmetric key.

On Windows the Certificate in the store can be a container for the Asymmetric key, for AKV, the encrypted DEK is sent to the Key vault for decryption etc.

In your model you have proposed a OpenSSL provider. I am interested in understanding what the user experience looks like here.

As you mentioned, the certificate (as an asymmetric key container) can be placed on the filesystem. (likely protected by file system permissions and also protected by an optional but recommended passphrase).

Will the Provider accept the Certificate file system path, and then use OpenSSL to decrypt/unwrap the encrypted Data encryption key using this certificate? Also is OpenSSL an implementation detail, that we happen to use OpenSSL to unwrap the payload on Linux?

If OpenSSL is an implementation detail, then should the provider be used to accept a file from the filesystem, which happens to use the OpenSSL APis on Linux, and its equivalent counterpart on Windows for decryption? This way the key provider can be certificate file based, and can work on both windows and linux, and can be orthogonal to operating system/platform.

[edwardneal]

Thanks @saurabh500.

There are two strands to this idea. The certificate provider is the most direct of the two: we can make this cross-platform, with the constraint that it only works for certificates in the user's personal certificate store (not the machine's.) From the user's perspective, the existing functionality lights up; the certificate is located by the thumbprint specified in the master key path passed to the provider. This provider doesn't directly involve itself with OpenSSL at all. It also doesn't involve working with file paths - I hadn't thought about that and can see the appeal. My only concern is a situation where SqlClient is trying to access the data on both OSes: at least one of the paths will be incompatible.

I'd specifically see an OpenSSL AlwaysEncrypted provider as analogous to the CNG and the CSP providers: a platform-specific way to retrieve the RSA key from hardware. This is the second strand: where it might not be practical to make exactly the same code path work on both OSes, we'd open a second path for users to retrieve their RSA key from a TPM or similar.

Technically, the use of OpenSSL is an implementation detail. OpenSSL looks up a provider (such as the tpm2 OpenSSL provider) and requests the first RSA key within that provider which matches a given URI. I think it's a judgement call whether we'd add a SqlColumnEncryptionOpenSslProvider class or modify the SqlColumnEncryptionCngProvider class. I'm slightly in favour of having a separate class though: we already expose the implementation detail of storing the key in a CNG or a CSP provider, and I think it'd be confusing for Linux to start instantiating an object which refers to the Windows-specific terminology of CNG. Having a class with multiple OS-dependent implementations might be difficult when both implementations need to run from the same master key path (for example, two instances of SqlClient running on both Windows and Linux.)

To crystallise this a little, pseudocode for the CNG, CSP and OpenSSL RSA key retrieval logic would look similar to the below. I'd expect the rest of the providers to remain broadly the same:

// masterKeyPath is [CNG provider name]/[CNG key identifier]
RSA GetKeyFromCng(string masterKeyPath)
{
	GetCngProviderAndKeyId(keyPath, out string cngProviderName, out string keyIdentifier);
	
	CngProvider cngProvider = new CngProvider(cngProviderName);
	CngKey cngKey = CngKey.Open(keyIdentifier, cngProvider);
	
	return new RSACng(cngKey);
}

// masterKeyPath is [CSP provider name]/[key name]
RSA GetKeyFromCsp(string masterKeyPath)
{
	GetCspProviderAndKeyName(keyPath, out string cspProviderName, out string keyName);
	
	RegistryKey providerKey = Registry.LocalMachine.OpenSubKey("SOFTWARE\Microsoft\Cryptography\Defaults\Provider\" + cspProviderName);
	int providerType = (int)key.GetValue("Type");
	CspParameters cspParams = new CspParameters(providerType, cspProviderName, keyName)
	{
		Flags = CspProviderFlags.UseExistingKey
	};
	
	return new RSACryptoServiceProvider(cspParams);
}

// masterKeyPath could be [OpenSSL provider name]/[key URI]
RSA GetKeyFromOpenSsl(string masterKeyPath)
{
	GetOpenSslProviderAndKeyUri(keyPath, out string osslProviderName, out string keyUri);
	
	SafeEvpPKeyHandle evpKey = SafeEvpPKeyHandle.OpenKeyFromProvider(osslProviderName, keyUri);
	
	return new RSAOpenSsl(evpKey);
}

It's completely dependent upon testing of the OpenSSL layer though. We'd need to confirm that RSAOpenSsl's Encrypt, Decrypt, SignData and VerifyData methods work, and I've not yet done that. The certificate provider is the easiest to implement!