Add Ability to Define Custom Metadata for KV Engines

[eternalyperplxed]

Is your feature request related to a problem? Please describe.
We currently have KV engines across 2 primary namespaces for individual git repositories, EKS clusters, and multiple applications/microservices. As our KV engine count grows (current count is >80), being able to group and track KV engines based on custom metadata would enable to us to track things such as internally-defined categories without relying on naming conventions.

Describe the solution you'd like
Similar to defining custom metadata on individual secrets, being able to define custom metadata at the KV engine level.

Describe alternatives you've considered
Relying on enforced naming conventions or static lists of KV engines maintained outside of Vault.

Explain any additional use-cases
Mostly the same use cases that exist for metadata on secrets.

[bosouza]

Hello! I went through the original proposal of adding custom key metadata in kv-v2, and found that it addressed the problem of associating metadata to a secret under a different ACL path, such that policies would be able to grant permissions to the metadata only. Before then users would add metadata as part of the secret itself, but then of course ACL policies could only grant permissions to both reading the data and metadata of the secret.

For your issue, couldn't you create an engine-metadata secret for each of your kv-v2 engines, then use the custom metadata endpoints to manage this secret as the metadata for the engine itself? It's not the prettiest but building a whole new metadata concept would need some strong justification in terms of use cases. If you'd like to discuss more about this feature request please describe in more detail why the workaround above wouldn't suffice, how you'd like this new metadata concept to interact with other kv-v2 APIs, and also if you see this being extended to a more general "mount custom metadata" for any kinds of mounts or if it make more sense for the kv-v2 engine only.