Vault v1.17.0: DynamoDB as HA storage causes random members to be sealed sometimes randomly.

[hajali-amine]

Describe the bug

We have a 3 node Vault cluster with DynamoDB as an HA backend.

Randomly, one of the members will get sealed all of a sudden thus making the cluster a 2 node cluster.

Once we unseal it, it gets back to normal.

To Reproduce

It happens randomly

Expected behavior

It doesn't get sealed.

Environment:
Vault v1.17.0
Ubuntu 20.04.6 LTS

Vault server configuration file(s):

cluster_name = "x"
max_lease_ttl = "768h"
default_lease_ttl = "768h"

disable_clustering = "False"
cluster_addr = "https://x:8201"
api_addr = "https://x:8200"

plugin_directory = "/usr/local/lib/vault/plugins"

listener "tcp" {
  address = "x:8200"
  cluster_address = "x:8201"
  tls_client_ca_file="xxxx.pem"
  tls_cert_file = "xxxx.pem"
  tls_key_file = "xxxx.key"
  tls_min_version  = "tls13"
  tls_disable = "false"
  }

backend "dynamodb" {
    table =          "xxxxx"
    ha_enabled =     "True"
    max_parallel =   "128"
    region =         ""
    access_key =     ""
    secret_key =     ""
    session_token =  ""
}


ui = true

log_format = "json"

telemetry {
    prometheus_retention_time = "720h"
    disable_hostname = true
  }

[stevendpclark]

Hello @hajali-amine,

Are there any logs you can post around when the node seals itself that could indicate the cause? With the information you've provided we don't have enough to investigate.

[hajali-amine]

here are the logs around the time it happened - there's nothing before or after these lines in here

vault-logs-pre-seal-20241201.log