Can SkyWalking OAP Handle 24TB/day with 30-Day Retention?

[linshenfanying]

What is the maximum data volume SkyWalking can support?
We have a requirement where we currently generate about 8 TB of data per day, expected to grow to 24 TB per day in three years. We need to retain data for 30 days. Can the SkyWalking OAP Server handle this scale?

[wu-sheng]

It is possible. I would recommend BanyanDB for this case. And you could put 7+ days data in cold data node.
Ref to https://skywalking.apache.org/docs/skywalking-banyandb/next/operation/benchmark/benchmark-hybrid/
You could use 16u16g node cluster(5-7 nodes) with ssd for hot and warn, sata for cold nodes.

If you prefer Elasticseaech, the cluster spec is crazy, at least 30+ high spec node(each 32u64g).

[wu-sheng]

Which industry and company are you working on? This spec is very special, and we only saw China top telecom and insurance companies having such large scale.

[linshenfanying]

Automotive manufacturer.

[wu-sheng]

Got it. Thanks for sharing. If you want to discuss more tech details, welcome to join slack channels.