ArcadeDB Deployment Architecture

[yyj8]

ArcadeDB only supports master-slave architecture and does not support true distributed clusters. Can we achieve horizontal expansion of cluster capabilities?

[lvca]

@yyj8 ArcadeDB has a master/slave (leader/followers) replication system, the most common replication across consistent DBMSs. If you are thinking to master/master replication, it's quite the opposite: there are many issues with consistency and performance.

ArcadeDB replication is super fast, compared with OrientDB, which is multi-master, and is up to 500X (!) faster with only 5 nodes.

I suggest you benchmark ArcadeDB across 3 or 5 nodes with any other DBMS and see the results :-)

[criminosis]

@lvca I might be circling around the same question the OP was asking and maybe others will end up here asking the same.

If I understand the leader/follower architecture you're referencing that seems to imply the graph still needs to "fit" on a single host. Is that accurate?

I've been using JanusGraph for a graph deployment experiment that leverages the full-text feature it provided via an additional backend of ElasticSearch. I'd say my average document size is 10MB but I wanted to model a graph of relationships from one document to another by traversing around the graph leveraging full-text as a entry point to arbitrary documents.

The disjoint systems however have become more a larger operational burden than I expected so I started looking around for something more vertically integrated and ended up at ArcadeDB. It seems to check a lot of boxes for my use case, except for expansion handling.

I think one path would be "phase in" larger nodes, to avoid downtime. I'd be running this in a self-managed k8s environment on top of EC2 instances. So I could see a path that I create new EC2 instances, with larger EBS volumes add them to the cluster as followers and let them naturally replicate and then phase out the older, smaller, instances. But that eventually hits the limit of a single EC2 instance's allowance of EBS volume mounts and the EBS volume sizes themselves)

I'm could imagine doing some kind of RAID setup with said EBS volumes so they appear to be a single volume to ArcadeDB, unless ArcadeDB supports multiple data directories, but the point is the same if the entire dataset needs to still fit on one host for ArcadeDB.

It seemed like one snippet in the docs (https://docs.arcadedb.com/#_add-and-remove-buckets-from-types) alluded to possibly bucket partitions being distributed around multiple servers in the future:

In the future could be possible to move buckets (and associated sub-indexes) to different servers for distributed partitioning.

Obviously the "in the future" is not lost on me, but I wanted to ask if I'm interpreting that possible future feature correctly that it would allow a dataset to exceed the confines of a single machine, or if I'm misunderstanding what that means.

[lvca]

That's right: ArcadeDB doesn't support multi node sharding. Internally each type is mapped on 1-N buckets, where a bucket is a shard. You can even plugin your bucket selection strategy (shard selection strategy). But it will not be distributed.

This was a reasoned choice to simplify the distributed architecture going with a RAFT election + master/slave handbook replication that just works. In OrientDB we weren't ever able to have a real multi-master + sharding that was rock solid.

Something could help in your use case, maybe, is implementing the federated database with a RID that is #<shard>:<bucket>:<position> and implement a remote lookup. This simple addition will allow to have distributed graphs, but then the update/insertion wouldn't be ACID cross servers, so it's the client that should take care of distributing stuff around shards/servers.

[criminosis]

Something could help in your use case, maybe, is implementing the federated database with a RID that is #:: and implement a remote lookup. This simple addition will allow to have distributed graphs, but then the update/insertion wouldn't be ACID cross servers, so it's the client that should take care of distributing stuff around shards/servers.

I'm assuming that's in reference to over here: #43 ?

I'll think it through, thanks for the fast response @lvca!

[lvca]

Correct