One Statamic, two websites, 10k+ entries per website

[carstenjaksch]

Hi,

I am considering Statamic for a client and played around a bit the last days. However, I didn't find much about the questions in my head.

The client has two websites (currently separate WordPress instances). I would like to move both to one Statamic installation, because although they are different brands, they share the design and some content.

Now comes the fun part: Each website will grow to 10k+ entries (landing pages) over time. My assumption is that this scenario cries out for a database instead of flat files.

My questions:

1. Is it, somehow, possible, to store these entries in one file (csv/json/yaml) instead of a database for easier content management and retrieve one entry (performant) when its page is visited/cached? There will be no index page. I have some idea about the CSV part, but the entire file will be crawled every time one entry is visited.
2. If the file approach is not feasible, how will the CP react to such high number of entries (stored in a DB)?
3. What should I consider otherwise to have a buttery smooth user experience (both front- and backend)? Maybe split the entries into multiple collections? Will that have a performance impact? Or something else?

Any experience with a high amount of entries would be much appreciated. 🙏

[ryanmitchell]

The honest answer to whether you need a database is 'it depends'. We have sites that have in that region of entries and they are flat file, and we have others that have less than needed the database.

Some factors that affect the decision:

1. Server spec - if you are doing a lot of reading from the disk you need a server with good i/o throughput, we have found the difference between different supplier to be massive in this regard (eg AWS bad, Hetzner good)

2. Amount of content in the entries - if you are using a page builder approach then it's likely you'll hit performance issues quicker. Still possible to work around it.

To answer your questions:

1. Yes, use a page builder approach like 'peak' provides an example of
2. If you switch to a database use the eloquent driver, its officially supported
3. You could consider setting up search indexes for each collection in the CP to make them perform faster in listing views

[carstenjaksch]

Thank you for the quick response @ryanmitchell!

Indeed, I plan to use Hetzner for hosting. Good to hear that their NVMe SSDs are performant enough for such a task.

Another thing is, what I forgot to mention, the LPs are based on a default template. So only one page uses a page builder (I will use Peak too, great batteries-included starter).

Each entry has a fraction of content (just fields) that will be merged with the page builder content. This should make each entry file rather small.

Perhaps flat files could be enough after all.

[ryanmitchell]

I would start there either way - you can easily migrate to eloquent if you find speed is an issue, there are import commands as part of the install.

[carstenjaksch]

True, thank you for the advice. Really appreciate that.

[duncanmcclean]

Hmm, your question seems unrelated to this discussion. Maybe ask on our Discord server or start your own discussion.