The loading times for pages often tend to be quite lengthy.

[Putarku]

LRR Version and OS
docker 0.9.28，The same issue occurs in version 0.9.30 as well.

Bug Details
Your Lanraragi has more than 50,000 manga documents, and sometimes it takes a long time to load the page. Additionally, searching for tags can also be slow. Given that your Docker container is running on a computer with 64GB of RAM and a CPU with the 10900K, there shouldn't be any hardware performance issues. All your manga documents are stored on a local 16TB mechanical hard drive.

At the moment, the most significant issue affecting usage is that after adding new manga documents, it takes a very long time to open the page. Even using the Tachiyomi app's Lanraragi plugin on a phone, there is a considerable delay (about 15-20 seconds) from opening until the actual gallery loads.

Matching Logs
There are no error logs, it's simply slow.

3.mp4

[Difegue]

There's no 0.9.28 so I'm assuming 0.9.22.

As for the issue itself, I'm guessing the indexing isn't fast enough for a search over 50k archives, so when the result isn't cached(after adding a new file or changing some metadata) it takes some time to get the results over.

This is just guessing though, I'd like to add some actual performance metrics to search at some point to try and pinpoint where it's slow.

[nspitko]

Happy to help track this down if/when metrics go in, there's a lot of major perf issues once the library gets large. I have a >100k library and it's not going well. Trying to migrate off ComicRack as it's no longer able to reliably save the database at this scale.

• Cold searches take ages. The server in question shouldn't be the limiting factor (64gb ram, all related storage is NVME, 16 core threadripper)
• Categories page takes an eternity to load and generally just hangs chrome. Sometimes the archives page will show "No categories" in the context menu (Presumably a timeout)
• Statistics page takes ages to load
• Batch tagging tool is almost entirely unusable (This should probably just be queued up on the server as even if the browser doesn't implode, metadata scans for the initial library import will take over a week at speeds that don't anger the servers)

During page loads most of the time is spent with /home/koyomi/la sitting on a core. Redis also sees some action in the first third or so of the query, but doesn't appear CPU bound like la is.

[psilabs-dev]

@nspitko There's several problems being mentioned, specifically:

1. Search: if search contains "$", it will take significantly longer, this is traced to a potentially very slow smembers Redis function. This can be the difference between a 3s search and a 50s search. I don't know if this is why search is slow for you.
2. The categories index generates an HTML that displays all archive entries. For 50k archives, that's a table with 50k rows. It only gets worse with more archives. See this line.
3. Batch tagging ops also have the same issue. The browser has to render a number of rows equal to the number of archives in your server.

I also have this issue on 2 LRR instances with 80k+ archives each; simply opening categories or batch ops breaks the browser. So I only do category operations via the API and delegate batch operations to background tasks executed by a service, because they take too much time.

But I'm not experiencing the slowdowns from the main issue, at most I have 10s loading times. Usually though they are 3-5s.

[Difegue]

See #570 for 2. and 3.

[maltbeverage]

I’m up to 536,912 items. Similar issues with long page loads but the thing is the issue is intermittent. Sometimes pages loads will be fine, one second or less for 100 items per page. Then for no apparent reason things will start loading slow, maybe 30-60 seconds per page.

I did notice at this size, import of new items just stops working. I have to call a rescan through the API to import new stuff. Maybe something with scanning is hosing up. In fact, after a reload of the container, pages load okay for a bit, then will start to slow down.

I’ve been meaning to deep dive further to see if I can isolate the issue. The size of my library is probably a bit out of scope of this software, but It certainly is a good performance test.