webui: add glossary preview buttons

[glowinthedark]

• and browse buttons 🔎 for input and output files that open browse.html displaying top NN entries, max GET param in the url controls the max number of entries as in e.g. http://127.0.0.1:1984/browse?path=/path/to/oald.mdx&max=42&format=OctopusMdict
• browse.html also allows some rudimentary search (the &word=something parameter in the URL)
• if the URL contains no max=123 parameter then DEFAULT_MAX_BROWSE_ENTRIES is used defined in server_ws_http.py
• add favicon.ico
• ui_controller.py: resolve home ~ placeholder in input/output paths
• server_ws_http.py: override standard library super.translate_path() to allow resource resolution from additional web roots, ie .js .css when browsing .mdx files
• add Browse link in webui menu for quickly opening browse.html in a new tab
• the dictionary format in browse.html is just a text field, didn't really bother about using a proper select with dropdown items, the way it's done in index.html; can be done in the future if there is a real need.

@ilius the entire idea came up after discovering view_glossary.py in the repo and thought it could be more useful and friendlier to use if hooked to the webui; searching in big files is of course slow and can take forever since it's just looping over entries; more of a poc, and not really meant to work as a dictionary since that will require some kind of indexing and better search approach

[ilius]

Interesting.

But "Browse" sounds like "browse files", something like a File Chooser or File Manager.

Should be renamed to something else, like "Glossary Preview".
In the menu, file name (glossary_preview.html), the code, etc.

searching in big files is of course slow and can take forever since it's just looping over entries

We use iterators / generators, so if you only read a small number of entries (then break out of loop), it should not take long.

[glowinthedark]

We use iterators / generators, so if you only read a small number of entries (then break out of loop), it should not take long.

yes, I noticed, clicking the 🔎 from index.html only takes the 42 top entries so it's generally pretty quick to render something; it's only the search that can take forever, thought it would be fun to be able to search, as a way of verifying the converted result or the input file.

I suppose that it would probably be mixing scopes between a conversion tool and a dictionary, but as a debugging extension it might be useful — for example, as a way to quickly preview a dictionary without installing it into a dedicated app before deciding if you need to convert it or not.

[ilius]

Right.
I don't think search makes sense here.

BTW I created this plugin just for purpose of viewing a dictionary in web browser (kinda like view_glossary.py but in browser).
You can find a certain headword in the index file (using grep in command line) and go to that page to see it.

[glowinthedark]

BTW I created this plugin just for purpose of viewing a dictionary in web browser

In fact, I've used the html_dir plugin before, but not as a way of browsing dictionaries but for extracting resources; for big dictionaries with hundreds of thousands of entries and additional resources like audio and images it generates way too many files, and I'd rather have them packed in some kind of container like sqlite or something optimized for searching like slob. With the current preview in the webui I do actually find it quite practical to be able to peek and inspect some dictionaries without needing to convert and install them.

[glowinthedark]

pushed update to use a static select dropdown for the format instead of a text field

---

[ilius]

So the Search function might need to read the entire glossary (if matching entries are fewer than then limit).
This is not nice.
Specially if glossary is big and user does not know this behavior.

[glowinthedark]

@ilius

So the Search function might need to read the entire glossary (if matching entries are fewer than then limit).
This is not nice.
Specially if glossary is big and user does not know this behavior.

yes, totally agree that it is not nice, and I appreciate that you raised this concern since imo it does need to be addressed properly, but don't have a 💯 clear idea of what would be an appropriate UI/UX here — (1) removing the search entirely will reduce the usability, (2) adding a persistent warning always displayed on the page will be ugly, reduce usability and be too annoying; how about (3) adding a modal dialog that will appear once and then if accepted by clicking the I understand button, a flag is stored in localStorage so that it doesn't appear anymore; the warning message displayed in the modal would need to be worded properly so that the message is clear, i.e. something along the lines of..

IMPORTANT The search functionality requires real-time parsing and converting the glossary data. This can take indefinite amount of time for large glossaries, and therefore this option is provided exclusively for testing purposes.
Please only use the search option for previewing and testing glossaries — it is NOT intended for regular dictionary lookups. Use a dedicated dictionary instead.

wdyt?

[glowinthedark]

btw, another side offtopic regarding web dictionary viewers, in the same category as 'goldendict-web', discovered the other day: https://github.com/liuyug/flask-mdict — made by the author of mdict-utils; the setup is easier than goldendict-web, although only mdict is supported, and nothing else. Overall I do believe the choice of sqlite is right for storage, indexing, searching, etc, no need to reinvent the wheel.

[ilius]

Sounds good.

I had starred flask-mdict, but don't remember it! Probably saw it a long time ago.

Can you make it so that searched entries will be loaded one at a time and not make the user wait for all?

I have also made wrappers around sqlite with a class that acts like a list, used for global sqlite mode and stardict writer.

[ilius]

https://github.com/ilius/pyglossary?tab=readme-ov-file#sqlite-mode

https://github.com/ilius/pyglossary/blob/master/pyglossary/sq_entry_list.py

https://github.com/ilius/pyglossary/blob/master/pyglossary/plugins/stardict/sqlist.py

[glowinthedark]

Can you make it so that searched entries will be loaded one at a time and not make the user wait for all?

hmmm, right, good idea, indeed, this would be much better than blocking the UI until all results are collected; will have to chew it a bit and find a clean for a way to do it with websockets

Only see SqList and wrappers being used in the stardict writer; can SqList be used with any format? is it supposed to be used as an alternative to pyglossary.glossary_v2.GlossaryCommon.directRead() or as a wrapper?

[ilius]

can SqList be used with any format?

No, it's specific to the use case.
If you mean the global sqlite mode, that's independent of formats (it's like a list of entries).

is it supposed to be used as an alternative to pyglossary.glossary_v2.GlossaryCommon.directRead() or as a wrapper?

No, actually I can add a sqlite=False argument to directRead method...
Edit: on second thought, it doesn't make sense, directRead does not cause loading anything in RAM. Anyway let's discuss it in a discussion page if you want.

[glowinthedark]

No, actually I can add a sqlite=False argument to directRead method that does that (just call _switchToSQLite)

aha, I see that _switchToSQLite() is already called since args.sqlite=true by default, right?

..sure, let's switch to a discussion page, although never used them before.

[glowinthedark]

switched to websockets to make entries load progressively on the client; the difference is not very huge — for dictionaries over 1GB the delay until the first entry is found is of course still noticeable, especially when a lot of resources need to be parsed (audio, images); to actually see the progressive loading in action had to set max to be over 500 and the dictionary must have enough entries for the given search term; or maybe I just need to try it on slower machine.

in any case; progressive loading now works and in the future if the backend responds quicker before the first record the client will reflect that

[ilius]

Thanks.