-
-
Notifications
You must be signed in to change notification settings - Fork 4
Open
Description
This library is a solid alternative to the long-unmaintained kuromoji.js, and it avoids many of pitfalls.
For example, kuromoji.js:
- Cannot load a user dictionary.
- Relies on
XMLHttpRequest, which breaks in Service Worker environments. - Uses callbacks heavily, leading to spaghetti code.
- Uses 1-based subscripts.
- Depends on
zlib.js, which doesn’t run in the browser. - Depends on
path, a Node.js core module. - Is not ESM-compatible.
Although there are several forks of kuromoji.js that try to fix these problems, none of them are well maintained.
I was able to replace kuromoji.js with this library, but it took a few hours and I ran into some issues:
- There were no examples of using this library with a Vite-based bundler. Since Vite has limitations around WASM bundling,
lindera-wasmwas not bundled automatically, causing the browser to throw atokenizerbuilder_new does not existerror. - After some research, I discovered I could exclude the WASM file from bundling by placing it in
public/. This solved the first issue, but then I faced a second one: how to telllindera-wasmwhere the WASM file is located? The documentation didn’t cover this. - Eventually, I inspected
lindera_wasm.jsand found that calling the default-exported__wbg_initfunction with the WASM path solved the problem.
The documentation and JSDoc should be improved to explain this setup more clearly.
I’d be happy to submit a PR.
Metadata
Metadata
Assignees
Labels
No labels