Skip to content

Improve documentation on how to use the default exported __wbg_init #58

@aiktb

Description

@aiktb

This library is a solid alternative to the long-unmaintained kuromoji.js, and it avoids many of pitfalls.

For example, kuromoji.js:

  1. Cannot load a user dictionary.
  2. Relies on XMLHttpRequest, which breaks in Service Worker environments.
  3. Uses callbacks heavily, leading to spaghetti code.
  4. Uses 1-based subscripts.
  5. Depends on zlib.js, which doesn’t run in the browser.
  6. Depends on path, a Node.js core module.
  7. Is not ESM-compatible.

Although there are several forks of kuromoji.js that try to fix these problems, none of them are well maintained.

I was able to replace kuromoji.js with this library, but it took a few hours and I ran into some issues:

  • There were no examples of using this library with a Vite-based bundler. Since Vite has limitations around WASM bundling, lindera-wasm was not bundled automatically, causing the browser to throw a tokenizerbuilder_new does not exist error.
  • After some research, I discovered I could exclude the WASM file from bundling by placing it in public/. This solved the first issue, but then I faced a second one: how to tell lindera-wasm where the WASM file is located? The documentation didn’t cover this.
  • Eventually, I inspected lindera_wasm.js and found that calling the default-exported __wbg_init function with the WASM path solved the problem.

The documentation and JSDoc should be improved to explain this setup more clearly.

I’d be happy to submit a PR.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions