How to use leann in a no gpu enviornment?

[Anaxagoras-bc]

I do my dev work in a vm, esp when using claude code incase it decides to transmit the contents of my filesystem back to anthropic for some reason.... I tried to run leann to index my codebase and it was so slow i gave up, it looked like it was going to need 20 hours...

Is it possible to leverage a local llm in my network? I run lm-studio on my m4 mbp which provides an openapi compatible endpoint, wondering if there was a way have lean utilize a remote device for creating the embedding?

[yichuan-w]

Yeah, that's a really good point, we have not think about that before, I guess local LM-studio to provide an endpoint to deal with the embedding is a cool idea, actually, what we have here to compute embedding is 3 methods:

1. sentencetransformer
   https://github.com/yichuan-w/LEANN/blob/main/packages/leann-core/src/leann/embedding_compute.py#L49
2. openai / gemeni:
   https://github.com/yichuan-w/LEANN/blob/main/packages/leann-core/src/leann/embedding_compute.py#L59
   You may not like that, it will upload file to openai
   3.ollama endpoint:
   https://github.com/yichuan-w/LEANN/blob/main/packages/leann-core/src/leann/embedding_compute.py#L63

I guess lm studio should be similar to ollama, but we only test them in the same server, it should be easy to add remote endpoint, maybe you can raise an issue to request that feature and we can make that asap.

[BuzzClanAI]

Here is my solution - if you have a gaming desktop in your hosehold. :)
In my latest article, I break down the exact steps I took to build this setup, including:

• The Magic of WSL2: Why it's the perfect bridge between Windows and Linux.
• Making GPU Passthrough Work: How to get CUDA acceleration inside WSL2.
• Remote Access: The simple networking tricks to SSH into your desktop from anywhere.

It's a guide for anyone who wants high-performance compute without the cloud bill.
Dive into the full story here:

How I Secretly Turned Our Family Gaming PC into an AI Powerhouse