May I use my local LLM (qwen2.5-7B in LM Studio) ?

[Asumer14]

Also I wanna rebuild the map change it to AI Family
I'm really interested in this project and I'm working on mine too.
Now I have some problem at building the map, so I want to have some inspiration and discussion in your project.

[chuanxin]

If you want to replace the default model, based on my experience, you can modify the LLM_CONFIG settings in the convex/util/llm.ts file in the project directory. For your needs, the modification will look something like this: chatModel: 'qwen2.5:7b' as const. You can specify other versions aside from 7b and also update the corresponding embeddingModel. I'm using 'znbang/bge:large-en-v1.5-q8_0'.

Additionally, I couldn't successfully set up a backend service for Convex locally. So, to use the local LLM, I set up a third-party tunnel service (e.g., ngrok) to connect to my localhost:11434.

[bruno686]

如果要修改替换默认模型，根据我的经验，可以修改LLM_CONFIG项目目录中凸/util/llm.ts文件中的设置。根据您的需要，将如下所示：chatModel: 'qwen2.5:7b' as const。您可以指定除 7b 之外的其他版本，并更新相应的embeddingModel。我正在使用'znbang/bge:large-en-v1.5-q8_0'。

另外，我无法在本地成功设置 Convex 的端口服务。因此，为了使用本地 LLM，我设置了第三方隧道服务（例如 ngrok）来连接到我的 localhost:11434。

Did you run the project successfully? I stuck in 12/26/2024 4:36:40 PM [CONVEX (aiTown/agentOperations: agentGenerateMessage)] A Uncaught Error: The Request to http://127.0.0.1:11434/api/embeddings is forbidden. I use local ollama. It's really strange.

[Ycx2000]

如果要修改替换默认模型，根据我的经验，可以修改LLM_CONFIG项目目录中凸/util/llm.ts文件中的设置。根据您的需要，将如下所示：chatModel: 'qwen2.5:7b' as const。您可以指定除 7b 之外的其他版本，并更新相应的embeddingModel。我正在使用'znbang/bge:large-en-v1.5-q8_0'。
另外，我无法在本地成功设置 Convex 的端口服务。因此，为了使用本地 LLM，我设置了第三方隧道服务（例如 ngrok）来连接到我的 localhost:11434。

Did you run the project successfully? I stuck in 12/26/2024 4:36:40 PM [CONVEX (aiTown/agentOperations: agentGenerateMessage)] A Uncaught Error: The Request to http://127.0.0.1:11434/api/embeddings is forbidden. I use local ollama. It's really strange.

Hi, I'm having the same problem... do you have any solutions now?

[bruno686]

如果要修改替换默认模型，根据我的经验，可以修改LLM_CONFIG项目目录中凸/util/llm.ts文件中的设置。根据您的需要，将如下所示：chatModel: 'qwen2.5:7b' as const。您可以指定除 7b 之外的其他版本，并更新相应的embeddingModel。我正在使用'znbang/bge:large-en-v1.5-q8_0'。
另外，我无法在本地成功设置 Convex 的端口服务。因此，为了使用本地 LLM，我设置了第三方隧道服务（例如 ngrok）来连接到我的 localhost:11434。

Did you run the project successfully? I stuck in 12/26/2024 4:36:40 PM [CONVEX (aiTown/agentOperations: agentGenerateMessage)] A Uncaught Error: The Request to http://127.0.0.1:11434/api/embeddings is forbidden. I use local ollama. It's really strange.

Hi, I'm having the same problem... do you have any solutions now?

No, so sad, I can't run this project locally. But, when I use together.AI, I find it works

[Ycx2000]

如果要修改替换默认模型，根据我的经验，可以修改LLM_CONFIG项目目录中凸/util/llm.ts文件中的设置。根据您的需要，将如下所示：chatModel: 'qwen2.5:7b' as const。您可以指定除 7b 之外的其他版本，并更新相应的embeddingModel。我正在使用'znbang/bge:large-en-v1.5-q8_0'。
另外，我无法在本地成功设置 Convex 的端口服务。因此，为了使用本地 LLM，我设置了第三方隧道服务（例如 ngrok）来连接到我的 localhost:11434。

Did you run the project successfully? I stuck in 12/26/2024 4:36:40 PM [CONVEX (aiTown/agentOperations: agentGenerateMessage)] A Uncaught Error: The Request to http://127.0.0.1:11434/api/embeddings is forbidden. I use local ollama. It's really strange.

Hi, I'm having the same problem... do you have any solutions now?

No, so sad, I can't run this project locally. But, when I use together.AI, I find it works

Thanks for your reply. I haven't found anyone else mentioning this problem except you. Could it be related to the fact that we are in China? Maybe some kind of internet issue... I'm not sure.

[Ycx2000]

如果要修改替换默认模型，根据我的经验，可以修改LLM_CONFIG项目目录中凸/util/llm.ts文件中的设置。根据您的需要，将如下所示：chatModel: 'qwen2.5:7b' as const。您可以指定除 7b 之外的其他版本，并更新相应的embeddingModel。我正在使用'znbang/bge:large-en-v1.5-q8_0'。
另外，我无法在本地成功设置 Convex 的端口服务。因此，为了使用本地 LLM，我设置了第三方隧道服务（例如 ngrok）来连接到我的 localhost:11434。

Did you run the project successfully? I stuck in 12/26/2024 4:36:40 PM [CONVEX (aiTown/agentOperations: agentGenerateMessage)] A Uncaught Error: The Request to http://127.0.0.1:11434/api/embeddings is forbidden. I use local ollama. It's really strange.

Hi, I'm having the same problem... do you have any solutions now?

No, so sad, I can't run this project locally. But, when I use together.AI, I find it works

Thanks for your reply. I haven't found anyone else mentioning this problem except you. Could it be related to the fact that we are in China? Maybe some kind of internet issue... I'm not sure.

But together.AI works for me too. Thanks.

[bruno686]

I have seen others also have this problem, but they don't have a solution. Up to now, I modified the LLM server to Together.AI, now it's working fine. Other solutions I tried, for example, ngrok, modifying local IP, and so on are all unuseful.

…

On Tue, Jan 7, 2025 at 10:46 PM Chengxiang ***@***.***> wrote: 如果要修改替换默认模型，根据我的经验，可以修改LLM_CONFIG项目目录中凸/util/llm.ts文件中的设置。根据您的需要，将如下所示：chatModel: 'qwen2.5:7b' as const。您可以指定除 7b 之外的其他版本，并更新相应的embeddingModel。我正在使用 'znbang/bge:large-en-v1.5-q8_0'。 另外，我无法在本地成功设置 Convex 的端口服务。因此，为了使用本地 LLM，我设置了第三方隧道服务（例如 ngrok）来连接到我的 localhost:11434。 Did you run the project successfully? I stuck in 12/26/2024 4:36:40 PM [CONVEX (aiTown/agentOperations: agentGenerateMessage)] A Uncaught Error: The Request to http://127.0.0.1:11434/api/embeddings is forbidden. I use local ollama. It's really strange. Hi, I'm having the same problem... do you have any solutions now? No, so sad, I can't run this project locally. But, when I use together.AI, I find it works Thanks for your reply. I haven't found anyone else mentioning this problem except you. Could it be related to the fact that we are in China? Maybe some kind of internet issue... I'm not sure. — Reply to this email directly, view it on GitHub <#263 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMPQLIRCHYSBHS77O7QZR2D2JPSE5AVCNFSM6AAAAABTMSOGT2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNZVGQ3TQNZSGY> . You are receiving this because you commented.Message ID: ***@***.***>

-- Zhuangzhuang He (贺壮壮) M.Eng., School of Computer Sci & Information Engineering Hefei University of Technology Hefei, Anhui, China

[bruno686]

能运行就完事了，要啥自行车

…

On Tue, Jan 7, 2025 at 11:10 PM Chengxiang ***@***.***> wrote: 如果要修改替换默认模型，根据我的经验，可以修改LLM_CONFIG项目目录中凸/util/llm.ts文件中的设置。根据您的需要，将如下所示：chatModel: 'qwen2.5:7b' as const。您可以指定除 7b 之外的其他版本，并更新相应的embeddingModel。我正在使用 'znbang/bge:large-en-v1.5-q8_0'。 另外，我无法在本地成功设置 Convex 的端口服务。因此，为了使用本地 LLM，我设置了第三方隧道服务（例如 ngrok）来连接到我的 localhost:11434。 Did you run the project successfully? I stuck in 12/26/2024 4:36:40 PM [CONVEX (aiTown/agentOperations: agentGenerateMessage)] A Uncaught Error: The Request to http://127.0.0.1:11434/api/embeddings is forbidden. I use local ollama. It's really strange. Hi, I'm having the same problem... do you have any solutions now? No, so sad, I can't run this project locally. But, when I use together.AI, I find it works Thanks for your reply. I haven't found anyone else mentioning this problem except you. Could it be related to the fact that we are in China? Maybe some kind of internet issue... I'm not sure. But together.AI works for me too. Thanks. — Reply to this email directly, view it on GitHub <#263 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMPQLIX6UYT4OC5MN25AXCL2JPU45AVCNFSM6AAAAABTMSOGT2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNZVGUZTMNJZGY> . You are receiving this because you commented.Message ID: ***@***.***>

-- Zhuangzhuang He (贺壮壮) M.Eng., School of Computer Sci & Information Engineering Hefei University of Technology Hefei, Anhui, China

[Ycx2000]

Yes hhh