✨ feat: support to customize Embedding model with env #5177

cookieY · 2024-12-25T10:17:02Z

💻 变更类型 | Change Type

🔀 变更说明 | Description of Change

close [Request] 自定义嵌入模型 Custom Embedding model #3785

📝 补充信息 | Additional Information

由于之前代码commit差距过大重新提交，提交说明参考 #4370

vercel · 2024-12-25T10:17:06Z

@cookieY is attempting to deploy a commit to the LobeHub Team on Vercel.

A member of the Team first needs to authorize it.

lobehubbot · 2024-12-25T10:17:18Z

👍 @cookieY

Thank you for raising your pull request and contributing to our Community
Please make sure you have followed our contributing guidelines. We will review it as soon as possible.
If you encounter any problems, please feel free to connect with us.
非常感谢您提出拉取请求并为我们的社区做出贡献，请确保您已经遵循了我们的贡献指南，我们会尽快审查它。
如果您遇到任何问题，请随时与我们联系。

codecov · 2024-12-25T10:19:55Z

Codecov Report

Attention: Patch coverage is 62.31884% with 78 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/libs/agent-runtime/bedrock/index.ts	31.25%	44 Missing ⚠️
src/libs/agent-runtime/ollama/index.ts	29.72%	26 Missing ⚠️
src/server/globalConfig/parseFilesConfig.ts	89.47%	6 Missing ⚠️
src/server/globalConfig/index.ts	66.66%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5177      +/-   ##
==========================================
- Coverage   93.09%   92.98%   -0.12%     
==========================================
  Files         631      634       +3     
  Lines       55424    55642     +218     
  Branches     2696     2904     +208     
==========================================
+ Hits        51596    51736     +140     
- Misses       3828     3906      +78

Flag	Coverage Δ
app	`92.98% <62.31%> (-0.12%)`	⬇️
server	`97.99% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

halexan · 2024-12-25T11:31:27Z

支不支持本地部署的embedding模型？例如 api 地址是 http://127.0.0.1/v1 的类 openai 接口？

lobehubbot · 2024-12-25T11:31:39Z

Does it support locally deployed embedding models? For example, the api address is an openai-like interface at http://127.0.0.1/v1?

cookieY · 2024-12-25T12:42:05Z

基于现有的框架设计无法实现这样的自定义需求，本地部署可以使用 ollama

lobehubbot · 2024-12-25T12:42:15Z

Such custom requirements cannot be achieved based on the existing framework design. For local deployment, ollama can be used.

src/config/knowledge.ts

src/const/settings/knowledge.ts

src/database/schemas/ragEvals.ts

src/libs/agent-runtime/types/embeddings.ts

src/libs/agent-runtime/utils/openaiCompatibleFactory/index.ts

src/server/globalConfig/parseSystemAgent.ts

src/server/modules/Files/index.ts

…dding

# Conflicts: # .env.example

cookieY · 2024-12-27T07:30:57Z

@arvinxx 已根据建议全部进行修改

lobehubbot · 2024-12-27T07:31:07Z

@arvinxx All modifications have been made based on suggestions

jiongjiongJOJO · 2024-12-31T02:58:45Z

期待尽快合并！！

lobehubbot · 2024-12-31T02:58:56Z

Looking forward to merging soon! !

huanyingtianhe · 2025-01-01T14:34:30Z

looking forward to seeing the change soon

bestZwei · 2025-01-03T06:09:57Z

期待尽快合并！！

lobehubbot · 2025-01-03T06:10:08Z

Looking forward to merging soon! !

arvinxx

再改一轮，就差不多了

src/const/settings/knowledge.ts

src/server/routers/async/file.ts

src/server/routers/lambda/chunk.ts

src/server/routers/async/file.ts

src/server/routers/lambda/chunk.ts

- Update the file configuration and standardize the model naming to camel case.

cookieY · 2025-01-07T05:32:00Z

@arvinxx 已修改完毕

lobehubbot · 2025-01-07T05:32:13Z

@arvinxx has been modified

lobehubbot · 2025-01-13T09:11:02Z

Can both custom baseurl and embedded model names be supported?

coolangcn · 2025-01-14T02:14:53Z

期待

lobehubbot · 2025-01-14T02:15:05Z

expectations

vercel · 2025-01-14T11:46:48Z

Deployment failed with the following error:

The provided GitHub repository does not contain the requested branch or commit reference. Please ensure the repository is not empty.

vercel · 2025-01-14T14:22:57Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated (UTC)
lobe-chat-database	✅ Ready (Inspect)	Visit Preview	Jan 14, 2025 4:22pm

lobehubbot · 2025-01-15T03:17:32Z

❤️ Great PR @cookieY ❤️

The growth of project is inseparable from user feedback and contribution, thanks for your contribution! If you are interesting with the lobehub developer community, please join our discord and then dm @arvinxx or @canisminor1990. They will invite you to our private developer channel. We are talking about the lobe-chat development or sharing ai newsletter around the world.
项目的成长离不开用户反馈和贡献，感谢您的贡献! 如果您对 LobeHub 开发者社区感兴趣，请加入我们的 discord，然后私信 @arvinxx 或 @canisminor1990。他们会邀请您加入我们的私密开发者频道。我们将会讨论关于 Lobe Chat 的开发，分享和讨论全球范围内的 AI 消息。

## [Version 1.46.0](v1.45.17...v1.46.0) <sup>Released on **2025-01-15**</sup> #### ✨ Features - **misc**: Add lm studio provider, support to customize Embedding model with env. <br/> <details> <summary><kbd>Improvements and Fixes</kbd></summary> #### What's improved * **misc**: Add lm studio provider, closes [#4483](#4483) ([0ac5802](0ac5802)) * **misc**: Support to customize Embedding model with env, closes [#5177](#5177) ([eed9357](eed9357)) </details> <div align="right"> [![](https://img.shields.io/badge/-BACK_TO_TOP-151515?style=flat-square)](#readme-top) </div>

lobehubbot · 2025-01-15T03:26:24Z

🎉 This PR is included in version 1.46.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

## [Version 1.88.0](v1.87.8...v1.88.0) <sup>Released on **2025-01-15**</sup> #### ✨ Features - **misc**: Add lm studio provider, support to customize Embedding model with env. <br/> <details> <summary><kbd>Improvements and Fixes</kbd></summary> #### What's improved * **misc**: Add lm studio provider, closes [lobehub#4483](https://github.com/bentwnghk/lobe-chat/issues/4483) ([0ac5802](0ac5802)) * **misc**: Support to customize Embedding model with env, closes [lobehub#5177](https://github.com/bentwnghk/lobe-chat/issues/5177) ([eed9357](eed9357)) </details> <div align="right"> [![](https://img.shields.io/badge/-BACK_TO_TOP-151515?style=flat-square)](#readme-top) </div>

disarticulate · 2025-01-19T13:14:57Z

great timing on this feature. is the docker images lobehub/lobe-chat-database updated?

disarticulate · 2025-01-19T13:53:21Z

was able to test this out.

After figuring out a correct embedding model for ollama: xiaoming/bce-embedding-base_v1 and trying both the proxy OpenAI Url and the ollama url:

DEFAULT_FILES_CONFIG=embedding_model=ollama/xiaoming/bce-embedding-base_v1
DEFAULT_FILES_CONFIG=embedding_model=openai/xiaoming/bce-embedding-base_v1

I get the following vague errors:

#DEFAULT_FILES_CONFIG=embedding_model=ollama/xiaoming/bce-embedding-base_v1

embeddingChunks error {
2025-01-19 07:47:39   message: '{"error":{"message":"[object Object]","name":"ResponseError","status_code":500},"errorType":"OllamaBizError","provider":"ollama"}',
2025-01-19 07:47:39   name: 'EmbeddingError'
2025-01-19 07:47:39 }

#DEFAULT_FILES_CONFIG=embedding_model=openai/xiaoming/bce-embedding-base_v1
embeddingChunks error {
2025-01-19 07:52:24   message: '{"endpoint":"http://vineyard_ollama:****/v1","error":{"headers":{"content-length":"0","content-type":"application/json; charset=utf-8","date":"Sun, 19 Jan 2025 13:52:24 GMT"},"stack":"Error: 500 status code (no body)\\n    at eV.generate (/app/.next/server/chunks/67863.js:41:23151)\\n    at r3.makeStatusError (/app/.next/server/chunks/67863.js:41:36093)\\n    at r3.makeRequest (/app/.next/server/chunks/67863.js:41:37037)\\n    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\\n    at async Object.embeddings (/app/.next/server/chunks/9992.js:1:7027)\\n    at async concurrency (/app/.next/server/app/(backend)/trpc/async/[trpc]/route.js:1:4431)\\n    at async /app/.next/server/chunks/42368.js:51:290507","status":500},"errorType":"ProviderBizError","provider":"openai"}',
2025-01-19 07:52:24   name: 'EmbeddingError'
2025-01-19 07:52:24 }

It does appear to be chunking, and takes ~a minute to respond. I'm running everything inside a container and have had to reroute some hosts requests.

SPLT12 · 2025-01-20T11:54:04Z

was able to test this out.

After figuring out a correct embedding model for ollama: xiaoming/bce-embedding-base_v1 and trying both the proxy OpenAI Url and the ollama url:

DEFAULT_FILES_CONFIG=embedding_model=ollama/xiaoming/bce-embedding-base_v1
DEFAULT_FILES_CONFIG=embedding_model=openai/xiaoming/bce-embedding-base_v1

I get the following vague errors:

#DEFAULT_FILES_CONFIG=embedding_model=ollama/xiaoming/bce-embedding-base_v1

embeddingChunks error {
2025-01-19 07:47:39   message: '{"error":{"message":"[object Object]","name":"ResponseError","status_code":500},"errorType":"OllamaBizError","provider":"ollama"}',
2025-01-19 07:47:39   name: 'EmbeddingError'
2025-01-19 07:47:39 }

#DEFAULT_FILES_CONFIG=embedding_model=openai/xiaoming/bce-embedding-base_v1
embeddingChunks error {
2025-01-19 07:52:24   message: '{"endpoint":"http://vineyard_ollama:****/v1","error":{"headers":{"content-length":"0","content-type":"application/json; charset=utf-8","date":"Sun, 19 Jan 2025 13:52:24 GMT"},"stack":"Error: 500 status code (no body)\\n    at eV.generate (/app/.next/server/chunks/67863.js:41:23151)\\n    at r3.makeStatusError (/app/.next/server/chunks/67863.js:41:36093)\\n    at r3.makeRequest (/app/.next/server/chunks/67863.js:41:37037)\\n    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\\n    at async Object.embeddings (/app/.next/server/chunks/9992.js:1:7027)\\n    at async concurrency (/app/.next/server/app/(backend)/trpc/async/[trpc]/route.js:1:4431)\\n    at async /app/.next/server/chunks/42368.js:51:290507","status":500},"errorType":"ProviderBizError","provider":"openai"}',
2025-01-19 07:52:24   name: 'EmbeddingError'
2025-01-19 07:52:24 }

It does appear to be chunking, and takes ~a minute to respond. I'm running everything inside a container and have had to reroute some hosts requests.

@disarticulate Same here, trying to figure it out but I get always those type of errors.

disarticulate · 2025-01-20T13:59:53Z

I also discovered a ollama error that returns errots based on embedding ctx. I got it to work wiyh ollama/quentinz/bge-large-zh-v1.5:latest

…

On Mon, Jan 20, 2025, 06:47 Henry Yee ***@***.***> wrote: was able to test this out. After figuring out a correct embedding model for ollama: xiaoming/bce-embedding-base_v1 and trying both the proxy OpenAI Url and the ollama url: DEFAULT_FILES_CONFIG=embedding_model=ollama/xiaoming/bce-embedding-base_v1 DEFAULT_FILES_CONFIG=embedding_model=openai/xiaoming/bce-embedding-base_v1 I get the following vague errors: #DEFAULT_FILES_CONFIG=embedding_model=ollama/xiaoming/bce-embedding-base_v1 embeddingChunks error { 2025-01-19 07:47:39 message: '{"error":{"message":"[object Object]","name":"ResponseError","status_code":500},"errorType":"OllamaBizError","provider":"ollama"}', 2025-01-19 07:47:39 name: 'EmbeddingError' 2025-01-19 07:47:39 } #DEFAULT_FILES_CONFIG=embedding_model=openai/xiaoming/bce-embedding-base_v1 embeddingChunks error { 2025-01-19 07:52:24 message: '{"endpoint":"http://vineyard_ollama:****/v1","error":{"headers":{"content-length":"0","content-type":"application/json; charset=utf-8","date":"Sun, 19 Jan 2025 13:52:24 GMT"},"stack":"Error: 500 status code (no body)\\n at eV.generate (/app/.next/server/chunks/67863.js:41:23151)\\n at r3.makeStatusError (/app/.next/server/chunks/67863.js:41:36093)\\n at r3.makeRequest (/app/.next/server/chunks/67863.js:41:37037)\\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)\\n at async Object.embeddings (/app/.next/server/chunks/9992.js:1:7027)\\n at async concurrency (/app/.next/server/app/(backend)/trpc/async/[trpc]/route.js:1:4431)\\n at async /app/.next/server/chunks/42368.js:51:290507","status":500},"errorType":"ProviderBizError","provider":"openai"}', 2025-01-19 07:52:24 name: 'EmbeddingError' 2025-01-19 07:52:24 } It does appear to be chunking, and takes ~a minute to respond. I'm running everything inside a container and have had to reroute some hosts requests. @disarticulate <https://github.com/disarticulate> Same here, trying to figure it out but I get always those type of errors. The issue seems to be related to not considering multiple slashes (/) when parsing configuration data. I will submit a PR to resolve this issue. — Reply to this email directly, view it on GitHub <#5177 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEFHWPPWTM2QSHRSZMCIRAL2LTV6FAVCNFSM6AAAAABUF2LTLWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMBSGMZTSOBUGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

SPLT12 · 2025-01-21T17:56:23Z

Mine is showing this

log.txt

steinvenic · 2025-01-22T02:58:18Z

DEFAULT_FILES_CONFIG="embedding_model=ollama/xiaoming/bce-embedding-base_v1"

* feat: 添加嵌入模型配置支持，更新相关文档和测试 * feat: 重构文件配置，更新默认设置和相关测试 * ♻ Code Refactoring - Update the file configuration and standardize the model naming to camel case.

## [Version 1.46.0](lobehub/lobe-chat@v1.45.17...v1.46.0) <sup>Released on **2025-01-15**</sup> #### ✨ Features - **misc**: Add lm studio provider, support to customize Embedding model with env. <br/> <details> <summary><kbd>Improvements and Fixes</kbd></summary> #### What's improved * **misc**: Add lm studio provider, closes [lobehub#4483](lobehub#4483) ([0ac5802](lobehub@0ac5802)) * **misc**: Support to customize Embedding model with env, closes [lobehub#5177](lobehub#5177) ([eed9357](lobehub@eed9357)) </details> <div align="right"> [![](https://img.shields.io/badge/-BACK_TO_TOP-151515?style=flat-square)](#readme-top) </div>

* feat: 添加嵌入模型配置支持，更新相关文档和测试 * feat: 重构文件配置，更新默认设置和相关测试 * ♻ Code Refactoring - Update the file configuration and standardize the model naming to camel case.

## [Version 1.46.0](lobehub/lobe-chat@v1.45.17...v1.46.0) <sup>Released on **2025-01-15**</sup> #### ✨ Features - **misc**: Add lm studio provider, support to customize Embedding model with env. <br/> <details> <summary><kbd>Improvements and Fixes</kbd></summary> #### What's improved * **misc**: Add lm studio provider, closes [lobehub#4483](lobehub#4483) ([0ac5802](lobehub@0ac5802)) * **misc**: Support to customize Embedding model with env, closes [lobehub#5177](lobehub#5177) ([eed9357](lobehub@eed9357)) </details> <div align="right"> [![](https://img.shields.io/badge/-BACK_TO_TOP-151515?style=flat-square)](#readme-top) </div>

feat: 添加嵌入模型配置支持，更新相关文档和测试

f8f64e0

dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. 🌠 Feature Request New feature or request | 特性与建议 labels Dec 25, 2024

Merge branch 'main' into embedding

e85f2e5

arvinxx requested changes Dec 26, 2024

View reviewed changes

cookieY added 3 commits December 27, 2024 15:19

feat: 重构文件配置，更新默认设置和相关测试

dd26797

Merge branch 'main' of https://github.com/lobehub/lobe-chat into embe…

bcf67b6

…dding

Merge branch 'embedding' of github.com:cookieY/lobe-chat into embedding

597a3e3

# Conflicts: # .env.example

Sun-drenched mentioned this pull request Dec 29, 2024

[Request] 向量化模型现只支持openai？ #5225

Closed

RS-Imagine mentioned this pull request Dec 29, 2024

[Request] Custom Embedded Model RS-Imagine/lobe-chat#15

Closed

arvinxx requested changes Jan 7, 2025

View reviewed changes

arvinxx and others added 3 commits January 7, 2025 11:03

Merge branch 'main' into embedding

085505b

♻ Code Refactoring

91ed1dc

- Update the file configuration and standardize the model naming to camel case.

Merge branch 'embedding' of github.com:cookieY/lobe-chat into embedding

e57656e

Merge branch 'main' into embedding

dc80f34

Merge branch 'main' into embedding

51d530b

vercel bot deployed to Preview – lobe-chat-database January 14, 2025 15:23 View deployment

Merge branch 'main' into embedding

972c0db

arvinxx changed the title ~~✨ feat: 允许用户自行定义 Embedding 模型~~ ✨ feat: support to Embedding 模型 Jan 14, 2025

arvinxx changed the title ~~✨ feat: support to Embedding 模型~~ ✨ feat: support to customize Embedding model with env Jan 14, 2025

vercel bot deployed to Preview – lobe-chat-database January 14, 2025 16:22 View deployment

arvinxx merged commit eed9357 into lobehub:main Jan 15, 2025
5 of 7 checks passed

lobehubbot added the released label Jan 15, 2025

zc16607 mentioned this pull request Jan 15, 2025

[Bug] 知识库分块报错（自定义嵌入模型zhipu） #5457

Closed

Sun-drenched mentioned this pull request Jan 25, 2025

[Bug] Embeding failed in both OpenAI compatible API and Ollama API #5583

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

✨ feat: support to customize Embedding model with env #5177

✨ feat: support to customize Embedding model with env #5177

cookieY commented Dec 25, 2024 •

edited by arvinxx

Loading

vercel bot commented Dec 25, 2024

lobehubbot commented Dec 25, 2024

codecov bot commented Dec 25, 2024 •

edited

Loading

halexan commented Dec 25, 2024

lobehubbot commented Dec 25, 2024

cookieY commented Dec 25, 2024

lobehubbot commented Dec 25, 2024

cookieY commented Dec 27, 2024

lobehubbot commented Dec 27, 2024

jiongjiongJOJO commented Dec 31, 2024

lobehubbot commented Dec 31, 2024

huanyingtianhe commented Jan 1, 2025

bestZwei commented Jan 3, 2025

lobehubbot commented Jan 3, 2025

arvinxx left a comment

cookieY commented Jan 7, 2025

lobehubbot commented Jan 7, 2025

lobehubbot commented Jan 13, 2025

coolangcn commented Jan 14, 2025

lobehubbot commented Jan 14, 2025

vercel bot commented Jan 14, 2025

vercel bot commented Jan 14, 2025 •

edited

Loading

lobehubbot commented Jan 15, 2025

lobehubbot commented Jan 15, 2025

disarticulate commented Jan 19, 2025

disarticulate commented Jan 19, 2025

SPLT12 commented Jan 20, 2025

disarticulate commented Jan 20, 2025 via email

SPLT12 commented Jan 21, 2025 •

edited

Loading

steinvenic commented Jan 22, 2025

✨ feat: support to customize Embedding model with env #5177

✨ feat: support to customize Embedding model with env #5177

Conversation

cookieY commented Dec 25, 2024 • edited by arvinxx Loading

💻 变更类型 | Change Type

🔀 变更说明 | Description of Change

📝 补充信息 | Additional Information

vercel bot commented Dec 25, 2024

lobehubbot commented Dec 25, 2024

codecov bot commented Dec 25, 2024 • edited Loading

Codecov Report

halexan commented Dec 25, 2024

lobehubbot commented Dec 25, 2024

cookieY commented Dec 25, 2024

lobehubbot commented Dec 25, 2024

cookieY commented Dec 27, 2024

lobehubbot commented Dec 27, 2024

jiongjiongJOJO commented Dec 31, 2024

lobehubbot commented Dec 31, 2024

huanyingtianhe commented Jan 1, 2025

bestZwei commented Jan 3, 2025

lobehubbot commented Jan 3, 2025

arvinxx left a comment

Choose a reason for hiding this comment

cookieY commented Jan 7, 2025

lobehubbot commented Jan 7, 2025

lobehubbot commented Jan 13, 2025

coolangcn commented Jan 14, 2025

lobehubbot commented Jan 14, 2025

vercel bot commented Jan 14, 2025

vercel bot commented Jan 14, 2025 • edited Loading

lobehubbot commented Jan 15, 2025

lobehubbot commented Jan 15, 2025

disarticulate commented Jan 19, 2025

disarticulate commented Jan 19, 2025

SPLT12 commented Jan 20, 2025

disarticulate commented Jan 20, 2025 via email

SPLT12 commented Jan 21, 2025 • edited Loading

steinvenic commented Jan 22, 2025

cookieY commented Dec 25, 2024 •

edited by arvinxx

Loading

codecov bot commented Dec 25, 2024 •

edited

Loading

vercel bot commented Jan 14, 2025 •

edited

Loading

SPLT12 commented Jan 21, 2025 •

edited

Loading