PaddleX/latest/pipeline_deploy/serving #3524
giscus[bot]
bot
Replies: 45 comments 133 replies
-
|
您好，paddlex --serve --pipeline image_classification这样启动服务之后，关闭服务的命令是什么呀？ |
Beta Was this translation helpful? Give feedback.
-
|
ctrl+c |
Beta Was this translation helpful? Give feedback.
-
|
paddlex --serve --pipeline {产线名称或产线配置文件路径} [{其他命令行选项}] 如果在部署多个产线， 是否要启动多个服务吗？ |
Beta Was this translation helpful? Give feedback.
-
|
您好，需要启动多个服务 |
Beta Was this translation helpful? Give feedback.
-
|
服务化部署支持实时数据的高性能推理？ |
Beta Was this translation helpful? Give feedback.
-
|
暂时没有支持流式返回的服务。不过，对于实时数据，可以考虑多实例+高性能推理增加并发、降低延迟～ |
Beta Was this translation helpful? Give feedback.
-
|
我在测试‘高稳定性服务化部署’，使用通用OCR SDK，本地docker启动成功，client.py测试GRPCInferenceService通过，Metrics Service访问成功，但是HTTPService请求失败，一直报错400 Bad Request。想问一下HTTPService的接口调用文档有没有，请求参数是什么，怎么才能正确访问HTTPService |
Beta Was this translation helpful? Give feedback.
-
|
你好，我们暂时没有提供HTTPService的调用文档，目前只支持gRPC～ |
Beta Was this translation helpful? Give feedback.
-
|
SDK的下载链接都失效了，麻烦修复下吧 |
Beta Was this translation helpful? Give feedback.
-
|
你好，目前这块正在升级中，文档先更新了，最新的3.0.1 SDK预计今天之内上传～ |
Beta Was this translation helpful? Give feedback.
-
您好，想问下此次更新主要是更新了什么内容呢？ |
Beta Was this translation helpful? Give feedback.
-
|
这次主要是bug修复，去除了一些无效的参数，另外PP-ChatOCR产线也补充了一个 |
Beta Was this translation helpful? Give feedback.
-
|
您好 |
Beta Was this translation helpful? Give feedback.
-
|
前者是部署镜像，用于高稳定性服务化部署，内置了相关环境；后者是开发镜像，主要用来开发调试paddlex代码，如果对镜像体积不关心，也可以用在部署（高性能推理、基础服务化部署等）。 |
Beta Was this translation helpful? Give feedback.
-
|
docker部署的，为什么λ localhost ~/PaddleX paddlex --serve --pipeline OCR |
Beta Was this translation helpful? Give feedback.
-
|
可能需要先确认一下本地是否能跑通 |
Beta Was this translation helpful? Give feedback.
-
|
我使用阿里云或者腾讯云的linux系统就可以，同样的命令，但是使用麒麟系统的这个就不太行 是不是不太支持这个版本呀？ |
Beta Was this translation helpful? Give feedback.
-
|
建议可以提一个issue，我们会跟进这个问题 |
Beta Was this translation helpful? Give feedback.
-
|
高稳定性服务化部署如何通过http方式调用 |
Beta Was this translation helpful? Give feedback.
-
|
暂不支持，后续我们会考虑添加HTTP调用示例～ |
Beta Was this translation helpful? Give feedback.
-
|
文档中说到 手动构造http请求 的方式，好像不能用呀，端口都没通。 |
Beta Was this translation helpful? Give feedback.
-
|
请问具体报什么错呢？ |
Beta Was this translation helpful? Give feedback.
-
|
已解决，是我机器端口号冲突了。把--network host改为-p 7090:8000 -p 7091:8001 -p 7092:8002就可以了。 |
Beta Was this translation helpful? Give feedback.
-
|
感谢你的反馈，后续我们将考虑增加端口占用检查，并优化错误提示 |
Beta Was this translation helpful? Give feedback.
-
|
您好，这个SDK又不能下载了 |
Beta Was this translation helpful? Give feedback.
-
|
不好意思，这几天在更新中，预计今天之内会上传。 |
Beta Was this translation helpful? Give feedback.
-
|
您好，想请教下CUDA版本为12.4的话 怎么获取ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/hps:paddlex3.1-gpu的版本 这个只支持CUDA版本为11.8的吧？ |
Beta Was this translation helpful? Give feedback.
-
|
你好，我用paddlex3.0.0rc1-gpu版本能跑成功了 但是OCR识别的结果还是和cpu一样 需要10秒左右；显卡是Tesla T4 请问还有其它调优方式吗？config_gpu.pbtxt配置是这样： |
Beta Was this translation helpful? Give feedback.
-
|
请使用paddlex和paddle框架最新版本吧 3.0.0rc1应该是很早以前的版本～ |
Beta Was this translation helpful? Give feedback.
-
|
之前只是paddlex3.1-gpu会报如下错误：Traceback (most recent call last): |
Beta Was this translation helpful? Give feedback.
-
|
这个问题看起来是因为我们底层使用的pypdfium2库无法正确解析这个PDF，建议可以到pypdfium2库提一个issue～ |
Beta Was this translation helpful? Give feedback.
-
|
This could also be a threading issue. pypdfium2 only supports single-threaded use, otherwise you have to guard it with a mutex. |
Beta Was this translation helpful? Give feedback.
-
|
服务化部署如何开启高性能推理？ |
Beta Was this translation helpful? Give feedback.
-
|
基础服务化部署在启动服务时指定参数 |
Beta Was this translation helpful? Give feedback.
-
|
高性能推理插件已经安装，但是加上这个参数启动时报错，不加这个参数是可以正常启动的。 paddlex --serve --use_hpip --pipeline OCRCreating model: ('PP-LCNet_x1_0_doc_ori', None) |
Beta Was this translation helpful? Give feedback.
-
|
请确认你使用的是最新版本的paddlex～ |
Beta Was this translation helpful? Give feedback.
-
|
执行 1.1 paddlex --install serving 一直报错啊，os 版本 AlmaLinux9.6， 在linux 上你们强烈建议使用 docker安装paddlex ，这里怎么没有使用docekr 安装的paddlex 执行服务部署的教程呀， 请大佬指教， 谢谢！ Using cached future-1.0.0-py3-none-any.whl (491 kB) [notice] A new release of pip is available: 25.0.1 -> 25.1.1 During handling of the above exception, another exception occurred: Traceback (most recent call last): |
Beta Was this translation helpful? Give feedback.
-
|
好的，谢谢大佬， 对于 2.4.2 可以使用grpc的方式吗， 监听的有端口， 是不是按http 的方式传入json就可以呢， 如果前台要出图片流， 或者 base64 字符， grpc 如何调用呢，谢谢大佬指导 |
Beta Was this translation helpful? Give feedback.
-
|
不好意思，我刚贴的文档是之前的版本的……可以参考这里： grpc的话，建议用Python客户端来调用；HTTP的话，可以手动构造请求。 |
Beta Was this translation helpful? Give feedback.
-
|
谢谢大佬指导，我们的业务环境比较特殊， 我们前台app 是安卓自动循环调用相机每帧去识别的，所以有大量的请求发送到server端，而且每次识别要尽量快，既然容器监听的有grpc的端口， 那么这种场景grpc 应该比http 性能要好很多吧，可以指导一下grpc的方式吗？ |
Beta Was this translation helpful? Give feedback.
-
|
这个版本 ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/hps:paddlex3.0.3-gpu \ 我运行正常，但是运行 docker run E0716 02:02:04.653384 7 model_repository_manager.cc:1890] Poll failed for model directory 'ocr': instance group ocr_0 of model ocr has kind KIND_GPU but server does not support GPUs I0716 02:02:04.653444 7 server.cc:549] I0716 02:02:04.653449 7 server.cc:592] I0716 02:02:04.653482 7 tritonserver.cc:1920] I0716 02:02:04.653493 7 server.cc:252] Waiting for in-flight requests to complete. |
Beta Was this translation helpful? Give feedback.
-
|
因为你使用的是CPU镜像～请使用GPU镜像吧（以 |
Beta Was this translation helpful? Give feedback.
-
|
https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/serving.html#23 调整后的 pipeline_config.yaml pipeline_name: OCR text_type: general use_doc_preprocessor: True SubPipelines: SubModules: |
Beta Was this translation helpful? Give feedback.
-
|
明白了，看起来是因为部分模型使用的后端（比如paddle）不支持这个配置，所以出现了上面的报错，在不手动固定后端的情况下，可能还是需要逐模型配置～ |
Beta Was this translation helpful? Give feedback.
-
|
{ 大佬，ocr识别的话，这个file必须必须是个url地址吗？ |
Beta Was this translation helpful? Give feedback.
-
|
不是，请参考产线使用文档吧，其中有详细的API参数说明。 |
Beta Was this translation helpful? Give feedback.
-
|
启用hpi时，启动服务发生以下错误 Creating model: ('PP-DocLayout_plus-L', None) |
Beta Was this translation helpful? Give feedback.
-
|
如果这个错误不影响正常使用（只是程序卡住打印这些错误），可以忽略～ |
Beta Was this translation helpful? Give feedback.
-
|
对于这种启动服务后还需下载的模型，因为网络原因无法下载怎么办，可以手动挂载吗 I0718 07:56:17.887946 7 grpc_server.cc:4117] Started GRPCInferenceService at 0.0.0.0:8001 The above exception was the direct cause of the following exception: Traceback (most recent call last): The above exception was the direct cause of the following exception: Traceback (most recent call last): |
Beta Was this translation helpful? Give feedback.
-
|
再问一下，字体文件又该挂载到哪个位置？ |
Beta Was this translation helpful? Give feedback.
-
|
解决了 |
Beta Was this translation helpful? Give feedback.
-
|
2.4.2 手动构造 HTTP 请求，构造的请求体格式不对，"data"中的json应该是String类型。 |
Beta Was this translation helpful? Give feedback.
-
|
pipeline_name: OCR text_type: general use_doc_preprocessor: True SubPipelines: SubModules: Inference backend config: precision='fp16' use_dynamic_shapes=True dynamic_shapes={'x': [[1, 3, 224, 224], [1, 3, 224, 224], [8, 3, 224, 224]]} pipeline_config.yaml配置修改下模型为PP-OCRv5_server_det, PP-OCRv5_server_rec 启动报以上错误 原来使用的mobile模型， 我就修改了这个地方，起订报错了，这两个模型不可用吗？ |
Beta Was this translation helpful? Give feedback.
-
|
NFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(572)::BuildTrtEngine Start to building TensorRT Engine... [INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(572)::BuildTrtEngine Start to building TensorRT Engine... 字后还有两个报错，ERROR] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(239)::log 4: Could not find any implementation for node Conv.140 + BatchNormalization.86 + Relu.59 due to insufficient workspace. 这个是空间不足吗？ |
Beta Was this translation helpful? Give feedback.
-
|
这些错误如果不影响正常直行的话可以忽略～ |
Beta Was this translation helpful? Give feedback.
-
|
我这边调用报错了，没法调用 如果ocr mobile5模型有些内容的话识别不出来，换成ocr server5的话，识别精度会不会好点呢 |
Beta Was this translation helpful? Give feedback.
-
|
请问是在PaddleX官方镜像中安装和使用高性能推理插件的吗？ server模型的识别精度更高，不过可能需要消耗更多的计算资源。 |
Beta Was this translation helpful? Give feedback.
-
|
镜像ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/hps:paddlex3.2-gpu |
Beta Was this translation helpful? Give feedback.
-
|
调整实例的时候需要考虑本地的资源情况，上面的配置会创建5个实例，理论上显存用量是单实例情况的5倍，同时如果本地计算资源有限，不同的实例可能会挤占资源，导致推理时间变长。 |
Beta Was this translation helpful? Give feedback.
-
|
您好，我已经按照教程在CUDA11.8的镜像中安装了 hpip-gpu和paddle2onnx。但是 paddlex --serve --pipeline OCR --port 8118 --use_hpip 这条命令行会提示： The Paddle Inference backend is selected with the default configuration. This may not provide optimal performance. paddlex --serve --pipeline OCR --port 8118 --use_hpip --hpi_config '{"backend": "onnxruntime"}' 这条命令行会报错：No inference backend and configuration could be suggested. Reason: 'onnxruntime' is not a supported inference backend. 请问我是遗漏了什么步骤吗？ |
Beta Was this translation helpful? Give feedback.
-
|
还有一个问题，请问高稳定性部署是否可以使用其他端口，如何设置呢？我在部署时，可以看到服务成功启动，但无法请求8000端口 |
Beta Was this translation helpful? Give feedback.
-
|
服务启动成功后，调用接口，SetInputs TRTBackend SetInputs not find name:x 错误，请问应该如何设置呢 |
Beta Was this translation helpful? Give feedback.
-
|
http请求有文档嘛 |
Beta Was this translation helpful? Give feedback.
-
我下载【通用版面解析 v3】SDK ，使用docker run At: E1020 10:19:27.707704 7 model_repository_manager.cc:1186] failed to load 'layout-parsing' version 1: Internal: UnboundLocalError: local variable 'transpose_weight_keys' referenced before assignment At: I1020 10:19:27.707830 7 server.cc:522] I1020 10:19:27.707855 7 server.cc:549] I1020 10:19:27.707892 7 server.cc:592] I1020 10:19:27.707954 7 tritonserver.cc:1920] I1020 10:19:27.707966 7 server.cc:252] Waiting for in-flight requests to complete. |
Beta Was this translation helpful? Give feedback.
-
|
新镜像已经推上去了，可以重新拉取试试～ |
Beta Was this translation helpful? Give feedback.
-
|
好的，感谢！ |
Beta Was this translation helpful? Give feedback.
-
|
大佬，请问一下，我使用https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/PP-StructureV3.html#3提供的api Java版本测试，端口8000->8089，返回400 使用docker logs 查看 代码如下： import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.Base64;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.ObjectNode;
import okhttp3.MediaType;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import okhttp3.RequestBody;
import okhttp3.Response;
public class PpstructrueV3Test {
public static void main(String[] args) throws IOException {
final String API_URL = "http://localhost:8089/layout-parsing";
final String imagePath = "E:\\tmp\\ai\\QQ20251016-164915.jpg";
final File file = new File(imagePath);
final byte[] fileContent = java.nio.file.Files.readAllBytes(file.toPath());
final String base64Image = Base64.getEncoder().encodeToString(fileContent);
final ObjectMapper objectMapper = new ObjectMapper();
final ObjectNode payload = objectMapper.createObjectNode();
payload.put("file", base64Image);
payload.put("fileType", 1);
System.out.println(payload);
final OkHttpClient client = new OkHttpClient();
final MediaType JSON = MediaType.get("application/json; charset=utf-8");
final RequestBody body = RequestBody.create(JSON, payload.toString());
final Request request = new Request.Builder()
.url(API_URL)
.post(body)
.build();
try (Response response = client.newCall(request).execute()) {
if (response.isSuccessful()) {
final String responseBody = response.body().string();
final JsonNode root = objectMapper.readTree(responseBody);
final JsonNode result = root.get("result");
final JsonNode layoutParsingResults = result.get("layoutParsingResults");
for (int i = 0; i < layoutParsingResults.size(); i++) {
final JsonNode item = layoutParsingResults.get(i);
final int finalI = i;
final JsonNode prunedResult = item.get("prunedResult");
System.out.println("Pruned Result [" + i + "]: " + prunedResult.toString());
final JsonNode outputImages = item.get("outputImages");
outputImages.fieldNames().forEachRemaining(imgName -> {
try {
final String imgBase64 = outputImages.get(imgName).asText();
final byte[] imgBytes = Base64.getDecoder().decode(imgBase64);
final String imgPath = imgName + "_" + finalI + ".jpg";
try (FileOutputStream fos = new FileOutputStream(imgPath)) {
fos.write(imgBytes);
System.out.println("Saved image: " + imgPath);
}
} catch (final IOException e) {
System.err.println("Failed to save image: " + e.getMessage());
}
});
}
} else {
System.err.println("Request failed with HTTP code: " + response.code());
}
}
}
}会是哪里出问题呢 |
Beta Was this translation helpful? Give feedback.
-
|
如果要对tritonserver直接做HTTP请求的话，得在请求体外面再包一层（可以参考文档），基础服务化部署的调用示例无法直接适用。 |
Beta Was this translation helpful? Give feedback.
-
|
谢谢大佬，已通了 |
Beta Was this translation helpful? Give feedback.
-
|
您好，使用paddlex3.0.1 镜像，安装服务化部署插件 paddlex --install serving Traceback (most recent call last): |
Beta Was this translation helpful? Give feedback.
-
|
重新拉取镜像后，启动没有问题了，训练 报错 |
Beta Was this translation helpful? Give feedback.
-
|
请问有对应的 api 文档吗？ |
Beta Was this translation helpful? Give feedback.
-
|
如果是http调用，可以参数： “2.4.2 手动构造 HTTP 请求” 这一节说明 |
Beta Was this translation helpful? Give feedback.
-
|
CPU 模式 请求后 返回响应码404 请问是API的请求有改动了吗 |
Beta Was this translation helpful? Give feedback.
-
|
忽略我的评论 |
Beta Was this translation helpful? Give feedback.
-
|
一言难尽的文档，要啥没啥。 |
Beta Was this translation helpful? Give feedback.
-
|
我在使用高稳定性服务化部署（docker）时，高并发请求访问调用grpc接口，约会有10%左右的请求返回空结果，但是实际上这些图片中有文字内容。当我将这些图片再次调用grpc接口进行计算时，其中又有一部分可以正常输出结果。我检查了docker容器日志，发现会偶发性产生如下错误： [ ERROR] [2025-11-03 06:21:12,130] [60098771ab20491380763bb02c35a93b] [b70b0af0-16c0-4e41-a52c-53d81ae7b4ea] - Unhandled exception
Traceback (most recent call last):
File "/paddlex/py310/lib/python3.10/site-packages/paddlex_hps_server/base_model.py", line 88, in execute
result_or_output = self.run(input_, log_id)
File "/paddlex/var/paddlex_model_repo/ocr/1/model.py", line 80, in run
images, data_info = utils.file_to_images(
File "/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/serving/infra/utils.py", line 252, in file_to_images
data_info = get_image_info(images[0])
File "/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/serving/infra/utils.py", line 261, in get_image_info
return ImageInfo(width=image.shape[1], height=image.shape[0])
AttributeError: 'NoneType' object has no attribute 'shape'想请问这种情况是什么原因产生，以及如何解决？谢谢！ |
Beta Was this translation helpful? Give feedback.
-
|
能不能提供详细的http请求文档，想要用本机图片测试路径怎么设置，以及batch推理是否支持，该怎么传递图片 |
Beta Was this translation helpful? Give feedback.
-
|
ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/paddlex:paddlex3.3.4-paddlepaddle3.2.0-gpu-cuda12.9-cudnn9.9这个镜像太大了，接近60G，有没有小一点的，适合部署的，我只需要OCR相关产线 |
Beta Was this translation helpful? Give feedback.
-
|
大佬，请问一下，你是在哪里可以查看paddlex有哪些镜像及版本？ |
Beta Was this translation helpful? Give feedback.
-
|
您好，我今天测试了paddlex_hps_PaddleOCR-VL_sdk，其中使用client.py测试一个pdf文件，该pdf文件多达几十页，运行client.py后，报错，报错信息为 请问如何解决 |
Beta Was this translation helpful? Give feedback.
-
|
高稳定性服务化部署在triton，但是OCRVL模型不支持加速好像，那么用什么最快 用原始的VLLm方式最快吗 |
Beta Was this translation helpful? Give feedback.
-
|
高稳定性部署会自动转化为onnx的吗 |
Beta Was this translation helpful? Give feedback.
-
|
我部署PP-chatocrV4，里面的server中的pipeline_config.yaml的chat-botllm怎么修改成其他的api呢，还是只能用百度智能云中的api |
Beta Was this translation helpful? Give feedback.
and others
Uh oh!
There was an error while loading. Please reload this page.
-
PaddleX/latest/pipeline_deploy/serving
https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/serving.html
Beta Was this translation helpful? Give feedback.
All reactions