Bug: 文档提及的PaddleOCRv4（英语）文字识别模型未上传到HuggingFace

[cainmagi]

问题描述 / Problem Description

查阅此处文档：
https://rapidai.github.io/RapidOCRDocs/model_list/

可知有一个名叫en_PP-OCRv4_rec_infer.onnx的模型。

然而该模型并未上传到HuggingFace的仓库：

https://huggingface.co/SWHL/RapidOCR/tree/main/PP-OCRv4

为什么需要这个模型？

据说该模型在处理空白字符的问题上表现更好。参见：
PaddlePaddle/PaddleOCR#5448

运行环境 / Runtime Environment

N/A

复现代码 / Reproduction Code

N/A

可能解决方案 / Possible solutions

N/A

[SWHL]

收到，这块我会统一整理

[leminhyen2]

@cainmagi I couldn't find "en_PP-OCRv4_rec_infer.onnx" in the link you shared

[cainmagi]

@cainmagi I couldn't find "en_PP-OCRv4_rec_infer.onnx" in the link you shared

Hello, @leminhyen2 ! I am meaning that this model supposes to be uploaded to the repository (the link I attached to this thread) but it was actually not.

There is indeed a v4 model named en_PP-OCRv4_rec for Paddle OCR English recognition. However, it is very strange that this model is only mentioned in Chinese documentation and not in English documentation.

https://paddlepaddle.github.io/PaddleOCR/main/ppocr/model_list.html#22

Note that this model is for PaddleOCR. It is not an ONNX model. If you want to have an ONNX model that is available for this project, you may still have to wait the author to upload the ONNX version later.

[leminhyen2]

@cainmagi Thanks for the link, I converted it to ONNX but couldn't get it to work with RapidOCR. The error indicated that Rapid OCR identified this model as v3_rec model. So maybe we'll need to wait for a new version of RapidOCR to support en V4 model. I wanted to update this model because the en v3 model did a bad job at recognizing english sentences while the zh-en model doesn't have white space.

File "\Modules\Lightweight_OCR_Server\Paddle_API.py", line 29, in
english_ocr_model = RapidOCR(det_model_path=det_path,rec_model_path=english_rec_path,intra_op_num_threads=-1,inter_op_num_threads=-1)
File "\Python39\lib\site-packages\rapidocr_onnxruntime\main.py", line 54, in init
self.text_rec = TextRecognizer(config["Rec"])
File "\Python39\lib\site-packages\rapidocr_onnxruntime\ch_ppocr_v3_rec\text_recognize.py", line 35, in init
self.postprocess_op = CTCLabelDecode(
File "\Python39\lib\site-packages\rapidocr_onnxruntime\ch_ppocr_v3_rec\utils.py", line 16, in init
self.character = self.get_character(character, character_path)
File "\Python39\lib\site-packages\rapidocr_onnxruntime\ch_ppocr_v3_rec\utils.py", line 31, in get_character
raise ValueError("character must not be None")
ValueError: character must not be None

[SWHL]

In the RapidOCR, I had written the rec dict to the onnx model. So you met the error. There are two solutions:

Firstly, you can use the PaddleOCRModelConvert tool to convert the model, which can finish the step automatically.

Secondly, you need to add the rec dict to onnx model as the following code:

import onnx
import onnxruntime as ort
from onnx import ModelProto
class ONNXMetaOp:
    @classmethod
    def add_meta(
        cls,
        model_path: Union[str, Path],
        key: str,
        value: List[str],
        delimiter: str = "\n",
    ) -> ModelProto:
        model = onnx.load_model(model_path)
        meta = model.metadata_props.add()
        meta.key = key
        meta.value = delimiter.join(value)
        return model

    @classmethod
    def get_meta(
        cls, model_path: Union[str, Path], key: str, split_sym: str = "\n"
    ) -> List[str]:
        sess = ort.InferenceSession(model_path)
        meta_map = sess.get_modelmeta().custom_metadata_map
        key_content = meta_map.get(key)
        key_list = key_content.split(split_sym)
        return key_list

    @classmethod
    def del_meta(cls, model_path: Union[str, Path]) -> ModelProto:
        model = onnx.load_model(model_path)
        del model.metadata_props[:]
        return model

    @classmethod
    def save_model(cls, save_path: Union[str, Path], model: ModelProto):
        onnx.save_model(model, save_path)


rec_dict_content = [
    "Text",
    "Title",
    "Figure",
    "Figure caption",
    "Table",
    "Table caption",
    "Header",
    "Footer",
    "Reference",
    "Equation",
]
model_path = "models/paper-8n.onnx"
model = ONNXMetaOp.add_meta(model_path, key="character", value=rec_dict_content)

new_model_path = "models/with_meta_onnx/1.onnx"
ONNXMetaOp.save_model(new_model_path, model)

t = ONNXMetaOp.get_meta(new_model_path, key="character")
print(t)

[cainmagi]

@leminhyen2 Thank you for letting me know this! Let's wait for the next update. ;-)

By the way, I have tried to use v4 model by using PaddleOCR only. It indeed works better on the text space. However, I spot that it still has the text space missing issue, especially when the paragraph is aligned by justify and a line has a wider space than the other lines. For example,

It is highly possible that the space is missing. Maybe I should submit an issue to PaddleOCR about this.

[SWHL]

Regarding the space loss problem, this is a known issue. The root cause of this problem is that there are fewer data sets in this area during model training.