Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

版面分析模型doclayout 输入shape是固定的1024x1024问题 #3

Open
SWHL opened this issue Nov 4, 2024 · 13 comments
Open

版面分析模型doclayout 输入shape是固定的1024x1024问题 #3

SWHL opened this issue Nov 4, 2024 · 13 comments

Comments

@SWHL
Copy link
Member

SWHL commented Nov 4, 2024

新建的版面分析模型doclayout 输入shape是固定的1024x1024, 这样会有两个问题：
1、有的图片会因非等比例缩放， 导致结果不准
2、增加计算量
建议， 将onnx模型的输入shape转成dynamic, 图像使用letterBox预处理后送入模型

Originally posted by @jesse01 in RapidAI/RapidOCR#246 (comment)
@SWHL
Copy link
Member Author

SWHL commented Nov 11, 2024

可以参考资料：https://github.com/opendatalab/DocLayout-YOLO/blob/119996a102e383c15a9fb0897ff038215ccfaf22/assets/script.sh#L17

@FrankWhh
Copy link

确实存在此问题，按照官方处理的预测精度更高

https://github.com/opendatalab/DocLayout-YOLO/blob/119996a102e383c15a9fb0897ff038215ccfaf22/doclayout_yolo/data/augment.py#L684

@SWHL
Copy link
Member Author

SWHL commented Nov 29, 2024 via email •
edited
Loading

@FrankWhh
Copy link

FrankWhh commented Nov 29, 2024 via email

@SWHL
Copy link
Member Author

SWHL commented Nov 29, 2024 via email

@FrankWhh
Copy link

FrankWhh commented Nov 29, 2024 via email

@SWHL
Copy link
Member Author

SWHL commented Nov 29, 2024 via email •
edited
Loading

@FrankWhh
Copy link

FrankWhh commented Nov 30, 2024 via email •
edited by SWHL
Loading

@SWHL
Copy link
Member Author

SWHL commented Nov 30, 2024 via email

@FrankWhh
Copy link

FrankWhh commented Dec 3, 2024

一、 基于您的onnx模型，我对图片增加padding预处理可以解决框的精度问题
二、我尝试重新导出onnx模型，跟官方模型基本可以对上，但是跟您这边的onnx不一样，而且无法被本项目使用，get_charater_list()报错，无character这个key，请问下，您这边onnx的导出参数是啥？

@SWHL
Copy link
Member Author

SWHL commented Dec 3, 2024 via email •
edited
Loading

@SWHL
Copy link
Member Author

SWHL commented Dec 3, 2024 •
edited
Loading

参考下面代码，把类别添加进去就可以用了。欢迎提一个PR，助力更新该项目模型。

from pathlib import Path
from typing import List, Union

import onnx
import onnxruntime as ort
from onnx import ModelProto
class ONNXMetaOp:
    @classmethod
    def add_meta(
        cls,
        model_path: Union[str, Path],
        key: str,
        value: List[str],
        delimiter: str = "\n",
    ) -> ModelProto:
        model = onnx.load_model(model_path)
        meta = model.metadata_props.add()
        meta.key = key
        meta.value = delimiter.join(value)
        return model

    @classmethod
    def get_meta(
        cls, model_path: Union[str, Path], key: str, split_sym: str = "\n"
    ) -> List[str]:
        sess = ort.InferenceSession(model_path)
        meta_map = sess.get_modelmeta().custom_metadata_map
        key_content = meta_map.get(key)
        key_list = key_content.split(split_sym)
        return key_list

    @classmethod
    def del_meta(cls, model_path: Union[str, Path]) -> ModelProto:
        model = onnx.load_model(model_path)
        del model.metadata_props[:]
        return model

    @classmethod
    def save_model(cls, save_path: Union[str, Path], model: ModelProto):
        onnx.save_model(model, save_path)

paper_label = [
    "Text",
    "Title",
    "Figure",
    "Figure caption",
    "Table",
    "Table caption",
    "Header",
    "Footer",
    "Reference",
    "Equation",
]
model_path = "models/paper-8n.onnx"
model = ONNXMetaOp.add_meta(model_path, key="character", value=paper_label)

new_model_path = "models/with_meta_onnx/1.onnx"
ONNXMetaOp.save_model(new_model_path, model)

t = ONNXMetaOp.get_meta(new_model_path, key="character")
print(t)

@FrankWhh
Copy link

FrankWhh commented Dec 3, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@SWHL @FrankWhh