Skip to content

Commit

Permalink
Merge branch 'ci' into layoutlm
Browse files Browse the repository at this point in the history
  • Loading branch information
gaotongxiao committed Oct 20, 2023
2 parents ae8c426 + 1d3b1ca commit ab14bb0
Show file tree
Hide file tree
Showing 34 changed files with 1,623 additions and 13 deletions.
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
[![Average time to resolve an issue](https://isitmaintained.com/badge/resolution/open-mmlab/mmocr.svg)](https://github.com/open-mmlab/mmocr/issues)
[![Percentage of issues still open](https://isitmaintained.com/badge/open/open-mmlab/mmocr.svg)](https://github.com/open-mmlab/mmocr/issues)
<a href="https://console.tiyaro.ai/explore?q=mmocr&pub=mmocr"> <img src="https://tiyaro-public-docs.s3.us-west-2.amazonaws.com/assets/try_on_tiyaro_badge.svg"></a>
[![Open in OpenXLab](https://cdn-static.openxlab.org.cn/app-center/openxlab_demo.svg)](https://openxlab.org.cn/apps?search=mmocr)

[📘Documentation](https://mmocr.readthedocs.io/en/dev-1.x/) |
[🛠️Installation](https://mmocr.readthedocs.io/en/dev-1.x/get_started/install.html) |
Expand Down Expand Up @@ -151,6 +152,7 @@ Supported algorithms:
- [x] [ABINet](configs/textrecog/abinet/README.md) (CVPR'2021)
- [x] [ASTER](configs/textrecog/aster/README.md) (TPAMI'2018)
- [x] [CRNN](configs/textrecog/crnn/README.md) (TPAMI'2016)
- [x] [MAERec](configs/textrecog/maerec/README.md) (ICCV'2023)
- [x] [MASTER](configs/textrecog/master/README.md) (PR'2021)
- [x] [NRTR](configs/textrecog/nrtr/README.md) (ICDAR'2019)
- [x] [RobustScanner](configs/textrecog/robust_scanner/README.md) (ECCV'2020)
Expand Down
2 changes: 2 additions & 0 deletions README_zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
[![Average time to resolve an issue](https://isitmaintained.com/badge/resolution/open-mmlab/mmocr.svg)](https://github.com/open-mmlab/mmocr/issues)
[![Percentage of issues still open](https://isitmaintained.com/badge/open/open-mmlab/mmocr.svg)](https://github.com/open-mmlab/mmocr/issues)
<a href="https://console.tiyaro.ai/explore?q=mmocr&pub=mmocr"> <img src="https://tiyaro-public-docs.s3.us-west-2.amazonaws.com/assets/try_on_tiyaro_badge.svg"></a>
[![Open in OpenXLab](https://cdn-static.openxlab.org.cn/app-center/openxlab_demo.svg)](https://openxlab.org.cn/apps?search=mmocr)

[📘文档](https://mmocr.readthedocs.io/zh_CN/dev-1.x/) |
[🛠️安装](https://mmocr.readthedocs.io/zh_CN/dev-1.x/get_started/install.html) |
Expand Down Expand Up @@ -150,6 +151,7 @@ mim install -e .
- [x] [ABINet](configs/textrecog/abinet/README.md) (CVPR'2021)
- [x] [ASTER](configs/textrecog/aster/README.md) (TPAMI'2018)
- [x] [CRNN](configs/textrecog/crnn/README.md) (TPAMI'2016)
- [x] [MAERec](configs/textrecog/maerec/README.md) (ICCV'2023)
- [x] [MASTER](configs/textrecog/master/README.md) (PR'2021)
- [x] [NRTR](configs/textrecog/nrtr/README.md) (ICDAR'2019)
- [x] [RobustScanner](configs/textrecog/robust_scanner/README.md) (ECCV'2020)
Expand Down
65 changes: 65 additions & 0 deletions configs/textrecog/_base_/datasets/union14m_benchmark.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
union14m_root = 'data/Union14M-L/'
union14m_benchmark_root = 'data/Union14M-L/Union14M-Benchmarks'

union14m_benchmark_artistic = dict(
type='OCRDataset',
data_prefix=dict(img_path=f'{union14m_benchmark_root}/artistic'),
ann_file=f'{union14m_benchmark_root}/artistic/annotation.json',
test_mode=True,
pipeline=None)

union14m_benchmark_contextless = dict(
type='OCRDataset',
data_prefix=dict(img_path=f'{union14m_benchmark_root}/contextless'),
ann_file=f'{union14m_benchmark_root}/contextless/annotation.json',
test_mode=True,
pipeline=None)

union14m_benchmark_curve = dict(
type='OCRDataset',
data_prefix=dict(img_path=f'{union14m_benchmark_root}/curve'),
ann_file=f'{union14m_benchmark_root}/curve/annotation.json',
test_mode=True,
pipeline=None)

union14m_benchmark_incomplete = dict(
type='OCRDataset',
data_prefix=dict(img_path=f'{union14m_benchmark_root}/incomplete'),
ann_file=f'{union14m_benchmark_root}/incomplete/annotation.json',
test_mode=True,
pipeline=None)

union14m_benchmark_incomplete_ori = dict(
type='OCRDataset',
data_prefix=dict(img_path=f'{union14m_benchmark_root}/incomplete_ori'),
ann_file=f'{union14m_benchmark_root}/incomplete_ori/annotation.json',
test_mode=True,
pipeline=None)

union14m_benchmark_multi_oriented = dict(
type='OCRDataset',
data_prefix=dict(img_path=f'{union14m_benchmark_root}/multi_oriented'),
ann_file=f'{union14m_benchmark_root}/multi_oriented/annotation.json',
test_mode=True,
pipeline=None)

union14m_benchmark_multi_words = dict(
type='OCRDataset',
data_prefix=dict(img_path=f'{union14m_benchmark_root}/multi_words'),
ann_file=f'{union14m_benchmark_root}/multi_words/annotation.json',
test_mode=True,
pipeline=None)

union14m_benchmark_salient = dict(
type='OCRDataset',
data_prefix=dict(img_path=f'{union14m_benchmark_root}/salient'),
ann_file=f'{union14m_benchmark_root}/salient/annotation.json',
test_mode=True,
pipeline=None)

union14m_benchmark_general = dict(
type='OCRDataset',
data_prefix=dict(img_path=f'{union14m_root}/'),
ann_file=f'{union14m_benchmark_root}/general/annotation.json',
test_mode=True,
pipeline=None)
38 changes: 38 additions & 0 deletions configs/textrecog/_base_/datasets/union14m_train.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
union14m_data_root = 'data/Union14M-L/'

union14m_challenging = dict(
type='OCRDataset',
data_root=union14m_data_root,
ann_file='train_annos/mmocr1.0/train_challenging.json',
test_mode=True,
pipeline=None)

union14m_hard = dict(
type='OCRDataset',
data_root=union14m_data_root,
ann_file='train_annos/mmocr1.0/train_hard.json',
pipeline=None)

union14m_medium = dict(
type='OCRDataset',
data_root=union14m_data_root,
ann_file='train_annos/mmocr1.0/train_medium.json',
pipeline=None)

union14m_normal = dict(
type='OCRDataset',
data_root=union14m_data_root,
ann_file='train_annos/mmocr1.0/train_normal.json',
pipeline=None)

union14m_easy = dict(
type='OCRDataset',
data_root=union14m_data_root,
ann_file='train_annos/mmocr1.0/train_easy.json',
pipeline=None)

union14m_val = dict(
type='OCRDataset',
data_root=union14m_data_root,
ann_file='train_annos/mmocr1.0/val_annos.json',
pipeline=None)
21 changes: 21 additions & 0 deletions configs/textrecog/_base_/schedules/schedule_adamw_cos_10e.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# optimizer
optim_wrapper = dict(
type='OptimWrapper',
optimizer=dict(
type='AdamW',
lr=4e-4,
betas=(0.9, 0.999),
eps=1e-08,
weight_decay=0.01))
train_cfg = dict(type='EpochBasedTrainLoop', max_epochs=10, val_interval=1)
val_cfg = dict(type='ValLoop')
test_cfg = dict(type='TestLoop')

# learning policy
param_scheduler = [
dict(
type='CosineAnnealingLR',
T_max=10,
eta_min=4e-6,
convert_to_iter_based=True)
]
16 changes: 16 additions & 0 deletions configs/textrecog/abinet/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,22 @@ Linguistic knowledge is of great benefit to scene text recognition. However, how
2. Facts about the pretrained model: MMOCR does not have a systematic pipeline to pretrain the language model (LM) yet, thus the weights of LM are converted from [the official pretrained model](https://github.com/FangShancheng/ABINet). The weights of ABINet-Vision are directly used as the vision model of ABINet.
```

We also provide ABINet trained on [Union14M](https://github.com/Mountchicken/Union14M)

- Evaluated on six common benchmarks

| methods | pretrained | | Regular Text | | | | Irregular Text | | download |
| :---------------------------------------------------------------: | :--------: | :----: | :----------: | :-------: | :-: | :-------: | :------------: | :----: | :----------------------------------------------------------------- |
| | | IIIT5K | SVT | IC13-1015 | | IC15-2077 | SVTP | CT80 | |
| [ABINet-Vision](configs/textrecog/abinet/abinet-vision_10e_union14m.py) | - | 0.9730 | 0.9645 | 0.9552 | | 0.8536 | 0.8977 | 0.9479 | [model](https://download.openmmlab.com/mmocr/textrecog/abinet/abinet_union14m-cbf19742.pth) |

- Evaluated on [Union14M-Benchmark](https://github.com/Mountchicken/Union14M)

| Methods | | Unsolved Challenges | | | | | Additional Challenges | | General | download |
| ------------------------------------------------------ | ----- | ------------------- | -------- | ----------- | --- | ------- | --------------------- | ---------- | ------- | ------------------------------------------------------- |
| | Curve | Multi-Oriented | Artistic | Contextless | | Salient | Multi-Words | Incomplete | General | |
| [ABINet-Vision](configs/textrecog/abinet/abinet-vision_10e_union14m.py) | 0.750 | 0.615 | 0.653 | 0.711 | | 0.729 | 0.591 | 0.026 | 0.794 | [model](https://download.openmmlab.com/mmocr/textrecog/abinet/abinet_union14m-cbf19742.pth) |

## Citation

```bibtex
Expand Down
118 changes: 118 additions & 0 deletions configs/textrecog/abinet/abinet-vision_10e_union14m.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
_base_ = [
'../_base_/datasets/union14m_train.py',
'../_base_/datasets/union14m_benchmark.py',
'../_base_/datasets/cute80.py',
'../_base_/datasets/iiit5k.py',
'../_base_/datasets/svt.py',
'../_base_/datasets/svtp.py',
'../_base_/datasets/icdar2013.py',
'../_base_/datasets/icdar2015.py',
'../_base_/default_runtime.py',
'../_base_/schedules/schedule_adamw_cos_10e.py',
'_base_abinet.py',
]

load_from = 'https://download.openmmlab.com/mmocr/textrecog/abinet/abinet_pretrain-45deac15.pth' # noqa

_base_.pop('model')
dictionary = dict(
type='Dictionary',
dict_file= # noqa
'{{ fileDirname }}/../../../dicts/english_digits_symbols_space.txt',
with_padding=True,
with_unknown=True,
same_start_end=True,
with_start=True,
with_end=True)

model = dict(
type='ABINet',
backbone=dict(type='ResNetABI'),
encoder=dict(
type='ABIEncoder',
n_layers=3,
n_head=8,
d_model=512,
d_inner=2048,
dropout=0.1,
max_len=8 * 32,
),
decoder=dict(
type='ABIFuser',
vision_decoder=dict(
type='ABIVisionDecoder',
in_channels=512,
num_channels=64,
attn_height=8,
attn_width=32,
attn_mode='nearest',
init_cfg=dict(type='Xavier', layer='Conv2d')),
module_loss=dict(type='ABIModuleLoss'),
postprocessor=dict(type='AttentionPostprocessor'),
dictionary=dictionary,
max_seq_len=26,
),
data_preprocessor=dict(
type='TextRecogDataPreprocessor',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375]))

# dataset settings
train_list = [
_base_.union14m_challenging, _base_.union14m_hard, _base_.union14m_medium,
_base_.union14m_normal, _base_.union14m_easy
]
val_list = [
_base_.cute80_textrecog_test, _base_.iiit5k_textrecog_test,
_base_.svt_textrecog_test, _base_.svtp_textrecog_test,
_base_.icdar2013_textrecog_test, _base_.icdar2015_textrecog_test
]
test_list = [
_base_.union14m_benchmark_artistic,
_base_.union14m_benchmark_multi_oriented,
_base_.union14m_benchmark_contextless,
_base_.union14m_benchmark_curve,
_base_.union14m_benchmark_incomplete,
_base_.union14m_benchmark_incomplete_ori,
_base_.union14m_benchmark_multi_words,
_base_.union14m_benchmark_salient,
_base_.union14m_benchmark_general,
]

train_dataset = dict(
type='ConcatDataset', datasets=train_list, pipeline=_base_.train_pipeline)
test_dataset = dict(
type='ConcatDataset', datasets=test_list, pipeline=_base_.test_pipeline)
val_dataset = dict(
type='ConcatDataset', datasets=val_list, pipeline=_base_.test_pipeline)

train_dataloader = dict(
batch_size=128,
num_workers=24,
persistent_workers=True,
sampler=dict(type='DefaultSampler', shuffle=True),
dataset=train_dataset)

test_dataloader = dict(
batch_size=128,
num_workers=4,
persistent_workers=True,
drop_last=False,
sampler=dict(type='DefaultSampler', shuffle=False),
dataset=test_dataset)

val_dataloader = dict(
batch_size=128,
num_workers=4,
persistent_workers=True,
pin_memory=True,
drop_last=False,
sampler=dict(type='DefaultSampler', shuffle=False),
dataset=val_dataset)

val_evaluator = dict(
dataset_prefixes=['CUTE80', 'IIIT5K', 'SVT', 'SVTP', 'IC13', 'IC15'])
test_evaluator = dict(dataset_prefixes=[
'artistic', 'multi-oriented', 'contextless', 'curve', 'incomplete',
'incomplete-ori', 'multi-words', 'salient', 'general'
])
16 changes: 16 additions & 0 deletions configs/textrecog/aster/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,22 @@ A challenging aspect of scene text recognition is to handle text with distortion
| [ASTER](/configs/textrecog/aster/aster_resnet45_6e_st_mj.py) | ResNet45 | 0.9357 | 0.8949 | 0.9281 | | 0.7665 | 0.8062 | 0.8507 | [model](https://download.openmmlab.com/mmocr/textrecog/aster/aster_resnet45_6e_st_mj/aster_resnet45_6e_st_mj-cc56eca4.pth) \| [log](https://download.openmmlab.com/mmocr/textrecog/aster/aster_resnet45_6e_st_mj/20221214_232605.log) |
| [ASTER-TTA](/configs/textrecog/aster/aster_resnet45_6e_st_mj.py) | ResNet45 | 0.9337 | 0.8949 | 0.9251 | | 0.7925 | 0.8109 | 0.8507 | |

We also provide ASTER trained on [Union14M](https://github.com/Mountchicken/Union14M)

- Evaluated on six common benchmarks

| Methods | pretrained | | Regular Text | | | | Irregular Text | | download |
| :------------------------------------------------------------: | :--------: | :----: | :----------: | :-------: | :-: | :-------: | :------------: | :----: | :-------------------------------------------------------------------- |
| | | IIIT5K | SVT | IC13-1015 | | IC15-2077 | SVTP | CT80 | |
| [ASTER](configs/textrecog/aster/aster_resnet45_6e_union14m.py) | - | 0.9437 | 0.8903 | 0.9360 | | 0.7857 | 0.8093 | 0.9097 | [model](https://download.openmmlab.com/mmocr/textrecog/aster/aster_union14m/aster_union14m-230eb471.pth) |

- Evaluated on [Union14M-Benchmark](https://github.com/Mountchicken/Union14M)

| Methods | | Unsolved Challenges | | | | | Additional Challenges | | General | download |
| ------------------------------------------------------ | ----- | ------------------- | -------- | ----------- | --- | ------- | --------------------- | ---------- | ------- | ------------------------------------------------------- |
| | Curve | Multi-Oriented | Artistic | Contextless | | Salient | Multi-Words | Incomplete | General | |
| [ASTER](configs/textrecog/aster/aster_resnet45_6e_union14m.py) | 0.384 | 0.130 | 0.418 | 0.529 | | 0.319 | 0.498 | 0.013 | 0.667 | [model](https://download.openmmlab.com/mmocr/textrecog/aster/aster_union14m/aster_union14m-230eb471.pth) |

## Citation

```bibtex
Expand Down
Loading

0 comments on commit ab14bb0

Please sign in to comment.