ABINet large performance difference in word_acc and word_acc_ignore_case

[hugotong6425]

I was training a ABINet from abinet_20e_st-an_mj pretrained weight but with a new dictionary english_digits_symbols_space.txt. Train set includes the train set of Cocotextv2, ICDAR2015, SVT, Textocr, Totaltext and Wildreceipt. I used the test set of above 6 datasets to evaluate.

I choose these 6 datasets because their labels are already case-sensitive. But I am curious why there would be a large difference in word_acc and word_acc_ignore_case. Please let me know if you have any insight.

	word_acc	word_acc_ignore_case	word_acc_ignore_case_symbol
Cocotextv2	0.2273	0.6763	0.6998
ICDAR2015	0.1406	0.8349	0.8536
SVT	0.9320	0.9320	0.9505
Textocr	0.3806	0.7616	0.7884
Totaltext	0.1153	0.7789	0.8278
Wildreceipt	0.3799	0.8611	0.8825

[gaotongxiao]

Not sure if you have changed the configuration here:

mmocr/configs/textrecog/abinet/_base_abinet-vision.py

Line 32 in f820470

module_loss=dict(type='ABIModuleLoss', letter_case='lower'),

letter_case should be "unchanged" if you want the model to be case-sensitive.

[hugotong6425]

Thanks a lot! I think it is the problem.