Is there a way to optimize transfomers backend binary size?

[yupm]

Hello..
Currently I am trying to bundle a deb package for Poricom which uses manga_ocr.
However, the torch dependencies of the transformers backend are making the deb excessively large.

$ find . -type f -size +50M -exec du -sh {} \;

318M    ./opt/poricom/libcublasLt.so.11
145M    ./opt/poricom/libcublas.so.11
179M    ./opt/poricom/unidic_lite/dicdir/sys.dic
69M     ./opt/poricom/unidic_lite/dicdir/matrix.bin
438M    ./opt/poricom/libcudnn_cnn_infer.so.8
88M     ./opt/poricom/libcudnn_ops_infer.so.8
57M     ./opt/poricom/assets/languages/chi_tra.traineddata
526M    ./opt/poricom/torch/lib/libtorch_cpu.so
253M    ./opt/poricom/torch/lib/libtorch_cuda_cpp.so
728M    ./opt/poricom/torch/lib/libtorch_cuda_cu.so

I wonder if you could advise or help regarding reducing the size of the bundle by using another backend such as TF or JAX.

[kha-white]

It's not a priority for me right now, but I agree that it would be nice to drop the torch dependency. Another option is ONNX.

[HighLiuk]

Hey there @yupm I'm currently trying to replicate MangaOcr outside of python so I can give you some hints. You can find here #45 a way to export an ONNX model and drop both the transformers & torch dependencies. However consider that this model alone (ONNX file) is more than 500M and I guess you cannot lower that.