Very large proto file takes a long time to compile

[fireyang]

Contains a large number of message proto file, compiled very slowly

[bitwalker]

Do you have an example file I can test with? How many messages are defined in the proto?

[fireyang]

https://github.com/fireyang/exprotobuf_performance
40 message and 3 enum; Add a new message every time recompile all

[bitwalker]

Thanks! I'll take a look as soon as possible

[dweinstein]

similar experience here w/ https://gist.github.com/dweinstein/5c2fa9a197974ee0fcf766e67d683854

[bitwalker]

It looks like the bulk of the time is split between gpb parsing and the Elixir compiler - ultimately there isn't much I can do to ease this with the current architecture.

I'm not sure if there is anyway to generate the Elixir source such that compilation is meaningfully faster (at least from a clean state), however one approach would be to generate Elixir source files, and which are then subsequently compiled. This is how one consumes gpb directly, and I am interested in going that route, because it would be much easier to ensure that only those things which actually changed are recompiled.

I'm considering a rewrite of exprotobuf for that reason, amongst others, but time is elusive, so I haven't been able to take a good crack at it yet.

[mustafaturan]

@bitwalker Would you please consider to use enif_protobuf? It looks it is at least 3 times faster than gbp