Keying runners / passing args

[barrelltech]

Hey FLAME team!

I'm planning to move my NLP pipeline over to use FLAME. What I would like to do is have one FLAME runner for each possible language - if the language pipeline is not loaded, spin up a FLAME instance, process all the documents that come in for that language, then after an idle time shut them down.

I'm not seeing any way to key tasks, or pass in an arg, besides making a custom flame runner for each language and adding all ~100 to my application supervision tree.

Is this something currently supported / could potentially be supported by FLAME? Happy to take a crack and submit a PR if the answers are no and yes respectively :)

[chrismccord]

I'm not seeing any way to key tasks, or pass in an arg, besides making a custom flame runner for each language and adding all ~100 to my application supervision tree.

I actually think this is quite reasonable. You can have your own supervisor that starts all the pools, one for reach language, so your app supervision tree remains tidy. You effectively want a local registry to call into each pool, which this already gives you, so I would roll with that and see if you need anything higher level, but I'm willing to be you'll be well served. Let us know how it goes! Thanks!

[barrelltech]

It was pretty straightforward indeed! BEAM never ceases to amaze. Thanks for the input!

If anyone finds this thread the code is included in another issue raised: #64