Allow parallel email creation to raise speed

[bernhardreiter]

Right now only one mailgen variant can run at a time
and mailgen only uses one (python thread).

If email sending speed becomes an issue, it may be possible to enable
more email creation processes to work in parallel.

Possibilities:

• allow to start several mailgen workers (from different machines)
• use threads within mailgen (because sending and crypto will be i/o bound from mailgen's side)
• make sure crypto processes can run on a different thread/core.

Both ideas will allow a machine with several cores to utilise them better.

If implemented, the sql interactions must be checked for race conditions.

Technically the use of SQL selections should right now prevent
an active second mailgen script to run.

@bernhard-herzog
https://github.com/Intevation/intelmq-mailgen/blob/master/intelmq-mailgen#L715
has FOR UPDATE NOWAIT
does this prevent mailmen from running twice like you've said?

[bernhard-herzog]

That SELECT ... FOR UPDATE together with the fact that mailgen uses the same database transaction for reading the notification information and marking a notification as sent, takes care of the serialization. That is, only one mailgen process can work with a notification at any point in time.

The NOWAIT has the effect that if you try to start a second mailgen process while another one is still running, you will get an error immediately. Without it, the second one would block until the first one finishes (and then proceed to process at least some of the same notifications again, IIRC).