-
Notifications
You must be signed in to change notification settings - Fork 19
Description
After switching to platformatic kafka, we have found that messages are being published to different partitions than they were before.
This breaks compaction and cases of multiple producers with different tech stacks.
The murmur2 implementation seems to be compatible in all three cases, however after applying the murmur2 function, the java and kafkajs libraries use a toPositive implementation as follows:
const toPositive = x => x & 0x7fffffff
(See https://github.com/tulios/kafkajs/blob/master/src/producer/partitioners/legacy/partitioner.js#L12 and https://github.com/apache/kafka/blob/0.10.2/clients/src/main/java/org/apache/kafka/common/utils/Utils.java#L770)
The platformatic library instead uses this:
partition = murmur2(key) >>> 0
(https://github.com/platformatic/kafka/blob/main/src/clients/producer/producer.ts#L337)
If this is intended, it should possibly be added to the readme.