Dependency graph: per topic aggregation.

[Karchevskiy]

Hi Jorge! I am really inspired by the work you have done in the direction of building distributed systems traces for systems contains kafka as message broker service. You probably also explored the issue of building a diagram of component relationships. It seems that kafka should not be a single node on such graphs, since it is often used as a transport for asynchronous operations and working on different way if compare with rest-dependent nodes. The default connectors provide information about the topics and types of messages sent (at least for java implementations). Whether someone uses aggregation on diagram not by the service name (kafka), but by the attribute indicating the topic or even message type. Were there any similar solutions?

[jeqo]

Thanks @Karchevskiy !

I think this issue is related openzipkin/brave#895.

I look forward to put some time on bump the discussion again but here are some thoughts on what's missing:

If you follow the discussion, there seem to be a (potential) mismatch between what the dependency graph currently express and what we want to achieve: a topic-based relation between producers and consumers, instead of cluster-based.

We have to look over the full pipeline to see where the mismatch might be:

• Brave's Kafka instrumentation on producer/consumer defines once a remoteServiceName that is currently used to set the Kafka cluster name (default: kafka).
• Dependency graph model uses this remoteServiceName to create a link between producer to cluster, and consumer to cluster.

As this value--remote service name-- is defined once, we can't redefine it on-the-fly, when creating spans. Though, initially the referenced issue is a feature-request to support this as a concat between cluster name and topic instead of only cluster name.
In the discussion this is been compared to service:span granularity, which is fair.

Another initial/naive alternatives that I have in mind:

• Propose an alternative model to dependency graph: path-based graph where instead of service to service, we could get service:span to service:span
• In addition, consider to define also remoteSpanName from instrumentation, so in the case of messaging instrumentations, we could use this field to store topic names and use the path-based graph to represent: producer:send to kafka:topic1 to consumer:poll.

Both involve a considerable change and additions to the current model.

[Karchevskiy]

Thank you @jeqo for the detailed answer. It seems like the easiest workaround (may be not fast, but should working, if I understood correctly) is to simply overwrite zipkin2.collector.Collector#record endpoint data (add concatenation with any related to Kafka tags data) without changing any client configurations. sorry if this barbarism breaks Zipkin's philosophy (:

[jeqo]

Depends what you're trying to achieve. If you are looking to change the kafka service name to something like kafka+[topic_name] and you have 1 topic per client, you can set that directly when instatiating KafkaTracing:

kafkaTracing = KafkaTracing.newBuilder(messagingTracing)
                           .remoteServiceName("my-broker_topic1")
                           .build();

Then service dependency graph will show relationships based on kafka topics and not just kafka. Will this work on your context?

[codefromthecrypt]

PS If you are using Brave, FinishedSpanHandler was made exactly for things like this.

…

On Thu, Mar 26, 2020 at 10:47 PM Jorge Quilcate Otoya < ***@***.***> wrote: Depends what you're trying to achieve. If you are looking to change the kafka service name to something like kafka+[topic_name] and you have 1 topic per client, you can set that directly when instatiating KafkaTracing : kafkaTracing = KafkaTracing.newBuilder(messagingTracing) .remoteServiceName("my-broker_topic1") .build(); Then service dependency graph will show relationships based on kafka topics and not just kafka. Will this work on your context? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#5 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAPVVY4KHJE2C47K5JLRSDRJNTJBANCNFSM4LQQH4FQ> .

[jeqo]

@Karchevskiy ⬆️