Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dataflow-Kafka Consumer Group - UnAuthorized Consumer Group #78

Open
RohanM07 opened this issue Feb 22, 2022 · 0 comments
Open

Dataflow-Kafka Consumer Group - UnAuthorized Consumer Group #78

RohanM07 opened this issue Feb 22, 2022 · 0 comments

Comments

@RohanM07
Copy link

Issue Summary:
the warning in data flow worker is misleading, I tried to dig deep into Beam KafkaIO, looks like this is a dataflow runner

What is the Business Impact you are facing?

Observed Behavior:
I m able to read the data from confluent Kafka with the provided Consumer group development.opp-ds.svc-app-event-ingest.omni-cg But the warning I'm getting is misleading but the Kafka Consumer Lag is going down as intended.

I was successfully able to read the 98,637+1,190,217 events from two different dataflow jobs and the consumer group Lag went down to Zero.

But the warning in the data flow worker is misleading, I tried to dig deep into Beam KafkaIO.
Warning :
"org.apache.kafka.common.errors.GroupAuthorizationException: Not authorized to access group: Reader-7_offset_consumer_136115595_development.opp-ds.svc-app-event-ingest.omni-cg-kafka-test-14

Expected Behavior (jobID if applicable): We don't have a JobID but I should not be seeing these unauthorized exceptions.

Please include screenshots. Yes

If Applicable:

  • Beam SDK (Language and Version): 2.32.0
  • Apache Kafka Version - 2.3.1
    NotAuthorizedAccessError
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant