Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kafka Default SSH Tunnel #366

Merged
merged 9 commits into from
Dec 5, 2023
Merged

Kafka Default SSH Tunnel #366

merged 9 commits into from
Dec 5, 2023

Conversation

dehume
Copy link
Contributor

@dehume dehume commented Nov 15, 2023

Fixes #357

Allow users to set a default ssh connections for brokers in Kafka connections. Currently ssh_tunnel is a top level attribute for Kafka connections so only a single ssh connection can be applied to all brokers. Adding ssh_tunnel as an attribute within the broker block (the same way aws privatelink is handled):

resource "materialize_connection_kafka" "kafka_connection" {
  kafka_broker {
      broker = "broker:9092"
      ssh_tunnel {
          name = materialize_connection_ssh_tunnel.tunnel.name
    }
  }
}

Will apply the ssh connection to that broker:

CREATE CONNECTION kafka_connection TO KAFKA (
    BROKERS ('broker1:9092' USING SSH TUNNEL ssh_connection)
);

And setting the ssh_tunnel at as a top level attribute will set the default:

resource "materialize_connection_kafka" "kafka_connection" {
  ssh_tunnel {
      name = materialize_connection_ssh_tunnel.tunnel.name
  }
  kafka_broker {
      broker = "broker:9092"
  }
}
CREATE CONNECTION kafka_connection TO KAFKA (
    BROKERS ('broker1:9092'),
    SSH TUNNEL ssh_connection
);

@dehume dehume added the breaking change Will cause a breaking change in the next release label Nov 15, 2023
@dehume dehume changed the title [WIP] Kafka Default SSH Tunnel Kafka Default SSH Tunnel Nov 20, 2023
@dehume dehume marked this pull request as ready for review November 20, 2023 15:29
@dehume dehume requested a review from bobbyiliev as a code owner November 20, 2023 15:29
@dehume dehume added the enhancement New feature or request label Nov 20, 2023
@dehume dehume removed the breaking change Will cause a breaking change in the next release label Dec 5, 2023
@dehume dehume merged commit 97e59fd into main Dec 5, 2023
5 checks passed
@dehume dehume deleted the Kafka-Default-SSH-Tunnel branch December 5, 2023 15:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Kafka source: support default SSH TUNNEL option
2 participants