Skip to content

Latest commit

 

History

History
132 lines (88 loc) · 3.06 KB

CONTRIBUTING.md

File metadata and controls

132 lines (88 loc) · 3.06 KB

Pre-requisites

  • Git
  • Make (see Makefile for all available commands)
  • Go 1.22+

Install Protoc and Go plugin

Download the protocol buffer compiler suitable for your platform from the releases page and install it to $HOME/.local.

For example (Linux x86_64):

unzip protoc-25.2-linux-x86_64.zip -d $HOME/.local

Make sure that you have export PATH="$HOME/.local/bin:$PATH" in your .bashrc/.zshrc.

(macOS only): try to run protoc from the terminal. If protoc cannot be executed due to the OS security policy, go to Settings -> Privacy & Security, find "protoc" there, and click "Allow anyway". Run protoc one more time from the terminal, and click "Open".

Then, install the Go plugin for the protocol buffer compiler:

make install-protoc-gen-go

Generate Go code from the Protobuf definitions

Download the Fivetran SDK protobuf files first. These files contain the types definitions for the destination GRPC server.

You can do it by executing the following command:

make prepare-fivetran-sdk

Then, generate the Go code:

make generate-proto

Running Go tests

Start ClickHouse server in Docker:

docker compose up -d

Test files marked as _integration_test require ClickHouse connection.

Test files marked as _e2e_test (like destination/main_e2e_test.go) require ClickHouse connection, and will also start the destination GRPC server.

Run the tests:

make test

Running tests with Fivetran SDK tester

Fivetran SDK tests are part of the normal Go test run, see destination/main_e2e_test.go; however, it is possible to execute them in a stand-alone mode, which might be useful for debugging purposes, if you don't want to modify the test code in Go.

Start ClickHouse server in Docker:

docker compose up -d

Start the destination app:

make run

Run the SDK tester with a particular input file from sdk_tests directory, for example, input_all_data_types.json:

make recreate-test-db && TEST_ARGS=--input-file=input_all_data_types.json make sdk-test

Run the SDK tester with all input JSON files (see sdk_tests directory):

make recreate-test-db && make sdk-test

See also: Fivetran SDK tester documentation.

Lint

Runs golangci-lint in Docker to lint the Go code:

make lint

Building a Docker image

make build
docker build . -t clickhouse-fivetran-destination

You should be able to run the destination app:

docker run clickhouse-fivetran-destination

Available flags

List of available flags for the destination app:

make build
./out/clickhouse_destination -h

Check the flags.go file for more details.