Add transformations support

CHANGELOG.md

@@ -9,10 +9,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [1.5.0](https://github.com/fivetran/terraform-provider-fivetran/compare/v1.4.2...v1.5.0)
 
+## Added
+Support for a new [Transformation Management API](https://fivetran.com/docs/rest-api/api-reference/transformation-management)
+- New resource `fivetran_transformation_project` instead of deprecated `fivetran_dbt_project`
+- New resource `fivetran_transformation` instead of deprecated `fivetran_dbt_transformation`
+- New data source `fivetran_transformation_project` instead of deprecated `fivetran_dbt_project`
+- New data source `fivetran_transformation_projects` instead of deprecated `fivetran_dbt_projects`
+- New data source `fivetran_transformation` instead of deprecated `fivetran_dbt_transformation`
+
 - New data source `fivetran_connectors` that allows to retrieve the list of existing Connections available for the current account.
 - New data source `fivetran_destinations` that allows to retrieve the list of existing Destinations available for the current account.
 - New data source `fivetran_external_logs` that allows to retrieve the list of existing External Logging Services available for the current account.
 
+## Deprecated
+- Datasources `fivetran_dbt_project`, `fivetran_dbt_projects`, `fivetran_dbt_transformation`, `fivetran_dbt_models`
+- Resources `fivetran_dbt_project`, `fivetran_dbt_transformation`
+
 ## [1.4.2](https://github.com/fivetran/terraform-provider-fivetran/compare/v1.4.1...v1.4.2)
 
 ## Added

docs/data-sources/quickstart_package.md

@@ -0,0 +1,29 @@
+---
+page_title: "Data Source: fivetran_quickstart_package"
+---
+
+# Data Source: fivetran_quickstart_package
+
+This data source returns the metadata details of the Quickstart transformation package if a valid identifier is provided
+
+## Example Usage
+
+```hcl
+data "fivetran_quickstart_package" "test" {
+    id = "id"
+}
+```
+
+<!-- schema generated by tfplugindocs -->
+## Schema
+
+### Required
+
+- `id` (String) The unique identifier for the Quickstart transformation package definition within the Fivetran system
+
+### Read-Only
+
+- `connector_types` (Set of String) The set of connector types
+- `name` (String) The Quickstart transformation package name
+- `output_model_names` (Set of String) The list of transformation output models
+- `version` (String) The Quickstart package definition version

docs/data-sources/quickstart_packages.md

@@ -0,0 +1,32 @@
+---
+page_title: "Data Source: fivetran_quickstart_packages"
+---
+
+# Data Source: fivetran_quickstart_packages
+
+Returns a list of available Quickstart transformation package metadata details
+
+## Example Usage
+
+```hcl
+data "fivetran_quickstart_packages" "all_packages_metadata" {
+}
+```
+
+<!-- schema generated by tfplugindocs -->
+## Schema
+
+### Read-Only
+
+- `packages` (Block List) (see [below for nested schema](#nestedblock--packages))
+
+<a id="nestedblock--packages"></a>
+### Nested Schema for `packages`
+
+Read-Only:
+
+- `connector_types` (Set of String) The set of connector types
+- `id` (String) The unique identifier for the Quickstart transformation package definition within the Fivetran system
+- `name` (String) The Quickstart transformation package name
+- `output_model_names` (Set of String) The list of transformation output models
+- `version` (String) The Quickstart package definition version

docs/data-sources/transformation.md

@@ -0,0 +1,68 @@
+---
+page_title: "Data Source: fivetran_transformation"
+---
+
+# Data Source: fivetran_transformation
+
+Returns transformation details if a valid identifier was provided
+
+## Example Usage
+
+```hcl
+data "fivetran_transformation" "test" {
+	id = "id"
+}
+```
+
+<!-- schema generated by tfplugindocs -->
+## Schema
+
+### Required
+
+- `id` (String) The unique identifier for the Transformation within the Fivetran system.
+
+### Read-Only
+
+- `created_at` (String) The timestamp of when the transformation was created in your account.
+- `created_by_id` (String) The unique identifier for the User within the Fivetran system who created the transformation.
+- `output_model_names` (Set of String) Identifiers of related models.
+- `paused` (Boolean) The field indicating whether the transformation will be set into the paused state. By default, the value is false.
+- `schedule` (Block, Read-only) (see [below for nested schema](#nestedblock--schedule))
+- `status` (String) Status of transformation Project (NOT_READY, READY, ERROR).
+- `transformation_config` (Block, Read-only) (see [below for nested schema](#nestedblock--transformation_config))
+- `type` (String) Transformation type. The following values are supported: DBT_CORE, QUICKSTART.
+
+<a id="nestedblock--schedule"></a>
+### Nested Schema for `schedule`
+
+Read-Only:
+
+- `connection_ids` (Set of String) The list of the connection identifiers to be used for the integrated schedule. Not expected for QUICKSTART transformations
+- `cron` (Set of String) Cron schedule: list of CRON strings. Used for for CRON schedule type
+- `days_of_week` (Set of String) The set of the days of the week the transformation should be launched on. The following values are supported: MONDAY, TUESDAY, WEDNESDAY, THURSDAY, FRIDAY, SATURDAY, SUNDAY. Used for for INTEGRATED schedule type
+- `interval` (Number) The time interval in minutes between subsequent transformation runs. Used for for INTERVAL schedule type
+- `schedule_type` (String) The type of the schedule to run the Transformation on. The following values are supported: INTEGRATED, TIME_OF_DAY, INTERVAL, CRON.
+- `smart_syncing` (Boolean) The boolean flag that enables the Smart Syncing schedule
+- `time_of_day` (String) The time of the day the transformation should be launched at. Supported values are: "00:00", "01:00", "02:00", "03:00", "04:00", "05:00", "06:00", "07:00", "08:00", "09:00", "10:00", "11:00", "12:00", "13:00", "14:00", "15:00", "16:00", "17:00", "18:00", "19:00", "20:00", "21:00", "22:00", "23:00". Used for for TIME_OF_DAY schedule type
+
+
+<a id="nestedblock--transformation_config"></a>
+### Nested Schema for `transformation_config`
+
+Read-Only:
+
+- `connection_ids` (Set of String) The list of the connection identifiers to be used for the integrated schedule. Also used to identify package_name automatically if package_name was not specified
+- `excluded_models` (Set of String) The list of excluded output model names
+- `name` (String) The transformation name
+- `package_name` (String) The Quickstart transformation package name
+- `project_id` (String) The unique identifier for the dbt Core project within the Fivetran system
+- `steps` (Attributes List) (see [below for nested schema](#nestedatt--transformation_config--steps))
+- `upgrade_available` (Boolean) The boolean flag indicating that a newer version is available for the transformation package
+
+<a id="nestedatt--transformation_config--steps"></a>
+### Nested Schema for `transformation_config.steps`
+
+Read-Only:
+
+- `command` (String) The dbt command in the transformation step
+- `name` (String) The step name

docs/data-sources/transformation_project.md

@@ -0,0 +1,47 @@
+---
+page_title: "Data Source: fivetran_transformation_project"
+---
+
+# Data Source: fivetran_transformation_project
+
+Returns transformation project details if a valid identifier was provided
+
+## Example Usage
+
+```hcl
+data "fivetran_transformation_project" "test" {
+	id = "id"
+}
+```
+
+<!-- schema generated by tfplugindocs -->
+## Schema
+
+### Required
+
+- `id` (String) The unique identifier for the transformation Project within the Fivetran system.
+
+### Read-Only
+
+- `created_at` (String) The timestamp of the transformation Project creation.
+- `created_by_id` (String) The unique identifier for the User within the Fivetran system who created the dbt Project.
+- `errors` (Set of String) List of environment variables defined as key-value pairs in the raw string format using = as a separator. The variable name should have the DBT_ prefix and can contain A-Z, 0-9, dash, underscore, or dot characters. Example: "DBT_VARIABLE=variable_value"
+- `group_id` (String) The unique identifier for the group within the Fivetran system.
+- `project_config` (Block, Read-only) (see [below for nested schema](#nestedblock--project_config))
+- `status` (String) Status of transformation Project (NOT_READY, READY, ERROR).
+- `type` (String) Transformation project type.
+
+<a id="nestedblock--project_config"></a>
+### Nested Schema for `project_config`
+
+Read-Only:
+
+- `dbt_version` (String) The version of transformation that should run the project
+- `default_schema` (String) Default schema in destination. This production schema will contain your transformed data.
+- `environment_vars` (Set of String) List of environment variables defined as key-value pairs in the raw string format using = as a separator. The variable name should have the DBT_ prefix and can contain A-Z, 0-9, dash, underscore, or dot characters. Example: "DBT_VARIABLE=variable_value"
+- `folder_path` (String) Folder in Git repo with your transformation project
+- `git_branch` (String) Git branch
+- `git_remote_url` (String) Git remote URL with your transformation project
+- `public_key` (String) Public key to grant Fivetran SSH access to git repository.
+- `target_name` (String) Target name to set or override the value from the deployment.yaml
+- `threads` (Number) The number of threads transformation will use (from 1 to 32). Make sure this value is compatible with your destination type. For example, Snowflake supports only 8 concurrent queries on an X-Small warehouse.

docs/data-sources/transformation_projects.md

@@ -0,0 +1,32 @@
+---
+page_title: "Data Source: fivetran_transformation_projects"
+---
+
+# Data Source: fivetran_transformation_projects
+
+Returns a list of all transformation projects available via API within your Fivetran account.
+
+## Example Usage
+
+```hcl
+data "fivetran_transformation_projects" "test" {
+}
+```
+
+<!-- schema generated by tfplugindocs -->
+## Schema
+
+### Read-Only
+
+- `projects` (Attributes List) (see [below for nested schema](#nestedatt--projects))
+
+<a id="nestedatt--projects"></a>
+### Nested Schema for `projects`
+
+Read-Only:
+
+- `created_at` (String) The timestamp of when the project was created in your account.
+- `created_by_id` (String) The unique identifier for the User within the Fivetran system who created the transformation Project.
+- `group_id` (String) The name of the group within your account related to the project.
+- `id` (String) The unique identifier for the transformation project within the Fivetran system.
+- `type` (String) Transformation project type.

docs/data-sources/transformations.md

@@ -0,0 +1,74 @@
+---
+page_title: "Data Source: fivetran_transformations"
+---
+
+# Data Source: fivetran_transformations
+
+Returns a list of all transformations available via API within your Fivetran account.
+
+## Example Usage
+
+```hcl
+data "fivetran_transformations" "test" {
+}
+```
+
+<!-- schema generated by tfplugindocs -->
+## Schema
+
+### Optional
+
+- `transformations` (Block List) (see [below for nested schema](#nestedblock--transformations))
+
+<a id="nestedblock--transformations"></a>
+### Nested Schema for `transformations`
+
+Required:
+
+- `id` (String) The unique identifier for the Transformation within the Fivetran system.
+
+Read-Only:
+
+- `created_at` (String) The timestamp of when the transformation was created in your account.
+- `created_by_id` (String) The unique identifier for the User within the Fivetran system who created the transformation.
+- `output_model_names` (Set of String) Identifiers of related models.
+- `paused` (Boolean) The field indicating whether the transformation will be set into the paused state. By default, the value is false.
+- `schedule` (Block, Read-only) (see [below for nested schema](#nestedblock--transformations--schedule))
+- `status` (String) Status of transformation Project (NOT_READY, READY, ERROR).
+- `transformation_config` (Block, Read-only) (see [below for nested schema](#nestedblock--transformations--transformation_config))
+- `type` (String) Transformation type. The following values are supported: DBT_CORE, QUICKSTART.
+
+<a id="nestedblock--transformations--schedule"></a>
+### Nested Schema for `transformations.schedule`
+
+Read-Only:
+
+- `connection_ids` (Set of String) The list of the connection identifiers to be used for the integrated schedule. Not expected for QUICKSTART transformations
+- `cron` (Set of String) Cron schedule: list of CRON strings. Used for for CRON schedule type
+- `days_of_week` (Set of String) The set of the days of the week the transformation should be launched on. The following values are supported: MONDAY, TUESDAY, WEDNESDAY, THURSDAY, FRIDAY, SATURDAY, SUNDAY. Used for for INTEGRATED schedule type
+- `interval` (Number) The time interval in minutes between subsequent transformation runs. Used for for INTERVAL schedule type
+- `schedule_type` (String) The type of the schedule to run the Transformation on. The following values are supported: INTEGRATED, TIME_OF_DAY, INTERVAL, CRON.
+- `smart_syncing` (Boolean) The boolean flag that enables the Smart Syncing schedule
+- `time_of_day` (String) The time of the day the transformation should be launched at. Supported values are: "00:00", "01:00", "02:00", "03:00", "04:00", "05:00", "06:00", "07:00", "08:00", "09:00", "10:00", "11:00", "12:00", "13:00", "14:00", "15:00", "16:00", "17:00", "18:00", "19:00", "20:00", "21:00", "22:00", "23:00". Used for for TIME_OF_DAY schedule type
+
+
+<a id="nestedblock--transformations--transformation_config"></a>
+### Nested Schema for `transformations.transformation_config`
+
+Read-Only:
+
+- `connection_ids` (Set of String) The list of the connection identifiers to be used for the integrated schedule. Also used to identify package_name automatically if package_name was not specified
+- `excluded_models` (Set of String) The list of excluded output model names
+- `name` (String) The transformation name
+- `package_name` (String) The Quickstart transformation package name
+- `project_id` (String) The unique identifier for the dbt Core project within the Fivetran system
+- `steps` (Attributes List) (see [below for nested schema](#nestedatt--transformations--transformation_config--steps))
+- `upgrade_available` (Boolean) The boolean flag indicating that a newer version is available for the transformation package
+
+<a id="nestedatt--transformations--transformation_config--steps"></a>
+### Nested Schema for `transformations.transformation_config.steps`
+
+Read-Only:
+
+- `command` (String) The dbt command in the transformation step
+- `name` (String) The step name

docs/guides/transformation_private_git_deploy_key.md

@@ -0,0 +1,60 @@
+----
+page_title: "Transformation Project Setup With Git Private Repo"
+subcategory: "Getting Started"
+---
+
+# How to set up a Transformation Project with private Git Repo.
+
+To be able to use private Transformation Project Git repository you have to grant Fivetran access to this repo.
+To do that you need to add a Deploy Key to your repository. 
+To get SSH key from Fivetran create `fivetran_transformation_project` resource:
+
+```hcl
+resource "fivetran_group" "my_group" {
+    name = "My_Group"
+}
+
+resource "fivetran_transformation_project" "project" {
+    provider = fivetran-provider
+    group_id = "group_id"
+    type = "DBT_GIT"
+    run_tests = true
+
+    project_config {
+        git_remote_url = "git_remote_url"
+        git_branch = "git_branch"
+        folder_path = "folder_path"
+        dbt_version = "dbt_version"
+        default_schema = "default_schema"
+        threads = 0
+        target_name = "target_name"
+        environment_vars = ["environment_var"]
+    }
+}
+```
+
+Then you need to set up the Transformation Project public key (field `public_key` in created resource) as a deploy key into your repo using:
+
+[GitHub Provider Repository Deploy Key Resource](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/repository_deploy_key):
+```hcl
+resource "github_repository_deploy_key" "example_repository_deploy_key" {
+  title      = "Repository test key"
+  repository = "repo-owner/repo-name"
+  key        = fivetran_transformation_project.test_project.project_config.public_key
+  read_only  = true
+}
+```
+
+or
+
+[Bitbucket Provider Repository Deploy Key Resource]https://registry.terraform.io/providers/DrFaust92/bitbucket/latest/docs/resources/deploy_key)
+```hcl
+resource "bitbucket_deploy_key" "test" {
+  workspace  = "repo-owner"
+  repository = "repo-name"  
+  key        = fivetran_transformation_project.test_project.project_config.public_key
+  label      = "Repository test key"
+}
+```
+
+Since we recommend using third-party providers in this case, please make sure that access to the repositories is provided correctly and the providers are configured correctly for connection.