Add Datastream Spanner Source Support #16116

alonj-google · 2026-01-12T14:25:14Z

datastream: added `spanner_profile` field to `google_datastream_connection_profile`

datastream: added `backfill_all.spanner_excluded_objects` and  `source_config.spanner_source_config` fields to `google_datastream_stream`

… Spanner source

github-actions · 2026-01-12T14:26:20Z

Hello! I am a robot. Tests will require approval from a repository maintainer to run.

Googlers: For automatic test runs see go/terraform-auto-test-runs.

@slevenick, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

modular-magician · 2026-01-12T14:36:39Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 1737 insertions(+), 69 deletions(-))
google-beta provider: Diff ( 6 files changed, 1849 insertions(+), 77 deletions(-))
terraform-google-conversion: Diff ( 7 files changed, 1961 insertions(+), 123 deletions(-))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_datastream_connection_profile (19 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_datastream_connection_profile" "primary" {
  spanner_profile {
    database = # value needed
    host     = # value needed
  }
}

Resource: google_datastream_stream (10 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_datastream_stream" "primary" {
  backfill_all {
    spanner_excluded_objects {
      schemas {
        schema = # value needed
        tables {
          columns {
            column = # value needed
          }
          table = # value needed
        }
      }
    }
  }
  rule_sets {
    object_filter {
      source_object_identifier {
        spanner_identifier {
          schema = # value needed
          table  = # value needed
        }
      }
    }
  }
  source_config {
    spanner_source_config {
      backfill_data_boost_enabled = # value needed
      change_stream_name          = # value needed
      exclude_objects {
        schemas {
          schema = # value needed
          tables {
            columns {
              column = # value needed
            }
            table = # value needed
          }
        }
      }
      fgac_role = # value needed
      include_objects {
        schemas {
          schema = # value needed
          tables {
            columns {
              column = # value needed
            }
            table = # value needed
          }
        }
      }
      max_concurrent_backfill_tasks = # value needed
      max_concurrent_cdc_tasks      = # value needed
      spanner_rpc_priority          = # value needed
    }
  }
}

modular-magician · 2026-01-12T15:00:27Z

Tests analytics

Total tests: 19
Passed tests: 8
Skipped tests: 10
Affected tests: 1

Click here to see the affected service packages

datastream

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccDatastreamStream_mongoDb

Get to know how VCR tests work

modular-magician · 2026-01-12T15:04:51Z

🟢 Tests passed during RECORDING mode:
TestAccDatastreamStream_mongoDb [Debug log]

🟢 No issues found for passed tests after REPLAYING rerun.

🟢 All tests passed!

View the build log or the debug log for each test

… Spanner source

modular-magician · 2026-01-12T16:04:16Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 1747 insertions(+), 69 deletions(-))
google-beta provider: Diff ( 6 files changed, 1859 insertions(+), 77 deletions(-))
terraform-google-conversion: Diff ( 7 files changed, 1961 insertions(+), 123 deletions(-))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_datastream_connection_profile (19 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_datastream_connection_profile" "primary" {
  spanner_profile {
    database = # value needed
    host     = # value needed
  }
}

Resource: google_datastream_stream (10 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_datastream_stream" "primary" {
  backfill_all {
    spanner_excluded_objects {
      schemas {
        schema = # value needed
        tables {
          columns {
            column = # value needed
          }
          table = # value needed
        }
      }
    }
  }
  rule_sets {
    object_filter {
      source_object_identifier {
        spanner_identifier {
          schema = # value needed
          table  = # value needed
        }
      }
    }
  }
  source_config {
    spanner_source_config {
      backfill_data_boost_enabled = # value needed
      change_stream_name          = # value needed
      exclude_objects {
        schemas {
          schema = # value needed
          tables {
            columns {
              column = # value needed
            }
            table = # value needed
          }
        }
      }
      fgac_role = # value needed
      include_objects {
        schemas {
          schema = # value needed
          tables {
            columns {
              column = # value needed
            }
            table = # value needed
          }
        }
      }
      max_concurrent_backfill_tasks = # value needed
      max_concurrent_cdc_tasks      = # value needed
      spanner_rpc_priority          = # value needed
    }
  }
}

modular-magician · 2026-01-12T16:27:14Z

Tests analytics

Total tests: 19
Passed tests: 9
Skipped tests: 10
Affected tests: 0

Click here to see the affected service packages

datastream

🟢 All tests passed!

View the build log

zli82016 · 2026-01-13T21:33:44Z

/gcbrun

modular-magician · 2026-01-13T21:46:00Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 1747 insertions(+), 69 deletions(-))
google-beta provider: Diff ( 6 files changed, 1859 insertions(+), 77 deletions(-))
terraform-google-conversion: Diff ( 7 files changed, 1961 insertions(+), 123 deletions(-))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_datastream_connection_profile (19 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_datastream_connection_profile" "primary" {
  spanner_profile {
    database = # value needed
    host     = # value needed
  }
}

Resource: google_datastream_stream (10 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_datastream_stream" "primary" {
  backfill_all {
    spanner_excluded_objects {
      schemas {
        schema = # value needed
        tables {
          columns {
            column = # value needed
          }
          table = # value needed
        }
      }
    }
  }
  rule_sets {
    object_filter {
      source_object_identifier {
        spanner_identifier {
          schema = # value needed
          table  = # value needed
        }
      }
    }
  }
  source_config {
    spanner_source_config {
      backfill_data_boost_enabled = # value needed
      change_stream_name          = # value needed
      exclude_objects {
        schemas {
          schema = # value needed
          tables {
            columns {
              column = # value needed
            }
            table = # value needed
          }
        }
      }
      fgac_role = # value needed
      include_objects {
        schemas {
          schema = # value needed
          tables {
            columns {
              column = # value needed
            }
            table = # value needed
          }
        }
      }
      max_concurrent_backfill_tasks = # value needed
      max_concurrent_cdc_tasks      = # value needed
      spanner_rpc_priority          = # value needed
    }
  }
}

modular-magician · 2026-01-13T22:10:07Z

Tests analytics

Total tests: 19
Passed tests: 9
Skipped tests: 10
Affected tests: 0

Click here to see the affected service packages

datastream

🟢 All tests passed!

View the build log

slevenick

Can you write an update test for this field? It should go in this file: https://github.com/GoogleCloudPlatform/magic-modules/blob/main/mmv1/third_party/terraform/services/datastream/resource_datastream_stream_test.go

slevenick · 2026-01-14T17:01:05Z

mmv1/products/datastream/Stream.yaml

+                        description: |
+                          Table name.
+                        required: true
+                      - name: 'columns'


Should this be required?

slevenick · 2026-01-14T17:01:21Z

mmv1/products/datastream/Stream.yaml

+                            Spanner column.
+                          type: NestedObject
+                          properties:
+                            - name: 'column'


This should be required if all other properties are output-only

slevenick · 2026-01-14T17:01:53Z

mmv1/products/datastream/ConnectionProfile.yaml

+    primary_resource_id: 'default'
+    vars:
+      source_connection_profile_id: 'source-profile'
+    exclude_test: true


Why are we excluding this test? Can we include it so we can make sure this all works

slevenick · 2026-01-14T17:02:18Z

mmv1/products/datastream/Stream.yaml

+      stream_id: 'spanner-stream'
+      source_connection_profile_id: 'source-profile'
+      destination_connection_profile_id: 'destination-profile'
+    exclude_test: true


Same, why exclude?

Add Datastream ConnectionProfile and Stream configuration options for…

c259069

… Spanner source

github-actions bot requested a review from slevenick January 12, 2026 14:26

modular-magician added the service/datastream label Jan 12, 2026

Add Datastream ConnectionProfile and Stream configuration options for…

5b4ef3e

… Spanner source

slevenick requested changes Jan 14, 2026

View reviewed changes

Add Datastream Spanner Source Support #16116

Are you sure you want to change the base?

Add Datastream Spanner Source Support #16116

Conversation

alonj-google commented Jan 12, 2026

Uh oh!

github-actions bot commented Jan 12, 2026

Uh oh!

modular-magician commented Jan 12, 2026

Diff report

Missing test report

Uh oh!

modular-magician commented Jan 12, 2026

Tests analytics

Action taken

Uh oh!

modular-magician commented Jan 12, 2026

Uh oh!

modular-magician commented Jan 12, 2026

Diff report

Missing test report

Uh oh!

modular-magician commented Jan 12, 2026

Tests analytics

Uh oh!

zli82016 commented Jan 13, 2026

Uh oh!

modular-magician commented Jan 13, 2026

Diff report

Missing test report

Uh oh!

modular-magician commented Jan 13, 2026

Tests analytics

Uh oh!

slevenick left a comment

Choose a reason for hiding this comment

Uh oh!

slevenick Jan 14, 2026

Choose a reason for hiding this comment

Uh oh!

slevenick Jan 14, 2026

Choose a reason for hiding this comment

Uh oh!

slevenick Jan 14, 2026

Choose a reason for hiding this comment

Uh oh!

slevenick Jan 14, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants