✨Source Strava: Migrate Python CDK to Low-code CDK (#31007)

Co-authored-by: Marcos Marx <[email protected]>
Co-authored-by: marcosmarxm <[email protected]>

airbyte-integrations/connectors/source-strava/.dockerignore

@@ -1,6 +1,5 @@
 *
 !Dockerfile
-!Dockerfile.test
 !main.py
 !source_strava
 !setup.py

airbyte-integrations/connectors/source-strava/Dockerfile

@@ -34,5 +34,5 @@ COPY source_strava ./source_strava
 ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
 ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]
 
-LABEL io.airbyte.version=0.1.4
+LABEL io.airbyte.version=0.2.0
 LABEL io.airbyte.name=airbyte/source-strava

airbyte-integrations/connectors/source-strava/README.md

@@ -1,35 +1,10 @@
 # Strava Source
 
-This is the repository for the Strava source connector, written in Python.
-For information about how to use this connector within Airbyte, see [the documentation](https://docs.airbyte.io/integrations/sources/strava).
+This is the repository for the Strava configuration based source connector.
+For information about how to use this connector within Airbyte, see [the documentation](https://docs.airbyte.com/integrations/sources/strava).
 
 ## Local development
 
-### Prerequisites
-**To iterate on this connector, make sure to complete this prerequisites section.**
-
-#### Minimum Python version required `= 3.7.0`
-
-#### Build & Activate Virtual Environment and install dependencies
-From this connector directory, create a virtual environment:
-```
-python -m venv .venv
-```
-
-This will generate a virtualenv for this module in `.venv/`. Make sure this venv is active in your
-development environment of choice. To activate it from the terminal, run:
-```
-source .venv/bin/activate
-pip install -r requirements.txt
-pip install '.[tests]'
-```
-If you are in an IDE, follow your IDE's instructions to activate the virtualenv.
-
-Note that while we are installing dependencies from `requirements.txt`, you should only edit `setup.py` for your dependencies. `requirements.txt` is
-used for editable installs (`pip install -e`) to pull in Python dependencies from the monorepo and will call `setup.py`.
-If this is mumbo jumbo to you, don't worry about it, just put your deps in `setup.py` but install using `pip install -r requirements.txt` and everything
-should work as you expect.
-
 #### Building via Gradle
 You can also build the connector in Gradle. This is typically used in CI and not needed for your development workflow.
 
@@ -39,22 +14,14 @@ To build using Gradle, from the Airbyte repository root, run:
 ```
 
 #### Create credentials
-**If you are a community contributor**, follow the instructions in the [documentation](https://docs.airbyte.io/integrations/sources/strava)
-to generate the necessary credentials. Then create a file `secrets/config.json` conforming to the `source_strava/spec.json` file.
+**If you are a community contributor**, follow the instructions in the [documentation](https://docs.airbyte.com/integrations/sources/strava)
+to generate the necessary credentials. Then create a file `secrets/config.json` conforming to the `source_strava/spec.yaml` file.
 Note that any directory named `secrets` is gitignored across the entire Airbyte repo, so there is no danger of accidentally checking in sensitive information.
-See `integration_tests/invalid_config.json` for a sample config file with fake tokens.
+See `integration_tests/sample_config.json` for a sample config file.
 
 **If you are an Airbyte core member**, copy the credentials in Lastpass under the secret name `source strava test creds`
 and place them into `secrets/config.json`.
 
-### Locally running the connector
-```
-python main.py spec
-python main.py check --config secrets/config.json
-python main.py discover --config secrets/config.json
-python main.py read --config secrets/config.json --catalog integration_tests/configured_catalog.json
-```
-
 ### Locally running the connector docker image
 
 #### Build
@@ -79,32 +46,15 @@ docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-strava:dev discover --
 docker run --rm -v $(pwd)/secrets:/secrets -v $(pwd)/integration_tests:/integration_tests airbyte/source-strava:dev read --config /secrets/config.json --catalog /integration_tests/configured_catalog.json
 ```
 ## Testing
-Make sure to familiarize yourself with [pytest test discovery](https://docs.pytest.org/en/latest/goodpractices.html#test-discovery) to know how your test files and methods should be named.
-First install test dependencies into your virtual environment:
-```
-pip install .[tests]
-```
-### Unit Tests
-To run unit tests locally, from the connector directory run:
-```
-python -m pytest unit_tests
-```
 
-### Integration Tests
-There are two types of integration tests: Acceptance Tests (Airbyte's test suite for all source connectors) and custom integration tests (which are specific to this connector).
-#### Custom Integration tests
-Place custom tests inside `integration_tests/` folder, then, from the connector root, run
-```
-python -m pytest integration_tests
-```
 #### Acceptance Tests
-Customize `acceptance-test-config.yml` file to configure tests. See [Connector Acceptance Tests](https://docs.airbyte.io/connector-development/testing-connectors/connector-acceptance-tests-reference) for more information.
+Customize `acceptance-test-config.yml` file to configure tests. See [Connector Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference) for more information.
 If your connector requires to create or destroy resources for use during acceptance tests create fixtures for it and place them inside integration_tests/acceptance.py.
-To run your integration tests with acceptance tests, from the connector root, run
+
+To run your integration tests with Docker, run:
 ```
-python -m pytest integration_tests -p integration_tests.acceptance
+./acceptance-test-docker.sh
 ```
-To run your integration tests with docker
 
 ### Using gradle to run tests
 All commands should be run from airbyte project root.

airbyte-integrations/connectors/source-strava/__init__.py

@@ -0,0 +1,3 @@
+#
+# Copyright (c) 2023 Airbyte, Inc., all rights reserved.
+#

airbyte-integrations/connectors/source-strava/acceptance-test-docker.sh

@@ -1,2 +1,3 @@
 #!/usr/bin/env sh
+
 source "$(git rev-parse --show-toplevel)/airbyte-integrations/bases/connector-acceptance-test/acceptance-test-docker.sh"

airbyte-integrations/connectors/source-strava/integration_tests/__init__.py

@@ -1,3 +1,3 @@
 #
-# Copyright (c) 2021 Airbyte, Inc., all rights reserved.
+# Copyright (c) 2023 Airbyte, Inc., all rights reserved.
 #

airbyte-integrations/connectors/source-strava/integration_tests/acceptance.py

@@ -11,4 +11,6 @@
 @pytest.fixture(scope="session", autouse=True)
 def connector_setup():
     """This fixture is a placeholder for external resources that acceptance test might require."""
+    # TODO: setup test dependencies if needed. otherwise remove the TODO comments
     yield
+    # TODO: clean up test dependencies

airbyte-integrations/connectors/source-strava/integration_tests/expected_records.jsonl

@@ -1,4 +1,3 @@
-{"stream": "athlete_stats", "data": {"biggest_ride_distance": 37825.1, "biggest_climb_elevation_gain": 102.30000000000001, "recent_ride_totals": {"count": 0, "distance": 0.0, "moving_time": 0, "elapsed_time": 0, "elevation_gain": 0.0, "achievement_count": 0}, "all_ride_totals": {"count": 46, "distance": 794446, "moving_time": 178718, "elapsed_time": 267074, "elevation_gain": 8366}, "recent_run_totals": {"count": 0, "distance": 0.0, "moving_time": 0, "elapsed_time": 0, "elevation_gain": 0.0, "achievement_count": 0}, "all_run_totals": {"count": 124, "distance": 841523, "moving_time": 302192, "elapsed_time": 342187, "elevation_gain": 9425}, "recent_swim_totals": {"count": 0, "distance": 0.0, "moving_time": 0, "elapsed_time": 0, "elevation_gain": 0.0, "achievement_count": 0}, "all_swim_totals": {"count": 0, "distance": 0, "moving_time": 0, "elapsed_time": 0, "elevation_gain": 0}, "ytd_ride_totals": {"count": 0, "distance": 0, "moving_time": 0, "elapsed_time": 0, "elevation_gain": 0}, "ytd_run_totals": {"count": 6, "distance": 29359, "moving_time": 13577, "elapsed_time": 14526, "elevation_gain": 379}, "ytd_swim_totals": {"count": 0, "distance": 0, "moving_time": 0, "elapsed_time": 0, "elevation_gain": 0}}, "emitted_at": 1681112894112}
-{"stream": "activities", "data": {"resource_state": 2, "athlete": {"id": 17831421, "resource_state": 1}, "name": "Afternoon Run", "distance": 5180.3, "moving_time": 1558, "elapsed_time": 1570, "total_elevation_gain": 41.7, "type": "Run", "sport_type": "Run", "workout_type": 0, "id": 732659975, "start_date": "2016-10-03T00:23:29Z", "start_date_local": "2016-10-02T17:23:29Z", "timezone": "(GMT-08:00) America/Los_Angeles", "utc_offset": -25200.0, "location_city": null, "location_state": null, "location_country": null, "achievement_count": 3, "kudos_count": 2, "comment_count": 1, "athlete_count": 1, "photo_count": 0, "map": {"id": "a732659975", "summary_polyline": "ijdfFnfhiVu@uCmaAnEwAlByAkDiLnBvI~EjDyB`BhBjw@oEnBvRpOA|ApWqAfJvAeH}AiZiAyKeCQaAgG", "resource_state": 2}, "trainer": false, "commute": false, "manual": false, "private": false, "visibility": "everyone", "flagged": false, "gear_id": null, "start_latlng": [37.874456, -122.271915], "end_latlng": [37.874412, -122.272033], "average_speed": 3.325, "max_speed": 5.3, "has_heartrate": false, "heartrate_opt_out": false, "display_hide_heartrate_option": false, "elev_high": 81.9, "elev_low": 37.5, "upload_id": 807763220, "upload_id_str": "807763220", "external_id": "51fbc33d23a743089b1e732118e37bc5", "from_accepted_tag": false, "pr_count": 1, "total_photo_count": 0, "has_kudoed": false}, "emitted_at": 1684220833770}
-{"stream": "activities", "data": {"resource_state": 2, "athlete": {"id": 17831421, "resource_state": 1}, "name": "New shorts, who dis", "distance": 8374.9, "moving_time": 2605, "elapsed_time": 2659, "total_elevation_gain": 75.0, "type": "Run", "sport_type": "Run", "workout_type": 0, "id": 735803811, "start_date": "2016-10-06T01:24:28Z", "start_date_local": "2016-10-05T18:24:28Z", "timezone": "(GMT-08:00) America/Los_Angeles", "utc_offset": -25200.0, "location_city": null, "location_state": null, "location_country": null, "achievement_count": 3, "kudos_count": 1, "comment_count": 1, "athlete_count": 1, "photo_count": 0, "map": {"id": "a735803811", "summary_polyline": "kjdfFlfhiVc@eD}aAvEsAxByAoDaLlBdIhF`IiGpi@sDeAeZxY_DAuKbIqB@{F~B_F~HaI|Hw@fAnKoJ`AeApCJ`S}SbBXjVbGr{@uA`JtA}G_E_o@wD_B", "resource_state": 2}, "trainer": false, "commute": false, "manual": false, "private": false, "visibility": "everyone", "flagged": false, "gear_id": null, "start_latlng": [37.874465, -122.271906], "end_latlng": [37.874449, -122.271743], "average_speed": 3.215, "max_speed": 5.4, "has_heartrate": false, "heartrate_opt_out": false, "display_hide_heartrate_option": false, "elev_high": 84.3, "elev_low": 37.5, "upload_id": 811413504, "upload_id_str": "811413504", "external_id": "f77a35ec330412183402e6fac2b5be20", "from_accepted_tag": false, "pr_count": 0, "total_photo_count": 0, "has_kudoed": false}, "emitted_at": 1684220833771}
-{"stream": "activities", "data": {"resource_state": 2, "athlete": {"id": 17831421, "resource_state": 1}, "name": "Afternoon Run", "distance": 5457.4, "moving_time": 1598, "elapsed_time": 1659, "total_elevation_gain": 12.8, "type": "Run", "sport_type": "Run", "workout_type": 0, "id": 744774691, "start_date": "2016-10-15T00:51:50Z", "start_date_local": "2016-10-14T17:51:50Z", "timezone": "(GMT-08:00) America/Los_Angeles", "utc_offset": -25200.0, "location_city": null, "location_state": null, "location_country": null, "achievement_count": 5, "kudos_count": 0, "comment_count": 0, "athlete_count": 1, "photo_count": 0, "map": {"id": "a744774691", "summary_polyline": "{z_oEngunUx@pJfeCV}}@mBudAPy@q@GwFiG_@tDV", "resource_state": 2}, "trainer": false, "commute": false, "manual": false, "private": false, "visibility": "everyone", "flagged": false, "gear_id": null, "start_latlng": [34.083181, -117.914954], "end_latlng": [34.083389, -117.914936], "average_speed": 3.415, "max_speed": 4.7, "has_heartrate": false, "heartrate_opt_out": false, "display_hide_heartrate_option": false, "elev_high": 140.8, "elev_low": 127.8, "upload_id": 821881164, "upload_id_str": "821881164", "external_id": "4f6161a407a088bc4db49f5959e9673a", "from_accepted_tag": false, "pr_count": 2, "total_photo_count": 0, "has_kudoed": false}, "emitted_at": 1684220833771}
+{"stream": "athlete_stats", "data": {"biggest_ride_distance": null, "biggest_climb_elevation_gain": null, "recent_ride_totals": {"count": 1, "distance": 0.0, "moving_time": 2300, "elapsed_time": 2300, "elevation_gain": 0.0, "achievement_count": 0}, "all_ride_totals": {"count": 1, "distance": 0, "moving_time": 2300, "elapsed_time": 2300, "elevation_gain": 0}, "recent_run_totals": {"count": 0, "distance": 0.0, "moving_time": 0, "elapsed_time": 0, "elevation_gain": 0.0, "achievement_count": 0}, "all_run_totals": {"count": 1, "distance": 0, "moving_time": 3600, "elapsed_time": 3600, "elevation_gain": 0}, "recent_swim_totals": {"count": 0, "distance": 0.0, "moving_time": 0, "elapsed_time": 0, "elevation_gain": 0.0, "achievement_count": 0}, "all_swim_totals": {"count": 0, "distance": 0, "moving_time": 0, "elapsed_time": 0, "elevation_gain": 0}, "ytd_ride_totals": {"count": 1, "distance": 0, "moving_time": 2300, "elapsed_time": 2300, "elevation_gain": 0}, "ytd_run_totals": {"count": 1, "distance": 0, "moving_time": 3600, "elapsed_time": 3600, "elevation_gain": 0}, "ytd_swim_totals": {"count": 0, "distance": 0, "moving_time": 0, "elapsed_time": 0, "elevation_gain": 0}}, "emitted_at": 1698179192900}
+{"stream": "activities", "data": {"resource_state": 2, "athlete": {"id": 95370757, "resource_state": 1}, "name": "My example activity", "distance": 0.0, "moving_time": 3600, "elapsed_time": 3600, "total_elevation_gain": 0, "type": "Run", "sport_type": "Run", "workout_type": null, "id": 10097971825, "start_date": "2023-01-01T06:00:00Z", "start_date_local": "2023-01-01T00:00:00Z", "timezone": "(GMT-06:00) America/Chicago", "utc_offset": -21600.0, "location_city": null, "location_state": null, "location_country": "United States", "achievement_count": 0, "kudos_count": 0, "comment_count": 0, "athlete_count": 1, "photo_count": 0, "map": {"id": "a10097971825", "summary_polyline": "", "resource_state": 2}, "trainer": false, "commute": false, "manual": true, "private": false, "visibility": "everyone", "flagged": false, "gear_id": null, "start_latlng": [], "end_latlng": [], "average_speed": 0.0, "max_speed": 0, "has_heartrate": false, "heartrate_opt_out": false, "display_hide_heartrate_option": false, "upload_id": null, "external_id": null, "from_accepted_tag": false, "pr_count": 0, "total_photo_count": 0, "has_kudoed": false}, "emitted_at": 1698179195177}
+{"stream": "activities", "data": {"resource_state": 2, "athlete": {"id": 95370757, "resource_state": 1}, "name": "Ride example", "distance": 0.0, "moving_time": 2300, "elapsed_time": 2300, "total_elevation_gain": 0, "type": "Ride", "sport_type": "Ride", "workout_type": null, "id": 10097974186, "start_date": "2023-10-01T05:00:00Z", "start_date_local": "2023-10-01T00:00:00Z", "timezone": "(GMT-06:00) America/Chicago", "utc_offset": -18000.0, "location_city": null, "location_state": null, "location_country": "United States", "achievement_count": 0, "kudos_count": 0, "comment_count": 0, "athlete_count": 1, "photo_count": 0, "map": {"id": "a10097974186", "summary_polyline": "", "resource_state": 2}, "trainer": false, "commute": false, "manual": true, "private": false, "visibility": "everyone", "flagged": false, "gear_id": null, "start_latlng": [], "end_latlng": [], "average_speed": 0.0, "max_speed": 0, "has_heartrate": false, "heartrate_opt_out": false, "display_hide_heartrate_option": false, "upload_id": null, "external_id": null, "from_accepted_tag": false, "pr_count": 0, "total_photo_count": 0, "has_kudoed": false}, "emitted_at": 1698179195185}

airbyte-integrations/connectors/source-strava/metadata.yaml

@@ -2,26 +2,27 @@ data:
   allowedHosts:
     hosts:
       - strava.com
+  registries:
+    oss:
+      enabled: true
+    cloud:
+      enabled: true
   connectorSubtype: api
   connectorType: source
   definitionId: 7a4327c4-315a-11ec-8d3d-0242ac130003
-  dockerImageTag: 0.1.4
+  dockerImageTag: 0.2.0
   dockerRepository: airbyte/source-strava
   githubIssueLabel: source-strava
   icon: strava.svg
   license: MIT
   name: Strava
-  registries:
-    cloud:
-      enabled: true
-    oss:
-      enabled: true
+  releaseDate: 2021-10-18
   releaseStage: beta
+  supportLevel: community
   documentationUrl: https://docs.airbyte.com/integrations/sources/strava
   tags:
-    - language:python
+    - language:low-code
   ab_internal:
     sl: 100
     ql: 300
-  supportLevel: community
 metadataSpecVersion: "1.0"

airbyte-integrations/connectors/source-strava/setup.py

@@ -5,13 +5,11 @@
 
 from setuptools import find_packages, setup
 
-MAIN_REQUIREMENTS = [
-    "airbyte-cdk",
-]
+MAIN_REQUIREMENTS = ["airbyte-cdk"]
 
 TEST_REQUIREMENTS = [
     "requests-mock~=1.9.3",
-    "pytest~=6.1",
+    "pytest~=6.2",
     "pytest-mock~=3.6.1",
 ]
 
@@ -22,7 +20,7 @@
     author_email="[email protected]",
     packages=find_packages(),
     install_requires=MAIN_REQUIREMENTS,
-    package_data={"": ["*.json", "schemas/*.json", "schemas/shared/*.json"]},
+    package_data={"": ["*.json", "*.yaml", "schemas/*.json", "schemas/shared/*.json"]},
     extras_require={
         "tests": TEST_REQUIREMENTS,
     },

airbyte-integrations/connectors/source-strava/source_strava/__init__.py

@@ -1,5 +1,5 @@
 #
-# Copyright (c) 2021 Airbyte, Inc., all rights reserved.
+# Copyright (c) 2023 Airbyte, Inc., all rights reserved.
 #