Python API MisconfiguredError: Missing credentials configuration for provider cop_cds

[geograFie]

It looks like an interesting tool to find and access EO data and I am working with Big EO data, so I would love to switch my current setup to EODAG. However, I have not been able to get it running to download data and I am struggling to configure the authentication for the different providers. Specifically, I want to download ERA5 data from the climate data store. Doing so via the CDS api works, so it does not seem to be an issue of my credentials, but perhaps how it is configured via the eodag.yml.

I used the eodag.yml template, but I keep running into the MisconfiguredError: Missing credentials configuration for provider cop_cds error. From the docs it is unclear to me how the eodag.yml config file must look like, as some entries use auth and api, or apikey, and others username and password credentials. This is very confusing.

 auth:
     credentials:
         apikey: <token>

 api:
     credentials:
         username: <email>
         password: <token>

I have tried different combinations, without success. EODAG either returns the above mentioned error or the following error: SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1007).

Searching works well and returns a valid entry, but downloading fails.

In all cases, the errors are returned in the last line of:

from eodag import EODataAccessGateway

dag = EODataAccessGateway()
dag.set_preferred_provider(provider='cop_cds')

search_criteria = {
    'productType': 'reanalysis-era5-single-levels-monthly-means',
    'start': '2022-01-01',
    'end': '2022-12-31',
    'geometry': {
        'type': 'Polygon',
        'coordinates': [[[-180, 90], [180, 90], [180, 0], [-180, 0], [-180, 90]]]
    }
}
products = dag.search_all(**search_criteria)

path = dag.download(products[0], outputs_prefix='.')

% python --version
Python 3.10.15
% eodag version
eodag (Earth Observation Data Access Gateway): version 3.0.1

[sbrunato]

Hello @geograFie , to configure authentication on cop_cds please follow these guidelines
cop_cds registration and configure with yaml user-configuration-file.
Please fill in your ~/.config/eodag/eodag.yml if entries already exist (it might be the case), or add the entries if your yaml file already existed from an older installation:

cop_cds:
    priority: # optional if you want to change provider priority
    download: # optional if you want to change provider download configuration
    auth:
        credentials:
            apikey: your-personal-access-token

Also, once credentials are set, you might have another issue with start and end parameters that do not seem to work when manually using a Copernicus ADS dataset (needs to be fixed on our side).
Until this is fixed, please use pre-configured product types:

guessed = dag.guess_product_type(keywords="era5 AND single AND monthly")
# you'll get ['ERA5_SL_MONTHLY']

# or check product types pre-configured for `cop_cds`
product_types = dag.list_product_types("cop_cds")

products = dag.search(
    provider="cop_cds",
    productType="ERA5_SL_MONTHLY",
    start="2022-01-01", end="2022-12-31"
)

[geograFie]

Thanks for the reply. Unfortunately, I still got an SSLError:

MaxRetryError: HTTPSConnectionPool(host='cds-beta.climate.copernicus.eu', port=443): Max retries exceeded with url: /api/retrieve/v1/processes/reanalysis-era5-single-levels-monthly-means/execution (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1007)')))

After some searching, I found out that the CDS API url is misconfigured inresources/providers.yml, as it was still using the cds beta url, which has been replaced end of October: ECMWF. I can see that this has been updated on GitHub, so I reinstalled EODAG from conda-forge and I have the most recent version 3.0.1 installed. However, the issue persists, as the providers.yml has not been updated and still contains the old invalid url https://cds-beta.climate.copernicus.eu.

Do you have any clue why it is not updating the providers.yml?

[sbrunato]

The URL was updated in #1397 that is only available trough v3.1.0b1 for the moment. (beta versions are on pypi but not conda-forge).

If you can't install v3.1.0b1, you can temporary fix it by overriding the provider configuration and updating URLs your user configuration file with:

cop_cds:
    download:
        order_on_response:
            metadata_mapping:
                orderStatusLink: https://cds.climate.copernicus.eu/api/retrieve/v1/jobs/{orderId}
                searchLink: https://cds.climate.copernicus.eu/api/retrieve/v1/jobs/{orderId}/results
    search:
        constraints_file_url: https://cds.climate.copernicus.eu/api/catalogue/v1/collections/{dataset}/constraints.json
        metadata_mapping:
            orderLink: 'https://cds.climate.copernicus.eu/api/retrieve/v1/processes/{dataset}/execution?{{"inputs": {qs#to_geojson}}}'
    auth:
        matching_url: https://cds.climate.copernicus.eu
        credentials:
            apikey: your-personal-access-token