Glue: Allow for assuming role for Glue #1299
Conversation
| ) | ||
| refreshable_credentials = DeferredRefreshableCredentials( | ||
| method="assume-role", | ||
| refresh_using=fetcher.fetch_credentials, |
There was a problem hiding this comment.
I'm relying on calling an in-house webservice that will provide new credentials. As I see it, this implementation will not allow a custom implementation of refresh_using that would enable me to call the webservice.
| ) | ||
| from botocore.session import Session as BotoSession | ||
|
|
||
| botocore_session = BotoSession() |
There was a problem hiding this comment.
How about accepting a boto3.Session as input? Then the client can configure it as needed.
There was a problem hiding this comment.
First of all, thanks @cshenrik for the quick reply!
I think we could do both, so you can also configure the Glue client similar to the S3 one (it uses similar properties).
@HonahX @kevinjqliu How strong are we on only passing str's? Otherwise, I think we might start jumping through hoops to get it working (eg. do something similar as when loading a custom FileIO).
There was a problem hiding this comment.
Im leaning towards loading custom clients instead of passing everything through properties. Every time we add another property, we need to map a name, a possible default, and pass to the underlying object.
The logic above is already pretty complicated, might as well split it out and call it something like RefreshableGlueClient
| credentials = Credentials( | ||
| access_key=get_first_property_value(properties, GLUE_ACCESS_KEY_ID, AWS_ACCESS_KEY_ID), | ||
| secret_key=get_first_property_value(properties, GLUE_SECRET_ACCESS_KEY, AWS_SECRET_ACCESS_KEY), | ||
| token=get_first_property_value(properties, GLUE_SESSION_TOKEN, AWS_SESSION_TOKEN), |
There was a problem hiding this comment.
took me a long time to find this, but it looks like token is correct here
https://github.com/boto/botocore/blob/179e8b5361cd83d4c4acdf0c1bc1708b4a6966e9/botocore/session.py#L944-L948
| ) | ||
| from botocore.session import Session as BotoSession | ||
|
|
||
| botocore_session = BotoSession() |
There was a problem hiding this comment.
Im leaning towards loading custom clients instead of passing everything through properties. Every time we add another property, we need to map a name, a possible default, and pass to the underlying object.
The logic above is already pretty complicated, might as well split it out and call it something like RefreshableGlueClient
| @@ -61,7 +61,7 @@ | |||
| AWS_SECRET_ACCESS_KEY = "client.secret-access-key" | |||
| AWS_SESSION_TOKEN = "client.session-token" | |||
| AWS_ROLE_ARN = "aws.role-arn" | |||
| AWS_SESSION_NAME = "aws.session-name" | |||
| AWS_ROLE_SESSION_NAME = "aws.role-session-name" | |||
There was a problem hiding this comment.
This is originally from #1296, which has not been released yet. So we don't need to worry about backwards compatibility
Fixes #1104
PTAL @BTheunissen @cshenrik