You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Initially posted in Slack but w/o any traction. This question is two-fold:
Is there a way to use the LivyOperator with a proxy? My first attempt at using the operator with a custom livy_default connection resulted in max retries exceeded errors. Then I noticed LivyHook was inheriting from HttpHook, so I tried a simple GET request to the same endpoint using SimpleHttpOperator and it failed with the same error. I then re-tried SimpleHttpOperator with proxies passed in and it worked. The problem is that the extra_options argument is not available in the LivyOperator; is there any other way to pass proxy information to the operator?
Assuming there is no way to pass proxy info to the LivyOperator through code, is there any way to add proxy info to an HTTP Airflow Connection object itself? (Perhaps through the "extras" option in the UI?).
I am using Airflow 1.10.12 and the below two packages to interact with Livy:
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Initially posted in Slack but w/o any traction. This question is two-fold:
LivyOperator
with a proxy? My first attempt at using the operator with a customlivy_default
connection resulted in max retries exceeded errors. Then I noticedLivyHook
was inheriting fromHttpHook
, so I tried a simple GET request to the same endpoint usingSimpleHttpOperator
and it failed with the same error. I then re-triedSimpleHttpOperator
with proxies passed in and it worked. The problem is that theextra_options
argument is not available in theLivyOperator
; is there any other way to pass proxy information to the operator?LivyOperator
through code, is there any way to add proxy info to an HTTP Airflow Connection object itself? (Perhaps through the "extras" option in the UI?).I am using Airflow
1.10.12
and the below two packages to interact with Livy:Beta Was this translation helpful? Give feedback.
All reactions