Save commonly used Param values as Presets #42192
Unanswered
matthewblock
asked this question in
Ideas
Replies: 1 comment
-
Also looking for this feature. Also, would be nice to do more complex DAG scheduling with various parameter "presets" (e.g. do frequent smaller incremental DAG runs in addition to less frequent bigger runs). |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Some terminology first since params and param values can get confusing: I'll call a set of values for a DAG's Params a "Configuration". We call it this on the
Trigger DAG w/config
page already.Currently when you execute a DAG that has DAG Params, you can select previously used Param values (called "Recent Configurations" in the UI dropdown), as well as the Default parameters, which always appears first in the dropdown:
It would be useful to pre-define more "Configurations" - Call it Favorites, Presets, or similar - Especially for DAGs that have complex Params or long lists of checkboxes, for example.
The current "best" way to label a "Configuration" is to execute the DAG and provide an optional Run ID:
There are a few issues with this:
Maybe these Preset Configurations could appear in the dropdown after Default parameters but before other Recent Configurations.
This is the reason my team is still using Airflow Variables for complex configurations that can change, alongside custom Pydantic validators and upstream validation task that does the same thing as Params. Right now we copy and paste our config (Airflow variable value) to an IDE and then do find and replace to change our complex configs.
Beta Was this translation helpful? Give feedback.
All reactions