Skip to main content

DAG-level parameters in Airflow

In Airflow, you can configure when and how your DAG runs by setting parameters in the DAG object. DAG-level parameters affect how the entire DAG behaves, as opposed to task-level parameters which only affect a single task or Airflow configs which affect the entire Airflow instance.

This guide covers all user-relevant DAG-level parameters in Airflow.

Basic DAG-level parameters

There are four basic DAG-level parameters. It is best practice to always set these parameters in any DAG:

ParameterDescription
dag_idThe name of the DAG. This must be unique for each DAG in the Airflow environment. When using the @dag decorator and not providing the dag_id parameter name, the function name is used as the dag_id. When using the DAG class, this parameter is required.
start_dateThe date and time after which the DAG starts being scheduled. Note that the first actual run of the DAG may be later than this date depending on how you define the schedule. See DAG scheduling and timetables in Airflow for more information. This parameter may be required depending on your Airflow version and schedule.
scheduleThe schedule for the DAG. There are many different ways to define a schedule, see Scheduling in Airflow for more information. Defaults to timedelta(days=1). This parameter replaces the deprecated schedule_interval and timetable parameters.
catchupWhether the scheduler should backfill all missed DAG runs between the current date and the start date when the DAG is unpaused. This parameter defaults to True. It is a best practice to always set it to False unless you specifically want to backfill missed DAG runs, see Catchup for more information.

UI parameters

Some parameters add documentation to a DAG or change its appearance in the Airflow UI:

ParameterDescription
descriptionA short string that is displayed in the Airflow UI next to the DAG name.
doc_mdA string that is rendered as DAG documentation in the Airflow UI. Tip: use __doc__ to use the docstring of the Python file. It is a best practice to give all your DAGs a descriptive DAG documentation.
owner_linksA dictionary with the key being the DAG owner and the value being a URL to link to when clicking on the owner in the Airflow UI. Commonly used as a mailto link to the owner's email address. Note that the owner parameter is set at the task level, usually by defining it in the default_args dictionary.
tagsA list of tags shown in the Airflow UI to help with filtering DAGs.
default_viewThe default view of the DAG in the Airflow UI. Defaults to grid.
orientationThe orientation of the DAG graph in the Airflow UI. Defaults to LR (left to right).

Jinja templating parameters

There are parameters that relate to Jinja templating, such as:

ParameterDescription
template_searchpathA list of folders where Jinja looks for templates. The path of the DAG file is included by default.
template_undefinedThe behavior of Jinja when a variable is undefined. Defaults to StrictUndefined.
render_template_as_native_objWhether to render Jinja templates as native Python objects instead of strings. Defaults to False.
user_defined_macrosA dictionary of macros that are available in the DAG's Jinja templates. Use user_defined_filters to add filters and jinja_environment_kwargs for additional Jinja configuration. See Macros: using custom functions and variables in templates.

Scaling

Some parameters can be used to scale your DAG's resource usage in Airflow. See Scaling Airflow to optimize performance for more information.

ParameterDescription
max_active_tasksThe number of task instances allowed to run concurrently for all DAG runs of this DAG. This parameter replaces the deprecated concurrency.
max_active_runsThe number of active DAG runs allowed to run concurrently for this DAG.
max_consecutive_failed_dag_runs(experimental, added in Airflow 2.9) The maximum number of consecutive failed DAG runs, after which the scheduler will disable this DAG.

Callback parameters

These parameters help you configure the behavior of Airflow callbacks.

ParameterDescription
on_success_callbackA function to be executed after completion of a successful DAG run.
on_failure_callbackA function to be executed after a failed DAG run.
sla_miss_callbackA function to be executed when a DAG misses its defined Service Level Agreement (SLA)
tip

On Astro, you can use Astro alerts instead of or in addition to Airflow callbacks. See When to use Airflow or Astro alerts for your pipelines on Astro for more information.

Other parameters

Other DAG parameters include:

ParameterDescription
end_dateThe date beyond which no further DAG runs will be scheduled. Defaults to None.
default_argsA dictionary of parameters that are applied to all tasks in the DAG. These parameters are passed directly to each operator, so they must be parameters that are part of the BaseOperator. You can override default arguments at the task level.
paramsA dictionary of DAG-level Airflow params. See Airflow params for more information.
dagrun_timeoutThe time it takes for a DAG run of this DAG to time out and be marked as failed.
access_controlSpecify optional permissions for roles specific to an individual DAG. See DAG-level permissions. This cannot be implemented on Astro. Astronomer recommends customers to use Astro's RBAC features instead.
is_paused_upon_creationWhether the DAG is paused when it is created. When not set, the Airflow config core.dags_are_paused_at_creation is used, which defaults to True.
auto_registerDefaults to True and can be set to False to prevent DAGs using a with context from being automatically registered which can be relevant in some advanced dynamic DAG generation use cases. See Registering dynamic DAGs.
fail_stopIn Airflow 2.7+, you can set this parameter to True to stop DAG execution as soon as one task in this DAG fails. Any tasks that are still running are marked as failed, and any tasks that have not run yet are marked as skipped. Note that you cannot have any trigger rule other than all_success in a DAG with fail_stop set to True.
dag_display_nameAirflow 2.9+ allows you to override the dag_id to display a different DAG name in the Airflow UI. This parameter allows special characters.

Was this page helpful?