From the course: Apache Airflow Essential Training

Unlock this course with a free trial

Join today to access over 24,300 courses taught by industry experts.

Complete branching pipeline using TaskFlow

Complete branching pipeline using TaskFlow - Apache Airflow Tutorial

From the course: Apache Airflow Essential Training

Start my 1-month free trial Buy for my team

Complete branching pipeline using TaskFlow

“

- [Instructor] In the previous demo, we only used the task flow API for the branching task. In this demo, we'll see how you can construct the entire branching pipeline using the task flow API. I still have my function definition branching using task flow, which is annotated using the @dag decorator. Now within this function, I have the read CSV file function, which is again annotated using @task. This is a single task within my DAG. By default, the ID for this task will have the same name as the Python function. But here I've explicitly specified a task ID, read_csv_file_task. This is on line 25. Determine branch is annotated using @task.branch. This is the same as before. You can see that both filter two seaters and filter front wheel drives are annotated using the @task decorator, on lines 44 and 57. Now is a good time for me to tell you that the task flow API, at the time of this recording, is still very new.…

Contents