![]() ![]() Notice that in Airflow 2.0, you don’t have to use the parameter provide_context anymore. Ti is the task instance object required for accessing your XComs. If you don’t know what I’m talking about, check this article.įor instance, let’s say you want to fetch the accuracy by pulling a XCom, you can do that: def _choose_best_model(ti):Īccuracy = ti.xcom_pull(key='accuracy', task_ids=) You can give additional arguments through op_kwargs and op_args. You can access the context of the task instance to pull XComs. As a result, parameters of PythonOperator are accessible in the BranchPythonOperator. The BranchPythonOperator inherits from the PythonOperator. ![]() Behind the scene of the BranchPythonOperator Here, the function returns “accurate”, therefore, the next task to tigger is “accurate”. You can see the condition returning the task id, either “accurate” or “inaccurate”. If you take a look at the python function _choose_best_result(). It expects a task_id and a python_callable function. Pay attention to the arguments of the BranchPythonOperator. Copy paste the code in that file and execute the command docker-compose up -d in the folder docker-airflow. Create a file branching.py in the folder airflow-data/dags. Airflow 2.0, not 1.10.14 □ Clone the repo, go into it. Then, go to my beautiful repository to get the docker compose file that will help you running Airflow on your computer. To run the code, install Docker on your computer. The code above gives you the same data pipeline as shown before. With DAG('branching', default_args=default_args, catchup=False) as dag:Ĭhoose_best_model = BranchPythonOperator(Ĭhoose_best_model > What about a bit of code to implement it? from airflow import DAGįrom import BranchPythonOperatorįrom import DummyOperator How to Use AsyncTask for Android In practiceĪll right, you know the BranchPythonOperator and you know how it works. Can you guess which task is executed next? “Is accurate” or “Is inaccurate”? Like the PythonOperator, the BranchPythonOperator executes a Python function returning the task id of the next task to execute. That condition is evaluated in a python callable function. The BranchPythonOperator allows to follow a specific path according to a condition. Ok, we are to happy to meet with the BranchPythonOperator but what does it do? Give a warm welcome to the BranchPythonOperator! The BranchPythonOperator ![]() Yes there is! And you know it, otherwise you wouldn’t be there □ How can you do this? Is there a mechanism to achieve this? Let’s say, if the accuracy is above 5.0 we trigger “Is accurate”, otherwise, “Is inaccurate” is run. Then, either the task “is accurate” or “is inaccurate” should get executed according to the accuracy of the best ML model. Once they all complete, “Choosing Best ML” task is getting triggered. The first three tasks are training machine learning models. ![]()
0 Comments
Leave a Reply. |