How to Setup Astro CLI and deploy to Astro (Windows)

Setup:

1. Download the optimal version of Astro for your Windows system from link.
2. Rename the downloaded file to “astro.exe” and save it.
3. Add the file path to environment variables.
4. To check if Astro has been configured correctly, run “astro” command

On cmd. After the successful configuration of Astro CLI, you should get a response like the one below:

How to make your first airflow DAG:

1. Make a project directory for e.g. “test” and run “astro dev init” . What this command will do is it will initialize an Astro project in the local directory with all necessary airflow files and folders that are required to run airflow.
2. Open the project directory “test”  in any IDE.
3. In “test” go to “dags”  and create a Python script of airflow dag “test_dag.py” in my case.
4. Below is an example:

Deploy DAGs on Astro:

1. Assuming we have access to an Astro account, we will run “Astro login” command in CLI and then enter.
2. A pop-up window will appear in the default browser to ask for a login to Astro account.
3. Login using your preferred way and allow access.
4. Below are the screen grab of the process.

   

5. After successful login, we can create an Astro workspace using
   “astro workspace create –name “name of workspace” –enforce-ci-cd ON”.
   In our case, we are sticking with the “data engineer” workspace.
6. To switch to a preferred workspace, use “astro workspace switch” and select the number.
7. Post that, use Astro UI to create the deployment. See below

   

8. Now use “Astro deploy” command to check for errors and send out all of your dags to Astro.

   

   

9. After successful deployment, we can see all of our dags in “dags“ section of Astro UI.

   

10. Now, we can trigger them manually or schedule them to perform our tasks.
11. Adding below screen grabs of the output of our dag.

    

For local testing and development:

1. To test a dag in the local environment before deploying it to Astro, we can use “astro dev start”. But to run airflow on the local machine, we will have to set up docker refer to this link for further process.
2. Now go to “localhost:8080” and use credentials username=admin,password=admin to login.
3. Now, you can try all of your dag for further debugging and optimization before deploying it to Astro.

References:

1. https://docs.astronomer.io/
2. https://academy.astronomer.io/your-onboarding-journey