Using the Airflow REST API

Since Airflow 2.0, the REST API of Airflow is fully documented and no longer experimental. You can try it out in your browser, by clicking the Stable REST API in the Airflow docs. The following documentation describes how to use it programmatically.

For Conveyor, we have protected the Airflow API with the same authentication mechanism as we use in the Conveyor CLI. This means you can ask the CLI for a Conveyor token and use this to communicate with the Airflow API. To get started, you first have to make sure you are logged in:

conveyor auth login

After this, you can fetch your access token from the Conveyor CLI to talk to the Airflow 2.0 API.

conveyor auth get --quiet | jq ".access_token"

This token you can be used when you make API requests to Airflow. Add the token as a Bearer token in the Authorization header. For example, if your environment is called dev, you can use the following curl request to get a list of all DAGs.

curl -X GET "https://app.conveyordata.com/environments/dev/airflow/api/v1/dags?limit=100" \
  -H "accept: application/json" \
  -H "Authorization: Bearer $(conveyor auth get --quiet | jq -r '.access_token')"

The URL of your Airflow environment can be constructed as: https://app.conveyordata.com/environments/ENVIRONMENT/airflow/api/v1/dags?limit=100

Replace the ENVIRONMENT name with the name of your Conveyor environment.

caution

When using ACCESS KEY and SECRET KEY for authentication and doing frequent calls (more than 1 call per hour), we recommend caching the access token until the token is expired.