Skip to main content

conveyor project run

Run a project

Synopsis​

This command automatically builds a project and then runs it in a certain environment. By default it will pass the current date-time as execution date to Airflow, this can be modified. The command will not change the projects deployed on the environment you are running on. If you are using conveyor resources these won't be deployed. You have to do a conveyor deploy to change these.

conveyor project run [flags]

Examples​

To execute the command you need to specify the environment, dag and task you want to run:

$ conveyor project run --env ENV --dag DAG --task TASK
$ conveyor run --env ENV --dag DAG --task TASK (alias)

If you have only one dag and task you can simplify this to:

$ conveyor project run --env ENV
$ conveyor run --env ENV (alias)

The execution date can be passed as an rfc3339 compliant timestamp. For example a UTC timestamp can be specified as:

$ conveyor project run --env ENV --execution-date 2006-01-02T15:04:05Z

You can pass airflow variables by specifying the --dag-env-var parameter multiple times. If you want to specify the airflow variable foo with value bar, use:

$ conveyor project run --env ENV --dag-env-var AIRFLOW_VAR_FOO=bar

You can also execute dynamic tasks using the conveyor run command. We supplied the variable --dynamic-task-args to support this.
Using this variable you can pass the correct arguments.

$ conveyor project run --env ENV --dynamic-task-args arguments="echo"

By default the command shows less output then the build option. To show the same output as build, you can use the --debug flag.

Options​

      --build-arg list                            Set build-time variables for the containers. Same as Docker build-arg argument
      --build-args list                           (Deprecated) use build-arg instead
      --build-opts string                         Pass additional options to the container runtime (only if BuildKit enabled)
      --dag string                                The dag to run
      --dag-env-var list                          Set environment variable used for DAG validation or generation
      --dag-validation-command string             Set this if you use a custom dag validation image and want to change the command that is ran (default "dag_validation")
      --dag-validation-image string               Set this if you want to use a custom dag validation image
      --dynamic-task-args stringToString          The dynamic task args for a dynamic airflow task (default [])
      --env string                                The environment to run in
      --execution-date Time                       The execution date to run the task on, the format is rfc3339. For example 2006-01-02T15:04:05Z (default current date-time)
  -h, --help                                      help for run
      --project-dependencies-environment string   Overwrite default project dependencies environment
      --secret stringArray                        Secret file to expose to the build (only if BuildKit enabled): id=mysecret,src=/local/secret
      --skip-dag-validation                       Set to skip the Airflow DAG validation
      --ssh stringArray                           SSH agent socket or keys to expose to the build (only if BuildKit enabled) (format: default|<id>[=<socket>|<key>[,<key>]])
      --task string                               The task of the dag to run

Options inherited from parent commands​

      --debug                        Show debug output
      --no-browser NO_BROWSER=true   Do not automatically open a browser at login instead print the url to the command line. You can also use the environment variable NO_BROWSER=true.
  -o, --output string                Change the output. Valid options are table or json (default "table")
      --quiet QUIET=true             Quiet down the output. You can also use the environment variable QUIET=true.

See also​