3. Deploy your project

The project and environment are now in a state where the project can be build and deployed to the environment.

3.1. Build the project

The first step is to validate and package your project code in a docker image and upload it. To execute the command, make sure you are using a terminal that is opened in the project folder.

conveyor build

Once done, this project is ready to deploy to any environment.

info

The first time build of a project can take some time depending on your internet connection. Subsequent builds should be significantly faster, since multiple steps are cached.

3.2. Deploy the project

Next, deploy the project to the environment:

conveyor deploy --env $ENVIRONMENT_NAME --wait

The --wait flag makes the command wait until the deploy is finished, this is certainly useful in a CI/CD context but also if you want to easily know when a deploy is finished.

3.3. Activate the workflow

In the Conveyor UI navigate to your environment, by clicking environment on the left and clicking on your environment in the list. This will automatically open Airflow.

By default, your workflow will be disabled. Enable it by selecting the toggle on the left next to your DAG. Airflow will now start scheduling your project, one run for each day since the start date you specified when creating the project. For more details on Airflow as well as how Conveyor integrates with Airflow, take a look here.

3.4. Explore logs and metrics

Next we can see our runs in the Task executions tab of our environment. Select one of the task executions, and you will be able to see all the settings of your task. And underneath you see the logs of your task. For longer running task CPU and Memory metrics are also available.

3.5. Run a task from the CLI

To shorten the feedback loop during development, you can run a task directly from the CLI using the conveyor run command. This command performs a build and runs the job on the cluster, avoiding the need to deploy your project to Airflow and showing the logs directly in your console instead.

The command does refer to your workflow definition in order to determine which job to launch. In the command below, the --dag parameter refers to the name of the DAG where to look for a task, and the --task parameter refers to the name of the task within the chosen DAG that you want to execute.

conveyor run --env $ENVIRONMENT_NAME --dag $PROJECT_NAME --task sample

Great! We just built and deployed our project to an environment for the first time. This is the main way used by developers to deploy their project to development or production. Using conveyor run we are able to quickly test out changes without having to deploy our project.