Skip to main content

Astro CI/CD templates for Harness

Use the following CI/CD template to automate a deploy with a single branch implementation, which requires only one Astro Deployment, from a Git repository to Astro with Harness.

If you use the DAG-only deploy feature on Astro or you're interested in a multiple-branch implementation, see Template overview to configure your own. To learn more about CI/CD on Astro, see Choose a CI/CD strategy.

Prerequisites​

Deploy template​

  1. Create a Harness CI pipeline and configure it as Remote with a Third-party Git provider.

  2. Authenticate to your Git provider.

  3. Choose Cloud for infrastructure, and Linux and ARM64 for platform. If you want to use other infrastructure, see Harness’s guide for implementation.

  4. Add a step, then choose Run as the template type from the step library. Use sh for your shell. Only one step is needed to deploy code from your Git repository to Astro, but you can add more if your use case requires.

  5. Paste the following code into the command prompt and apply changes.

#!/bin/bash
set -ex
# Prerequisites: Set variables
ORGANIZATION_ID=$ASTRO_ORG_ID
DEPLOYMENT_ID=$ASTRO_DEPLOYMENT_ID
ASTRO_API_TOKEN=$ASTRO_API_TOKEN
ASTRO_PROJECT_PATH=$ASTRO_PROJECT_PATH
# Step 1: Initialize deploy
echo -e "Initiating Deploy Process for deployment $DEPLOYMENT_ID\n"
CREATE_DEPLOY=$(curl --location --request POST "https://api.astronomer.io/platform/v1beta1/organizations/$ORGANIZATION_ID/deployments/$DEPLOYMENT_ID/deploys" \
--header "X-Astro-Client-Identifier: script" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $ASTRO_API_TOKEN" \
--data '{
"type": "IMAGE_AND_DAG"
}' | jq '.')
DEPLOY_ID=$(echo $CREATE_DEPLOY | jq -r '.id')
REPOSITORY=$(echo $CREATE_DEPLOY | jq -r '.imageRepository')
TAG=$(echo $CREATE_DEPLOY | jq -r '.imageTag')
DAGS_UPLOAD_URL=$(echo $CREATE_DEPLOY | jq -r '.dagsUploadUrl')
# Step 2: Log in to Docker
docker login images.astronomer.cloud -u cli -p $ASTRO_API_TOKEN
echo -e "\nBuilding Docker image $REPOSITORY:$TAG for $DEPLOYMENT_ID from $ASTRO_PROJECT_PATH"
# Step 3: Build image
docker build -t $REPOSITORY:$TAG --platform=linux/amd64 $ASTRO_PROJECT_PATH
# Step 4: Push image
echo -e "\nPushing Docker image $REPOSITORY:$TAG to $DEPLOYMENT_ID"
docker push $REPOSITORY:$TAG
# Step 5: Create tar file wd
echo -e "\nCreating a dags tar file from $ASTRO_PROJECT_PATH/dags and stored in $ASTRO_PROJECT_PATH/dags.tar\n"
cd $ASTRO_PROJECT_PATH
tar -cvf "$ASTRO_PROJECT_PATH/dags.tar" "dags"
# Step 6: Upload DAGs tar file
echo -e "\nUploading tar file $ASTRO_PROJECT_PATH/dags.tar\n"
VERSION_ID=$(curl -i --request PUT $DAGS_UPLOAD_URL \
--header 'x-ms-blob-type: BlockBlob' \
--header 'Content-Type: application/x-tar' \
--upload-file "$ASTRO_PROJECT_PATH/dags.tar" | grep x-ms-version-id | awk -F': ' '{print $2}')
VERSION_ID=$(echo $VERSION_ID | sed 's/\r//g') # Remove unexpected carriage return characters
echo -e "\nTar file uploaded with version: $VERSION_ID\n"
# Step 7: Finalizing Deploy
FINALIZE_DEPLOY=$(curl --location --request POST "https://api.astronomer.io/platform/v1beta1/organizations/$ORGANIZATION_ID/deployments/$DEPLOYMENT_ID/deploys/$DEPLOY_ID/finalize" \
--header "X-Astro-Client-Identifier: script" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $ASTRO_API_TOKEN" \
--data '{"dagTarballVersion": "'$VERSION_ID'"}')
ID=$(echo $FINALIZE_DEPLOY | jq -r '.id')
if [[ "$ID" != null ]]; then
       echo -e "\nDeploy is Finalized. Image and DAG changes for deployment $DEPLOYMENT_ID should be live in a few minutes"
       echo "Deployed Image tag: $TAG"
       echo "Deployed DAG Tarball Version: $VERSION_ID"
else
       MESSAGE=$(echo $FINALIZE_DEPLOY | jq -r '.message')
       if  [[ "$MESSAGE" != null ]]; then
               echo $MESSAGE
       else
               echo "Something went wrong. Reach out to astronomer support for assistance"
       fi
fi
# Cleanup
echo -e "\nCleaning up the created tar file from $ASTRO_PROJECT_PATH/dags.tar"
rm -rf "$ASTRO_PROJECT_PATH/dags.tar"
  1. Add the following environment variables in Harness:

ORGANIZATION_ID: The ID for your Organization. DEPLOYMENT_ID: The ID for your Deployment. ASTRO_API_TOKEN: The value for your API token as a secret. ASTRO_PROJECT_PATH: The default value is /harness/ and is followed by the folder name used in the Git provider for Astro, if applicable.

  1. Run the pipeline. Harness requires you to save changes before executing. Logs will display any errors.

Was this page helpful?