Deploying to Tinybird through CI/CD

After you create your data project in Git, you can implement continuous integration (CI) and continuous deployment (CD) workflows to automate interaction with Tinybird.

When you create a project using tb create, Tinybird generates CI/CD templates you can use in GitHub and GitLab to automate testing and deployment.

The Tinybird Local container is a key part of the CI workflow. See Local container for more information.

CI workflow

As you expand and iterate on your data projects, you can continuously validate your changes. In the same way that you write integration and acceptance tests for source code in a software project, you can write automated tests for your API endpoints to run on each pull or merge request.

A potential CI workflow could run the following steps when you open a pull request:

  1. Install Tinybird CLI: Sets up dependencies and installs the Tinybird CLI to run the required commands.
  2. Build project: Checks the datafile syntax and correctness.
  3. Test project: Runs fixture tests, data quality tests, or both to validate changes.
  4. Deployment check: Validates the deployment before creating it, similar to a dry run.

The following templates are available for GitHub and GitLab:

name: Tinybird - CI Workflow

on:
  workflow_dispatch:
  pull_request:
    branches:
      - main
      - master
    types: [opened, reopened, labeled, unlabeled, synchronize]

concurrency: ${{ github.workflow }}-${{ github.event.pull_request.number }}

env:
  TINYBIRD_HOST: ${{ secrets.TINYBIRD_HOST }}
  TINYBIRD_TOKEN: ${{ secrets.TINYBIRD_TOKEN }}

jobs:
  ci:
    runs-on: ubuntu-latest
    defaults:
      run:
        working-directory: '.'
    services:
      tinybird:
        image: tinybirdco/tinybird-local:beta
        ports:
          - 7181:7181
    steps:
      - uses: actions/checkout@v3
      - name: Install Tinybird CLI
        run: curl https://tinybird.co | sh
      - name: Build project
        run: tb build
      - name: Test project
        run: tb test run
      - name: Deployment check
        run: tb --cloud --host ${{ env.TINYBIRD_HOST }} --token ${{ env.TINYBIRD_TOKEN }} deploy --check

CD workflow

Once your changes are validated by the CI pipeline, you can automate the deployment process and let Tinybird handle the migration for you with no downtime.

A potential CD workflow could run the following steps when you merge a pull request:

  1. Install Tinybird CLI: Sets up dependencies and installs the Tinybird CLI to run the required commands.
  2. Deploy project: Creates a staging deployment in Tinybird Cloud, migrates data, promotes to live, and removes previous deployment.

The following templates are available for GitHub and GitLab:

name: Tinybird - CD Workflow

on:
  push:
    branches:
      - main
      - master

concurrency: ${{ github.workflow }}-${{ github.event.ref }}

env:
  TINYBIRD_HOST: ${{ secrets.TINYBIRD_HOST }}
  TINYBIRD_TOKEN: ${{ secrets.TINYBIRD_TOKEN }}

jobs:
  cd:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Install Tinybird CLI
        run: curl https://tinybird.co | sh
      - name: Deploy project
        run: tb --cloud --host ${{ env.TINYBIRD_HOST }} --token ${{ env.TINYBIRD_TOKEN }} deploy

Secrets

Make sure to provide the values for the following secrets in your CI/CD settings:

  • TINYBIRD_HOST
  • TINYBIRD_TOKEN

Run tb info to get the values for the secrets in Cloud. For example:

tb info

» Tinybird Cloud:
--------------------------------------------------------------------------------------------
user: tinybird@domain.co
workspace_name: forward
workspace_id: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXX
token: YOUR-ADMIN-TOKEN
user_token: YOUR-USER-TOKEN
api: https://api.tinybird.co
ui: https://cloud.tinybird.co/gcp/europe-west2/forward
--------------------------------------------------------------------------------------------

» Tinybird Local:
--------------------------------------------------------------------------------------------
user: tinybird@domain.co
workspace_name: forward
workspace_id: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXX
token: YOUR-LOCAL-ADMIN-TOKEN
user_token: YOUR-LOCAL-USER-TOKEN
api: http://localhost:7181
ui: http://cloud.tinybird.co/local/7181/forward
--------------------------------------------------------------------------------------------

» Project:
---------------------------------------------------
current: /path/to/your/project
.tinyb: /path/to/your/project/.tinyb
project: /path/to/your/project
---------------------------------------------------

tb_secrets replacements happen at parser time in the server. If a secret is changed after a deployment is done, Tinybird won’t detect it automatically and will require an extra deployment.

Next steps

Updated