Common use cases¶

The following uses cases illustrate how Tinybird CLI solve common situations using available commands.

Download Pipes and data sources from your account¶

There are two ways you can start working with the CLI. You can either start a new data project from scratch, or if you already have some data and API Endpoints in your Tinybird account, pull it to your local disk to continue working from there.

For this second option, use the --match flag to filter Pipes or data sources containing the string passed as parameter.

For instance, to pull all the files named project:

tb pull --match project
[D] writing project.datasource(demo)
[D] writing project_geoindex.datasource(demo)
[D] writing project_geoindex_pipe.pipe(demo)
[D] writing project_agg.pipe(demo)
[D] writing project_agg_API_endpoint_request_log_pipe_3379.pipe(demo)
[D] writing project_exploration.pipe(demo)
[D] writing project_moving_avg.pipe(demo)

The pull command does not preserve the directory structure, so all your datafiles are downloaded to your current directory.

Once the files are pulled, you can diff or push the changes to your source control repository and continue working from the command line.

Push the entire data project¶

tb push --push-deps

Push a Pipe with all its dependencies¶

tb push pipes/mypipe.pipe --push-deps

Adding a new column to a Data Source¶

Data Source schemas are mostly immutable, but you have the possibility to append new columns at the end of an existing Data Source with an Engine from the MergeTree Family or Null Engine.

If you want to change columns, add columns in other positions, or modify the engine, you must first create a new version of the Data Source with the modified schema. Then ingest the data and finally point the Pipes to this new API Endpoint. To force a Pipe replacement use the --force flag when pushing it.

Append new columns to an existing Data Source¶

As an example, imagine you have the following Data Source defined, and it has been already pushed to Tinybird:

SCHEMA >
    `test` Int16,
    `local_date` Date,
    `test3` Int64

If you want to append a new column, you must change the *.datasource file to add the new column new_column. You can append as many columns as you need at the same time:

SCHEMA >
    `test` Int16,
    `local_date` Date,
    `test3` Int64,
    `new_column` Int64

After appending the new column, execute tb push my_datasource.datasource --force and confirm the addition of the column(s). The --force parameter is required for this kind of operation.

Existing imports will continue working once the new columns are added, even if those imports don't carry values for the added columns. In those cases, the new columns contain empty values like 0 for numeric values or '' for Strings, or if defined, the default values in the schema.

Create a new version of the Data Source to make additional add/change column operations¶

To create a new version of a Data Source, create a separate datafile with a different name. You can choose a helpful naming convention such as adding a _version suffix (e.g. my_ds_1.datasource).

Debug mode¶

When you work with Pipes that use several versions of different data sources, you might need to double check which version of which Data Source the Pipe is pointing at before you push it to your Tinybird account.

To do so, use the --dry-run --debug flags like this:

tb push my_pipe.pipe --dry-run --debug

After you've validated the content of the Pipe, push your Pipe as normal.

Automatic regression tests for your API Endpoints¶

Any time you --force push a Pipe which has a public API Endpoint that has received requests, some automatic regression tests are executed.

If the previous version of the API Endpoint returns the same data as the version you are pushing, the CLI checks for the top ten requests. This can help you validate whether you are introducing a regression in your API.

Other times, you are consciously --force pushing a new version which returns different data. In that case you can avoid the regression tests with the --no-check flag:

tb push my_may_view_pipe.pipe --force --no-check