Sep 03, 2019

Improved Support for Replacing or Appending Data

One of the foundational ideas of Tinybird Analytics is resiliency and consistency under high frequency or big data updates.
Javier Álvarez Medina
Co-founder

One of the foundational ideas of Tinybird Analytics is resiliency and consistency under high frequency or big data updates. We understand data as something that evolves and changes over time, therefore our product must treat data like that: as something that is alive.

Whenever you build a data pipe in Tinybird Analytics, your transformation nodes will always remain connected to the Data Sources they use. As soon as those Data Sources change, all their related transformation nodes will be recalculated in real-time. No delays.

Also, we’ve taken care of implementing everything needed to ensure your APIs keep performing in real-time as new data gets ingested. You will not need to worry about how and when you should be updating your data.

Although our API already supported appending data to an existing Data Source or replacing it entirely, during the last week we’ve added support for both replacing and appending data in our User Interface.

SCHEMA > Evolution
Want to stay updated? Subscribe to our bi-weekly newsletter.
Append or replace data to an existing Data Source
Append or replace data to an existing Data Source

As you can see in the gif above, whenever Tinybird Analytics detects you are trying to import a Data Source with a name that already exists in your account, the advanced data import options are shown.

In case you need to append data to an existing Data Source directly, we’ve enabled a shortcut to do exactly that; you can find it in the Data Source modal window, as you can see in the image below.

Append data to a Data Source
Append data to a Data Source

Take a look at our documentation and let us know what you think, or sign up for early access to Tinybird (we’re gradually opening up early access to companies and developers).

Do you like this post?

Related posts

A big performance boost, adding columns and more

Tinybird

Team

May 31, 2021
Iterate your real-time data pipelines with Git
A new way to create intermediate Data Sources in Tinybird

Tinybird

Team

Jun 15, 2023
Iterating terabyte-sized ClickHouse®️ tables in production
Iterating terabyte-sized ClickHouse®️ tables in production
Tinybird is out of beta and open to everyone
Ship data as you ship code: Deploy changes with a single command
Changelog: Revamping the API endpoints workflow and boosting your productivity
Simplifying event sourcing with scheduled data snapshots in Tinybird
Changelog #18: High-frequency ingestion, handling NDJSON files and more product enhancements

Tinybird

Team

Feb 01, 2022

Build fast data products, faster.

Try Tinybird and bring your data sources together and enable engineers to build with data in minutes. No credit card required, free to get started.