Ingest with Estuary

In this guide, you'll learn how to use Estuary to push data streams to Tinybird.

Estuary is a real-time ETL tool that allows you capture data from a range of source, and push it to a range of destinations. Using Estuary's Dekaf, you can connect Tinybird to Estuary as if it was a Kafka broker - meaning you can use Tinybird's native Kafka Connector to consume data from Estuary.

Read more about Estuary Dekaf.

Prerequisites

  • An Estuary account & collection
  • A Tinybird account & Workspace

Connecting to Estuary

In Estuary, create a new token to use for the Tinybird connection. You can do this from the Estuary Admin Dashboard.

In your Tinybird Workspace, create a new Data Source and use the Kafka Connector.

To configure the connection details, use the following settings (these can also be found in the Estuary Dekaf docs).

  • Bootstrap servers: dekaf.estuary.dev
  • SASL Mechanism: PLAIN
  • SASL Username: {}
  • SASL Password: Estuary Refresh Token (Generate your token in the Estuary Admin Dashboard)

Tick the Decode Avro messages with Schema Register box, and use the following settings:

  • URL: https://dekaf.estuary.dev
  • Username: {}
  • Password: The same Estuary Refresh Token as above

Click Next and you will see a list of topics. These topics are the collections you have in Estuary. Select the collection you want to ingest into Tinybird, and click Next.

Configure your consumer group as needed.

Finally, you will see a preview of the Data Source schema. Feel free to make any modifications as required, then click Create Data Source.

This will complete the connection with Estuary, and new data from the Estuary collection will arrive in your Tinybird Data Source in real-time.

Updated