Send Python logs to Tinybird¶
You can send logs from a Python application or service to Tinybird using the standard Python logging library and the tinybird-python-sdk.
Prerequisites¶
To use the Tinybird Python SDK you need Python 3.11 or higher.
Configure the logging handler¶
First, configure a Tinybird logging handler in your application. For example:
import logging
from multiprocessing import Queue
from tb.logger import TinybirdLoggingQueueHandler
logger = logging.getLogger('your-logger-name')
handler = TinybirdLoggingHandler(<YOUR_TB_API_URL>, <YOUR_TB_WRITE_TOKEN>, 'your-app-name')
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)
logger.addHandler(handler)
Each time you call the logger, the SDK sends an event to the tb_logs Data Source in your Workspace.
To configure the Data Source name, initialize the TinybirdLoggingHandler like this:
handler = TinybirdLoggingHandler(<YOUR_TB_API_URL>, <YOUR_TB_WRITE_TOKEN>, 'your-app-name', ds_name="your_tb_ds_name")
Non-blocking logging¶
If you want to avoid blocking the main thread, use a queue to send the logs to a different thread. For example:
import logging
from multiprocessing import Queue
from tb.logger import TinybirdLoggingQueueHandler
from dotenv import load_dotenv
load_dotenv()
TB_API_URL = os.getenv("<YOUR_TB_API_URL>")
TB_WRITE_TOKEN = os.getenv("<YOUR_TB_WRITE_TOKEN>")
logger = logging.getLogger('your-logger-name')
handler = TinybirdLoggingQueueHandler(Queue(-1), TB_API_URL, TB_WRITE_TOKEN, 'your-app-name', ds_name="your_tb_ds_name")
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)
logger.addHandler(handler)