Send Python logs to Tinybird

You can send logs from a Python application or service to Tinybird using the standard Python logging library and the tinybird-python-sdk.

Prerequisites

To use the Tinybird Python SDK you need Python 3.11 or higher.

Configure the logging handler

First, configure a Tinybird logging handler in your application. For example:

import logging
from multiprocessing import Queue
from tb.logger import TinybirdLoggingQueueHandler

logger = logging.getLogger('your-logger-name')
handler = TinybirdLoggingHandler(<YOUR_TB_API_URL>, <YOUR_TB_WRITE_TOKEN>, 'your-app-name')
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)
logger.addHandler(handler)

Each time you call the logger, the SDK sends an event to the tb_logs data source in your workspace.

To configure the data source name, initialize the TinybirdLoggingHandler like this:

handler = TinybirdLoggingHandler(<YOUR_TB_API_URL>, <YOUR_TB_WRITE_TOKEN>, 'your-app-name', ds_name="your_tb_ds_name")

Non-blocking logging

If you want to avoid blocking the main thread, use a queue to send the logs to a different thread. For example:

import logging
from multiprocessing import Queue
from tb.logger import TinybirdLoggingQueueHandler
from dotenv import load_dotenv

load_dotenv()
TB_API_URL = os.getenv("<YOUR_TB_API_URL>")
TB_WRITE_TOKEN = os.getenv("<YOUR_TB_WRITE_TOKEN>")

logger = logging.getLogger('your-logger-name')
handler = TinybirdLoggingQueueHandler(Queue(-1), TB_API_URL, TB_WRITE_TOKEN, 'your-app-name', ds_name="your_tb_ds_name")
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)
logger.addHandler(handler)
Updated