About the company
The Linux Foundation uses Tinybird to power LFX Insights, their open source project analytics platform. Data flows from PostgreSQL via Sequin CDC into Kafka, then into Tinybird where it's modeled and served through APIs.
“In the initial version of Insights, it could take up to 12 hours for changes in Postgres to appear in Snowflake. That wasn't going to work for the development team or the product.”
Joana Maia
Engineering Manager at The Linux Foundation
Problem
The Linux Foundation's LFX Insights platform tracks project health across 6,000+ open source projects, ingesting data from GitHub, GitLab, Gerrit, and community channels into Postgres. To power analytics, they synced Postgres to Snowflake via Fivetran, but cost constraints limited syncs to every twelve hours. Developers waited half a day to validate changes, and users saw stale data. The batch window wasn't a technical limitation; it was a budget decision.
Why Tinybird
For the Linux Foundation, Tinybird turned a twelve-hour feedback loop into a real-time analytics platform, letting open source maintainers see their project's impact as it happens.
Results
- Data freshness transformed. From 12-hour batch delays to near real-time. Maintainers and contributors see project impact as it happens.
- Development velocity unlocked. Changes in Postgres reflected immediately in the analytics layer. No more waiting half a day to validate.
- Cost structure improved. No more choosing between fresh data and reasonable costs. Real-time ingestion without linear cost scaling.
- Small team, big impact. Two engineers shipped a complete analytics product on unfamiliar technology.
- Future-ready architecture. Migration to Tinybird Forward will bring git-based workflows and branch-wise development as the platform scales to 10,000+ projects.
Postgres for transactions, Snowflake for analytics, twelve hours in between
LFX Insights tracks open source project health across 6,000+ critical projects, pulling data from various sources: GitHub, GitLab, Gerrit, Discord, Slack, Stack Overflow, Reddit, and more. Every commit, PR, issue, message, and contribution flows into the platform.
The Linux Foundation's data platform stores all of this in Postgres: activities, members, organizations, and the relationships between them. The architecture captures events in near real-time as they happen across these data sources.
The problem was getting that data into the hands of users.
Before Tinybird, the Linux Foundation used Snowflake for analytics. Snowflake was already in use across other products in the organization, so it seemed like the natural choice. They synced Postgres data to Snowflake via Fivetran, then built their analytic models on top.
The problem wasn't speed. It was cost. Both the Fivetran sync and the Snowflake transformations were expensive. Running them more frequently meant higher bills. The twelve-hour batch window wasn't a technical limitation; it was a budget decision. Freshness lost to economics.
This is a common pattern: the traditional stack of transactional database, scheduled batch ETL, and cloud data warehouse works until you need data faster than the budget allows.
And when costs pile up, teams batch more aggressively to stay within budget. Freshness becomes the casualty. The Linux Foundation needed a real-time OLAP layer that didn't force them to choose between fresh data and reasonable costs.
Twelve hours to know if your change worked
The twelve-hour delay wasn't just a user-facing problem. It was killing development velocity.
When the analytics team made a change in Postgres, they had to wait half a day to see if it worked correctly in the analytics layer. If something broke, they wouldn't know until the next batch ran. Debugging became guesswork. Iteration slowed to a crawl.
The team needed faster feedback, but faster batches meant higher costs. They were stuck.
From twelve hours to near real-time
The Linux Foundation evaluated alternatives before landing on Tinybird. They were already familiar with ClickHouse®'s analytical performance and evaluated managed providers. Tinybird fit their needs: managed infrastructure, Kafka-native ingestion, and a developer experience that wouldn't burden their small engineering team.
The architecture they built is straightforward: Postgres remains the source of truth, Sequin captures changes via CDC and publishes them to Kafka, and Tinybird consumes from Kafka to power the analytics layer. The same Kafka topics still feed Snowflake for other products that depend on it, but LFX Insights runs entirely on Tinybird.
The result: data that previously took twelve hours to sync now appears in near real-time. When something goes wrong, the team sees it immediately. They can diagnose issues, validate changes, and ship fixes without waiting for tomorrow's batch to confirm their work.
That real-time foundation also gives them flexibility they didn't have before. Not every feature needs up-to-the-second freshness. Leaderboards don't shift dramatically hour to hour, so they refresh daily. Other metrics update hourly. The team chooses the cadence per feature based on what users actually need, not what the budget allows.
Before, cost dictated a single twelve-hour refresh for everything. Now, real-time ingestion is the baseline, and they tune freshness per feature without penalty.
Small team, fast delivery
“We were able to deliver a first version of the product quickly with a technology that was pretty much unknown to all the developers. Just two engineers working on Tinybird had it up and running in a week.”
Joana Maia
Engineering Manager at The Linux Foundation
What stood out to Joana's team was how quickly they shipped despite Tinybird being unfamiliar technology. The developer experience let a small team build a complete analytics product without getting blocked on infrastructure concerns.
The team is currently running on Tinybird Classic and plans to migrate to Tinybird Forward. The git-based workflow and branch-wise development model in Forward will give them more controlled deployments and safer schema changes as the product scales to cover 10,000+ critical open source projects. As usage and team size grow, governance and safety will become the next bottleneck and that's where Forward shines.
For the Linux Foundation, Tinybird turned a twelve-hour feedback loop into a real-time analytics platform, letting open source maintainers see their project's impact as it happens.
