ShareChat NoSQL modernization

ShareChat runs one of India's largest social feeds. Public engineering talks describe 180 million monthly active users, 2.5 billion posts per month, and operational workloads that could not stay on a generic cloud NoSQL DBaaS once latency and cost stopped scaling. The modernization story that gets retweeted is ScyllaDB: 3–5× performance gains, 50–80% cost reduction, and 100 TB migrated with no downtime using dual writes and Apache Beam backfills.

That is an operational database win. It is not a full data platform win. Chat threads, like counters, notification state, and ML feature rows need single-digit millisecond reads. Executive dashboards, fraud investigations, and ad attribution need columnar SQL over months of history. Teams that copy ShareChat's NoSQL path without planning the analytics split rebuild the same pain six months later in a warehouse queue.

What ShareChat moved to ScyllaDB

Public ScyllaDB case studies and summit talks name these production shapes:

Geetish Nayak's team benchmarked alternatives after their managed NoSQL service could not hold single-digit millisecond targets under growth. They chose ScyllaDB for shard-per-core architecture and Cassandra-compatible drivers so Node.js, Go, and Java services could share libraries.

Charan Movva's counter aggregation talk describes a three-node cluster at 48 vCPUs and 350 GB RAM per node holding microsecond P99 even when load tests pushed toward 1.2 million ops/sec on a parallel cluster. That is the bar for user-facing engagement metrics, not for "run this GROUP BY over 90 days of events."

Migration mechanics worth copying

ShareChat's 100 TB zero-downtime migration is the part other platform teams ask about first. Public material describes three coordinated pieces:

1. Apache Beam batch jobs move historical rows from source tables to ScyllaDB with transforms shared across languages.
2. Dual-write drivers in application code write live traffic to source and destination until cutover confidence is high.
3. Re-sync jobs repair inconsistency when dual writes fail or compaction lag creates drift.

They tuned consistency levels, compaction strategy (incremental vs leveled), and data models per use case. Counter workloads stayed on counter tables. Feature store rows moved to protocol-buffer blobs per timestamp to cut row fetch volume from billions to hundreds of millions per second.

None of that replaces an analytics tier. It protects the product path users feel on every tap.

Kafka + NoSQL for real-time aggregates

ShareChat's engagement counter pipeline combines Kafka Streams windowed aggregation with ScyllaDB serving layers. Events land on Kafka, stream processors compute rolling counts, and ScyllaDB holds the latest values applications read.

That pattern excels when the question is "what is the count right now for this post?" It struggles when the question is "which cohort of creators drove retention lift in Q2 across language and region?" Windowed counters are not a substitute for historical SQL.

Teams often mirror Kafka topics into ClickHouse® for the second question while keeping ScyllaDB authoritative for serving. ClickHouse integration Confluent Cloud covers connector choices when the bus already exists.

Where analytical SQL enters the picture

Operational NoSQL answers point reads and small range scans. Analytics needs:

• Columnar compression on append-only event history
• Rollups (uniq, quantiles, funnels) without scanning every counter row
• HTTP endpoints product and ops tools call without opening CQL to BI users

ShareChat-scale social products generate both shapes from the same user actions. A view increments ScyllaDB and emits an event. If only the increment exists, you lose drill-down. If only the event stream exists, your homepage counter API pays warehouse latency.

Split responsibilities early:

ClickHouse for time series metrics, rollups, and SLOs describes tiered materialized views so dashboards never scan raw events for a badge count.

Tinybird beside a modernized NoSQL stack

Tinybird does not replace ScyllaDB for chat partitions. It gives you managed ClickHouse ingest, SQL Pipes as endpoints, and git-based deploys for the analytics half ShareChat's public talks do not cover.

Mirror Kafka once, query many endpoints

After ShareChat-style dual writes stabilize, point a Kafka connector (or Events API for app-generated JSON) at ClickHouse tables shaped for analytics, not for serving:

CREATE TABLE post_engagement_events (
    event_time DateTime64(3),
    post_id String,
    user_id String,
    event_type LowCardinality(String),
    language LowCardinality(String),
    region LowCardinality(String)
)
ENGINE = MergeTree
PARTITION BY toYYYYMM(event_time)
ORDER BY (post_id, event_time);

Rollups feed product analytics without touching ScyllaDB read paths:

NODE daily_engagement_by_language
SQL >
    SELECT
        language,
        toDate(event_time) AS day,
        countIf(event_type = 'view') AS views,
        uniq(user_id) AS viewers
    FROM post_engagement_events
    WHERE event_time >= today() - 30
    GROUP BY language, day
    ORDER BY day DESC, views DESC

TYPE endpoint

JWT fixed_params scope region or tenant_id server-side when partners hit the same endpoint with different contracts.

Branches for schema changes during migration chaos

NoSQL modernization runs for quarters. Analytics schema will change weekly while dual-write frameworks run. Tinybird Branches let you test new rollup columns against production-shaped data before deploy. Build real-time APIs on ClickHouse walks from SQL node to HTTPS endpoint without a custom Node wrapper around JDBC.

Service Data Sources expose ingest lag and endpoint p95 so you notice analytics drift while ScyllaDB on-call focuses on shard hotspots.

Canva reports 3.6 PB processed per month and 54 ms p99 query latency on Tinybird's product page. Resend reports 100 TB per month and 62 ms p90 without relying on cache.

Signs you stopped at NoSQL modernization too early

Every product question becomes a new CQL table. If each dashboard tile needs a bespoke counter cluster, you are denormalizing analytics into serving stores.

Kafka Streams owns your reporting logic. Window definitions that belong in SQL become brittle Java topology changes.

Data science reads the feature store for batch exports. ML serving and analyst SQL share storage but not access patterns. Feature store latency tuning is the wrong optimization for cohort analysis.

100 TB migrated, zero TB queryable in SQL. Migration success metrics hide the warehouse backlog building on the side.

Dual-write forever. ShareChat planned cutover and re-sync. Permanent dual paths double cost and consistency risk.

What ShareChat's path teaches

ShareChat's NoSQL modernization is a reference implementation for operational scale: shard-aware drivers, compaction tuning, Kafka aggregation, and migration frameworks that respect uptime. Copy the migration playbook for serving paths.

Plan analytics as a sibling system, not a phase-two afterthought. Columnar history, rollups, and published endpoints turn the same user growth into questions executives and models can answer without overloading the NoSQL tier that keeps the app fast.

Frequently Asked Questions (FAQs)

Did ShareChat replace all databases with ScyllaDB?

Public talks focus on operational NoSQL use cases: chat, counters, notifications, ads serving, feature store. They do not describe warehouse or BI elimination.

How long did ShareChat's migration take?

Public material emphasizes zero-downtime dual writes and 100 TB moved with re-sync jobs. Timelines vary by use case and language stack.

Is ScyllaDB enough for social analytics?

It excels at keyed reads and counters. Historical aggregates, funnels, and ad-hoc SQL typically need a columnar analytics engine alongside it.

What role does Kafka play?

Stream processing for windowed aggregates and decoupling writers from serving stores. Analytics often consumes the same topics downstream.

Can Tinybird read from ScyllaDB directly?

Typical pattern mirrors events into ClickHouse via Kafka or the Events API rather than querying CQL from BI tools.

When should we copy ShareChat's compaction tuning?

When read amplification on feature store or counter clusters shows up in P99 tails. Tuning is workload-specific, not universal defaults.