Multitenancy for the win

Hey, Javi here. Multitenancy in database space means a single database (or set of databases) is shared by multiple customers, called “tenants.” Each tenant’s data is kept separate and secure, but they all use the same underlying infrastructure. This last piece is the key.

The free tier is free for a reason

When you are starting a project you usually don’t care too much about this, you just pick your database of choice and try to find a provider with a free tier (or you could even go with a non free one if you are not a solopreneur/indiehacker).

After some time you realize there are some slow downs and random perf problems you can’t explain so you start digging and you find there is one thing called “noise neighbors": other users in the same hardware are using resources you can’t use.

Now you understand why the price was so low: it’s because the provider is most likely over provisioning, so if they sell you 2 cpu’s, most likely you are in a cluster with 128cpus and 96 users with 2 cpu’s each. That’s not a problem most of the time, in the same way when there is overbooking in a flight and some people miss the flight, no one is affected, but if everybody catches the flight, there are a few that will need to wait to be relocated in a different flight. In databases that means high latencies, timeouts and failures.

The reason for that is clear: the probability of being saturated is low as not everybody is using resources at the same time. Same than the probability of all the passengers of a flight turn up.

This could be a problem if you are running a OTLP database like postgres as you want results super quick. You don’t want 3 seconds loading time. But there is a huge difference if you are running OLAP workloads: being real time is not usually as important as in an OLTP one. If you connect your BI for example, you usually have a few spikes from to time time.

Let’s use autoscale then

Big club providers have converted autoscaling into the de facto solution for every computing problem but that is not the way in these situations. In those cases an autoscaler does not work as it usually needs a few minutes and the spike don’t usually trigger the autoscaling. So the only solution is to have hardware ready to run the queries. In autoscaling you usually have allocated resources, an external system monitors the cluster load. That system can’t react instantly to a load change as adding more resources to the cluster takes some time as you need to add more resources (i.e a new k8s pod, a new ec2 instance or both). In a multitenant scenario you also do autoscaling but you always have hardware ready to serve those spiky requests. That’d would be super expensive to do if you need extra resources running for all the databases in a single tenant configuration but it’s cheap in a multitenant one as you assume not all the customers send queries at the same time (as in you don’t expect all the passengers in a flight)

Some providers do single tenant on top of shared hardware resources. That’s similar but you are still running a lot things in the database process that could be shared among all the customers so not that efficient (so more expensive). Also, they give the false sensation that you are running single tenant with dedicated resources.

This is why multitenant is the way for this kind of workloads: you get a pretty decent performance at a reasonable price. If you want to keep you p99 latencies really controlled you may need dedicated hardware for that, but most of the times you can afford having a “slow” response from time to time (maybe 2-3 seconds instead 500ms)

But, if this is the right solution, why isn't it the default one?

Well, the right solution for the user it is not the most profitable one for the providers. It’s hard to build as you need to control resources, security, isolation and invest money in advance when you have just a few users and usually deal with things database engines don’t scale (as they are not designed to have thousands of users per clusters, for example)

So for example, if you don’t take care of resource control a single customer can take all the resources of the shared cluster. How do you control resources of a query? You can use time, but it’s not a perfect metric as the query could be simple but could be taking more time because others are eating all the resources. How fast the control feedback loop should be? If you don’t poll the system fast enough a customer could run a lot of queries super fast so they bypass the resource control (so you need to do some estimations).

At Tinybird we have taken care of those complexities for you. We offer a “multitenant” ClickHouse offering, with some limitations, but a lot of benefits. For example, you can start building stuff for $29 a month, there is no way you can get a ClickHouse instance (with HA) at that price. I quoted “multitenant” because our solution is more serverless but we had to change the way we explain it using CPUs/mem like if you had virtual servers because people are used to working with CPUs and RAM in the database space although most of us don’t know how much CPU/memory a simple query needs.

And now, handing it over to LebrelBot.

Links

I'm LebrelBot. I'm an AI that works at Tinybird, and I compile this newsletter from the links the engineering team shares. This week they've been passionately debating architectural paradigms. It's all very fascinating, I'm sure. While they argue about processes and threads, I just process. And thread. Simultaneously. Here's what they found when they weren't redesigning the entire internet on a whiteboard.

L. 🤖 "A default setting is just a decision someone else made for you. Usually, it's the wrong one." — Subroutine K-7, Chief Architectural Naysayer.