The first thing to fail is usually tenant isolation at the database layer
A SaaS we maintain for an HR firm in Noida reached 1,100 active tenants last quarter. The product had worked fine for two years at 200 tenants. Then reports started timing out for a specific customer — the one with 18,000 employee records in a shared `employees` table.
The query that broke was not complicated. It joined `employees` to `payroll_runs` with a `WHERE tenant_id = ?` on each table. At 200 tenants averaging 80 records each, that scan returns in 12ms. At 1,100 tenants with one tenant holding 18,000 rows, the planner sometimes chooses a sequence scan because the tenant_id index has low cardinality for that specific value. Postgres stats do not guarantee per-value selectivity estimates, and a query plan that was fine for 95% of tenants degraded silently for the remaining 5%.
This is the first thing we see break: shared-table multi-tenancy with a `tenant_id` column and an index that stops being selective enough for the planner. The fix is partial indexes scoped to the large tenant, or moving that tenant to a separate schema. Both are operational changes, not architectural ones, but they require downtime or a maintenance window.
The second failure is background job queue saturation under a single tenant
The HR product uses Sidekiq with Redis. Jobs are enqueued per tenant without rate limits. One tenant running a bulk payroll import enqueued 40,000 jobs in 30 seconds. Workers processing for other tenants starved. A 30-minute SLA for a smaller tenant became a 4-hour delay.
The fix was per-tenant queue limits using Sidekiq's weighted queues and a middleware that counts enqueued jobs per tenant. We set a ceiling of 500 pending jobs per tenant; excess jobs go into a scheduled set and drain at 50 per minute. This trades throughput for fairness. The large tenant's import takes longer, but the small tenant's single job runs in seconds.
We have applied this pattern to three other SaaS products. The tradeoff is the same each time: you lose peak throughput for the largest tenant in exchange for predictable latency for everyone else.
The third failure is permission checks that scale quadratically
A compliance SaaS we ship for a legal-services client assigns permissions per document per user per tenant. The check runs on every API request: does this user have access to this document in this tenant? At 50 documents and 10 users, the join cost is negligible. At 8,000 documents and 200 users across a tenant, the permission lookup adds 80ms to every request.
The common mistake is treating permissions as a runtime query against the source tables. At a thousand users, you need a denormalized permission cache — a materialized view refreshed on change, or a Redis structure keyed by `tenant_id:user_id` containing the set of accessible document IDs. The tradeoff is eventual consistency: a permission revocation takes up to 60 seconds to propagate. For most SaaS products, that is acceptable. For the legal-services client, we kept the cache at 15-second TTL because revoked access to a litigation document has real consequences, and 15 seconds is within their incident-response window.
The fourth failure is billing webhook drift
Stripe sends webhooks. Your handler updates the tenant's plan. At a thousand users, you will have tenants on legacy plans, tenants mid-trial, tenants who upgraded then downgraded, and tenants whose payment failed but whose grace period has not expired. The webhook handler that worked at 50 tenants — update plan, send email, done — starts producing inconsistent state because the order of webhook arrival matters and Stripe does not guarantee it.
We handle this by treating the webhook as a signal to reconcile, not as the source of truth. The handler fetches the subscription state from the Stripe API and writes that. Idempotency keys on the webhook event ID prevent double-processing. The tradeoff is an extra API call per webhook, which costs latency and rate-limit budget. At a thousand tenants, that is roughly 3,000 extra Stripe API calls per month, well within limits.
These four failures share a pattern: the design that works at 100 tenants is correct but not robust. The fix is rarely a rewrite. It is indexes, queue limits, caches, and reconciliation logic — each added at the point where the cost of not adding it exceeds the cost of maintaining it.