Developer infrastructure
Concept studyOne data platform, many tenants, no surprises
A multi-tenant ingestion and query platform where the hard requirement was that adding a tenant could never become a reason to break an existing one.
- Engagement
- Embedded engineer
- Duration
- Nine months
- Year
- 2024
- Client
- Not applicable
The challenge
What made this difficult.
Product data had outgrown per-customer databases. Isolation was inconsistent, cost attribution was impossible, and every new tenant triggered bespoke infrastructure work. The architectural question was how to share infrastructure without letting one tenant's behaviour, volume, or a noisy neighbour become a reliability event for everyone else.
Constraints
Non-negotiables we designed around.
Strict tenant isolation
A defect in one tenant's data path must not be able to read or corrupt another tenant's data, including during migrations and partial failures.
Noisy-neighbour containment
One customer's ingestion volume or expensive query must not degrade service for the rest.
Per-tenant cost attribution
Finance needed attributable cost, which meant metering had to be designed in rather than reconstructed from logs.
Ongoing onboarding
Adding a tenant had to become a routine operation, not a project.
Approach
How we would build it.
Made isolation a property of the data path
Tenant identity is carried in the schema and enforced at the storage layer, rather than applied as a filter in application queries where a missing predicate becomes a data leak.
Bounded resources at every tier
Per-tenant concurrency limits, query timeouts, and storage quotas are enforced at the platform boundary, and a tenant that exceeds its budget degrades for itself first.
Metered at the point of consumption
Ingestion, storage, and query cost are attributed as they occur, so invoices and internal cost questions are answered by the same instrumentation that runs the platform.
Turned onboarding into a reviewed template
A new tenant is provisioned from a versioned template with an explicit checklist, and the review is where genuine exceptions get examined — not the default path.
Migration as a rehearsed operation
Moving a tenant between isolation strategies was tested as a routine, reversible procedure with verification at each stage, because it is precisely the kind of change that is safe until it is not.
Architecture
How the pieces fit together.
Multi-tenant data platform. Tenant identity is enforced in the data path; metering is a first-class concern, not a derived report.
Ingest
- Per-tenant endpoints
- Schema validation
- Backpressure
- Quotas
Store
- Tenant-scoped schemas
- Row-level enforcement
- Tiered storage
- Encryption keys per tenant
Query
- Query planner
- Concurrency limits
- Statement timeouts
- Result caching
Meter
- Ingestion counters
- Storage accounting
- Query cost model
- Chargeback export
Operate
- Per-tenant dashboards
- Isolation alerts
- Migration tooling
- Onboarding templates
Stack
What it would run on.
Languages
- Go
- TypeScript
- Python
- SQL
Data
- PostgreSQL extensions
- Columnar store
- Object storage
- Kafka-compatible log
Infrastructure
- Kubernetes
- Terraform
- OpenTelemetry
- Key management
Expected outcomes
What success would look like.
Isolation
Enforced in path
Tenant scoping lives in the storage layer, so a missing application predicate cannot become a cross-tenant leak.
Containment
Per-tenant budgets
A tenant exceeding its budget degrades for itself first rather than consuming shared capacity.
Cost attribution
Metered inline
Cost data comes from the same instrumentation that runs the platform, not from a reconstruction after the fact.
Onboarding
Templated
Provisioning runs from a versioned template, so genuine exceptions get reviewed instead of becoming the default path.
What we learned
The conclusions we would carry forward.
Isolation enforced in application code is isolation that eventually fails; put it in the data path.
Cost attribution is trivial to add on day one and painful to reconstruct a year later — it belongs in the design, not the roadmap.
Making the common case routine is what freed review time for the cases that actually deserved it.
Related