Backend Scalability: Caching, Queues and Read Replicas

A flash sale, a WhatsApp campaign or a viral reel can send ten times your normal traffic in an hour. If your API slows down or your checkout times out at that moment, you lose the sales you paid to attract. Planning for backend scalability means knowing what to fix first, so you spend on the right thing instead of rebuilding everything.

This guide covers the practical order for Indian product teams: measure, tune the database, add caching, move slow work into queues, and only then add read replicas. Each step is cheaper and less risky than the next.

What is backend scalability?

Backend scalability is the ability of your API, server and database to handle more users and data without slower responses or failures. It is solved one bottleneck at a time, and the right fix depends on where your system is actually slow.

Two ideas help:

• Vertical scaling means a bigger server with more CPU and memory. It is quick and often enough for a long time.
• Horizontal scaling means more servers behind a load balancer. It needs stateless application servers, so sessions and files must live in shared storage rather than on one machine.

Most teams that feel "we need to scale" actually have a few slow queries and some work that should not happen inside the request. Start there.

How do you find the real bottleneck?

Measure response time per endpoint and query time per database call before you change anything. Without numbers, teams add caches and servers that do not touch the real problem.

Collect these basics:

1. Response time at the 95th percentile per route, not just the average.
2. Slowest database queries, using your database's slow query log or statistics views.
3. Error and timeout rate during peak hours.
4. CPU, memory and connection counts for the app server and database.
5. Queue backlog size, once you have background jobs.

Then run a load test on your two or three most important paths, such as login, product listing and checkout, at the traffic you expect for your next campaign. This tells you which limit you hit first.

Why fix queries and indexes before anything else?

Fixing indexes and query patterns usually gives the largest gain for the least money, so do it before adding any new service. A single missing index can turn a millisecond lookup into a full table scan as data grows.

Check these in order:

• Missing indexes on columns used in WHERE, JOIN and ORDER BY. Use your database's explain output to confirm an index is used.
• N+1 queries, where a list of 50 orders triggers 50 extra queries for customers. Load related data in one query or a join.
• Unbounded lists. Add pagination and a maximum page size to every list endpoint. Our guide to REST API design best practices covers pagination and filtering.
• Selecting everything. Fetch only the columns you return.
• Connection limits. Each app instance opens database connections, and a database allows a limited number. Use a connection pool with sensible limits so that adding app servers does not exhaust the database.

If your data model itself is the issue, revisit your database choice using our comparison of PostgreSQL, MySQL and MongoDB.

When does caching help, and how should you do it?

Caching helps when the same data is read much more often than it changes. A cache stores a copy of the result in fast memory, so repeated requests skip the database.

The most common pattern is cache-aside. Redis documents it like this: the application reads from Redis first, falls back to the database on a miss, writes the result back with a time-to-live (TTL), and invalidates the key when the data is updated.

Good candidates and risky ones:

Follow these rules:

1. Always set a TTL, so a missed invalidation fixes itself.
2. Delete or update the key after every write to that data.
3. Plan for stampedes. When a popular key expires, hundreds of requests can hit the database together. Use a short lock or refresh the key slightly before it expires.
4. Choose an eviction policy on purpose. Redis suggests allkeys-lru as a good default when a subset of keys is read far more than the rest.
5. Measure the hit rate. A cache with a low hit rate adds complexity without helping.

Which work should move into background queues?

Move any task the user does not need to wait for into a queue, so the request returns quickly and the work happens in a separate worker. A queue is a list of jobs that workers process at their own pace.

Typical jobs to move out of the request:

• Sending email, SMS and WhatsApp notifications
• Generating GST invoices and PDF reports
• Processing uploaded images and files
• Calling slow third-party APIs and retrying failed webhooks
• Syncing data to accounting tools such as Tally or to a CRM

A queue also protects you during spikes: the backlog grows for a few minutes, but the website stays responsive.

Design jobs carefully:

1. Make each job idempotent. A job may run twice after a crash or retry, so sending the same invoice number twice should not create two invoices or two charges.
2. Retry with backoff. Retry failed jobs a limited number of times with growing delays, then move them to a failed list that someone reviews.
3. Keep jobs small and pass IDs, not whole records, so a job reads fresh data when it runs.
4. Monitor queue length and job age, and alert when jobs wait too long.

If you use BullMQ with Node.js, the BullMQ production guide says Redis must run with maxmemory-policy set to noeviction, because evicting keys arbitrarily breaks queues. That conflicts with the eviction settings you may want for a cache, so run queues and caches on separate Redis instances. Laravel's queue system offers database and Redis drivers, and the same idempotency rules apply.

When should you add a read replica?

Add a read replica after tuning and caching, when a single database is still limited by read traffic. A read replica is a copy of your primary database that serves read-only queries, which frees the primary to handle writes.

In PostgreSQL, a replica is a hot standby that accepts read-only connections while it replays changes streamed from the primary. Streaming replication is asynchronous by default, so there is a small delay. The PostgreSQL documentation describes this as typically under one second when the standby can keep up, and says data on the standby is eventually consistent with the primary.

That delay decides which reads can use a replica:

• Safe for replicas: reports, dashboards, search pages, product listings and admin analytics.
• Keep on the primary: a user's own data right after they change it, payment confirmation and stock checks during checkout.

Other points to plan for:

1. Monitor lag on the primary, using the byte difference between the current write position and the replica's replay position, and alert if it keeps growing.
2. Expect cancelled queries. Long-running queries on a replica can conflict with incoming changes and be cancelled, so keep replica queries short or run heavy reports on a dedicated replica.
3. Send writes only to the primary and route reads in your data layer, not scattered through the code.

Managed databases from major cloud providers offer replicas as a setting, which removes much of the setup work. A replica adds a monthly infrastructure cost, so check it against what tuning and caching can still give you.

In what order should you scale your backend?

Scale in the order that gives the most benefit for the least complexity. Use this table as a guide:

Infrastructure costs vary a lot by cloud provider, region and traffic, so get an estimate for your own setup rather than relying on generic figures. As a rough project-experience range, steps 1 to 5 are often a few weeks of engineering effort for a typical product, and they delay the need for much larger spending.

How Arham Technology approaches backend scalability

Our backend development service treats scale as part of the design, not a late rescue. For each project we follow the same process:

1. Domain and threat sketch: we map the entities, the busiest workflows and the failure cases, such as a sale day or a webhook storm.
2. Contract first: API contracts include pagination, filters and error formats, so clients cannot request unbounded data.
3. Core services build: the main workflows ship with indexes, tests around critical rules and structured logs.
4. Integration and load reality: external systems get retries and idempotency, and key paths are exercised under the load you expect.
5. Runbook handover: deploy steps, migration notes and incident basics are documented for your team.

We build in Node.js, TypeScript, PostgreSQL, MongoDB and Redis, and in Laravel where it fits. For subscription products, the same patterns apply to billing jobs and tenant data, which our SaaS development service covers in more detail. Security belongs in the same plan, and our API security checklist shows the rate limits and logging that protect a busy API.

Next step

Start with measurement: list your five slowest endpoints and your five slowest queries, then fix the cheapest causes first. If you want a second opinion, see our backend development service or contact us with your current stack and traffic expectations, and we will suggest a practical scaling plan.