SaaS Job Queues: Measure Tenant Wait Before Adding Workers

Suppose one customer starts a large export just as another requests a small report. Both jobs land in your SaaS queue. The workers stay busy and the queue drains, yet the second customer waits long enough to press the button again. Total throughput looks healthy. The experience of that customer tells you something else.

Before adding workers, measure how long each tenant’s runnable jobs wait, how long they occupy a worker, and which tenants hold the active slots. Then compare those observations with CPU, database and external-service limits. If one tenant dominates the queue, changing how jobs get their turn may help. If every tenant waits while the underlying resource is saturated, capacity or job size needs attention.

AWS’s SaaS Lens guidance on noisy neighbors describes the risk of one tenant degrading another’s experience when resources are shared. A queue is one place where that happens. VM isolation between servers doesn’t decide how your application allocates the worker slots inside a shared server.

Give every attempt a job ID, tenant ID, work type, runnable time, start time and finish or failure time. Use a trusted application clock or consistently defined timestamps. Derive queue wait as start time minus runnable time, and service time as finish time minus start time. Keep deliberate scheduling delay separate. A report scheduled for tomorrow isn’t a customer waiting for an available worker today.

Also inspect the oldest runnable job that hasn’t started. Completed-job timing leaves that job out. Compare tenants doing similar work over the same interval, with the number of observations beside any percentile. A short window with one completed job can’t describe the tenant’s usual experience. Keep tenant identities in access-controlled diagnostic data and choose a bounded aggregation strategy for shared dashboards.

In the hypothetical export queue, check whether the small report was behind a burst of exports or whether it started promptly and then waited on the database. In the first case, inspect job selection and the number of slots the exporting tenant holds. In the second, inspect service time and the shared dependency. These two delays can look identical from the browser.

BullMQ’s global concurrency control limits active jobs across a queue’s workers. That can protect a shared dependency. To distribute turns between tenants, you also need a selection policy or tenant-level control. For example, BullMQ Pro Groups documents round-robin selection between groups. Groups are a Pro feature, and the application must map jobs to the intended tenant group.

Round-robin selection gives groups turns. Long jobs can still hold worker slots after selection. BullMQ Pro’s group concurrency offers a separate cap on simultaneous work for each group. Evaluate the equivalent controls in your own queue system, including what they cover across multiple worker processes. A per-process setting can have a different effect when another process starts.

Choose a response based on the constrained resource. Split very large jobs into bounded pieces where the operation supports it. Cap active work when a tenant can occupy most slots. Consider dedicated worker capacity when a tenant needs a stronger boundary, while checking whether the database or API remains shared. Retain retry limits and a recovery path for jobs that fail after partial work.

Rehearse with two disposable tenants. Send a burst of representative long jobs from one and occasional short jobs from the other. Record queue wait, service time, oldest runnable age, resource usage and successful outputs before and after one change. Repeat with comparable input sizes. Decide what delay is acceptable for the small report before using the result to choose a limit.

A self-managed Krova Cube can host a worker pool for that rehearsal. If the application accepts work asynchronously, the HTTP 202 and job-status guide shows how to expose progress to the caller. Use that status to distinguish accepted, waiting and completed work while the queue determines who gets the next turn.

This guide was prepared with AI assistance from the linked primary documentation. The export scenario and proposed rehearsal are hypothetical. No queue benchmark or production workload was run.