Load testing and benchmarks

How ITSM Engine is load-tested, the latest results, and how to run the same test on your own hardware. The system requirements are based on these results.

What is measured

The test answers one question: how many agents can work at the same time, on one server (Standard) or a few (Scale), before they notice?

"Notice" is taken to be the point where 95% of requests no longer finish within a few hundred milliseconds, or requests start to fail.

It doesn't measure requester traffic on portals and forms, incoming email, or bulk API use. Those are small next to agents' traffic at the sizes tested, and will get their own scenarios.

How the test works

The installation

The installation is the production setup from deploy/, built from source: Caddy with HTTPS, the application container, PostgreSQL 17 and MinIO, with the production compose file and default settings.

• For Scale, the Scale file is laid over it: two application servers, the worker, PgBouncer and Redis.
• Outgoing email goes to a local Mailpit instead of a real provider.
• The test talks to the desk through Caddy, as browsers do.

The data

The data is the same large data set used to check query plans:

Records	Count
Cases	200,000, spread over two years and all statuses
Messages	1,000,000 requester messages, replies and notes
Contacts	100,000
Accounts	20,000
Units	200, under one organization
Agents	3,000, each with the Agent role in one unit

The database is about 0.7 GB.

The agents

The agents are simulated by a script that sends the same requests the agent app does, with each agent signed in on its own session. Each agent:

• opens the inbox (the case list and the counts above it), on one of three common views: assigned to me, unassigned, or all open;
• keeps the live stream open and counts the sidebar again when it says something changed, and every 5 minutes, as the app does (--client poll instead refreshes every 60 seconds, as the app did before live updates);
• every 5 to 20 seconds opens a case from the list (the case with its conversation, the unit's branding and the canned replies);
• stays on it for 30 to 120 seconds, sending the presence heartbeat every 20 seconds, as the open case page does with the live stream (every 8 seconds with --client poll);
• after a case, adds an internal note in 1 case in 10, and sends an email reply in 1 in 33;
• searches before 1 case in 10.

The steps

The number of agents grows in steps of 250, 500, 1,000, 2,000 and 3,000. Each step ramps up over 60 seconds and then holds for 180 seconds while it is measured.

250 agents → 500 → 1,000 → 2,000 → 3,000

What is recorded at each step:

• requests per second;
• response times (median, 95th and 99th percentile) overall and per kind of request;
• errors: any response that isn't a success, or takes longer than 30 seconds;
• CPU and memory of each container, sampled every 5 seconds;
• open PostgreSQL connections.

Results: Scale

Version 0.1.0 (unreleased), Scale setup with two application servers, run on 26 October 2026 with the live client. The same host and data set as the Standard run below; another small project was running on the machine.

Agents online	Requests/s	Median	95%	99%	Errors	App servers CPU / memory (each)	PostgreSQL CPU / memory
250	27	12 ms	32 ms	82 ms	0	0.16 + 0.03 cores / 330 MB	0.16 cores / 256 MB
500	58	9 ms	26 ms	69 ms	0	0.19 + 0.09 cores / 350 MB	0.28 cores / 276 MB
1,000	125	7 ms	21 ms	59 ms	0	0.13 + 0.20 cores / 380 MB	0.39 cores / 302 MB
2,000	281	8 ms	37 ms	62 ms	0	0.20 + 0.31 cores / 790 MB	0.69 cores / 353 MB
3,000	467	11 ms	62 ms	97 ms	0	0.48 + 0.36 cores / 823 MB	1.06 cores / 352 MB

The worker stayed at about 0.03 cores and 280 MB, PgBouncer at 2 MB and Redis at 9 MB. Caddy used up to 770 MB at 3,000 agents, mostly the 3,000 open live streams.

Reading the Scale results

• 3,000 agents online on two application servers: 95% of requests finish within 62 ms, and nothing fails. Neither application server is near its limit (half a core each), and PostgreSQL uses about one core.
• Live updates cost less than polling. At 3,000 agents the desk answered 467 requests a second, against 537 with the older client that refreshed every minute, while agents saw changes as they happened.
• Background traffic is still most of it: presence heartbeats and sidebar recounts were about two thirds of all requests.

Results: Standard

Version 0.1.0 (unreleased), Standard setup, run on 26 October 2026 with the live client. The same host and data set as the Scale run above.

Part	Details
Host	Apple M5 Pro, 18 CPU cores, 24 GB memory
Containers	Docker (OrbStack), with 18 CPUs and 12 GB available
PostgreSQL	17, default settings (shared_buffers 128 MB, max_connections 100)
Application	DATABASE_POOL_SIZE 20 (the default)
Load generator	The same machine as the desk

Agents online	Requests/s	Median	95%	99%	Errors	Application CPU / memory	PostgreSQL CPU / memory
idle	0				0	0.03 cores / 307 MB	0.06 cores / 177 MB
250	27	13 ms	38 ms	82 ms	0	0.16 cores / 423 MB	0.19 cores / 256 MB
500	58	11 ms	34 ms	67 ms	0	0.22 cores / 426 MB	0.24 cores / 255 MB
1,000	124	9 ms	34 ms	63 ms	0	0.26 cores / 491 MB	0.35 cores / 262 MB
2,000	280	9 ms	61 ms	111 ms	0	0.38 cores / 889 MB	0.56 cores / 263 MB
3,000	479	18 ms	124 ms	225 ms	0	0.61 cores / 1,041 MB	0.89 cores / 266 MB

CPU is in cores: 1.0 is one core fully used. Memory is the highest value seen during the step. Caddy used up to 790 MB at 3,000 agents, mostly the open live-update connections; MinIO stayed at about 75 MB.

Reading the Standard results

• 3,000 agents online on one server: 95% of requests finish within 124 ms, and nothing fails. The application uses 0.6 of a core and PostgreSQL 0.9, so one server has room left at this size.
• Two days earlier it couldn't: the run of 24 October, before shared sidebar counts, the access cache and live updates, had requests waiting 3 to 5 seconds at 3,000 agents (95% within 5.2 s), with the application at a full core. Those changes took about half the work off the application and Postgres.
• Scale is still faster at the top (62 ms against 124 ms at 3,000) and keeps the desk up when one server stops, but one server is now enough for most desks up to that size.

Limits of these results

• The load generator ran on the same machine as the desk and shared its CPUs. On separate machines the desk has more room, so these results are on the cautious side. The scripts only target localhost for now, as a guard against pointing them at a real installation.
• Each step held for 3 minutes. Memory over a full working day hasn't been measured yet.
• One data set size. Larger databases (millions of cases) haven't been tested under load yet.

Running it yourself

The scripts are in the repository, under scripts/load/. They need Docker with Compose, Node.js 22, and about 4 GB of free memory for the containers.

[!CAUTION] Run them on a test machine, never on a server with a real installation: env.sh stops every other compose project on the machine, and the test fills its own database.

Build and fill → Run the steps → Check status → Remove

1. Build the production setup as compose project itsmengine-load, fill the data set and sign in 3,000 agents:

   sh scripts/load/env.sh up standard

   …or the Scale setup, with APP_SERVERS application servers (2 by default):

   APP_SERVERS=2 sh scripts/load/env.sh up scale

2. Run the steps: 250, 500, 1,000, 2,000 and 3,000 agents (--variant scale labels a Scale run's results):

   node scripts/load/agents.mjs

   A shorter run, for a first look:

   node scripts/load/agents.mjs --steps 100,500 --ramp 30 --hold 60

3. Containers, CPU and memory right now:

   sh scripts/load/env.sh status

4. Remove the test installation and its data:

   sh scripts/load/env.sh down

Options and output

• Data set size: env.sh up takes AGENTS, CASES, MESSAGES, CONTACTS, ACCOUNTS and UNITS from the environment, for example CASES=1000000 MESSAGES=5000000 sh scripts/load/env.sh up.
• Results: each run writes a summary table and the full figures (JSON) to scripts/load/results/.
• Clean measurements: env.sh up stops other compose projects first, so the measurements are not mixed with other workloads. Their containers are stopped, not removed; start them again with docker compose start.