On 20 September 2026 the LIONEX site finally started counting its own HTTP requests by route. The next day we looked at a full day of data for the first time and saw that the most popular address on the site was neither the home page nor the blog. It was /api/health, a service route that no human ever opens. Over the day it received 13,666 requests, or 60% of all application traffic.
Why request metrics were not recorded at all before 20 September is covered in the article about four alerts that never existed. This one is about the first finding those metrics produced, the two settings that removed it, and a way to check the same thing on your own setup in a few minutes.
What the first day showed
The health route exists for the platform, not for visitors. Docker, configured through Coolify, hits it periodically and uses the response to decide whether the container is alive. A few bad responses in a row, and the container gets restarted.
The check interval was set to 5 seconds. By the settings, that is 17,280 calls per day; the metrics for 21 September recorded 13,666, fewer than calculated but in the same ballpark. The frequency alone is not the problem: answering "the process is alive" costs almost nothing. The problem was in the route's code. On every call it ran SELECT 1 against the database, so in theory the database received the same 17,280 queries per day just to confirm it was still there.
That database runs on the same server as a dozen of our other projects. That day the server's load average hit 8.36, against an alert threshold of 8. To be clear: we did not measure what share of that load came from the healthcheck, and we do not call it the cause. What we did see is that the shared machine was running extra work around the clock that nobody knew about.
Why nobody noticed
A single call took milliseconds, produced no errors, and did not show up in the logs. There was only one way to spot it: count requests by route. Until 20 September nobody did, and without that breakdown a service address can take most of the traffic without a single graph showing it.
Worse, each piece looked reasonable on its own. The five seconds were a default in the deployment panel; nobody chose them deliberately. The database query had been added at some point with good intentions, "so the check honestly looks at the database": a health endpoint that never touches the database can report a container as healthy when it can no longer serve a single page. Each decision is defensible on its own. Together they produced a query to the shared database every five seconds, indefinitely.
Two levers, and you need both
The first is the interval. On 21 September we changed it from 5 to 30 seconds. We kept the retry count at 10, so a container now has to fail checks for about 5 minutes (10 × 30 s) before it is restarted. Previously 50 seconds (10 × 5 s) was enough.
This decision has a cost, and it is worth naming: we now detect a real failure more slowly. For an agency site with no order queue, that is acceptable. For a payment service, where every minute of downtime costs money, it may not be, and there we would weigh it differently.
The second lever is the route itself. A successful database check result is now remembered for 60 seconds in a process variable. The healthcheck gets its answer immediately, and the database gets at most one query per minute.
The interval alone would not have been enough: every call would still go to the database, just less often. The cache alone would have removed the database load, but the platform would still poke the process every five seconds: 17,280 calls per day, each one launching curl inside the container. Together they separate two different things: how often the platform wants to know the state, and how often the answer actually reaches the database.
A process variable worked here because the site runs as a long-lived Node server: the process stays up for hours, and the value survives between requests. In serverless, where a function spins up for each request, a cache like this would remember nothing.
Why we do not cache failures
We only remember success. If the database does not respond, the route returns 503 with the error text, clears the success flag, and the next call goes to the database again.
The asymmetry is deliberate. Success is the normal state, and caching it removes almost all the extra work. Failure is exactly the moment when the platform uses the response to decide whether to restart the container, so every check after it has to be fresh. If the success flag stayed set after a failure, the container would be considered healthy for another minute after the database went down, precisely when it needs to be restarted.
One limit remains regardless: up to 60 seconds pass between the last successful check and the next real database query, and health will not see a failure in that window right away. Against a five-minute window before restart, we consider that acceptable.
Verification
First, locally: 13 calls in a row produced one database query, and a call 65 seconds later went to the database again. On 22 September the route's tests were rewritten: the old test described a version that had not existed for a long time and did not run at all, because the project was missing the jsdom package. The new tests check a 200 response with "ok" when the database is up; no second database query within 60 seconds; a 503 with the error text when the database is unavailable; and that the failure is not cached.
The container config the same day: Interval 30 s, Timeout 5 s, Retries 10.
The "after" numbers, and a share that barely moved
On 22 September, over 15 minutes outside a deployment window, Prometheus showed 31.5 requests to /api/health (the fraction comes from how Prometheus calculates a counter's increase over an interval). That is about 126 per hour, or roughly 3,000 per day instead of 13,666.
By design, the cache caps database queries at one per minute, meaning at most 1,440 per day instead of 17,280. This figure is calculated from the settings; we did not measure database queries separately.
The share, however, barely moved. Over the same 15 minutes, health accounted for 31.5 of 55 application requests, still the majority. An agency site has little traffic, and the service address takes a noticeable part of it at any interval. The problem was never the share. The problem was that every one of those calls went to the shared database.
A measurement trap: a deploy in the window
The first attempt to measure "after" gave 690 requests per hour, as if we had changed nothing. The cause was in the window itself: a deploy of a new image had landed in it. During a rollout the platform itself polls the new container's health frequently, and for a while the metrics system holds two series, one for the old container and one for the new. In the 15 minutes after the deploy, the result was 31.5.
The takeaway is simple: measure outside the deployment window, or it is easy to "prove" the fix did not work.
How to check your own
Look at the interval. For Docker:
docker inspect -f '{{json .Config.Healthcheck}}' <container>The Interval there is in nanoseconds:
5000000000means 5 seconds. In Kubernetes, the same thing lives inperiodSecondsfor liveness and readiness.Read the health route's code. What does it do on every call: hit the database, Redis, an external API? Multiply each such action by the number of calls per day:
calls per day = 86 400 / interval in seconds 86 400 / 5 = 17 280Separate "the process is alive" from "ready to serve". Kubernetes explicitly splits liveness and readiness. Docker has a single check, so the practical compromise is this: cache success and do not cache failure.
Count requests by route at least once, from proxy logs or application metrics. Without that, things like this are simply invisible.
Multiply Retries by Interval. The result is how long a container runs broken before it is restarted. That number should be chosen deliberately, not inherited from the defaults.
Limits
Everything described here applies to one site on one platform, Coolify with Docker. The "before" numbers cover one day; the "after" numbers cover 15 minutes plus the container config. The number of database queries after the fix is calculated, not measured.
And the main caveat: we do not claim the site got faster or that server load dropped. We did not measure that. What we removed is constant extra work that nobody saw, and nothing more.
If you want someone to review the healthcheck, alerts and request accounting on your site, that is part of uptime monitoring. Container and server configuration are covered by server administration.
The rule we took away for ourselves: a service route is traffic too, and it needs to be counted just like pages for people.
