/status/v1/simple timing out regularly

[eyyit]

Describe the Bug

Queries to /status/v1/simple usually return in less than 1 second, but at times they do not return at all and timeout.

I assume a timeout is a catastrophic service check, but it would be nice to get some insight into what is happening here. It seems timeouts happen even when the service is responding to requests in a healthy manner.

Expected Behavior

Rapid and accurate responses according to the state of the service and stack for load balancers and state tracking to monitor.

Steps to Reproduce

1. Start puppetserver 7.x stack
2. Send traffic to it (in my case, 3 clients connecting per second)
3. Keep checking /status/v1/simple
4. Notice that at times it just won't respond, other times the responses can take 10s of seconds or more
5. Also notice that regardless of the reported health, the service is fine, even if the healthcheck times out.

Environment

• Version puppetserver 7.16.0
• Platform:
  • Dockerized Ubuntu 22.04
  • Custom docker image using deb from apt.puppetlabs.com with openjdk-jre-17-headless