High queue on public ranch, expect delays

Outage | 2026-02-05 11:00:00 UTC

We are experiencing high load on Testing Farm. Investigation in place. Expect delays.

Update | 2026-02-05 15:00:00 UTC

The situation is ongoing, current average queue time is 3.5 hours

Update | 2026-02-06 01:40:00 UTC

Things are improving, we had to kill testing from scap via atex. Seems like some new integration using reservations is not cleaning up resources, causing all the requests to stay up long and take all resources, blocking other users.

Update | 2026-02-06 02:00:00 UTC

queue is almost gone, we are monitoring the situation