Testing Farm Status

There are currently no known issues, if you have troubles, contact us :) Recently resolved Rawhide image update is blocked on BZ#2034336 - resolved on 2021-01-03 00:12:00 - seems underlying problems …

December 29, 2021 at 11:00 AM UTC

Expired RHEL-8.6.0 and RHEL-9.0.0 images in Red Hat Ranch and BaseOS CI

Resolved after 109h 12m

December 21, 2021 at 4:00 PM UTC

Issues with provisioning in Red Hat Ranch

Resolved after 20m

December 19, 2021 at 8:00 PM UTC

Issues Testing Fedora Rawhide

Resolved after 16h 0m

Incident history

Resilience Testing outage

artifacts.osci.redhat.com outage

RHEL-9.6.0-image-mode is broken

download.devel SSL certificate expired

changed script for postinstall broke jobs in beaker

RHEL 10 CTC outage on Red Hat Ranch - broken compose

Fedora 41 and Rawhide outage on public ranch - 0xe2 breaking ansible

Red Hat Ranch / BaseOS CI Outage - provisionig issues during the weekend

Misconfigured worker in public ranch

Testing Farm 2024-10.1 released

Testing Farm API down

Gitlab CEE outage affecting jobs run on BaseOS CI and Red Hat ranch.

Fedora 41 outage on Red Hat ranch

Fedora 41 outage on public ranch.

BaseOS CI and Red Hat Ranch affected by provisioning outages. BaseOS CI down.

BaseOS CI and Red Hat Ranch affected by OpenShift outage

BaseOS CI suspended for changes in UMB certificate.

Public Ranch Outage - no archive disk space

Public Ranch - broken Fedora repositories

Red Hat Ranch / BaseOS CI Outage - provisionig issues

Red Hat Ranch Outage - large logs consuming all the space. Testing-Farm dispatcher is disabled

Public ranch outage - workers not able to connect to Artemis.

Fedora 40 small disk size

Red Hat Ranch and BaseOS-CI Outage - dns misconfiguration, some crucial sites not accessible

Testing Farm API outage and longer queues on both ranches.

All Fedora images outdated, public ranch requests processing paused.

Fedora Rawhide outdated image

Beaker PPC64 Outage - large queue because of limited hardware

Public and Redhat Ranch Outage - broken deployment

Public Ranch Outage - artifacts out of space

Public Ranch Outage - complete outage, request will stay queued or error out

Cyborg Loki is down

Public Ranch - Large queue

Red Hat Ranch and BaseOS-CI Outage - broken krb infra

Red Hat Ranch Outage - large queue, jobs getting delayed

Red Hat Ranch Outage - large queue, jobs getting delayed

Public Ranch Outage - possible large queue due to deadlocked workers

Public Ranch Outage - artifacts storage out of space

Public Ranch Outage - F40 images expired

Public Ranch Outage - large queue, high error rate due to broken workers

Public Ranch Outage - Random issues with access to koji

Public Ranch Outage - Broken Fedora Rawhide images

Public Ranch Outage - Expired images

Public Ranch Outage - Fedora 40 expired

Red Hat Ranch Degraded

Public Ranch Outage - Expired images

Public Ranch Outage - Artemis requests in error state

Long queues in Public and Red Hat ranch

Public Ranch artifact storage ran out of disk space

Public Ranch large queue - expect delays

Public Ranch large queue - expect delays

Public Ranch large queue - expect delays

BaseOS CI suspended for urgent jenkins controller upgrade.

BaseOS CI suspended for urgent jenkins controller upgrade.

Public Ranch Outage - Artemis requests in error state

Public Ranch Outage

Public Ranch reservations are broken

Fedora CI installability test affected by systemd/selinux issue.

Testing Farm Public Ranch outage due to expired images.

Testing Farm Red Hat Ranch outage during the weekend

BaseOS CI limited due to outage due to BRQ network maintanance.

BaseOS CI and Testing Farm Red Hat Ranch outage due to Beaker unavailability

Expired images on Public ranch.

Testing Farm Red Hat ranch and BaseOS CI outage for RHEL9 and RHEL-8.1.0

BaseOS CI outage due to network issues

Console logs are disabled due to a discovered bug

Red Hat Ranch outage during weekend

Console log has wrong link in results viewer or not available at all.

Composes not updating on Red Hat ranch

Cyborg Loki is being migrated to S3 storage and better deployment

Beaker testing blocked on Red Hat Ranch and BaseOS CI

Large queue times on Red Hat Ranch

Red Hat Ranch affected by large queue

Public ranch testing blocked

Fedora Rawhide testing blocked on dnf5 breakage

Fedora Rawhide testing blocked on dnf5

Fedora Rawhide testing blocked on dnf5

Red Hat Ranch affected by RH infra outage

Expired Fedora-Rawhide images in Public Ranch