Testing Farm Status

There are currently no known issues, if you have troubles, contact us :) Recently resolved Rawhide image update is blocked on BZ#2034336 - resolved on 2021-01-03 00:12:00 - seems underlying problems …

December 29, 2021 at 11:00 AM UTC

Expired RHEL-8.6.0 and RHEL-9.0.0 images in Red Hat Ranch and BaseOS CI

Resolved after 109h 12m

December 21, 2021 at 4:00 PM UTC

Issues with provisioning in Red Hat Ranch

Resolved after 20m

December 19, 2021 at 8:00 PM UTC

Issues Testing Fedora Rawhide

Resolved after 16h 0m

Incident history

Public Ranch Outage - no archive disk space

Public Ranch - broken Fedora repositories

Red Hat Ranch / BaseOS CI Outage - provisionig issues

Red Hat Ranch Outage - large logs consuming all the space. Testing-Farm dispatcher is disabled

Public ranch outage - workers not able to connect to Artemis.

Fedora 40 small disk size

Red Hat Ranch and BaseOS-CI Outage - dns misconfiguration, some crucial sites not accessible

Testing Farm API outage and longer queues on both ranches.

All Fedora images outdated, public ranch requests processing paused.

Fedora Rawhide outdated image

Beaker PPC64 Outage - large queue because of limited hardware

Public and Redhat Ranch Outage - broken deployment

Public Ranch Outage - artifacts out of space

Public Ranch Outage - complete outage, request will stay queued or error out

Cyborg Loki is down

Public Ranch - Large queue

Red Hat Ranch and BaseOS-CI Outage - broken krb infra

Red Hat Ranch Outage - large queue, jobs getting delayed

Red Hat Ranch Outage - large queue, jobs getting delayed

Public Ranch Outage - possible large queue due to deadlocked workers

Public Ranch Outage - artifacts storage out of space

Public Ranch Outage - F40 images expired

Public Ranch Outage - large queue, high error rate due to broken workers

Public Ranch Outage - Random issues with access to koji

Public Ranch Outage - Broken Fedora Rawhide images

Public Ranch Outage - Expired images

Public Ranch Outage - Fedora 40 expired

Red Hat Ranch Degraded

Public Ranch Outage - Expired images

Public Ranch Outage - Artemis requests in error state

Long queues in Public and Red Hat ranch

Public Ranch artifact storage ran out of disk space

Public Ranch large queue - expect delays

Public Ranch large queue - expect delays

Public Ranch large queue - expect delays

BaseOS CI suspended for urgent jenkins controller upgrade.

BaseOS CI suspended for urgent jenkins controller upgrade.

Public Ranch Outage - Artemis requests in error state

Public Ranch Outage

Public Ranch reservations are broken

Fedora CI installability test affected by systemd/selinux issue.

Testing Farm Public Ranch outage due to expired images.

Testing Farm Red Hat Ranch outage during the weekend

BaseOS CI limited due to outage due to BRQ network maintanance.

BaseOS CI and Testing Farm Red Hat Ranch outage due to Beaker unavailability

Expired images on Public ranch.

Testing Farm Red Hat ranch and BaseOS CI outage for RHEL9 and RHEL-8.1.0

BaseOS CI outage due to network issues

Console logs are disabled due to a discovered bug

Red Hat Ranch outage during weekend

Console log has wrong link in results viewer or not available at all.

Composes not updating on Red Hat ranch

Cyborg Loki is being migrated to S3 storage and better deployment

Beaker testing blocked on Red Hat Ranch and BaseOS CI

Large queue times on Red Hat Ranch

Red Hat Ranch affected by large queue

Public ranch testing blocked

Fedora Rawhide testing blocked on dnf5 breakage

Fedora Rawhide testing blocked on dnf5

Fedora Rawhide testing blocked on dnf5

Red Hat Ranch affected by RH infra outage

Expired Fedora-Rawhide images in Public Ranch

Unexpected outage due to new release 2023-04.1

Red Hat Ranch & BaseOS CI - outage due to elevated cloud costs

Public Ranch - unexpected outage

Public Ranch - unexpected outage

Public Ranch - expired images

CentOS Stream 9 aarch64 broken in Public Ranch

Red Hat Ranch - deploying 2023-03.1 release - large queue expected

Red Hat Ranch - provisioning outage

Red Hat Ranch - larger queue

Public ranch - Artemis resource ended in 'error' state

CentOS Stream 8 broken in Public Ranch

Fedora-Rawhide broken in Red Hat Ranch

Red Hat internal infrastructure outage

BaseOS CI - large queue, expect delays.

Public Ranch - large queue, expect delays.

Public Ranch outage - requests stuck in queued

BaseOS CI Outage