Previous incidents

May 2024
May 14, 2024
1 incident

The v3 cluster is slow to accept new v3 tasks


Resolved May 14 at 08:00am BST

Runs are operating at normal speed again

There were pods in our cluster in the RunContainerError state, this happens when a run isn’t heartbeating back to the platform. We’re closely monitoring and have cleaned these. We’re determining which tasks caused this and what we can do to prevent this from happening in the future.

1 previous update

May 09, 2024
1 incident

Queues and runs have been processing at good speeds now for several hours on ...

Resolved May 09 at 10:00pm BST

Before we get into what happened I want to emphasise how important reliability is to us. This has fallen very short of providing you all with a great experience and a reliable service. We're really sorry for the problems this has caused for you all.

All paying customers will get a full refund for the entirety of May.

What caused this?

This issue started with a huge number of queued events in v2. Our internal system that handles concurrency limits on v2 was slowing down the pro...

April 2024
No incidents reported
March 2024
No incidents reported