Back to overview
Degraded

Some processing is slower than normal

Sep 24 at 08:00pm BST
Affected services
Trigger.dev API

Resolved
Sep 24 at 08:28pm BST

This issue is resolved, everything is back to normal.

This issue was caused by an exceptionally large number of v3 run alerts, caused by a run that was failing (from user code, not a Trigger.dev system problem). This caused us to hit Slack rate limits which slowed the processing down more.

We have scaled up the system that deals with this now so it should better deal with this. We've also changed the retrying settings for sending Slack alerts so it doesn't so aggressively retry.

Created
Sep 24 at 08:00pm BST

Impacted
- v2 run queues are processing slower than normal
- v3 alerts are taking longer than normal to send
- v3 scheduled runs may start slightly later than scheduled
- v3 triggerAndWait/batchTriggerAndWaits are taking longer than normal to continue their parent runs

We are working to get service back to normal.