Batches with more than 20 runs are slow to process
Resolved
Jul 15 at 03:15pm BST
Batches are processing as normal now. We have increased future capacity.
This was caused by a runaway loop of batches by a customer and this part of the system didn't have enough capacity to process them all fast enough.
We are updating how we process and rate-limit batches to prevent this from happening again, as well as improved internal alerts if similar issues happen in the future.
Affected services
Trigger.dev API
Updated
Jul 15 at 02:33pm BST
We're rapidly processing the backlog, batches should be back to normal soon. We'll update when it's running at normal throughput.
Affected services
Trigger.dev API
Created
Jul 15 at 02:05pm BST
We have a large backlog of batches to process (when you use our batchTrigger functions) caused by a huge influx of them. This only impacts batches with more than 20 runs in them.
We're working to increase the processing of these to get back up to date and to prevent this from happening in the future
Affected services
Trigger.dev API