Previous incidents

The issue is now resolved and DNS service is fully recovered. The degradation was caused by abnormal DNS load patterns, which caused intermittent issues with external resolvers.

We've already applied some changes to harden our networking infrastructure and will monitor closely to confirm stability.

Realtime is back to live.

We're really sorry for this extended period of large delays. The service couldn't keep up the number of runs being processed and was falling further behind. We have made some configuration changes and upgraded it so it can cope with a higher throughput of runs.

If you were using our React hooks that just did streaming, they were unimpacted by this.

DNS service is now back to fully operational. Increased traffic combined with a routine infrastructure rollout caused intermittent DNS resolution failures. We've tuned our DNS configuration to resolve the issue and are working on longer-term improvements to prevent recurrence.

The issue has been resolved. Dashboard and telemetry are now fully operational.

The issue is now resolved and dequeue times are back to normal. Mainly free-tier runs were affected. This was caused by a spike in the free-tier run volume.