The networking issues from our worker cluster cloud provider is no longer happening. Networking has been back to full speed for the past 10 minutes and run are processing fast.

v3 runs are processing slowly. We think this is due to an intermittent networking issue with our worker cluster cloud provider. We are investigating and escalating this issue with them.

This isn't due to a code change.

The API and dashboard have been back to normal for some time. Runs are processing fast.

We are working on stopping this from happening again. We have identified the JSON data that caused this Node.js crash but it's not at all clear why it crashed V8. as it's valid JSON.

A crash caused some dashboard instability and has slowed run processing down. We're working to get all run queues back to their normal nominal size.

We know what caused the crash (some unexpected user data that somehow crashed Node.js). We have stopped the user from doing this temporarily until we have a permanent fix in place.