Previous incidents

This has been resolved by rolling back to ElectricSQL 1.0.3.

We were on 1.0.20 which worked when we first deployed it yesterday. Then on a second deploy today it stopped receiving publications from Postgres. We tried restarting it a few times but it wouldn't receive any updates, so we rolled back.

v4 dequeue performance has now improved again, and we're working on two things:

A short term fix to prevent this from happening again in the short term, to be deployed today.

A long term fix for dequeue performance that will be worked on and hopefully shipped this week, which will vastly improve dequeue performance and scaling.

Queues are back to nominal length and have been for some time.

This issue was caused by a huge influx of queues, which meant we weren't considering them all when selecting queues for dequeuing.

We have increased some settings to make this better and we're looking at what we can do in the future to make this scale better for the next 10–100x multiple of queues.