current status?

[Dieterbe]

Hello!
i stumbled upon this project because i'm considering using powersync and was wondering if anyone has done serious correctness verification using jepsen or similar tools.

Seems you are working on exactly this. I checked your readme and https://github.com/nurturenature/jepsen-powersync/blob/main/doc/logbook.md
seems that for basic/single-client uses, the system works as designed; but your more advanced tests involving multiple users reveal issues, which may potentially be due to a bug the testing itself.

Is that basically the current status? I was wondering if you could give some more color or conclusions about your testing thus far, and whether you've found powersync to be a robust system including in multi-client situations. thanks!

[nurturenature]

Thanks for the interest!

It is very early in the test process.

So early that it's not really possible to meaningfully interpret any results yet.

The project is

• under active development
• a collaboration to build and verify a causal+ connector
• kept current with results if you'd like to follow or review occasionally

I can say that the documentation, examples/demos, and APIs have been nice to work with.

[nurturenature]

Hi,

Thanks for your patience.

This behavior was an artifact of the application's newly developed causally consistent backendConnector, and not inherent in PowerSync.

Characteristics of the test and backendConnector that exhibited this behavior:

• triggered by higher rates, transactions/second/client
• larger transactions involving multiple updates to more keys
  • so more likely to conflict with other client's concurrent transactions with shared keys
• key/value updates with an exponential key distribution, i.e. grind against individual keys to try and force update inconsistencies
• once the transactions/sec crossed a threshold, e.g. starting to challenge PostgreSQL's update latency
  • retries were triggered due to PostgreSQL deadlocks/concurrent updates against common keys
  • normal for all applications with multiple clients, same keys in a single transaction
• retry values were too hot, i.e. too small
• the end of test quiescence was inadequate to allow the backendConnector to complete the upload queue
  • so final reads weren't really final 😔

The backendConnector retry min/max values have been expanded and now the test explicitly waits for the upload queue == 0 during the end of test quiescence before final reads.

Still need to tune the application's backendConnector's retry logic.
For a fair(er) LWW, one almost wants a reverse exponential back-off, more like a Zeno's back-in 🙃

whether you've found powersync to be a robust system including in multi-client situations

With these changes, in no-fault/normal operation, even when overwhelming PostgreSQL, PowerSync performs well and is provably causally consistent.

The tests remain under active development:

• will update the logbook to reflect this
• will be moving on to fault injection next 😈