Skip to content

Concurrent local appends crash the apply pipeline (null batch) under live replication #88

@cayasso

Description

@cayasso

Environment:
autobee 1.0.9 · Node v22.22.2

The Problem:
When a writer makes two concurrent (un-awaited) append() calls while another writer is appending and replicating live, autobee crashes in the apply pipeline:

  TypeError: Cannot read properties of null (reading 'start')
      at getLocalBatch (lib/topo.js:228)
      at Object.sort (lib/topo.js:75)
      at Autobee.prepareBatch (index.js:636)
      at Autobee._processBatch (index.js:646)
      at Autobee._bumpPendingWriters (index.js:575)
      at Autobee._drain (index.js:389)

Expected result:
Concurrent append() calls on one writer should be queued/serialized safely, not crash the drain.

Failing test case

const test = require('brittle')
const { create, encode, apply, replicate, replicateAndSync } = require('./helpers')

// Two writers replicating live. While b appends continuously, a issues TWO
// concurrent (un-awaited) local appends. autobee crashes in the apply pipeline:
//   TypeError: Cannot read properties of null (reading 'start')  (lib/topo.js:230)
// The node's `batch` field (stamped only later, in flush()/next()) is still null
// when getLocalBatch reads it.
test('concurrent local appends under live replication crash the apply pipeline', async (t) => {
  let crash = null
  const onError = (e) => { crash = crash || e }

  const a = await create(t, null, { apply })
  const b = await create(t, a.key, { apply })
  a.on('error', onError)
  b.on('error', onError)

  await a.append(encode({ addWriter: b.local.id }))
  await replicateAndSync(a, b)
  const tear = replicate(a, b)

  let k = 0
  const pump = setInterval(() => b.append(encode({ v: k++ })).catch(onError), 0)
  await Promise.all([a.append(encode({ v: 'x' })), a.append(encode({ v: 'y' }))])
  await new Promise((r) => setTimeout(r, 100))

  clearInterval(pump)
  await tear()
  t.absent(crash, crash && crash.message)
})

All three conditions are required, removing any one passes: (1) live replication, (2) the other writer appending, (3) two concurrent appends on a.

Root cause (according to Claude)
A fresh node is created with batch: null (createNode copies the null batch arg of a plain append — lib/writers.js:527). node.batch is only stamped with { start, end } later, in a separate pass (flush() writers.js:506 / next() writers.js:406). Under concurrent appends, the apply pipeline reaches getLocalBatch and reads head.batch.start (lib/topo.js:228) on a pending node before that stamping pass runs → null.start.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions