Skip to content

Consumers, leases and workers

Every capability that reacts, projects or owns a sent command is a consumer of the outbox with a durable cursor. Delivery is exactly once per consumer: a reaction’s effects commit together with the advance of that capability’s cursor, so a crash or a second process can never apply it twice.

A capability’s cursor is named by consumer (default: its name). Rename a capability and keep its old name as consumer: 'store-requests': delivery resumes instead of replaying every fact. Two capabilities never share one (ATLAS026, and the runtime refuses to start). To split one, copy the cursor before deploying the new capability:

Terminal window
atlas cursor copy store-requests store-reviews --db <url>

On its first delivery the runtime traces a warning (consumer-starts-at-zero) when a consumer at 0 would replay a non-empty outbox while the store holds cursors no capability owns.

  • A rejected or failed reaction (including a decide that throws) advances and is traced.
  • A transient store error does not advance; the worker retries with backoff.
  • A refusal the database would repeat for the same bytes (poison) settles as failed with store-rejected; its projected rows still commit with the advance and the cursor passes the entry.
  • An entry deeper than maxDepth settles as failed with cascade-exceeded.

Decisions are pure, so I/O after a fact (a push, an email, a webhook) runs at the edge, in a subscriber with its own durable cursor, delivered by the same workers:

runtime.subscribe('push-delivery', ['PushRequested'], async (fact) => { // before runtime.start(); drain() delivers it too
await sendPush(fact.payload, { idempotencyKey: fact.id }); // fact = { id, type, payload (latest version), source, v }
});
Rule Why
At least once. The cursor advances past a fact only after the handler resolves; a throw keeps it there and workers back off (subscriber-failed). A side effect cannot commit with the cursor.
Idempotency is the handler’s, keyed by fact.id. A crash between the side effect and the advance delivers the fact again.
id is a delivery identity; types must be emitted by some capability; register before start(). Typos and late registration fail at boot.
A new subscriber starts at 0. On a database with history, atlas cursor copy <consumer> <id> --db <url>. Same rule as a new capability.
Not for business rules: write results back through commands. The map, check and examples cover capabilities only.

Any number of processes may share one database. Consumer leases make each consumer (every capability and every edge subscriber) run in one process at a time:

const runtime = createRuntime({ capabilities, store: createPostgresStore(pool), instance: process.env.HOSTNAME }); // default: a random UUID
const workers = runtime.start({ leaseMs: 30_000, renewMs: 10_000 }); // the defaults
const api = runtime.start({ workers: false }); // or: an API-only instance (no consumer, no timer loop)
  • A worker loop delivers a consumer only while its instance holds that consumer’s lease in atlas_leases (consumer, owner, expires_at), renewed every renewMs.
  • Fair share (since 0.5.0, with a store that implements holders): each instance holds at most ceil(consumers / instances) and hands the excess over within a renewMs.
  • stop() finishes the entry in flight, then releases its leases. A process that dies keeps nothing: its lease expires after leaseMs.
  • renewMs must be shorter than leaseMs (start throws otherwise). The lease clock is the database’s.
  • An edge handler that outlives its lease may run twice: dedupe by fact.id stays the contract.
  • drain() ignores leases. Do not call it in production next to running workers.
  • One outbox check per instance every pollMs (default 250), backing off to idleMs (default 1000) while nothing changes; it releases due timers and wakes only the consumers that take the new types.
  • A consumer holding its lease keeps its cursor in memory; entries it does not take are skipped without a query, and the cursor catches up within 5 s or on stop().
  • Consecutive entries commit in one transaction with one cursor advance: delivery: { projections: 100, reactions: 20, subscribers: 10 } (the defaults). A batch that conflicts or is refused falls back to one entry at a time.
  • Delivery is sequential per consumer: batching buys round trips, not parallelism.

See the production guide for pools, retries and deploy layouts.