Quickstart

A local database that works offline and converges with your other devices when it can.

npm install tangentfeed

import { openSpace, broadcast } from "tangentfeed";

const db = await openSpace({
  space: "kitchen-42",
  transports: [broadcast()],
});

const id = await db.insert("tasks", { title: "Order rice bran oil", done: false });
await db.update("tasks", id, { done: true });

db.subscribe(async () => {
  render(await db.list("tasks"));
});

Open that page in two tabs and they sync. Add a WebRTC transport and it syncs across devices. Nothing else in your code changes.

Mental model

Five ideas cover almost everything. Reading them once will save you from surprises later.

A space is the unit of replication

A space is a named set of tables. Peers only sync with peers in the same space. One user's data, one space, is the usual mapping.

Every write is an operation

Writes append to a log rather than overwriting anything. The tables you read are a cache materialized from that log, which is why a delete writes a tombstone operation instead of removing rows.

The cell is the unit of conflict

Conflicts are resolved per (table, row, column). Two devices editing the title and the status of the same row concurrently both keep their edit. Two devices editing the same field means the later stamp wins.

Clocks are logical, not wall clocks

Each operation carries a hybrid logical clock stamp. A device whose clock is an hour slow still sorts its writes correctly relative to everything it has seen, because receiving an operation advances the local clock past it.

Convergence does not depend on delivery

Merging is commutative, associative, and idempotent. Messages can arrive out of order, twice, or much later, and every replica still reaches identical state. This is why transports are allowed to be unreliable.

Installation

The tangentfeed package pulls in the engine, IndexedDB storage, encryption, and both browser transports. Install component packages directly if you want a narrower dependency footprint.

npm install tangentfeed

npm install tangentfeed @tangentfeed/adapter-sqlite better-sqlite3

npm install tangentfeed @tangentfeed/react

Every package ships ES modules with TypeScript declarations. There is no build step or bundler plugin to configure.

Reading and writing

Writing

Rows are identified by a client-generated ULID returned from insert. There are no server-assigned identifiers, which is what makes offline creation work without reconciliation later.

const id = await db.insert("tasks", { title: "Prep sambar", done: false, station: "tiffin" });

// each key becomes one operation on one cell
await db.update("tasks", id, { done: true });

// null clears a cell; the column disappears from the row
await db.update("tasks", id, { station: null });

// delete writes a tombstone, it does not erase history
await db.delete("tasks", id);

Reading

Reads hit local storage and never wait on the network, but they are asynchronous because storage engines are. list returns visible rows sorted by row id, which is insertion order since ULIDs are time-prefixed.

const row = await db.get("tasks", id);   // undefined if absent or deleted
const rows = await db.list("tasks");     // visible rows, oldest first

Reacting to change

subscribe fires after every committed batch, local or remote. The event carries which rows changed and where the change came from, so you can re-read only what you need.

const unsubscribe = db.subscribe((event) => {
  // event.origin is "local" or "remote"
  // event.changes is [{ table, row }, ...]
  // event.ops is the operations just committed
  if (event.changes.some((c) => c.table === "tasks")) refreshTasks();
});

Transports

A transport moves opaque messages between peers. It may lose, duplicate, or reorder them. Pass as many as you like; the engine deduplicates operations, so running several at once costs nothing but bandwidth.

Broadcast, for one device

Syncs tabs and workers of the same origin through BroadcastChannel. No infrastructure, no configuration, useful in almost every browser app.

import { openSpace, broadcast } from "tangentfeed";

const db = await openSpace({ space: "kitchen-42", transports: [broadcast()] });

WebRTC, across networks

Devices connect directly over data channels. A signaling server introduces them and relays connection descriptions, then plays no further part. Kill it after two peers connect and they keep syncing.

import { openSpace, broadcast, webrtc } from "tangentfeed";

const db = await openSpace({
  space: "kitchen-42",
  transports: [
    broadcast(),
    webrtc({
      signaling: "wss://sync.example.com",
      iceServers: [
        { urls: "stun:stun.l.google.com:19302" },
        { urls: "turn:turn.example.com", username: "u", credential: "p" },
      ],
      onSignalingState: (state) => showStatus(state),
    }),
  ],
});

QR pairing, with no server

Two devices exchange connection blobs by camera or copy and paste. The offering device mints the space id, the answering device adopts it. Everything after the handshake is identical to any other transport.

import { openSpace, manualPair, existing, generateDeviceId } from "tangentfeed";

const deviceId = generateDeviceId();
const pair = manualPair({ deviceId, onState: (s) => showStatus(s) });

// device A: show this as a QR code
const invite = await pair.createOffer();

// device B: scan the invite, show the answer back
const answer = await pair.acceptOffer(invite);

// device A: scan the answer, and the channel opens
await pair.acceptAnswer(answer);

const db = await openSpace({
  space: pair.space,
  deviceId,
  transports: [existing(pair)],
});

Storage adapters

Storage is chosen per replica and never affects the protocol. Replicas on different engines sync with each other and converge to identical state, which the test suite verifies directly.

SQLite

Construct the adapter yourself so the SQLite binding stays your choice, and so browser bundles never pull in a native dependency.

import Database from "better-sqlite3";
import { SqliteAdapter, betterSqliteDriver } from "@tangentfeed/adapter-sqlite";

const db = await openSpace({
  space: "kitchen-42",
  storage: SqliteAdapter.open(betterSqliteDriver(new Database("tasks.db"))),
});

The result is an ordinary SQLite file, readable while the app runs:

sqlite3 tasks.db "SELECT table_name, column_name, value FROM ops ORDER BY id;"

Drivers ship for better-sqlite3 and node:sqlite. Bun and Expo bindings fit the same four-method shape.

Writing an adapter

An adapter stores operations, stores the winning operation per cell, persists the frontier and clock, and applies batches atomically. That last requirement is not optional: a crash midway through a batch must leave the log and the materialized state agreeing.

Encryption

Cell values are encrypted with XChaCha20-Poly1305 before they enter the log, so storage, transports, and relays only ever hold ciphertext. Every peer in the space needs the same secret.

// derived from a passphrase, salted with the space id
encryption: { passphrase: "correct horse battery staple" }

// or supply 32 random bytes shared out of band
encryption: { secret: keyBytes }

What is protected

Values, and only values. The ciphertext is bound to the operation id, so a ciphertext lifted from one operation and pasted into another fails authentication rather than silently relocating data.

What is not

Table, row, and column names travel in the clear, along with timestamps and device identifiers. A relay learns the shape and timing of your activity, never its content. Row tombstones are also plaintext by design, so that peers without the key can still order deletes correctly.

Compaction

The log grows with every write, so superseded operations need reclaiming. Compaction is safe by construction: it never drops an operation that a known peer has not yet received.

const stats = await db.compact();
// { removed: 10000, rowsReclaimed: 0, blockedBy: [] }

// see what would happen without touching storage
await db.compact({ dryRun: true });

Reclamation is bounded by a horizon: the lowest point every known peer has acknowledged. One long-absent peer therefore blocks reclamation, which is why blockedBy names the peers responsible instead of leaving you guessing why nothing happened.

Deleted rows

Tombstones are kept by default. Reclaiming them early would let a peer that has been away reintroduce a deleted row, so it requires an explicit opt-in and happens only when the horizon has passed every operation belonging to the row.

await db.compact({ includeTombstones: true });

A reasonable schedule is on startup and then hourly while the app is open. It is cheap and safe to call often.

Signatures

Every operation carries an Ed25519 signature, and a device's identity is derived from its public key rather than chosen: deviceId is the first 16 bytes of SHA-256(publicKey). A device therefore cannot claim an identity it has no key for, and an operation cannot be forged or altered after the fact.

None of this needs configuration. A keypair is generated and persisted the first time a space is opened, and peers exchange public keys when they connect. Unsigned operations are rejected everywhere; there is no mode that accepts them, because a mode that did would offer no protection.

Signing happens after encryption, not before. Section 7 requires a peer without the key to merge and forward correctly, so the signature covers the ciphertext; signing the plaintext would make signatures unverifiable by exactly the peers that design exists to support.

This is protocol v0.2, and it is not wire-compatible with v0.1: operations gained a required signature field and deviceId widened from 64 to 128 bits, so a v0.1 and a v0.2 peer abort when they meet.

Typed schemas

The data API accepts any table name and any JSON, which is flexible and easy to get wrong. A schema declares the shape once: TypeScript infers the row types from it, and local writes are checked against it before they become operations.

import { s, defineSchema } from "@tangentfeed/schema";

const schema = defineSchema({
  tasks: {
    title:   s.string(),
    done:    s.boolean().default(false),
    station: s.string().optional(),
    tags:    s.array(s.string()).default([]),
  },
});

const db = await openSpace({ space: "kitchen-42", schema });

// done and tags are filled in; station may be omitted
await db.insert("tasks", { title: "Prep sambar" });

// compile error, and throws at runtime
await db.insert("tasks", { titel: "typo" });

const rows = await db.list("tasks");
// { id, title, done, station?, tags }[]

Field types are string, number, boolean, array, object and enum, each with .optional(), .nullable() and .default(value). A defaulted column may be omitted on insert but is always present on read, because the default is written as a real cell. An optional column may be absent from both. .nullable() widens the value rather than the presence, so you have to say when a null is meaningful rather than merely missing.

update takes a partial and never applies defaults. It writes individual cells, and inventing a default there would overwrite a value a peer had set.

Only local writes are checked

Validation runs in insert and update, before operations are generated. Rejected data never enters the log, which is what keeps a schema from affecting convergence: a peer running a different schema still syncs with you completely, and you simply cannot author rows you consider invalid.

Data arriving from peers is never inspected. Filtering reads through a local schema would make the visible state depend on which schema version a replica happens to be running, and two peers with identical logs would disagree about what they contain.

import { parseRow } from "@tangentfeed/schema";

const row = await db.get("tasks", id);
const checked = parseRow(schema.tasks, row);

if (!checked.ok) console.warn(checked.issues);

Objects are validated inside but remain one cell, matching how the merge already works: concurrent edits to different keys of the same object do not merge separately. Migrations are not included. Versioning a schema while peers still hold rows written under an older one is a genuinely hard distributed problem, and it is deliberately out of scope rather than half-solved.

React

The hooks subscribe to the engine and re-read only the slice that changed. Reads are local, so first paint arrives quickly, but they are still asynchronous and the hooks report a loading state for that first pass.

import { useSpace, useRows, useTable } from "@tangentfeed/react";
import { broadcast } from "tangentfeed";

function Tasks() {
  const db = useSpace({ space: "kitchen-42", transports: [broadcast()] });
  const { rows, loading } = useRows(db, "tasks");
  const { insert, update, remove } = useTable(db, "tasks");

  if (loading) return <p>Loading</p>;

  return (
    <ul>
      {rows.map((task) => (
        <li key={task.id}>
          <input
            type="checkbox"
            checked={task.done === true}
            onChange={(e) => update(task.id, { done: e.target.checked })}
          />
          {String(task.title)}
        </li>
      ))}
    </ul>
  );
}

Every hook is generic over the schema, so passing one to useSpace carries the types through the rest. Rows arrive typed and the mutation helpers take the declared shape, which removes the defensive String(task.title) and task.done === true in the example above. See Typed schemas.

API

openSpace(options)

The space object

Passing a schema narrows these: table names are checked, insert and update take the declared shape, and get and list return the inferred row type. Without one they keep the untyped signatures above.

Packages

Protocol

The protocol is specified independently of this implementation, so a client in another language can interoperate. What follows is an orientation; the specification itself is the authority.

Operation

Clock encoding

A stamp is millis-counter-device in fixed-width lowercase hex, 34 characters total. Plain string comparison equals logical ordering, which means storage engines can sort stamps without understanding them.

Drift protection

An operation stamped more than five minutes ahead of the receiving clock is rejected outright. Accepting it would let one broken clock win every conflict far into the future.

Sync session

Peers exchange a hello with their clocks, then frontiers, then the operations the other side is missing, then acknowledgements. After catch-up they forward new operations as they happen. The exchange repeats on every reconnect, which is how gaps heal.

Conformance

The suite is language-neutral JSON: a batch of operations plus the exact state and frontier that must result. An implementation is conforming when it produces identical results from every vector applied in each of these orders.

1. As given
2. Reversed
3. At least two independent shuffles
4. Shuffled with every operation duplicated
5. One operation at a time, shuffled

Passing all five is the practical expression of the core guarantee: delivery order, duplication, and batching cannot change the outcome. The reference implementation runs this matrix against both its storage engines.

Deployment

Signaling server

Stateless, small, and unable to read your data. It tracks who is present in a space and relays connection blobs, nothing else. Restart it freely.

npx @tangentfeed/signaling-server   # listens on :8787

Serve it behind TLS as wss:// in production. If you run several instances, route peers in the same space to the same instance, since presence is per instance.

An always-on replica

Direct peer sync needs both devices open at once. A small Node process holding a replica removes that constraint: each device syncs with it whenever convenient, and it is a backup as a side effect. It has no special authority, it just has good uptime.

import Database from "better-sqlite3";
import { SqliteAdapter, betterSqliteDriver } from "@tangentfeed/adapter-sqlite";
import { openSpace, webrtc } from "tangentfeed";

const db = await openSpace({
  space: process.env.SPACE,
  storage: SqliteAdapter.open(betterSqliteDriver(new Database("replica.db"))),
  transports: [webrtc({ signaling: process.env.SIGNALING })],
  encryption: { secret: keyBytes },  // optional: it holds only ciphertext
});

setInterval(() => db.compact(), 3600_000);

Hosting the app

The client is static files. Anything that serves HTML works. Serve over HTTPS so camera pairing and clipboard actions are available.

Recipes

Give a device a stable identity

Persist the id somewhere a duplicated tab will not inherit, which rules out sessionStorage.

import { generateDeviceId } from "tangentfeed";

function deviceId() {
  let id = localStorage.getItem("device-id");
  if (!id) {
    id = generateDeviceId();
    localStorage.setItem("device-id", id);
  }
  return id;
}

Share a space by link

A space id plus a secret is everything needed to join. Put the secret in the URL fragment, which browsers do not send to servers.

// https://app.example.com/#space=kitchen-42&key=BASE64KEY
const params = new URLSearchParams(location.hash.slice(1));

const db = await openSpace({
  space: params.get("space"),
  encryption: { secret: decodeKey(params.get("key")) },
  transports: [broadcast(), webrtc({ signaling: SIGNALING })],
});

Show sync status honestly

People tolerate being offline. They do not tolerate not knowing. Surface peer count and signaling state rather than a spinner.

Model rows for merging

Fields that get edited independently belong in separate columns, because merging happens per cell. A title and a status merge cleanly. The same two values nested inside one object do not, since the object is a single cell.

Troubleshooting

Peers stay at zero

Check signaling state first. If it never reaches connected, the server or URL is wrong. If it connects but no peers appear, the two clients are probably in different spaces. If peers appear but channels never open, the network is blocking direct traffic, which is common on guest Wi-Fi. Test on a phone hotspot to confirm, and add TURN to fix it properly.

Two tabs never see each other

Broadcast requires a real origin, so pages opened from the file system are isolated. Serve over http or https. If both tabs share a device id, one evicts the other; give each a distinct id.

Clock drift errors

A device's clock is more than five minutes off. Fix the system time. The protection is deliberate: accepting the operation would poison conflict resolution far into the future.

Compaction reclaims nothing

Read blockedBy. A peer that has been away pins the horizon, and nothing below it can be dropped until that peer catches up or you stop tracking it.

Storage keeps growing

Call compact() periodically. Without it the log keeps every historical write forever.

Status and limits

Version 0.2, published on npm. The conformance vectors pin the protocol's behaviour down and two independent implementations pass them, but the wire format is not frozen until 1.0 — v0.2 already broke compatibility with v0.1.

Known limits

• No membership, roles or revocation. A space name is effectively a bearer credential.
• The signaling server has no authentication, rate limiting or payload cap. Do not expose it to the open internet.
• Cross-network sync is unproven: every test so far had both peers on one local network. Traversing NAT generally needs a TURN relay.
• A space syncs whole. There is no partial replication or per-row access control.
• Values merge whole, so there are no collaborative text or ordered-list types yet.
• Key rotation requires rewriting data rather than a configuration change.
• Peers must overlap in time to sync directly. An always-on replica is the answer today.

Shipped since 0.1

• Signed operations, with identity derived from a keypair rather than chosen.
• A typed schema layer that infers TypeScript types and validates local writes.
• A second implementation in Dart, validated by the same conformance vectors.
• Conformance coverage for every normative rule, compaction included.

Planned

• Membership, roles and revocation — the missing half of write authorization.
• Store-and-forward mailboxes, so peers that never overlap still converge.
• React Native adapters.