Skip to content

feat: add WAL extension and logical decoding startup config#1

Open
sorenbs wants to merge 1 commit intomainfrom
wal
Open

feat: add WAL extension and logical decoding startup config#1
sorenbs wants to merge 1 commit intomainfrom
wal

Conversation

@sorenbs
Copy link
Member

@sorenbs sorenbs commented Feb 17, 2026

Summary

This PR adds a small, ppg-dev-focused WAL API to PGlite using built-in PostgreSQL logical decoding (pgoutput) and SQL decoding functions.

This is part of a temporary Prisma fork strategy to unblock ppg-dev while we prepare upstream PRs.

What we are building (ppg-dev scoped)

A minimal PGlite extension namespace with:

  • wal.ensure({ slot, publication, tables? })
  • wal.poll({ slot, publication, maxChanges?, protoVersion? }) -> WalEvent[]

And a pgoutput decoder that emits typed events:

  • begin
  • commit
  • relation
  • insert
  • update
  • delete

Why this approach

For ppg-dev we want the simplest reliable path:

  • Use PostgreSQL standard pgoutput format.
  • Use SQL logical decoding functions (pg_create_logical_replication_slot, pg_logical_slot_get_binary_changes) instead of implementing replication protocol streaming.
  • Avoid idle/background polling: ppg-dev polls only immediately after writes it already observes.

Changes in this PR

1) Startup Postgres config support in PGlite

  • Added postgresConfig?: Record<string, string | number | boolean> to PGliteOptions.
  • Startup args now append -c key=value entries from postgresConfig.

Files:

  • packages/pglite/src/interface.ts
  • packages/pglite/src/pglite.ts

2) WAL extension implementation

  • Added new extension module implementing:
    • ensure: validates wal_level=logical, ensures publication and logical replication slot (pgoutput plugin).
    • poll: fetches binary changes and decodes pgoutput into typed events.
  • Includes pgoutput binary decoder and relation cache handling.

File:

  • packages/pglite/src/wal/index.ts

3) Public exports

  • Exported wal extension and WalEvent type from package index.

File:

  • packages/pglite/src/index.ts

4) Test coverage

Added comprehensive coverage for:

  • postgresConfig startup behavior (wal_level, replication limits)
  • ensure behavior and guardrails
  • insert/update/delete decoding via logical decoding SQL path
  • table-scoped publication behavior
  • decoder strictness/error paths and tuple mapping edge cases

File:

  • packages/pglite/tests/wal.test.ts

5) Test config aliasing for local pg-protocol source resolution

File:

  • packages/pglite/vitest.config.ts

Validation

  • Ran full packages/pglite test suite successfully.
  • WAL-specific tests pass with logical slot creation + change polling + pgoutput decoding.

ppg-dev usage pattern this enables

For each write query:

  1. Execute write on PGlite.
  2. Call db.wal.poll(...) immediately.
  3. Broadcast/apply returned events.

No LISTEN/NOTIFY or idle polling loop is required for this scoped use case.

Temporary fork publishing strategy

This branch is intended to be published as a temporary Prisma-owned package (or npm alias target) so ppg-dev can depend on deterministic builds while upstream work proceeds.

Upstream merge plan

  1. Upstream runtime fixes first in electric-sql/postgres-pglite (paired repo PR).
  2. Upstream postgresConfig option and WAL extension in electric-sql/pglite with this same minimal scope.
  3. Keep follow-up items separate from initial merge:
    • richer batching semantics
    • optional expanded tuple behavior based on replica identity
    • any additional filtering ergonomics
  4. After upstream releases include these changes, switch ppg-dev dependency back to upstream package and drop fork override.

Cross-repo dependency

This PR depends on the wasm runtime/linking fixes in:

  • prisma/postgres-pglite branch wal

(Companion PR link will be attached in comments.)

Companion PR: prisma/postgres-pglite#1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Comments