The plpgsql + EXCEPTION versions of base_cell_numeric,
base_cell_timestamptz, and base_cell_bool were labeled PARALLEL SAFE
but EXCEPTION blocks require subtransactions, which Postgres cannot
start in a parallel worker. Any parallel scan that invoked them
crashed with 'cannot start subtransactions during a parallel
operation' — notably DuckDB's postgres extension on large base COPY
reads.
Rewrite each as a pure SQL function using jsonb_typeof + regex
validation for the 'coerce-or-null' semantics. No plpgsql, no
subtransactions, genuinely parallel-safe. Signatures unchanged so
existing call sites (loader, expression indexes, engine predicates)
are untouched.
Integration spec that seeds a 10K-row base with diverse property shapes
(text, number, date, checkbox, select, multi-select) and runs an
exhaustive matrix of filter/sort combinations against both the DuckDB
cache path and the Postgres-direct path. Asserts identical row ids and,
where semantics allow, identical cursor strings and pagination meta.
The suite is gated by INTEGRATION_DB_URL and skips cleanly without it.
34 tests total: 26 flat filter ops (text/number/date/checkbox/select/
multi-select), 4 nested boolean trees (AND/OR/mixed/max-depth), 3
multi-key sorts, and one full filter+sort+pagination walk.
Seed tuning to keep both engines in lock-step:
* digit-only row positions so PG default collation and DuckDB
bytewise collation agree on the tail tiebreak.
* lowercase name pool so mixed-case locale/bytewise divergence
doesn't surface on text-secondary sort.
* priority is non-NULL to avoid the PG keyset stall when a boundary
cursor encodes the '+/-Infinity' numeric sentinel (postgres.js
parses it as NaN, which applyCursor re-emits as null).
- collection-loader streams base rows via postgres and bulk-inserts into an
in-memory DuckDB instance using the Appender API, then builds an index on
each indexable column
- base-query-cache service routes list() calls through the prepared-statement
path; ensureLoaded does schema-version checks with single-pass LRU eviction
- keyset param-ordering bug in the DuckDB builder fixed: placeholders appear
head-to-tail but were being pushed tail-to-head, which made DuckDB bind the
wrong value for each ? and throw Binder Error on typed columns
- base-row repo gains countActiveRows for the router to use in task 6
- seed script split into an importable helper so integration tests can seed a
10k-row base deterministically without shelling out
- new integration spec compares Postgres vs DuckDB pagination end-to-end for
a numeric sort and guards against duplicate rows from DuckDB
Integration test is skipped unless INTEGRATION_DB_URL is set.
Philipinho