Commit Graph

236 Commits

Author SHA1 Message Date
Philipinho eea989260a test(base): filter/sort parity matrix against postgres
Integration spec that seeds a 10K-row base with diverse property shapes
(text, number, date, checkbox, select, multi-select) and runs an
exhaustive matrix of filter/sort combinations against both the DuckDB
cache path and the Postgres-direct path. Asserts identical row ids and,
where semantics allow, identical cursor strings and pagination meta.

The suite is gated by INTEGRATION_DB_URL and skips cleanly without it.
34 tests total: 26 flat filter ops (text/number/date/checkbox/select/
multi-select), 4 nested boolean trees (AND/OR/mixed/max-depth), 3
multi-key sorts, and one full filter+sort+pagination walk.

Seed tuning to keep both engines in lock-step:
  * digit-only row positions so PG default collation and DuckDB
    bytewise collation agree on the tail tiebreak.
  * lowercase name pool so mixed-case locale/bytewise divergence
    doesn't surface on text-secondary sort.
  * priority is non-NULL to avoid the PG keyset stall when a boundary
    cursor encodes the '+/-Infinity' numeric sentinel (postgres.js
    parses it as NaN, which applyCursor re-emits as null).
2026-04-23 05:00:17 +01:00
Philipinho fc08cffd37 test(server): init LRU test module so pg extension bootstraps 2026-04-23 04:40:30 +01:00
Philipinho fde0ccb3c7 refactor(base): replace streaming loader with pg-extension CREATE TABLE AS SELECT 2026-04-23 04:28:25 +01:00
Philipinho e663d7eecf test(server): align integration stubs with new config + pg-extension injection 2026-04-23 04:28:21 +01:00
Philipinho 96e875f1de test(base): tighten loader-sql mapping assertions to full projections 2026-04-23 03:37:23 +01:00
Philipinho 6544ff6d38 feat(base): pure SQL builder for pg-extension loader 2026-04-23 03:31:00 +01:00
Philipinho 7ca712c9ab fix(base): propagate pg-extension bootstrap failure reason; align closeSync style 2026-04-23 03:26:41 +01:00
Philipinho a798397af0 feat(base): postgres extension service with bootstrap install + per-connection attach 2026-04-23 03:17:36 +01:00
Philipinho 9ba6459427 feat(base): env vars for per-instance duckdb memory limit + threads 2026-04-23 03:09:58 +01:00
Philipinho 14827ec6a0 test(server): add getBaseQueryCacheDebug to integration test env stubs 2026-04-19 23:41:06 +01:00
Philipinho c931fa5ec9 perf(server): skip per-request row count when collection is resident 2026-04-19 23:39:27 +01:00
Philipinho 7e07d77510 chore(server): add per-request perf logs for base query cache diagnostics 2026-04-19 22:44:39 +01:00
Philipinho 55feb01249 test(server): assert duckdb cache matches postgres on a 100K-row base 2026-04-19 22:28:07 +01:00
Philipinho 4636af3870 feat(server): warm duckdb collections on boot from redis recent-access set 2026-04-19 22:16:20 +01:00
Philipinho c9adf84260 feat(server): evict least-recently-used duckdb collections when cap exceeded 2026-04-19 22:11:55 +01:00
Philipinho 4f38c61725 fix(server): avoid acquiring redis client when base query cache is disabled 2026-04-19 22:05:56 +01:00
Philipinho df22efb290 feat(server): propagate row mutations to duckdb cache via redis pubsub 2026-04-19 22:00:37 +01:00
Philipinho 7534b44e6e refactor(server): preserve cache-failure stack trace and reuse hasSearch 2026-04-19 21:50:34 +01:00
Philipinho cf6b48cd58 feat(server): route large base list queries through the duckdb cache 2026-04-19 21:46:27 +01:00
Philipinho 45000bbd8b fix(server): close duckdb resources on load failure, dedupe concurrent loads, drop unused cells projection 2026-04-19 21:39:05 +01:00
Philipinho 91ad3de258 feat(server): load bases into DuckDB and serve list queries from cache
- collection-loader streams base rows via postgres and bulk-inserts into an
  in-memory DuckDB instance using the Appender API, then builds an index on
  each indexable column
- base-query-cache service routes list() calls through the prepared-statement
  path; ensureLoaded does schema-version checks with single-pass LRU eviction
- keyset param-ordering bug in the DuckDB builder fixed: placeholders appear
  head-to-tail but were being pushed tail-to-head, which made DuckDB bind the
  wrong value for each ? and throw Binder Error on typed columns
- base-row repo gains countActiveRows for the router to use in task 6
- seed script split into an importable helper so integration tests can seed a
  10k-row base deterministically without shelling out
- new integration spec compares Postgres vs DuckDB pagination end-to-end for
  a numeric sort and guards against duplicate rows from DuckDB

Integration test is skipped unless INTEGRATION_DB_URL is set.
2026-04-19 21:31:05 +01:00
Philipinho b28597125d fix(server): use DuckDB json_contains for multi-select filters and expand builder coverage 2026-04-19 21:11:29 +01:00
Philipinho a9db3ef008 feat(server): add DuckDB SQL builder for base list queries 2026-04-19 21:06:41 +01:00
Philipinho 574c5316f0 feat(server): scaffold base query-cache module behind feature flag 2026-04-19 20:59:24 +01:00
Philipinho 3af2db7a8b feat(server): add property-type to DuckDB column-spec mapping 2026-04-19 20:54:59 +01:00
Philipinho 95d0457a7e refactor(base): drop /list suffix from base endpoints to match codebase convention 2026-04-18 23:36:52 +01:00
Philipinho f119d728a8 fix(base): handle csv export client abort and mid-stream errors 2026-04-18 18:18:34 +01:00
Philipinho 66f9194e96 feat(base): add csv export http endpoint 2026-04-18 18:14:41 +01:00
Philipinho 19b3f26cbb feat(base): register csv export service in module 2026-04-18 18:14:01 +01:00
Philipinho 56c57afff3 feat(base): add streaming csv export service 2026-04-18 18:13:20 +01:00
Philipinho d84aadadbb feat(base): add export base csv dto 2026-04-18 18:11:34 +01:00
Philipinho da0321b468 feat(base): add csv cell serializer with per-type rules 2026-04-18 18:10:47 +01:00
Philipinho e017209d76 feat(base): emit base:rows:deleted websocket event 2026-04-18 16:32:27 +01:00
Philipinho fc734475df feat(base): add POST /bases/rows/delete-many endpoint 2026-04-18 16:31:44 +01:00
Philipinho a7f9d66778 feat(base): add deleteMany service method for batch row delete 2026-04-18 16:31:11 +01:00
Philipinho 4a9e891582 feat(base): add BASE_ROWS_DELETED event type 2026-04-18 16:29:26 +01:00
Philipinho 65c5bb11b8 feat(base): add DeleteRowsDto for batch row delete 2026-04-18 16:29:02 +01:00
Philipinho ede1a799f2 feat(base): disable type-conversion API for v1, preserve engine for v2 2026-04-18 14:13:08 +01:00
Philipinho f5b19316af Base WIP 2026-04-18 13:13:53 +01:00
Philipinho 081bb67239 Merge branch 'main' into base 2026-04-17 13:48:49 +01:00
Philip Okugbe a6a7e4370a feat(ee): PDF export api (#2112)
* feat(ee): server side PDF export

* feat: pdf export queue

* sync

* sync
2026-04-14 16:26:54 +01:00
Philip Okugbe cc00e77dfb fix: space overview favorites (#2110) 2026-04-14 02:58:24 +01:00
Philip Okugbe 4056bd0104 feat: enhancements (#2107)
* refactor
* fix
* update packages
2026-04-13 23:34:40 +01:00
Philip Okugbe bd68e47e03 feat(ee): page verification workflow (#2102)
* feat: page verification workflow

* feat: refactor page-verification

* sync

* fix type

* fix

* fix

* notification icon

* use full word

* accept .license file

* - update templates
- update migration and notification

* fix copy

* update audit labels

* sync

* add space name
2026-04-13 20:20:34 +01:00
Philip Okugbe d42091ccb1 feat: favorites (#2103)
* feat: favorites and templates(ee)

* rename migrations

* fix sidebar

* cleanup tabs

* fix

* turn off templates

* cleanup

* uuid validation
2026-04-12 22:06:25 +01:00
Philip Okugbe 57efb91bd3 feat(ee): ai chat (#2098)
* feat: ai chat

* feat: ai chat

* sync

* cleanup

* view space button
2026-04-10 19:23:47 +01:00
Philip Okugbe da9b43681e feat: watch space (#2096) 2026-04-09 00:37:51 +01:00
Philip Okugbe 895c1817ae feat: bug fixes (#2084)
* handle enter in inline code

* fix: duplicate comment cache

* track link nodes (backlinks)

* fix en-US translation

* fix internal a-links

* overrides

* 0.71.1
2026-04-05 13:45:36 +01:00
Philip Okugbe 879aa2c3d8 feat: page update notifications (#2074)
* feat: watchers notification and email preferences

* fix: email copy

* digests

* clean up

* fix

* clean up

* move backlinks queue-up to history processor

* fix

* fix keys

* feat: group notifications

* filter

* adjust email digest window
2026-03-31 16:03:59 +01:00
Philip Okugbe c180d0e487 feat: ratelimits (#2073)
* feat: rate limits

* ip
2026-03-30 15:38:44 +01:00