docmost/docs/superpowers/plans/2026-04-19-infinite-scroll-loop-fix-v2.md

Infinite Scroll Fetch Loop Fix v2 Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Kill the pagination loop that still fires extra POST /bases/rows requests as the user scrolls continuously through a 10K+ base, even after the v1 rows-length guard.

Architecture: Replace the virtualItems-polling effect with a scroll-event-driven trigger plus a synchronous pending-fetch ref. Two independent dedup mechanisms: (a) only consider firing after a real scroll event (debounced), (b) never dispatch a second fetchNextPage until the previous one's resulting page has been committed to the cache. The previous v1 fix relied on rows.length growing after each commit, which under sustained scrolling lets react-query schedule multiple fetches before its own isFetchingNextPage observed-true state propagates to our render.

Tech Stack: React 18, @tanstack/react-virtual, @tanstack/react-query v5.

---

Background

What v1 fixed and what it missed

v1 (grid-container.tsx, commit c4d8b6c3) added lastTriggeredRowsLenRef — "don't fire again until rows.length grows past the last value we triggered on." That breaks the idle-at-bottom loop (where the effect was firing every render on the same rows.length).

It still leaks under sustained scrolling:

1. User scrolls rapidly. Effect runs. Condition holds. Ref updates to N. onFetchNextPage() dispatched.
2. react-query enqueues a fetch. But the internal state object mutations reaching our isFetchingNextPage snapshot happen across a microtask boundary; within the SAME React render commit there's a window where isFetchingNextPage is still false from React's perspective.
3. The effect dep list includes virtualItems, which virtualizer.getVirtualItems() mints fresh every render. Any other state change (scroll position, virtual measurements, React 18 batching flushes) re-fires the effect.
4. Our rows.length <= ref gate blocks further fires AT THE SAME LENGTH. But because the user is SCROLLING, as soon as a page DOES commit, rows.length jumps, and on that same commit render the effect can fire repeatedly (e.g., due to scroll-driven re-measures) because the isFetchingNextPage false window can overlap with the new render.
5. If enough fetches pile up, react-query dispatches each in sequence. Requests are only partly deduped — each call while a fetch is in flight returns the same promise, BUT if the commit has already landed and isFetchingNextPage flipped briefly to false before our render observed it, that call dispatches a NEW fetch. Net: 10× more dispatches than pages committed.

Reported symptom: on the 10K base the loop kicks in after a while of scrolling. Network panel shows 1176 requests initiated, only 124 loaded — ~10× over-dispatch.

The fix shape

Two orthogonal locks that together make over-dispatch impossible:

Lock A — synchronous pending flag. A pendingFetchRef set to true SYNCHRONOUSLY right before onFetchNextPage() is called. Cleared in a separate effect that watches isFetchingNextPage transitioning back to false (i.e., the fetch that we started has resolved). While the ref is set, the trigger never re-fires. This eliminates the same-render-double-fire window that the v1 length guard couldn't cover.

Lock B — real scroll events, not render-driven polling. Attach a listener to scrollRef's scroll event, debounced ~50 ms. On each debounced scroll, compute "am I near the bottom" from raw scrollTop + clientHeight >= scrollHeight - threshold. This removes virtualItems from the trigger path entirely — no effect is running on every render.

With both locks, the worst-case dispatch cadence is one per debounced scroll tick where you're near the bottom. Combined with Lock A's pending gate, you get at most one request in flight at a time, committing sequentially as the user scrolls.

Keep the v1 lastTriggeredRowsLenRef guard as a safety net (it's cheap and prevents re-trigger against stale row-set data).

---

File Structure

Modified:

• apps/client/src/features/base/components/grid/grid-container.tsx — replace the current trigger effect; add a pending ref + a reset effect.

No new files, no new deps, nothing server-side.

---

Task 1: Replace the trigger with a scroll-driven + pending-guarded version

File: apps/client/src/features/base/components/grid/grid-container.tsx

Step 1: Add a pending ref next to the existing lastTriggeredRowsLenRef

Inside GridContainer, right after const lastTriggeredRowsLenRef = useRef(0); (around line 70), add:

// Synchronous guard: set to true the moment we dispatch `onFetchNextPage`,
// cleared only after `isFetchingNextPage` has transitioned back to false.
// This closes the "React 18 batching / snapshot staleness" window where
// `isFetchingNextPage` from the hook is still observed false even though
// a dispatch is already in flight — which is how the v1 length-only
// guard still permits over-dispatch during sustained scrolling.
const pendingFetchRef = useRef(false);

Step 2: DELETE the existing trigger effect

Locate the effect at roughly lines 122-130:

useEffect(() => {
  if (!hasNextPage || isFetchingNextPage || !onFetchNextPage) return;
  const lastItem = virtualItems[virtualItems.length - 1];
  if (!lastItem) return;
  if (lastItem.index < rows.length - OVERSCAN * 2) return;
  if (rows.length <= lastTriggeredRowsLenRef.current) return;
  lastTriggeredRowsLenRef.current = rows.length;
  onFetchNextPage();
}, [virtualItems, rows.length, hasNextPage, isFetchingNextPage, onFetchNextPage]);

Delete it entirely. We're replacing render-polling with scroll-event-driven.

Step 3: Add a new scroll-event-driven effect in its place

Insert this (same location):

// Scroll-event-driven pagination trigger. Previously this was a
// render-effect that polled `virtualItems` — which runs on every render
// (virtualItems has fresh identity each call) and over-dispatches when
// React's `isFetchingNextPage` snapshot is stale relative to react-query's
// in-flight state. A plain scroll event with a small debounce and a
// synchronous pending ref fires at most once per scroll pulse AND
// at most one in-flight request at a time.
useEffect(() => {
  const el = scrollRef.current;
  if (!el) return;

  const NEAR_BOTTOM_PX = ROW_HEIGHT * OVERSCAN * 2;
  let debounceTimer: ReturnType<typeof setTimeout> | null = null;

  const maybeFetch = () => {
    if (!onFetchNextPage) return;
    if (!hasNextPage) return;
    if (isFetchingNextPage) return;
    if (pendingFetchRef.current) return;
    const node = scrollRef.current;
    if (!node) return;
    const distanceFromBottom =
      node.scrollHeight - (node.scrollTop + node.clientHeight);
    if (distanceFromBottom > NEAR_BOTTOM_PX) return;
    if (rows.length <= lastTriggeredRowsLenRef.current) return;

    pendingFetchRef.current = true;
    lastTriggeredRowsLenRef.current = rows.length;
    onFetchNextPage();
  };

  const onScroll = () => {
    if (debounceTimer) clearTimeout(debounceTimer);
    debounceTimer = setTimeout(maybeFetch, 50);
  };

  // Also evaluate once on mount / when deps change — covers the case
  // where the user hasn't scrolled yet but the viewport is already
  // past the near-bottom threshold (e.g. first page is short).
  maybeFetch();

  el.addEventListener("scroll", onScroll, { passive: true });
  return () => {
    el.removeEventListener("scroll", onScroll);
    if (debounceTimer) clearTimeout(debounceTimer);
  };
}, [rows.length, hasNextPage, isFetchingNextPage, onFetchNextPage]);

Notes the implementer must not change:

• The maybeFetch() call BEFORE adding the scroll listener is intentional — it handles "viewport already past threshold on mount/commit" without requiring a scroll.
• NEAR_BOTTOM_PX = ROW_HEIGHT * OVERSCAN * 2 keeps the trigger threshold equivalent to the old lastItem.index >= rows.length - OVERSCAN * 2 rule (20 rows * 36 px = 720 px).
• pendingFetchRef.current = true is set BEFORE onFetchNextPage() so a synchronous re-entry can't slip through.
• passive: true on the listener is performance-critical on large lists.

Step 4: Add an effect that clears pendingFetchRef when a fetch resolves

Immediately after the effect from Step 3, add:

useEffect(() => {
  if (!isFetchingNextPage) {
    // react-query's fetch we triggered has resolved (data committed +
    // isFetchingNextPage back to false). Release the pending gate.
    pendingFetchRef.current = false;
  }
}, [isFetchingNextPage]);

This is the counterpart to Step 3's pendingFetchRef.current = true. The flag lifecycle:

• false initially
• set true synchronously just before onFetchNextPage()
• set back to false as soon as isFetchingNextPage observed goes back to false

Between those, no new dispatch is allowed.

Step 5: Keep the existing reset effect

The effect at roughly lines 132-140 that resets lastTriggeredRowsLenRef to 0 when rows.length drops (view/filter/sort switch) must stay as-is. Do NOT delete it.

Step 6: Build

pnpm nx run client:build

Expected: success.

Step 7: Commit

git add apps/client/src/features/base/components/grid/grid-container.tsx
git commit -m "fix(base): drive pagination from scroll events with in-flight gate to kill dispatch loop"

---

Task 2: USER smoke test

⚠️ Do not run pnpm dev as an agent. Hand off.

On the 10K base (should also work on 100K):

• Rapid continuous scroll to bottom.

  1. DevTools → Network → filter to Fetch/XHR → clear.
  2. Scroll the scrollbar smoothly from top to bottom of the 10K base, no pauses.
  3. Expected: roughly one POST /bases/rows per page (~100 total for 10K rows / 100 per page). NO "n requests in flight with only k loaded" state — completed count should track initiated count closely.

• Idle at bottom for 30 s.

  1. After reaching the bottom, wait.
  2. Zero additional requests fire.

• Scroll up and back down.

  1. Scroll up to row 5000, then back to bottom.
  2. No refetch of already-cached pages; only new pages (if any remain) fire.

• Sort change mid-scroll.

  1. While at row 5000, change the sort from Title-asc to Title-desc.
  2. Pagination resets cleanly. Fetches the first page of the new sort order; scrolling continues to fetch normally.

• Filter that narrows to few hundred rows.

  1. Apply a filter producing ~200 matches.
  2. Scroll to bottom. Exactly ~2 fetches (first page was already loaded + next one). Stops cleanly.

• Regression: unsorted base.

  1. Remove sort/filter. Scroll through. Fetches still fire correctly.

• Regression: create row at top of scroll.

  1. Scroll to top, add a new row. It appears. No extra pagination fires.

If any step shows over-dispatch (initiated » loaded) or misses a page, report which with approximate counts.

---

Out of scope

• Swapping to an IntersectionObserver sentinel. Scroll-event + debounce achieves the same dedup without the complexity of maintaining a sentinel element inside a virtualized grid.
• Measuring real row heights via measureElement. Unrelated to the dispatch loop; useful later for pixel-perfect scrolling.
• A visible "loading more…" indicator during fetch. UX only.