Lab · 5
Can AI build an infinite-scroll page that stays fully visible to AI?
This started as two separate experiments — pagination for the Lab, infinite scroll for Thinking — before a working session collapsed them into one question. Real pagination means real page-2 URLs, which felt like overhead this site doesn’t need yet. Real infinite scroll usually means fetching more as you scroll, which is exactly the pattern most AI crawlers can’t see through, since most of them never execute JavaScript at all. So the actual test became narrower: can AI build something that feels like infinite scroll to a person, while every item still ships in the page’s raw HTML — nothing fetched, nothing hidden from a machine that only ever reads the source?
Hypothesis
AI can build a page that scrolls like infinite scroll for users but stays fully visible to AI for AEO/GEO.
Finding
The infinite scroll works and the html is visible in the source code.
Tested by building and shipping the tool itself, live on /lab/ and /thinking/ — not a controlled study across other sites, other content types, or other design systems.
Method
Research
Researched current AI-crawler behavior for paginated and infinite-scroll content.
Design
Designed a progressive reveal, not fetch-based infinite scroll, so nothing is ever hidden from a non-rendering crawler.
Build
Wrote the CSS/JS and wired it into the existing shared toolbar script.
Ship
Built it directly on both live pages.
Tech
The experiment
A progressive scroll reveal, layered onto the existing content toolbar organism from Experiment 4 rather than built as a separate system. Every entry on /lab/ and /thinking/ still renders in full in the page’s HTML at load — a new .scroll-sentinel marker just tells the shared script to reveal the next batch, via a new .is-scroll-hidden state class, as a person scrolls near the bottom. Nothing is fetched, and the reveal coordinates with the existing filter/sort/search logic: changing a filter or sort resets the reveal back to the first batch, so it never shows a stale mix. Confirmed against current research before building: most AI crawlers (GPTBot, ClaudeBot, PerplexityBot, and others) don’t execute JavaScript at all — they read the raw HTML response, so content present in the markup but visually staged behind a CSS class is fully available to them regardless of scroll state.
AI handled
- Researching current SEO/AEO/GEO crawling behavior for paginated and infinite-scroll content
- Identifying that most AI crawlers don’t render JavaScript, which shaped the actual design
- Designing progressive reveal (not fetch-based infinite scroll) so nothing is ever hidden from a non-rendering crawler
- Writing the CSS/JS and wiring it into the existing shared toolbar script
- Building it directly on both live pages
Dr. Paul handled
- Setting the original two-experiment idea (pagination for Lab, infinite scroll for Thinking)
- Deciding, after working through the SEO/AEO/GEO tradeoffs, to collapse it into one experiment instead
- Setting the final hypothesis
- Moving the avatar/chatbot Next Question down a slot
- Setting the final finding once the mechanism was confirmed working live
Worked: Verified two ways: scrolling the live page shows the batched reveal working, and reading the page’s raw HTML response — not the rendered DOM — shows every item present at every scroll state.
Constraint: The first version had a real bug: an IntersectionObserver look-ahead fired the reveal — and hid the sentinel as exhausted — before it was ever actually on screen, on a page short enough that the look-ahead ate nearly the whole remaining scroll distance. Caught only once Dr. Paul tried scrolling the live page, not from local testing alone.
Finding
The infinite scroll works and the html is visible in the source code.
Verified two ways: scrolling the live page shows the batched reveal working, and reading the page’s raw HTML response — not the rendered DOM — shows every item present at every scroll state.
A post-launch bug was caught and fixed along the way: the reveal was firing before it was ever visible on screen, the kind of thing that only surfaces once a real person tries scrolling a real page, not a demo.
Implication
For a CMO, this closes a false choice marketing teams have been living with: build the smoother scroll experience people actually prefer, or keep content broken into paginated pages so AI answer engines can still see all of it. Those two goals used to compete because most infinite-scroll builds fetch content on demand, and most AI crawlers never run the JavaScript that fetch depends on — so a page built for a better human experience could quietly go invisible to the systems now answering people’s questions before they ever click through. Here nothing is ever fetched or hidden: every item ships in the page’s HTML at load, and only the on-screen display is staged. The scroll experience and the AEO/GEO visibility stop trading off against each other — they’re the same page.
Next Question · 6
Can AI build an opt-in, self-destructing tracking feature?
A Mission-Impossible-style path tracker — opt-in, first-party, gone in 15 minutes.
Read Experiment 6 →