feat: Add ilike and trigram similarity search to pagination by wschurman · Pull Request #431 · expo/entity

wschurman · 2026-02-10T04:56:23Z

Why

This PR adds search capabilities to the existing cursor-based pagination system added in #422 which only supported orderBy ordering.

This adds:

Case-insensitive pattern matching (ILIKE) - Useful for basic text search across entity fields
Trigram similarity search - Provides fuzzy matching capabilities for more advanced search use cases like handling typos or finding similar text

This enables building user-facing search features while maintaining the benefits of cursor-based pagination (stable results, efficient queries).

How

The implementation introduces a unified PaginationSpecification that supports three strategies:

Standard pagination - The existing orderBy-based pagination
ILIKE search - Pattern matching with automatic wildcard escaping
Trigram search - PostgreSQL trigram similarity with configurable threshold

Key implementation details:

Search terms are properly parameterized to prevent SQL injection
ILIKE special characters (%, _, ) are escaped to prevent pattern injection
Results maintain stable cursor-based pagination with proper ordering:
- ILIKE: Ordered by search fields, then ID
- Trigram: Ordered by exact match priority, similarity score, optional extra fields, then ID

Test Plan

The PR includes comprehensive test coverage:

Unit tests for both AuthorizationResultBasedKnexEntityLoader and
EnforcingKnexEntityLoader
Integration tests covering:
- Forward/backward pagination with search
- Multi-field search
- Case-insensitive matching
- Trigram similarity with various thresholds
- Cursor stability across pages
- Edge cases (empty results, special characters)
- Combined with WHERE clauses
- Security validation ensuring proper escaping and parameterization

All existing tests pass, confirming backward compatibility for standard pagination when using the new API.

wschurman · 2026-02-10T04:56:52Z

codecov · 2026-02-10T05:23:06Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (7b8a49c) to head (89f4ba6).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##              main      #431    +/-   ##
==========================================
  Coverage   100.00%   100.00%            
==========================================
  Files          108       109     +1     
  Lines        15606     16045   +439     
  Branches       812      1410   +598     
==========================================
+ Hits         15606     16045   +439

Flag	Coverage Δ
integration	`27.02% <100.00%> (+2.05%)`	⬆️
unittest	`94.32% <41.00%> (-1.58%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

wschurman · 2026-02-13T02:19:29Z

(for reviewers that are extra curious, you can see the evolution of this over time by seeing the versions on graphite. claude helped quite a lot but required a massive amount of manual intervention)

packages/entity-database-adapter-knex/src/internal/EntityKnexDataManager.ts

ide · 2026-02-15T22:35:50Z

packages/entity-database-adapter-knex/src/internal/EntityKnexDataManager.ts

+    return sql`CASE WHEN ${SQLFragment.join(ilikeConditions, ' OR ')} THEN ${raw(
+      direction === PaginationDirection.FORWARD ? '1' : '0',
+    )} ELSE ${raw(direction === PaginationDirection.FORWARD ? '0' : '1')} END`;


Why does the pagination direction affect the matching score of each row? To me it seems like ASC/DESC should take care of reversing the order without flipping 1 and 0 here.

Good catch. Technically it worked because the order was also flipped in buildTrigramExactMatchPriority based on direction, but the flip wasn't necessary. Updated and existing tests still pass.

From claude:

You're right. Look at buildTrigramExactMatchPriority (line 672): it already flips both the CASE values and the sort direction. That's redundant — CASE WHEN ... THEN 1 ELSE 0 END DESC and CASE WHEN ... THEN 0 ELSE 1 END ASC produce the same ordering. And buildTrigramExactMatchCaseExpression (used in the cursor comparison) also flips the values based on direction, but the cursor comparison operator is already flipped at line 549 (< vs >). The direction-aware value flipping is doing nothing in both places. The sort direction / comparison operator already handles it. You should just always use 1 for match and 0 for no-match, and let DESC/ASC and </> do the work.

My concern was that it flipped the scores for matching but not any other sorting criteria, so the "backward" option wouldn't always be the reverse of "forward".

wschurman · 2026-02-16T20:58:16Z

Merge activity

Feb 16, 8:58 PM UTC: A user started a stack merge that includes this pull request via Graphite.
Feb 16, 8:58 PM UTC: @wschurman merged this pull request with Graphite.

…#453) # Why The one downside of using id-based cursor pagination and subqueries (added in #422, #431) is that it risks the row referenced by the cursor being deleted between pagination requests. It's an edge case though to be clear. Encoding the full cursor alleviates this, but has it's own downsides since trigram similarity isn't encodable and neither are things like Date objects. It is still preferable to use ID-based cursors for all pagination. But it becomes the library's responsibility to explicitly document what the behavior is when a row referenced by the cursor is no longer present. This PR does this. # How When the cursor row is no longer present, the tuple evaluates to `NULL`, which produces a empty page. The alternative to this behavior is to run an id check query ahead of the pagination query, and throw an error if it doesn't exist, or even return the first page of data if it doesn't exist. Both of these are less optimal since they cause unexpected behavior during results consumption. So we keep the behavior as is and explicitly document it. # Test Plan Run new tests.

wschurman mentioned this pull request Feb 10, 2026

feat: add includeTotal option to pagination #430

Closed

wschurman force-pushed the wschurman/02-09-feat_add_ilike_and_trigram_similarity_search_to_pagination branch from 16b0067 to ce977d7 Compare February 10, 2026 05:20

wschurman force-pushed the wschurman/02-09-feat_add_includetotalcount_to_pagination branch from 50a20a7 to e8b3570 Compare February 10, 2026 21:26

wschurman force-pushed the wschurman/02-09-feat_add_ilike_and_trigram_similarity_search_to_pagination branch 2 times, most recently from 4d3b6d9 to 2159470 Compare February 10, 2026 22:33

wschurman force-pushed the wschurman/02-09-feat_add_includetotalcount_to_pagination branch from e8b3570 to ee53a9f Compare February 10, 2026 22:33

wschurman force-pushed the wschurman/02-09-feat_add_ilike_and_trigram_similarity_search_to_pagination branch from 2159470 to ec9eae3 Compare February 10, 2026 22:58

wschurman force-pushed the wschurman/02-09-feat_add_includetotalcount_to_pagination branch 2 times, most recently from 2b437e1 to 4439b05 Compare February 10, 2026 23:12

wschurman force-pushed the wschurman/02-09-feat_add_ilike_and_trigram_similarity_search_to_pagination branch 5 times, most recently from ab5e72b to 1ba6320 Compare February 11, 2026 03:30

wschurman force-pushed the wschurman/02-09-feat_add_includetotalcount_to_pagination branch from 4439b05 to 9a0ae15 Compare February 11, 2026 03:30

wschurman force-pushed the wschurman/02-09-feat_add_ilike_and_trigram_similarity_search_to_pagination branch 4 times, most recently from b7927ab to 3a3f704 Compare February 11, 2026 15:20

wschurman force-pushed the wschurman/02-09-feat_add_ilike_and_trigram_similarity_search_to_pagination branch from 3ca4856 to 3188915 Compare February 12, 2026 20:17

wschurman changed the base branch from graphite-base/431 to main February 12, 2026 20:17

wschurman force-pushed the wschurman/02-09-feat_add_ilike_and_trigram_similarity_search_to_pagination branch 5 times, most recently from 6a2d709 to 28b5ab9 Compare February 13, 2026 02:10

wschurman requested review from ide and quinlanj February 13, 2026 02:18

This was referenced Feb 13, 2026

feat: Add pagination max page size configuration #436

Merged

fix: pass in augmentable classes to installation #437

Merged

wschurman marked this pull request as ready for review February 13, 2026 03:38

ide reviewed Feb 15, 2026

View reviewed changes

wschurman mentioned this pull request Feb 15, 2026

feat: batch mutation #446

Draft

feat: Add ilike and trigram similarity search to pagination

89f4ba6

wschurman force-pushed the wschurman/02-09-feat_add_ilike_and_trigram_similarity_search_to_pagination branch from 28b5ab9 to 89f4ba6 Compare February 16, 2026 00:06

wschurman requested a review from ide February 16, 2026 00:11

ide approved these changes Feb 16, 2026

View reviewed changes

wschurman mentioned this pull request Feb 16, 2026

fix: eliminate some unnecessary any casts #448

Merged

wschurman merged commit f72cbca into main Feb 16, 2026
5 checks passed

wschurman deleted the wschurman/02-09-feat_add_ilike_and_trigram_similarity_search_to_pagination branch February 16, 2026 20:58

wschurman mentioned this pull request Feb 17, 2026

fix: explicitly document behavior when cursor entity no longer exists #453

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add ilike and trigram similarity search to pagination#431

feat: Add ilike and trigram similarity search to pagination#431
wschurman merged 1 commit intomainfrom
wschurman/02-09-feat_add_ilike_and_trigram_similarity_search_to_pagination

wschurman commented Feb 10, 2026 •

edited

Loading

Uh oh!

wschurman commented Feb 10, 2026 •

edited

Loading

Uh oh!

codecov bot commented Feb 10, 2026 •

edited

Loading

Uh oh!

wschurman commented Feb 13, 2026

Uh oh!

Uh oh!

ide Feb 15, 2026

Uh oh!

wschurman Feb 16, 2026

Uh oh!

ide Feb 16, 2026

Uh oh!

wschurman commented Feb 16, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

wschurman commented Feb 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

How

Test Plan

Uh oh!

wschurman commented Feb 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Feb 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

wschurman commented Feb 13, 2026

Uh oh!

Uh oh!

ide Feb 15, 2026

Choose a reason for hiding this comment

Uh oh!

wschurman Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

ide Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

wschurman commented Feb 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge activity

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

wschurman commented Feb 10, 2026 •

edited

Loading

wschurman commented Feb 10, 2026 •

edited

Loading

codecov bot commented Feb 10, 2026 •

edited

Loading

wschurman commented Feb 16, 2026 •

edited

Loading