Skip to content

Conversation

@gbirman
Copy link
Contributor

@gbirman gbirman commented Dec 12, 2025

Summary

before, a search miss would cause us to keep hitting the soup endpoint for every single dss item, including emails, which ends up being dozens/hundreds of soup calls...every time.

searching on soup + using search service requires some careful consideration. notably, we want to able to fuzzy over what we have locally. we also want to be able to fetch more data from soup to get more search results. however, there's a cost to doing this with emails since there are so many. what this pr does:

  • enables emails in soup as a search source when it's not a valid search term (less than 3 chars) but prevents fetching more data if emails are also being fetched
  • searches over dss soup non-email items when there's a search miss, fetching more data if more pages are needed
  • uses these cached sources on subsequent search queries. i bump the stale time to 1 min so that when i clear the search and search the same term again i don't repeat the soup calls. in my dev macro that equates to around 10 soup calls for a non-exhaustive search over all non-email/non-channel items. the previous stale time was 5 seconds which would mean we would iterate over all dss items for every new search miss effectively. NOTE: THIS MAY HAVE SOME UNINTENDED CONSEQUENCES if the query is not invalidated when we need new data. i think it's fine though?

i think what we'll eventually want to do though, is separate a cached data source from our fetch more. since the search source should be relatively stable, we don't fetch more to keep getting more backend data every time when we already have a cached search source. having the search data fetch very tightly linked to the request body/search term state and response outcomes makes it difficult to reason about/keep the list stable

Screenshots, GIFs, and Videos

this is what we had in DEV

CleanShot.2025-12-12.at.16.34.36.mp4
CleanShot.2025-12-12.at.16.09.07.mp4

@gbirman gbirman requested a review from a team as a code owner December 12, 2025 21:29
@linear
Copy link

linear bot commented Dec 12, 2025

@gbirman gbirman merged commit 9343c12 into main Dec 15, 2025
20 checks passed
@gbirman gbirman deleted the gab/m-5399-infinite-soup-search-for-query-misses branch December 15, 2025 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants