Add a rate limiting option

[pandrRe]

This could be a entire new library, but I think the idea really fits TanStack Query.

I noticed the problem while developing a SPA using Vue Query. Constantly hot reloading and refreshing the page usually ended up in a few 429 Too Many Requests popping up. It's not impossible to implement a solution using some throttling, retryDelay, some external rate-limiting library, etc. but I just think that it would be much easier if we could quickly implement something right on useQuery that properly batched requests or delayed them according to a straightforward rate limit ruleset.

Example:

My page has connections with two services: a Infinite Timeline one and a General one for everything else. They have the following rules:

Infinite Timeline Service
Limited to 1 request per 2 seconds.
Limited to 15 requests per minute.

General Service
Limited to 1 request per 5 seconds.
Limited to 20 requests per minute.

To make TanStack Query respect these rules, I want to be able to just declare them like:

const SECOND = 1000
const MINUTE = 60 * SECOND

// Only one query using a rate-limited service.
export function useTimeline() {
  return useInfiniteQuery({
    queryKey: ['timeline'],
    queryFn: fetchTimeline,
    getNextPageParam: (lastPage, pages) => lastPage.nextCursor,
    rateLimits: {
      2 * SECOND: 1,
      1 * MINUTE: 15,
    },
  })
}

//Multiple queries using a rate-limited service.
const sharedLimiter = new QueryLimiter({
  requestsPerTime: {
    5 * SECOND: 1,
    1 * MINUTE: 20,
  }
})

export function useProfileData() {
  return useQuery({
    queryKey: ['profileData'],
    queryFn: fetchProfileData,
    rateLimits: sharedLimiter,
  })
}

export function usePostComments(postId) {
  return useQuery({
    queryKey: ['postComments', postId],
    queryFn: fetchPostComments(postId),
    rateLimits: sharedLimiter,
  })
}

And what happens inside a QueryLimiter is that, if a query has been invoked and the query function would break one of the rate limits if called, the query gets in a queue until the limiter has room to call it. useQuery could even get a new isQueued state.

I think the biggest advantage of this approach is that instead of needing to fine tune staleTimes around the application to try and reduce load on certain APIs, this just makes the application respect a sane limit defined by the API itself.

[TkDodo]

I would see something like rate-limiting or even debouncing / throttling to need to happen on the api layer. React Query only cares about the produced promise - it is not tied to "data fetching".

there are some similar discussions on related topics where the stance has always been that this will likely not make it into the library:

• Avoid consecutive mutation requests #2191
• Built in Debounce #3077

to quote myself:

It's a tradeoff. Every new feature increases api surface, learning curve and bundle size. We would also likely need to add a 3rd party dependency. Features that are not often requested and can be implemented in user-land in an easy manner will not be added.