How to Deal With Rate Limited APIs in NodeJS

Problem

I need to make 100k API calls to an API that errors-out if I send more than 1000 API calls per minute.

Solutions

From simplest to most complex

  1. Add a delay between calls
  2. Use RxJS
  3. Use a message queue

My Solution

We tried RxJS, and it was too complex. Instead, we just added a delay using setTimeout and inserted all the records into our own DB, so we don’t have to deal with the rate limiter at all.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
janac

janac

Most of my writing is about software. I enjoy summarizing and analyzing books and self-help videos. I am a full-time software engineer.