async-await-queue

Promise-based priority queues for throttling, rate- and concurrency limiting of Node.js or browser tasks

Zero-dependency, total size: 2.93 kB uncompressed and 1.16 kB gzip-compressed

There is a medium story about using this package to parallelize download loops : Parallelizing download loops in JS with async-await-queue

This is an interesting solution to the priority queues problem.

There are other Promise-based queues out there but they are not async/await compatible and do not support priorities.

It guarantees order and never wakes up contexts that won't run.

I use it with tens of thousands of jobs on the queue. O(log(n)) on the number of jobs, O(log(n)) on the number of different priorities. Just make sure to always call Queue.end(). Or, since 1.2, there is a safer, but less versatile method, Queue.run().

Typical uses:

• Rate-limit expensive external API requests - especially on ban-happy servers
• Avoiding to launch all the tasks in an async loop at the same time while allowing some degree of controlled concurrency

The queues keep references to the Promise resolve() function and resolve it from outside of the Promise constructor. This is a very unusual use of Promises to implement locks that I find interesting (this is what the medium story is about).

2024 Update: My technique is on track to become an official ECMAScript language: ES Promise.withResolvers

Install

npm install --save async-await-queue

Typical usage

Require as CJS

const { Queue } = require('async-await-queue');

Import as ES6 Module

import { Queue } from 'async-await-queue';

(or read the jsdoc)

IMPORTANT Keep in mind that when running asynchronous code without explicitly awaiting it, you should always handle the eventual Promise rejections by a .catch() statement.

Examples

Basic example

const { Queue } = require('async-await-queue');
/**
 * No more than 2 concurrent tasks with
 * at least 100ms between two tasks
 * (measured from task start to task start)
 */
const myq = new Queue(2, 100);
const myPriority = -1;

/**
 * This function will launch all tasks and will
 * wait for them to be scheduled, returning
 * only when all tasks have finished
 */
async function downloadTheInternet() {
  for (let site of Internet) {
    /**
     * The third call will wait for the previous two to complete
     * plus the time needed to make this at least 100ms
     * after the second call
     * The first argument needs to be unique for every
     * task on the queue
     */
    const me = Symbol();
    /* We wait in the line here */
    await myq.wait(me, myPriority);

    /**
     * Do your expensive async task here
     * Queue will schedule it at
     * no more than 2 requests running in parallel
     * launched at least 100ms apart
     */
    download(site)
      /* Signal that we are finished */
      /* Do not forget to handle the exceptions! */
      .catch((e) => console.error(e))
      .finally(() => myq.end(me));
  }
  return await myq.flush();
}

Using a function

/**
 * This is the new style API introduced in 1.2
 * It is equivalent to the previous example
 */
async function downloadTheInternet() {
  const q = [];
  for (let site of Internet) {
     /** The third call will wait for the previous two to complete
      * plus the time needed to make this at least 100ms
      * after the second call
      */
    q.push(myq.run(() => download(site).catch((e) => console.error(e))));
  }
  return Promise.all(q);
}

Running sequentially

/**
 * This function will execute a single task at a time
 * waiting for its place in the queue
 */
async function downloadTheInternet() {
  let p;
  /**
   * The third call will wait for the previous two to complete
   * plus the time needed to make this at least 100ms
   * after the second call
   * The first argument needs to be unique for every
   * task on the queue
   */
  const me = Symbol();
  /* We wait in the line here */
  await myq.wait(me, myPriority);

  /**
   * Do your expensive async task here
   * Queue will schedule it at
   * no more than 2 requests running in parallel
   * launched at least 100ms apart
   */
  try {
    await download(site);
  } catch (e) {
    console.error(e);
  } finally {
    /* Signal that we are finished */
    /* Do not forget to handle the exceptions! */
    myq.end(me);
  }
}

Fire-and-forget

/**
 * This function will schedule all the tasks and
 * then will return immediately a single Promise
 * that can be awaited upon
 */
async function downloadTheInternet() {
  const q = [];
  for (let site of Internet) {
    /**
     * The third call will wait for the previous two to complete
     * plus the time needed to make this at least 100ms
     * after the second call
     * The first argument needs to be unique for every
     * task on the queue
     */
    const me = Symbol();
    q.push(
      myq
        .wait(me, myPriority)
        .then(() => download(site))
        .catch((e) => console.error(e))
        .finally(() => myq.end(me))
    );
  }
  return Promise.all(q);
}

Unresolvable Promises in Node.js

When using this package, something that you should be aware of is that Node.js has a very particular behavior when dealing with unresolvable Promises: nodejs/node#43162

When awaiting an unresolvable Promise, Node.js will simply exit - instead of blocking indefinitely - which would probably be what most people expect.

If you are using this package in Node.js and it seems to simply unexpectedly exit without reaching the program's normal end and without reporting any errors, you most probably have an unresolvable Promise.