Refactor delete requests to use rate limiter

[sebtoombs]

What

Rate limit delete requests (why) to avoid getting rate limit banned

How

Using rate-limiter-flexible with an in memory queue to limit requests. This allows us to tell Node to fire all our requests in parallel and then rely on the queue to handle meeting the rate limit requirement.

What this means is we can fire off as many requests as possible as quickly as possible (possibly taking advantage of multiple threads) until we hit the rate limit and backoff.

The rate limiter config (index.js lines 9 - 16) needs to be set, or come from come config maybe? Currently (as an example) it is set as max. 1 request (points) per 2 seconds (duration). And requests beyond that will get added to the queue (maxQueueSize 100). If the queue overflows, these requests will return an error, same as if the request had failed (but message something like "Queue is full"). There's probably no limit on the queue size (beyond the max integer in Node & memory on the runner), so could probably up that to some ridiculous value.

Note

• Although I added a node_module dep, I didn't check in the files because I wanted to keep the PR neat, so if you want to stick with checking in the deps will need to do another npm install and then commit that.
• We can avoid checking all the deps by bundling all the actual deps at build time (happy to add that if you want)

[apkelly]

Does this slice keep the head of the array or the tail? We want to keep the head (as these are the latest versions).

[sebtoombs]

Yep, this will remove the last toKeep items from the array - which I have just realised is incorrect. Pushed a change 🙏