RECAP Fetch issue: Cookie cache expired before task could run for user

[habeasdockets]

With the huge increase in the number of immigrant habeas corpus cases being filed, we've been using the Fetch API a significant amount on a daily basis! It runs from a cron in our back end. The purpose is to ensure that RECAP is aware of all the immigrant habeas cases, has their nature of suit (463), and has the docket entries for the initial filings. We then use RECAP as our source of truth for docket entries, case info, etc.

Today we had dozens of fetches that didn't go through, all with the error "Cookie cache expired before task could run for user: [userid]." We also had some yesterday, and I recall seeing the error once before that. In general, the Fetch queue seems to have slowed down significantly. I think that fetches are getting through the queue though, if they don't hit this error.

[mlissner]

This could also be due to a general influx of Fetch users and we may need to get more IP addresses to cover it. This will have to take backseat to a big architecture project we're working on, but @albertisfu, can you or Eduardo take a look and see if anything jumps out?

[habeasdockets]

It may be that we've been hogging the capacity that you have for bulk usage (at least at certain times of day) and that any additional users is enough to push it over the edge. Today it's performing well. Been running for an hour and it's about half way through.

Really appreciate anything that you all can do! FYI- these days, not just lawyers are interested. Having the cases up in a timely fashion, without the need to get a PACER account, really helps out the family members. Based on the emails I get, they're often trying to figure out if the habeas has really been filed or if they're possibly getting scammed. Which unfortunately, is a real thing. :(