Skip to content

minConcurrency is not being respected #1741

Description

@LahmerIlyas

Which package is this bug report for? If unsure which one to select, leave blank

None

Issue description

I'm using the cheerio crawler and I'm setting both the maxConcurrency and the minConcurrency to 20 but I'm always seeing currentConcurrency as 1:

INFO  CheerioCrawler:AutoscaledPool: state {"currentConcurrency":1,"desiredConcurrency":20,"systemStatus":{"isSystemIdle":true,"memInfo":{"isOverloaded":false,"limitRatio":0.2,"actualRatio":0},"eventLoopInfo":{"isOverloaded":false,"limitRatio":0.6,"actualRatio":0.024},"cpuInfo":{"isOverloaded":false,"limitRatio":0.4,"actualRatio":0},"clientInfo":{"isOverloaded":false,"limitRatio":0.3,"actualRatio":0}}}

Code sample

this.crawler = new CheerioCrawler({
      ignoreSslErrors: true,
      autoscaledPoolOptions: {
        desiredConcurrency: 20,
        maxConcurrency: 20,
        minConcurrency: 20,
      },
      requestHandler: this.router,
      keepAlive: true,
      requestQueue: this.requestQueue,
      failedRequestHandler: this.onFailedRequest.bind(this),
    });

Package version

3.1.4

Node.js version

16

Operating system

ubuntu

Apify platform

  • Tick me if you encountered this issue on the Apify platform

I have tested this on the next release

No response

Other context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working.

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions