Remove Amazon CloudFront from crawler list ?

[loadkpi]

see the initial point loadkpi/crawler_detect#61

I probably agree with it because Amazon CloudFront is more like a reverse proxy than crawler

But I also found that it's not strict opinion:

• it was added on Aug 2020 Add pattern for Amazon CloudFront #392
• after that it was remove on Sep 2020 Remove Amazon CloudFront from the crawlers #410
• and then again was added on Aug 2023 Amazon Cloudfront #504

[byroot]

it was added on Aug 2020 #392

Interestingly, not justification was given.

after that it was remove on Sep 2020 #410

Justified because CloudFront main use case is being a caching proxy. Many apps running on AWS are behind it.

and then again was added on Aug 2023 #504

They say they get a "ton of it" lately, but I wonder what that actually means. Perhaps there is a way I'm not aware of, to use CloudFront as an ongoing proxy?

Either way, I'd argue it's not clear cut a crawler enough to justify being in the list by default.

To be clear, this isn't a huge problem for us, I easily worked around it, but I think it's a footgun for many people running their app on AWS.

[JayBizzle]

So, what we saying, keep or remove?