feat: Add Cloudflare bypass test and enable native-tls-alpn feature…

[evanstern]

The Issue

The website is protected by Cloudflare, which was blocking the default reqwest HTTP client used by the application. This resulted in the application receiving a "Just a moment..." challenge page instead of the actual recipe content, causing the JSON-LD extractor to fail.

The Fix

The root cause was identified as a known issue with reqwest's default TLS configuration when interacting with Cloudflare's HTTP/2 implementation. The fix involved:

Enabling native-tls-alpn: This feature ensures correct Application-Layer Protocol Negotiation (ALPN) for HTTP/2, which Cloudflare requires to validate the client.

Changes

Cargo.toml
Enabled native-tls-alpn feature for reqwest.

reqwest = { version = "0.12", features = ["blocking", "json", "native-tls-vendored", "native-tls-alpn"] }

Verification

I verified the fix by running the application against the reported URL:

cargo run https://www.acouplecooks.com/apple-cranberry-crisp/

Result: The application successfully extracted the recipe.

Automated Test

I also added a new integration test tests/test_cloudflare_bypass.rs to ensure this remains fixed. You can run it with:

cargo test --test test_cloudflare_bypass -- --ignored

[evanstern]

I've noticed this issue in on more than a few websites. The application/ld+json section is present in the headers (verified via cUrl and manual inspection of page source) but Cloudflare was blocking the reqwest library from seeing anything so the import consistently failed.

I looked around for reasons why and found this post: seanmonstar/reqwest#2227 and specifically this comment seanmonstar/reqwest#2227 (reply in thread)

So, I went that route.

It seems like a simple fix for a problem that will affect more than a few websites.

This is my first PR to this project. Please let me know if I should do things differently 😊

[dubadub]

Wow, that's a simple fix for that annoying issue, great find!