docs(agent): update skill with the latest changes

Author: D4Vinci

agent-skill/Scrapling-Skill.zip

@@ -302,6 +302,8 @@ QuotesSpider(crawldir="./crawl_data").start()
 ```
 Press Ctrl+C to pause gracefully - progress is saved automatically. Later, when you start the spider again, pass the same `crawldir`, and it will resume from where it stopped.
 
+While iterating on a spider's `parse()` logic, set `development_mode = True` on the spider class to cache responses to disk on the first run and replay them on subsequent runs - so you can re-run the spider as many times as you want without re-hitting the target servers. The cache lives in `.scrapling_cache/{spider.name}/` by default and can be overridden with `development_cache_dir`. Don't ship a spider with this enabled.
+
 ### Advanced Parsing & Navigation
 ```python
 from scrapling.fetchers import Fetcher

agent-skill/Scrapling-Skill/SKILL.md

@@ -85,6 +85,49 @@ async def on_start(self, resuming: bool = False):
         self.logger.info("Starting fresh crawl")
 ```
 
+## Development Mode
+
+When you're iterating on a spider's `parse()` logic, re-hitting the target servers on every run is slow and noisy. Development mode caches every response to disk on the first run and replays them from disk on subsequent runs, so you can tweak your selectors and re-run the spider as many times as you want without making a single network request.
+
+Enable it by setting `development_mode = True` on your spider:
+
+```python
+class MySpider(Spider):
+    name = "my_spider"
+    start_urls = ["https://example.com"]
+    development_mode = True
+
+    async def parse(self, response: Response):
+        yield {"title": response.css("title::text").get("")}
+```
+
+The first run fetches normally and stores each response on disk. Every subsequent run serves the same requests from the cache, skipping the network entirely.
+
+### Cache Location
+
+By default, responses are cached in `.scrapling_cache/{spider.name}/` relative to the current working directory (where you ran the spider from, **not** where the spider script lives). You can override the location with `development_cache_dir`:
+
+```python
+class MySpider(Spider):
+    name = "my_spider"
+    start_urls = ["https://example.com"]
+    development_mode = True
+    development_cache_dir = "/tmp/my_spider_cache"
+```
+
+### How It Works
+
+1. **Cache key**: Each response is keyed by the request's fingerprint, so any change to fingerprint-affecting attributes (`fp_include_kwargs`, `fp_include_headers`, `fp_keep_fragments`) will produce a fresh fetch.
+2. **Storage format**: One JSON file per response, named `{fingerprint_hex}.json`. The body is base64-encoded so binary content is preserved exactly. Writes are atomic (temp file + rename).
+3. **Replay**: On a cache hit, the engine skips the network entirely, including `download_delay`, rate limiting, and the `is_blocked()` retry path. The cached response goes straight to your callback.
+4. **Stats**: Cached requests still count toward `requests_count`, `response_bytes`, and the per-status counters, so your stat output looks the same as a normal crawl. Two extra counters, `cache_hits` and `cache_misses`, let you see how the cache performed.
+
+### Clearing the Cache
+
+There's no automatic expiration. To force a fresh crawl, delete the cache directory or call the manager's `clear()` method directly.
+
+**Warning:** Development mode is meant for development, not production. Cached responses never expire, and replay bypasses rate limiting and blocked-request retries. Don't ship a spider with `development_mode = True`.
+
 ## Streaming
 
 For long-running spiders or applications that need real-time access to scraped items, use the `stream()` method instead of `start()`:
@@ -220,6 +263,8 @@ print(f"Failed: {stats.failed_requests_count}")
 print(f"Blocked: {stats.blocked_requests_count}")
 print(f"Offsite filtered: {stats.offsite_requests_count}")
 print(f"Robots.txt disallowed: {stats.robots_disallowed_count}")
+print(f"Cache hits: {stats.cache_hits}")
+print(f"Cache misses: {stats.cache_misses}")
 print(f"Items scraped: {stats.items_scraped}")
 print(f"Items dropped: {stats.items_dropped}")
 print(f"Response bytes: {stats.response_bytes}")

agent-skill/Scrapling-Skill/references/spiders/advanced.md

@@ -60,6 +60,10 @@ When a request comes in, the Session Manager routes it to the correct session ba
 
 An optional system that, if enabled, saves the crawler's state (pending requests + seen URL fingerprints) to a pickle file on disk. Writes are atomic (temp file + rename) to prevent corruption. Checkpoints are saved periodically at a configurable interval and on graceful shutdown. Upon successful completion (not paused), checkpoint files are automatically cleaned up.
 
+### Response Cache
+
+An optional cache that, when development mode is enabled, stores every fetched response on disk and replays it on subsequent runs. Each response is keyed by request fingerprint and serialized as JSON (with the body base64-encoded so binary content survives). It's meant for iterating on `parse()` logic without re-hitting the target servers, not for production use.
+
 ### Output
 
 Scraped items are collected in an `ItemList` (a list subclass with `to_json()` and `to_jsonl()` export methods). Crawl statistics are tracked in a `CrawlStats` dataclass which contains a lot of useful info.