1.8.1

1.8.1 (2026-07-08)

🐛 Bug Fixes

• Keep named request queues across runs (#2015) (3b9f8c6) by @vdusek
• Make adaptive-crawler extra importable (#2016) (b850cd6) by @vdusek
• deps: Raise tldextract floor to >=5.3.0 to avoid AttributeError on same-domain enqueue (#2031) (8850a86) by @anxkhn
• statistics: Handle exceptions without a traceback in ErrorTracker (#2035) (be04c17) by @anxkhn
• sessions: Preserve cookie expiry of 0 (Unix epoch) on serialization (#2034) (49665e2) by @anxkhn
• sitemap: Ignore sitemap URLs with an empty <loc> (#2032) (54bce08) by @anxkhn

1.8.0

1.8.0 (2026-07-02)

🚀 Features

• Add PydanticAiCrawler with AI-powered HTML extraction (#1964) (ccfff76) by @Mantisus

🐛 Bug Fixes

• Add is_finished and correct is_empty semantics in RequestQueueClient (#1982) (0ef1078) by @Mantisus
• Keep minutes segment in format_duration for sub-minute remainders (#1994) (b322ace) by @anxkhn
• templates: Use Playwright version locked base docker images (#1997) (b5f0cb4) by @Pijukatel
• Ignore negative Retry-After header instead of disabling 429 back-off (#1995) (5027018) by @anxkhn
• sitemap: Prevent sitemap parser from leaking metadata between <url> entries (#1992) (a58687e) by @anxkhn
• templates: Pin Playwright to base image version for templates (#1996) (917b6b6) by @Pijukatel

1.7.3

1.7.3 (2026-06-22)

🐛 Bug Fixes

• memory-storage: Apply skip_empty after pagination in MemoryDatasetClient.get_data (#1937) (7e807aa) by @vdusek
• redis: Return all items from Redis dataset get_data with desc=True and limit=None (#1939) (f7cde2e) by @vdusek
• Retry sitemap fetching on error and raise when retries are exhausted (#1943) (76927d7) by @vdusek
• Validate storage name and alias values (#1950) (0cac092) by @vdusek
• Constrain default sitemap loading (#1956) (3fd5ace) by @Pijukatel
• Remove extra sleep in ImpitHttpClient.stream (#1980) (149a6aa) by @Mantisus
• Retry unprocessed requests in RequestQueue.add_request (#1976) (cfee910) by @vdusek
• Gracefully close sitemap stream on SitemapRequestLoader abort (#1979) (202726d) by @Mantisus

1.7.2

1.7.2 (2026-06-04)

🐛 Bug Fixes

• templates: Install pinned playwright into system env in uv Dockerfile (#1922) (0fd0f3a) by @vdusek
• memory-storage: Avoid duplicate processed requests in memory request queue client (#1941) (d343d0e) by @vdusek
• redis: Preserve shared Redis index hashes when dropping a storage (#1942) (4729dd1) by @vdusek
• Do not raise KeyError in parse_sitemap when partial options are provided (#1940) (8ab3f95) by @vdusek
• Reset private state correctly in sitemap parsers (#1938) (7db517a) by @vdusek

1.7.1

1.7.1 (2026-05-26)

🐛 Bug Fixes

• Include sql_mysql in the all extra (#1895) (4023314) by @vdusek
• Update push_data and user_data annotation with JsonSerializable instead of Any (#1889) (662b93b) by @Mantisus
• stagehand: Inject --no-sandbox into Stagehand's Chromium launch when sandbox is disabled (#1906) (041b92a) by @vdusek
• templates: Pin playwright to base image version in uv Dockerfile template (#1904) (8d902c9) by @vdusek

1.7.0

1.7.0 (2026-05-12)

🚀 Features

• Add use to Router for middleware support with pre-handler execution (#1857) (23d7d6c) by @Mantisus
• Add opt-in per-domain request throttling for HTTP 429 backoff (#1762) (c17f4d5) by @MrAliHasan
• Add pre/post launch hooks to BrowserPool (#1879) (00ffb7e) by @Mantisus
• Add StagehandCrawler with AI-powered browser automation (#1854) (da84db1) by @Mantisus
• cli: Add Adaptive and Stagehand crawler templates (#1888) (39b2d24) by @vdusek

🐛 Bug Fixes

• Reject non-http(s) URL schemes in HTTP clients (#1862) (ac66b2a) by @vdusek
• Filter sitemap-derived URLs by enqueue strategy (#1864) (b3db0dc) by @vdusek
• Bump BrowserPool default operation_timeout to 60 seconds (#1877) (38e7dd2) by @vdusek
• redis: Prevent counter corruption from concurrent mark handled in Redis RQ (#1878) (50d70f0) by @Mantisus
• Fall back to drop+recreate when RequestQueue.purge is unsupported (#1883) (cd15dce) by @vdusek

1.6.3

1.6.3 (2026-04-27)

🐛 Bug Fixes

• Fix potential deadlocks in SitemapRequestLoader and RequestManagerTandem (#1843) (6226d93) by @Mantisus
• Add retry logic for RedisStorageClient and SqlStorageClient (#1838) (b80f562) by @Mantisus
• Fix StorageInstanceManager cache eviction (#1855) (983f14f) by @janbuchar
• Report integer count in 'Experiencing problems' status log (#1860) (40170a6) by @vdusek
• Preserve forefront flag on RequestQueue retry path (#1861) (dc1073a) by @vdusek

1.6.2

1.6.2 (2026-04-08)

🐛 Bug Fixes

• file-system: Reclaim orphaned in-progress requests on RQ recovery (#1825) (e86794a) by @vdusek
• Prevent premature EventManager shutdown when multiple crawlers share it (#1810) (2efb668) by @Mantisus
• Apply SQLite optimizations to the custom connection_string in SqlStorageClient (#1837) (8b53e27) by @Mantisus
• Apply SharedTimeout to post-navigation hooks (#1839) (88bd05a) by @vdusek

1.6.1

1.6.1 (2026-03-30)

🐛 Bug Fixes

• Handle invalid URLs in RequestList (#1803) (0b2e3fc) by @Mantisus
• playwright: Filter unsupported context options in persistent browser (#1796) (69ad22e) by @sushant-mutnale
• Remove double usage_count increment in Session.retire() (#1816) (c40d411) by @vdusek
• Defer page object cleanup to make it accessible in error handlers (#1814) (7eeb500) by @janbuchar

⚡ Performance

• Offload BeautifulSoup parsing to a thread via asyncio.to_thread (#1817) (d612ffa) by @vdusek

1.6.0

1.6.0 (2026-03-20)

🚀 Features

• Allow non-href links extract & enqueue (#1781) (6db365d) by @kozlice
• Add post_navigation_hooks to crawlers (#1795) (38ceda6) by @Mantisus
• Add page lifecycle hooks to BrowserPool (#1791) (6f2ac13) by @Mantisus
• Expose BrowserType and CrawleePage (#1798) (b50b9f2) by @Mantisus
• Expose use_state in BasicCrawler (#1799) (d121873) by @Mantisus

🐛 Bug Fixes

• redis: Do not remove handled request data from request queue (#1787) (3008c61) by @kozlice
• redis: Update actual Request state in request queue Redis storage client (#1789) (787231c) by @Mantisus