With the release of Crawlee v3.18, type-safe routers and schema validation are now available. How are you planning to integrate these features into your existing scraping workflows?
Original source
Crawlee v3.18: Type-safe routers | Crawlee for JavaScript · Build reliable crawlers. Fast.
What the source supports
The source details the release of Crawlee v3.18, highlighting type-safe routers, runtime schema validation for userData, Puppeteer 25 support, and various bug fixes. It also covers v3.17 features like dynamic memory snapshots and custom load signals for autoscaling.
What AtlasRepo adds
AtlasRepo structures the release notes into actionable sections for developers, emphasizing the practical benefits of type safety and schema validation in reducing runtime errors. It provides concrete steps for implementation and highlights the impact of bug fixes on crawler stability.
Key takeaways
- Type-safe routers allow defining specific userData shapes per label, catching typos and mismatches at compile time.
- Runtime schema validation using libraries like Zod ensures data integrity before handlers execute, failing fast on invalid inputs.
- Puppeteer 25 is now supported, with backward compatibility maintained for versions down to 21.
- Memory autoscaling now dynamically adjusts to container resizing, preventing under-utilization or crashes.
- Custom load signals enable developers to integrate external metrics like proxy health into autoscaling decisions.
- Bug fixes address issues with off-domain redirects, sitemap filtering, and backpressure deadlocks.
Questions for the community
- Which schema validation library (Zod, Valibot, ArkType) do you prefer for integrating with Crawlee’s new router features?
- Have you encountered issues with memory autoscaling in dynamic container environments, and how has the new dynamic snapshot feature helped?