crawlee icon indicating copy to clipboard operation
crawlee copied to clipboard

Improve type safety

Open janbuchar opened this issue 5 months ago • 1 comments

  • There are multiple opportunities throughout Crawlee to improve type safety and optionally introduce runtime validation:
    • Dataset items (CrawlingContext.pushData)
    • Key-value store content
    • Request.userData
    • Request routing labels
    • ? possibly somewhere else
  • One option is allowing the user to declare some special types (DatasetItem, RequestLabel, ...) in a .d.ts file and using it for type checking
  • A safer and more versatile approach would be to accept standard-schema models (either via BasicCrawler or Configuration) and infer the types from those
    • this would also add runtime validation (in contrast to the .d.ts approach which is more of a gentleman agreement)

janbuchar avatar Jul 22 '25 13:07 janbuchar

Related: #1485

B4nan avatar Jul 23 '25 15:07 B4nan