crawlee
crawlee copied to clipboard
Simplify `Configuration`
- a library may exist to make working with environment-based configuration easier (similar to
pydantic-settingsin Python) - we should reconsider the current configuration priority where environment variables take precedence over arguments to the
Configurationconstructor - we should consider removing support for the
crawlee.jsonconfiguration file - we should move strictly Apify-specific configuration options to the SDK (e.g.,
ACTOR_DEFAULT_DATASET_ID)
a library may exist to make working with environment-based configuration easier (similar to
pydantic-settingsin Python) we should reconsider the current configuration priority where environment variables take precedence over arguments to theConfigurationconstructor
To be decided later.
we should consider removing support for the
crawlee.jsonconfiguration file
Let's confirm with our users (mainly but not only delivery) if they use this, likely nope so fine to remove.
we should move strictly Apify-specific configuration options to the SDK (e.g.,
ACTOR_DEFAULT_DATASET_ID)
Most important bit, python version already works this way.