fix: cleanup purging behavior for CLI
Closes #590
BREAKING CHANGE:
Purging in projects using Crawlee for SDK v3+ now purge by default, so using --purge flag is no longer necessary. Use --no-purge to keep the storage folder intact.
The purge-queue, purge-dataset and purge-key-value-store flags have been removed, and the logic of all three was combined into the purge flag.
Hi guys, I've just realized that if we want the CLI to behave consistently with how the Apify platform works, we shouldn't purge entirely by default, but instead the CLI should just keep adding local runs for later introspection and comparison. I know you're making it on par with Crawlee's behavior, but I think this will introduce further divergence.
I am not sure about that, regular actor developer runs, runs, and runs. They would end up with a lot of junk locally, I am not sure people would like such a default.
Also, I am not sure if this is a "divergence", we are not doing that now either, the CLI never considered multiple storage folders, neither the actual consumers of the storage folder (crawlee/sdk). We could surely just rename the existing storage folder instead of wiping it, but should that really be a default?
@B4nan Understood. Divergence from the platform behavior is what I'm concerned about. You run it there and the results always persist by default under a new run. But unifying the behavior of our local tooling is definitely a way to go.
@vladfrangu lets describe the BCs in the PR description
I also found out you can group flags in the help menu. Hows this look? @netmilk @B4nan
nit: should be Node.js, not node.js
also i double checked with others, python works the same as JS, auto purge for crawlee or SDK (even without crawlee) v2+