apify-sdk-python icon indicating copy to clipboard operation
apify-sdk-python copied to clipboard

fix: Make `ApifyFileSystemStorageClient` fully compatible with the apify cli

Open Pijukatel opened this issue 4 months ago • 0 comments

Description

  • Make sure that storage from ApifyFileSystemStorageClient does not get purged twice due to storage from FileSystemStorageClient pointing to the same location. (Those storage clients will have the same cache key and thus there can be only one.)
  • Ensure that Actor will open input containing KVS on initialization to ensure that an aware storage client is used.
  • Support any possible pre-existing input key and file that is defined through Configuration.input_key. Different input files will have different handling based on their suffix:
    • ".json" is parsed as json.
    • ".txt" is opened as plain text
    • everything else is opened as bytes
    • without extension is tried to be parsed as json first, but falls back to bytes
  • Create a metadata file for the valid pre-existing input file, without modifying the input file (otherwise, cli might detect the change to the input, which would be a false positive)
  • Raise an error if two valid pre-existing input files exist in the expected storage directory.
  • CLI does not respect env variables with the input key so far. TODO: https://github.com/apify/apify-cli/issues/960

Issues

Closes: #621 Related to: #INPUT.json Automatically Deleted on Each Run (Python SDK Local Storage Issue)

Testing

  • Added unit tests.
  • Manually tested with [email protected]
    • npx [email protected] run -i {"a":"c"} with pre-existing input file or without input and multiple times in a row
    • npx [email protected] run with pre-existing input file or without input and multiple times in a row

Pijukatel avatar Nov 11 '25 15:11 Pijukatel