before_run should be a separate bash session than run

[kengz]

For instance, any directory change in before_run is persisted in the main run using file. Example:

  - name: dataset
    provider: python
    requirements: requirements.txt
    before_run:
      - apt-get install -y unzip
      - mkdir -p data/amex-default-prediction
      - cd data/amex-default-prediction
      - kaggle competitions download -c amex-default-prediction
      - unzip amex-default-prediction.zip
    file: dl/datamodule.py
    artifacts:
      - data

This will result in an error python: can't open file '/workflow/data/amex-default-prediction/dl/datamodule.py': [Errno 2] No such file or directory

In general before_run should be its own bash session.