scrapydweb icon indicating copy to clipboard operation
scrapydweb copied to clipboard

[Bug Report] Setting doesn't take effect in scrapydweb_settings_v10.py

Open laggardkernel opened this issue 5 years ago • 0 comments

Describe the bug I set the DATA_PATH value in scrapydweb_settings_v10.py but it doesn't take effect.

  1. There's no files created under the DATA_PATH
  2. It could be consolidated on the Web page "Settings"

To Reproduce Steps to reproduce the behavior:

  1. Make and cd an empty directory
  2. Run scrapydweb once to generate a scrapydweb_settings_v10.py file
  3. Open scrapydweb_settings_v10.py and set DATA_PATH.
  4. Run scrapydweb again and the service starts normally.

It's weird that the SCRAPYD_SERVERS I added did appear in the webpage, but the DATA_PATH didn't reflect the value i set.

Seems related to the vars.py about import_module

SCRAPYDWEB_SETTINGS_PY = 'scrapydweb_settings_v10.py'
try:
    custom_settings_module = importlib.import_module(os.path.splitext(SCRAPYDWEB_SETTINGS_PY)[0])
except ImportError:
    custom_data_path = ''
    custom_database_url = ''
else:
    custom_data_path = getattr(custom_settings_module, 'DATA_PATH', '')
    custom_data_path = custom_data_path if isinstance(custom_data_path, str) else ''
    custom_database_url = getattr(custom_settings_module, 'DATABASE_URL', '')
    custom_database_url = custom_database_url if isinstance(custom_database_url, str) else ''

Environment (please complete the following information):

  • Operating system: Ubuntu 16.04
  • Python version: 3.7.3
  • ScrapydWeb version: 1.4.0
  • Scrapyd version: 1.2.1
  • Scrapyd amount 2
  • Scrapy version: 2.2.1
  • Browser Chrome 70

Temporary solution: set DATA_PATH as an environment variable by export on *nix.

laggardkernel avatar Aug 14 '20 10:08 laggardkernel