AndyTheFactory
Deployment issues with TOP_DIRECTORY
First off, thanks for the library @codelucas, it's awesome!
I am experiencing an exception on deployment to AWS Lambda, and deployment to Elastic Beanstalk results in a build failure. The cause is a permissions issue due to os.path.expanduser("~") being set to root, but the application process is executed under a non-login, non super-user. In both cases, I was able to work around the problem by setting TOP_DIRECTORY = DATA_DIRECTORY in settings.py.
In terms of a general solution, if the code checked for root in os.path.expanduser and in that case used the local directory with no absolute path, that should work for AWS deploys. I imagine an alternative would be to set TOP_DIRECTORY in config.py instead of settings.py. I'm sure there are other ways to resolve this as well.
I'm happy to write the code, but didn't want to open a PR without checking on the best way to implement. Thanks for your attention to this matter. I am looking forward to removing the sed replacements from my post-build scripts!
@ekingery I have the exact same problem. Could you please share how you applied sed replacements?
Thanks!
You might just need the sed line / command, but below are all of the related lines in my bash install script. You will probably need to update the path to the settings.py file based on your virtualenv / pip install paths.
# hack to fix https://github.com/codelucas/newspaper/issues/384
# must be run with virtualenv activated if in use
np_path=`pip3 show newspaper3k | grep 'Location: ' | awk '{print $2 "/newspaper/"}'`
sed -i '/^TOP_DIRECTORY =/c\TOP_DIRECTORY = "/tmp/.newspaper_scraper"' $np_path"settings.py"
mkdir -p /tmp/.newspaper_scraper/
That works for local development and deployment to AWS Elastic Beanstalk. It would be great to have @codelucas or someone else chime in on the preferred solution to this. I'd be happy to make the code change once I know it's likely to be accepted.