tarbell icon indicating copy to clipboard operation
tarbell copied to clipboard

Extra context not merged when generating site

Open ghing opened this issue 9 years ago • 8 comments

The publish command conveniently sets the ROOT_URL context variable based on the S3 bucket:

        extra_context = {
            "ROOT_URL": bucket_url,
            "S3_BUCKET": bucket_url.root,
            "BUCKET_NAME": bucket_name,
        }

This extra context gets passed to tarbell_generate().

However, TarbellSite.generate_static_site(), which does the heavy lifting, doesn't do anything with the extra context when it calls freeze().

A hook in a blueprint like this works as a workaround, but it seems like the extra context should get merged by default:

@register_hook('generate')    
def merge_extra_context(site, output_root, extra_context):
    context = site.get_context()
    context.update(**extra_context)

If all that needs to happen is essentially what's in the hook above, I'm happy to submit a pull request for this.

ghing avatar Sep 11 '15 16:09 ghing

@eads: Have you had a chance to look at this?

ghing avatar Dec 15 '15 19:12 ghing

Hey Geoff; I'm sorry to say I haven't. To be honest, I won't have a lot of time for the next month or so, but I will try to take a look this weekend.

On Tue, Dec 15, 2015 at 2:50 PM, Geoffrey Hing [email protected] wrote:

@eads https://github.com/eads: Have you had a chance to look at this?

— Reply to this email directly or view it on GitHub https://github.com/tarbell-project/tarbell/issues/370#issuecomment-164875251 .

David Eads | http://recoveredfactory.net

"Medical statistics will be our standard of measurement: we will weigh life for life and see where the dead lie thicker, among the workers or among the privileged." -- Rudolf Virchow

eads avatar Jan 08 '16 14:01 eads

No worries. I can also just send a pull request. I'm mostly just looking for someone from another org to replicate this bug.

ghing avatar Jan 08 '16 17:01 ghing

David, hello;

a few months ago, I was trying to create my first tarbell and made the install process until the end. After a few months with no time to spare, I decided to get back to it. However, when I tried to start a new project, I've received the following message:

Winterfell-2:~ MacBookPro$ tarbell newproject -bash: tarbell: command not found

All supporting software (Python, Pip, Virtualenv) are installed. So I tried to install it again:

Winterfell-2:~ MacBookPro$ pip install tarbell

But a long error message came out after an apparently successful start (hightlighting in red the part of the Terminal error message that had this color):

Collecting tarbell Using cached tarbell-1.0.4.tar.gz Requirement already satisfied (use --upgrade to upgrade): Flask==0.10.1 in /Library/Python/2.7/site-packages (from tarbell) Requirement already satisfied (use --upgrade to upgrade): Jinja2==2.7.3 in /Library/Python/2.7/site-packages (from tarbell) Requirement already satisfied (use --upgrade to upgrade): Markdown==2.4.1 in /Library/Python/2.7/site-packages (from tarbell) Requirement already satisfied (use --upgrade to upgrade): MarkupSafe==0.23 in /Library/Python/2.7/site-packages (from tarbell) Requirement already satisfied (use --upgrade to upgrade): Frozen-Flask==0.11 in /Library/Python/2.7/site-packages (from tarbell) Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.11 in /Library/Python/2.7/site-packages (from tarbell) Requirement already satisfied (use --upgrade to upgrade): boto==2.38.0 in /Library/Python/2.7/site-packages (from tarbell) Requirement already satisfied (use --upgrade to upgrade): clint==0.4.1 in /Library/Python/2.7/site-packages (from tarbell) Requirement already satisfied (use --upgrade to upgrade): gnureadline==6.3.3 in /Library/Python/2.7/site-packages (from tarbell) Requirement already satisfied (use --upgrade to upgrade): google-api-python-client==1.3.1 in /Library/Python/2.7/site-packages (from tarbell) Requirement already satisfied (use --upgrade to upgrade): keyring==5.3 in /Library/Python/2.7/site-packages (from tarbell) Collecting python-dateutil>=2.2 (from tarbell) Using cached python_dateutil-2.4.2-py2.py3-none-any.whl Collecting requests==2.3.0 (from tarbell) Using cached requests-2.3.0-py2.py3-none-any.whl Collecting sh==1.09 (from tarbell) Using cached sh-1.09.tar.gz Requirement already satisfied (use --upgrade to upgrade): wsgiref==0.1.2 in /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7 (from tarbell) Collecting xlrd==0.9.3 (from tarbell) Using cached xlrd-0.9.3.tar.gz Requirement already satisfied (use --upgrade to upgrade): Werkzeug>=0.7 in /Library/Python/2.7/site-packages (from Flask==0.10.1->tarbell) Requirement already satisfied (use --upgrade to upgrade): itsdangerous>=0.21 in /Library/Python/2.7/site-packages (from Flask==0.10.1->tarbell) Requirement already satisfied (use --upgrade to upgrade): args in /Library/Python/2.7/site-packages (from clint==0.4.1->tarbell) Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8 in /Library/Python/2.7/site-packages (from google-api-python-client==1.3.1->tarbell) Requirement already satisfied (use --upgrade to upgrade): oauth2client>=1.3 in /Library/Python/2.7/site-packages (from google-api-python-client==1.3.1->tarbell) Requirement already satisfied (use --upgrade to upgrade): uritemplate>=0.6 in /Library/Python/2.7/site-packages (from google-api-python-client==1.3.1->tarbell) Requirement already satisfied (use --upgrade to upgrade): six>=1.5 in /Library/Python/2.7/site-packages (from python-dateutil>=2.2->tarbell) Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in /Library/Python/2.7/site-packages (from oauth2client>=1.3->google-api-python-client==1.3.1->tarbell) Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 in /Library/Python/2.7/site-packages (from oauth2client>=1.3->google-api-python-client==1.3.1->tarbell) Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in /Library/Python/2.7/site-packages (from oauth2client>=1.3->google-api-python-client==1.3.1->tarbell) Requirement already satisfied (use --upgrade to upgrade): simplejson>=2.5.0 in /Library/Python/2.7/site-packages (from uritemplate>=0.6->google-api-python-client==1.3.1->tarbell) Installing collected packages: python-dateutil, requests, sh, xlrd, tarbell Found existing installation: python-dateutil 1.5 Uninstalling python-dateutil-1.5: Exception: Traceback (most recent call last): File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/basecommand.py", line 211, in main status = self.run(options, args) File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/commands/install.py", line 311, in run root=options.root_path, File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/req/req_set.py", line 640, in install requirement.uninstall(auto_confirm=True) File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/req/req_install.py", line 716, in uninstall paths_to_remove.remove(auto_confirm) File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/req/req_uninstall.py", line 125, in remove renames(path, new_path) File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/utils/init.py", line 315, in renames shutil.move(old, new) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 299, in move copytree(src, real_dst, symlinks=True) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 208, in copytree raise Error, errors Error: [('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/init.py', '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/init.py', "[Errno 1] Operation not permitted: '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/init.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/init.pyc', '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/init.pyc', "[Errno 1] Operation not permitted: '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/init.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.py', '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.py', "[Errno 1] Operation not permitted: '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.pyc', '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.pyc', "[Errno 1] Operation not permitted: '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.py', '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.py', "[Errno 1] Operation not permitted: '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.pyc', '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.pyc', "[Errno 1] Operation not permitted: '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.py', '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.py', "[Errno 1] Operation not permitted: '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.pyc', '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.pyc', "[Errno 1] Operation not permitted: '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.py', '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.py', "[Errno 1] Operation not permitted: '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.pyc', '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.pyc', "[Errno 1] Operation not permitted: '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.py', '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.py', "[Errno 1] Operation not permitted: '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.pyc', '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.pyc', "[Errno 1] Operation not permitted: '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.py', '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.py', "[Errno 1] Operation not permitted: '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.pyc', '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.pyc', "[Errno 1] Operation not permitted: '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/init.py', '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/init.py', "[Errno 1] Operation not permitted: '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/init.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/init.pyc', '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/init.pyc', "[Errno 1] Operation not permitted: '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/init.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/zoneinfo-2010g.tar.gz', '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/zoneinfo-2010g.tar.gz', "[Errno 1] Operation not permitted: '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/zoneinfo-2010g.tar.gz'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo', '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo', "[Errno 1] Operation not permitted: '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil', '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil', "[Errno 1] Operation not permitted: '/var/folders/3z/kd_ybxm91fdgtnm85782x2v00000gn/T/pip-S0uXRS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil'")]

Woudl you be able to tell me what I can do here?

Thanks a lot and my best wishes

Cassiano

On 8 January 2016 at 15:25, Geoffrey Hing [email protected] wrote:

No worries. I can also just send a pull request. I'm mostly just looking for someone from another org to replicate this bug.

— Reply to this email directly or view it on GitHub https://github.com/tarbell-project/tarbell/issues/370#issuecomment-170064303 .

cgobbet avatar Jan 18 '16 19:01 cgobbet

@moxphere It looks like you might not have permissions to install Tarbell in the default directory. Try this:

sudo pip install tarbell

Also, in the future it would be great if you opened a separate ticket for issues like this. That way it's more likely for someone to see it, rather than have it burried in the comments of an unrelated issue. Thanks!

ghing avatar Jan 18 '16 20:01 ghing

On 18 January 2016 at 18:19, Geoffrey Hing [email protected] wrote:

sudo pip install tarbell

​Geoffrey, no luck but this time, there is a message about the 'sudo':

The directory '/Users/MacBookPro/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. The directory '/Users/MacBookPro/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

What should I do to follow this advice? Can you tell me what I should type in Terminal?

Thanks a lot and my best wishes

cgobbet avatar Jan 18 '16 22:01 cgobbet

@moxphere I moved your issue over to https://github.com/tarbell-project/tarbell/issues/395. To avoid confusing what you're running into with the original issue here (#370). I'll follow up with you over in #395.

ghing avatar Jan 18 '16 23:01 ghing

@eads, I've always installed Python using Homebrew, which is also what's suggested by the Hitchhiker's Guide to Python. This avoids some (most?) of the permissions issues with trying to install packages using the system Python. Do you think this is a reasonable suggestion for @moxphere/others?

ghing avatar Jan 18 '16 23:01 ghing