ValueError: unmarshallable object (python 3)

[kwist-sgr]

Python 2.7.12

>>> import marshal
>>> import unipath
>>> path=unipath.Path('/home')
>>> marshal.dumps(path)
's\x05\x00\x00\x00/home'

Python 3.5.2

>>> import marshal
>>> import unipath
>>> path=unipath.Path('/home')
>>> marshal.dumps(path)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: unmarshallable object

[levic]

I'm not convinced this should work. From the python documentation (2 and 3)

The marshal module exists mainly to support reading and writing the “pseudo-compiled” code for Python modules of .pyc files Not all Python object types are supported; in general, only objects whose value is independent from a particular invocation of Python can be written and read by this module. The following types are supported: .... If you’re serializing and de-serializing Python objects, use the pickle module instead ... pickle supports a substantially wider range of objects than marshal

It appears to work in python 2 only because unipath.Path is a subclass of AbstractPath which is a subclass of unicode. Unmarshalling does not work in python 2:

>>> marshal.loads(marshal.dumps('/hello/world'))
'/hello/world'
>>> marshal.loads(marshal.dumps(unipath.Path('/hello/world')))
'/\x00h\x00e\x00l\x00l\x00o\x00/\x00w\x00o\x00r\x00l\x00d\x00'

Additionally, python 2 just treats a string subclass as a string and throws away any extra attributes:

import marshal
class ExtraString(unicode):
    def __init__(self):
        self.extra = 5

>>> marshal.dumps(u'') == marshal.dumps(ExtraString())
True

Finally, this isn't specific to unipath, you can't marshal arbitrary class instances:

>>> marshal.dumps(object())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: unmarshallable object

In short: the problem is actually that python 2 is silently accepting something it shouldn't

[levic]

It appears that marshal is in fact intended to throw an exception if you try to marshal a built-in subclass, but the actual implementation in 2.6 & 2.7 doesn't actually do this for string subtypes.

It does fail loudly at least from 3.0 onwards -- I can't find any relevant change but presumably had something to do with all the 3.x unicode changes.

[kwist-sgr]

I use unipath in Jinja2 Bytecode Cache for Django project

Python 2.7

from unipath import Path

# Absolute filesystem path to the Django project directory:
DJANGO_ROOT = Path(__file__).ancestor(2)

# Absolute filesystem path to the top-level project folder:
SITE_ROOT = DJANGO_ROOT.parent

TEMPLATES = [
    {
        'BACKEND': 'service.core.jinja2.engine.Jinja2',
        'NAME': 'jinja',
        'DIRS': [SITE_ROOT.child('templates')],
        'APP_DIRS': False,
        'OPTIONS': {
            'context_processors': CONTEXT_PROCESSORS,
            'environment': 'service.core.jinja2.environment',
            'autoescape': False,
            'newstyle_gettext': False,
            'debug': DEBUG,
            
        },
    },

This error [ValueError: unmarshallable object] appeared after migration to Python 3.x

workaround:

'DIRS': [unicode(SITE_ROOT.child('templates'))],

[levic]

@kwist-sgr see the link in my previous comment: if jinja you were not supposed to be able to do this in python 2 but python didn't check for subtypes of built-in classes.

Python 3 specifically checks for this. Unipath can't change this -- it's core python behaviour. If jinja's cache is using marshall then you need to convert it to a built-in type

Try replacing

'DIRS': [SITE_ROOT.child('templates')],

with

'DIRS': [str(SITE_ROOT.child('templates'))],