Easy Audit throws exception when saving model with a BinaryField

[pmi123]

I have a model that stores data of unknown type in a BinaryField. The data is saved using pickle.dumps and read using pickle.loads. I need to do this because the APIs that return the image's descriptor returns the data in different formats depending on the algorithm and type of image. New algorithms come along frequently, so checking the type on each model save is not viable.

class ImageDescriptor(Model):
    image_descriptor_id = models.AutoField(primary_key=True)
    document_id = models.ForeignKey(Document, on_delete=models.CASCADE, )
    algorithm = models.CharField(max_length=255)
    _descriptor = models.BinaryField(default = 0)
    language = models.CharField(max_length=255, default=settings.BASE_LANGUAGE)
    created = models.DateTimeField(auto_now_add=True, editable=False, verbose_name="date created")
    updated = models.DateTimeField(auto_now=True, editable=False, verbose_name="last update")  
    
    def set_data(self, data):
        self._descriptor = pickle.dumps(data)
            
    def get_data(self):
        return pickle.loads(self._descriptor)
    
    descriptor = property(get_data, set_data) 

Easy Audit throws an exception on each model save:

[2021-12-04 09:01:50,686: ERROR/ForkPoolWorker-6] easy audit had a pre-save exception.
Traceback (most recent call last):
  File "/home/mark/.virtualenvs/archive/lib/python3.6/site-packages/django/utils/encoding.py", line 62, in force_str
    s = str(s, encoding, errors)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mark/.virtualenvs/archive/lib/python3.6/site-packages/easyaudit/signals/model_signals.py", line 69, in pre_save
    delta = model_delta(old_model, instance)
  File "/home/mark/.virtualenvs/archive/lib/python3.6/site-packages/easyaudit/utils.py", line 53, in model_delta
    old_value = get_field_value(old_model, field)
  File "/home/mark/.virtualenvs/archive/lib/python3.6/site-packages/easyaudit/utils.py", line 30, in get_field_value
    value = smart_text(getattr(obj, field.name, None))
  File "/home/mark/.virtualenvs/archive/lib/python3.6/site-packages/django/utils/encoding.py", line 107, in smart_text
    return smart_str(s, encoding, strings_only, errors)
  File "/home/mark/.virtualenvs/archive/lib/python3.6/site-packages/django/utils/encoding.py", line 31, in smart_str
    return force_str(s, encoding, strings_only, errors)
  File "/home/mark/.virtualenvs/archive/lib/python3.6/site-packages/django/utils/encoding.py", line 66, in force_str
    raise DjangoUnicodeDecodeError(s, *e.args)
django.utils.encoding.DjangoUnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte. You passed in b'\x80\x03cimagehash\nImageHash\nq\x00)\x81q\x01}q\x02X\x04

As a work around, I can just use DJANGO_EASY_AUDIT_UNREGISTERED_CLASSES_EXTRA = ['memorabilia.ImageDescriptor',], but I thought you would interested in this use case for Easy Audit when data is stored in a BinaryField.

[jheld]

good idea, yes if it's possible that would be quite beneficial.

I don't have all of the supporting functions and background in my head but I wonder if this would be a good opportunity to leverage a custom processing function, provided/referenced at the project level, and thus allows flexible customization of this type of request. I thought we had made a custom serialization ability, but even if we did, I don't know if that would be the right fit for this, either. Can you think on that a bit and see if either of those solutions would solve the problem?

[pmi123]

Forgot to include the version of Easy Audit. Same exception thrown for both Easy Audit 1.2 and 1.3.

I assume @jheld, that your question was directed to someone else?

[jheld]

No. I was just asking if an improvement like I describe above, facilitated by the library, would be useful for solving your issue.

[pmi123]

@jheld I am not sure I completely understand your proposed solutions. Can you elaborate a little more on these two solutions?

Thanks!

[jheld]

Sure thing.

I think it would be beneficial for the library to provide a custom callable (set at the project level in settings) for allowing different implementations of the logic contained in such things as pre_save.

So for instance, get_field_value could have an override. Or potentially whatever larger flow happening within pre_save could. Would have to see an implementation before deciding which is better and still keeps things stable.

[pmi123]

@jheld I think that will work. I am sorry, I just don't have the bandwidth to give this the serious thought it deserves. I defer to your better judgement. I also want to thank you for looking at this issue so quickly!