pysolr icon indicating copy to clipboard operation
pysolr copied to clipboard

NaN values are not working when simplejson >= 3.19.0 is used

Open andreasmueller92 opened this issue 1 year ago • 0 comments

I have

  • [X] Tested with the latest release
  • [ ] Tested with the current master branch
  • [X] Searched for similar existing issues

Expected behaviour

NaN values should be allowed for upload when using simplejson

Actual behaviour

[...]
  File "<software-path>/lib/python3.11/site-packages/simplejson/encoder.py", line 379, in iterencode
    return _iterencode(o, 0)
           ^^^^^^^^^^^^^^^^^
ValueError: Out of range float values are not JSON compliant

Steps to reproduce the behaviour

  1. Have simplejson >= 3.19.0 available on the system
  2. pysolr.Solr(<SOLR_URL>).add([{'key': float('nan')}])

Reason

In version 3.19.0, simplejson deciced to change the default value for allow_nan to False for simplejson.dumps (Ctrl+F The default for allow_nan), which is used in pysolr 3.9.0 and simplejson.JSONEncoder (Ctrl+F 3.19.0: This default), used in pysolr 3.10 betas.

When not using simplejson but the default Python-JSON (which is done when simplejson is not available), this is not an issue, as NaN is allowed by default. This should be consistent across the different en-/decoders.

Possible solution

Explicitly set allow_nan=True when using simplejson

Configuration

  • Operating system version:
  • Search engine version:
  • Python version: 3.11.3
  • pysolr version: 3.9.0
  • simplejson version: 3.19.1

andreasmueller92 avatar Apr 19 '24 07:04 andreasmueller92