arctic icon indicating copy to clipboard operation
arctic copied to clipboard

Chunkstore update with a Passthrough chunker duplicates segments when converting from Python 2.7->3

Open TomTaylorLondon opened this issue 5 years ago • 2 comments

Arctic Version

# All

Arctic Store

# ChunkStore

Platform and version

2.7 vs 3

Description of problem and/or code sample that reproduces the issue

Segments are duplicated when:

  • Data is written in Python 2.7 using Chunkstore Passthrough chunker
  • Data is updated in Python 3

Root cause is:

  • start_date/end_date is b'NA' which in Python 2.7 == str('NA') but in Python 3.0 != str('NA')

Possible fixes:

  1. Remove support for update with Passthrough chunker. It's unclear to me what functionality it provides above a write(..)
  2. Infer if start_date/end_date is str('NA') or bytes('NA') in python 3 and maintain backwards compatibility

@bmoscon thoughts?

TomTaylorLondon avatar Feb 09 '20 18:02 TomTaylorLondon

https://api.mongodb.com/python/current/python3.html indicates we will most likely have the reverse issue too

TomTaylorLondon avatar Feb 09 '20 18:02 TomTaylorLondon

@TomTaylorLondon if you want the history on the passthrough chunker, we can talk offline. Someone internal needed it to write non timeseries data. It may or not be needed anymore.

bmoscon avatar Feb 09 '20 19:02 bmoscon