[Enhancement Proposal] Logarithmic Filter

[GibbonJojo]

Hey,

For a project, I used a logarithmic filter. I though others might find that useful as well.

Code:

class Logarithmic(Codec):
    
    codec_id="logarithmic"
    
    def __init__(self, dtype, astype=None):
        self.dtype = np.dtype(dtype)
        if astype is None:
            self.astype = self.dtype
        else:
            self.astype = np.dtype(astype)
            
        if self.astype.kind != 'f':
            raise ValueError('only floating point data types are supported for astype')

    
    def encode(self, buf):        
        # log10(0) = -inf
        buf[buf==0] += 0.01
        
        return np.log10(buf, dtype=self.astype)
    
    def decode(self, buf):
        return (10**buf).astype(self.dtype)
    
    def get_config(self):
        return dict(
        id=self.codec_id,
        dtype=self.dtype.str,
        astype=self.astype.str
        )
    
    def __repr__(self):
        r = f"{type(self).__name__}(dtype={self.dtype.str}"
        if self.astype != self.dtype:
            r += f', astype={self.astype.str}'
        r += ')'
        return r

Documentation would follow, if you would accept that proposal

Example:

>>> x = np.linspace(0.1, 100, 15)
>>> x
array([  0.1       ,   7.23571429,  14.37142857,  21.50714286,
        28.64285714,  35.77857143,  42.91428571,  50.05      ,
        57.18571429,  64.32142857,  71.45714286,  78.59285714,
        85.72857143,  92.86428571, 100.        ])
>>> codec = Logarithmic(np.float32)
>>> y = codec.encode(x)
>>> y
array([-1.       ,  0.8594814,  1.1574999,  1.3325827,  1.4570163,
        1.553623 ,  1.6326019,  1.6994041,  1.7572875,  1.8083557,
        1.8540456,  1.8953831,  1.9331256,  1.9678488,  2.       ],
      dtype=float32)
>>> z = codec.decode(y)
>>> z
array([  0.1     ,   7.235714,  14.371428,  21.507143,  28.642859,
        35.77857 ,  42.914284,  50.050003,  57.185707,  64.32143 ,
        71.45714 ,  78.592865,  85.72858 ,  92.864296, 100.      ],
      dtype=float32)

If you want that filter in your module, I can implement it according to you standards :)

[alimanfoo]

Thanks @GibbonJojo, very interesting.

I'd love to know more about how this filter can be useful. Did you use it to improve compression ratio, or for some other purpose?

[GibbonJojo]

I mainly needed the decode function, since I had data on a logarithmic scale after decoding the data with another filter. And due to the project structure, it was better to chain these filters together.

Granted, I dont think it improves compression ratio, since using another dtype would result in a precision loss. But I thought, there could be use cases for other people as well; you probably know that better than me