permute grayscale data as augmentation

[kaczmarj]

To approximate something like "stylized imagenet", let's try to permute the grayscale data in the features volumes. Hopefully this will cause the models to rely more on the shape of the features and less on the texture.

For example, values of 0 can become 0.5 and 0.5 can become 0.2, etc. Permutation ensures there are no clashes.

[kaczmarj]

sample code

def permute_values(x, quantize=True):
    x = tf.convert_to_tensor(x)

    if quantize:
        x = tf.quantization.quantize(
            x, min_range=tf.reduce_min(x), max_range=tf.reduce_max(x), T=tf.qint8)
        x = tf.cast(x.output, tf.int32)
        
    uniq, _ = tf.unique(tf.reshape(x, (-1,)))
    uniq_shuffled = tf.random.shuffle(uniq)
    
    def _replace(x, keys, values):
        sidx = tf.argsort(keys)
        ks = tf.gather(keys, sidx)
        vs = tf.gather(values, sidx)
        idx = tf.searchsorted(ks, tf.reshape(x, (-1,)))
        idx = tf.reshape(idx, x.shape)
        return tf.gather(vs, idx)
    
    return _replace(x, keys=uniq, values=uniq_shuffled)

[kaczmarj]

this is the updated permutation code:

def permute_values(x):

    def normalize(x, max_value=12):
        min_ = tf.reduce_min(x)
        max_ = tf.reduce_max(x)
        return (x - min_) * (max_value / (max_ - min_))

    def _replace(x, keys, values):
        sidx = tf.argsort(keys)
        ks = tf.gather(keys, sidx)
        vs = tf.gather(values, sidx)
        idx = tf.searchsorted(ks, tf.reshape(x, (-1,)))
        idx = tf.reshape(idx, x.shape)
        return tf.gather(vs, idx)

    x = tf.convert_to_tensor(x)
    x = normalize(x)
    x = tf.round(x)
    uniq, _ = tf.unique(tf.reshape(x, (-1,)))
    uniq_shuffled = tf.random.shuffle(uniq)
    return _replace(x, keys=uniq, values=uniq_shuffled)

[kaczmarj]

the random permutation makes images like this:

[Aakanksha-Rana]

Isnt this somewhat realized with intensity transformations available in intensity transformations such as noise addition ?

[satra]

@Aakanksha-Rana - is there a quantized intensity transform? this augmentation would be something similar to that. this is different from noise addition. it's binning intensities and then swapping them with parameters being perhaps number of bins or edges of bins. so not a simple gray level quantization, although that can be one of the cases.

[Aakanksha-Rana]

We don't do any quantize-ing yet in any of our augmentation, for sure. But this can be done and should be easy. Quantize-ing intensity and not using the continous gray-scale values, we would be simply compressing the intensity range further and remove important textured based information (as jakub also pointed). Indeed computationally, it is definitely useful, and may also be useful for localization and detection tasks where coarse intensity values make some important description (but not for parcellation or segmentation, I think).