center_shift=nan

[eghouti]

Hello,

Sometimes I get center_shift=nan and I don't understand why and how can I fix this.

[subhadarship]

center_shift can be a very large number when the centroids change a lot (in the initial iterations of the K-means algorithm). I am not sure why it would be nan though. Is it possible for you to reproduce the case when center_shift=nan?

[eghouti]

yes I obtained it several times

[subhadarship]

It would be very useful if you can share the code snippet which you are using so that I can reproduce it at my end.

[eghouti]

I just took resnet18 and compute a kmeans on its weight (independently on each layer)

[subhadarship]

So if I understand correctly, you are trying to cluster the weight vectors from different layers. Can you confirm?

clustering the weights of resnet sounds really cool btw !!

[eghouti]

yes this is exactly what I am trying to do

[subhadarship]

@eghouti what distance metric are you using? euclidean or cosine?

[eghouti]

I used euclidean distance

[NIRVANALAN]

same problem reproduced every time code here https://github.com/NIRVANALAN/Centroid_GCN python train.py --dataset cora --self-loop cluster logits produced by dgl(a grpah ml lib, easy to install)

[HolmesShuan]

Toy problem failed, code as follow

import torch
import numpy as np
from kmeans_pytorch import kmeans

# data
data_size, dims, num_clusters = 1000, 1, 3
x = np.random.randn(data_size, dims) 
x[x<0.] = 0.
x = torch.from_numpy(x)

# kmeans
cluster_ids_x, cluster_centers = kmeans(
    X=x, num_clusters=num_clusters, distance='euclidean', device=torch.device('cuda:0')
)

[GenjiB]

The problem comes from: if a cluster_center is an outliner, there are no neighbor points to calculate the mean point. To calculate the mean of none points results in Nan.

For Line80 in init.py

if torch.isnan(selected.mean(dim=0)).sum()==0:
    initial_state[index] = selected.mean(dim=0)

[YNNEKUW]

In my case, initializing cluster_centers will obtain that error each time, while @GenjiB 's method does work.

[markrmiller]

Was hitting this constantly trying to switch over from a working sklearn kmeans use case. Thanks for the workaround.