Warning message for Quick-TRANSfer on >300k cells

[jzhou88]

Dear Harmony people,

I got the following warning message when running Harmony on >300k cells, which indicated that the k-means algorithm inside Harmony didn't converge.

"Warning: Quick-TRANSfer stage steps exceeded maximum (= 17686000)"

Before, I came across the same issue when running Harmony on ~200k cells. At that time, I was able to overcome this problem by increasing the initial maximum number of iterations from 100 to 5000 in k-means, which was suggested by you guys in the following page:

https://github.com/immunogenomics/harmony/issues/25

However, this time when I was dealing with >300k cells, no matter how large the number I used (e.g., 50000), the iteration trick just didn't work. I also tried to increase the initial number of start clusters from 20 to 100 in k-means, which didn't work either. By the way, I was using 100GB memory to run Harmony. So, I don't think memory was the issue.

If possible, would you please help me solve this problem? Thanks in advance.

Best, J

[yingyonghui]

Hi @jzhou88 , I just came up with the same question. Have you solved this problem?

Best, Yingyong

[jzhou88]

Hi @yingyonghui ,

No, I haven't. However, I outputted the convergence plot, which looked fine to me. So, I just moved on. Please let me know if you have any thoughts.

Best, J

[amjass12]

any update on this? i too have seen convergence that looks fine...

[CJ-Liu]

I added the options " kmeans_init_nstart=20, kmeans_init_iter_max=100"[ #25] , but still encountered the similar error. According to R documentation of kmeas,when some of the points (rows of x) are extremely close, the algorithm may not converge in the “Quick-Transfer” stage, signalling a warning (and returning ifault = 4). Slight rounding of the data may be advisable in that case. so how to rounding of the data slightly?

https://www.rdocumentation.org/packages/stats/versions/3.6.2/topics/kmeans

[D90-DS]

round slightly? just copy and paste this into R and you'll think of how to change it for whatever level of rounding would fit your needs. tiny_numbers <- seq(.01,.02, by = .0001) tiny_numbers rounded_number = round(tiny_numbers/.002)*.002 rounded_number