plotHeatmap not working with NMF development version

[rufletch]

Error in strsplit(newName, "\\.") : non-character argument

This error occurs with all plotHeatmap commands that I've tested. It does not prevent producing the heat map, for instance, making a pdf of the heatmap, but it will stop R from knitting an Rmd file. The error can be traced to line 817, caused by an issue in line 816.

[diyadas]

I've also had the same issue on my machine and gotten past the error by wrapping the plotHeatmap command in a try statement, but this is obviously not ideal.

[epurdom]

What version of clusterExperiment is this? Does it happen on the develop branch or the current release?

Any chance you have some example I could run to reproduce? (I have made a small change to develop around this code, but I don't see how it would effect this. Otherwise, this is very old code so I'm a bit baffled.)

[diyadas]

I think it may be linked to the development version of NMF (which we installed to get support for the na.color argument). In this version of NMF, line 816 of plotHeatmap (NMF:::vplayout(NULL)) produces FALSE, which is causing the error in line 817. (I have clusterExperiment 1.4.0 which is loading NMF 0.23.6.)

Here's sample code which produces the same error for me:

mat=matrix(runif(10, 5.0, 10), 2,5)
mat[c(4,7,10)] <- NA
plotHeatmap(mat, clusterSamples=F, clusterFeatures=F, labRow = c("A","B"), 
sampleData=data.frame(clusters=c(1,1,2,3,4)), 
clusterLegend=list(clusters=c("red","blue","green","purple")), cexRow = .4, annLegend=F, 
layout='_', na.color="black", labCol=rep("", ncol(mat)))

As far as a temporary patch is concerned, would it be possible to alter the code so a warning is generated instead of an error? We are getting the heatmaps we want; it's just that the error stops the knitting of Rmd files.

[epurdom]

Thanks Diya, that is very helpful, because I wouldn't have found that since I'm not using the development version. I have pushed a patch to the develop branch of cluster experiment to skip this step if it hits an error there (this is what puts names next to the colored rectangles next to the samples). Try it and see if it at least clears without giving an error.

[rufletch]

Thanks, Elizabeth. I'll update to the latest clusterExperiment and give it a shot later today.

Best, Russell

On Sun, Jan 28, 2018 at 3:32 PM, Elizabeth Purdom [email protected] wrote:

Thanks Diya, that is very helpful, because I wouldn't have found that since I'm not using the development version. I have pushed a patch to the develop branch of cluster experiment to skip this step if it hits an error there (this is what puts names next to the colored rectangles next to the samples). Try it and see if it at least clears without giving an error.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/epurdom/clusterExperiment/issues/235#issuecomment-361106698, or mute the thread https://github.com/notifications/unsubscribe-auth/APdpML6m8rO6viaYWT1eHHFup2NF0sz7ks5tPQOSgaJpZM4Ru7hi .

[rufletch]

Works! Thank you.

On Mon, Jan 29, 2018 at 9:24 AM, Russell Fletcher [email protected] wrote:

Thanks, Elizabeth. I'll update to the latest clusterExperiment and give it a shot later today.

Best, Russell

On Sun, Jan 28, 2018 at 3:32 PM, Elizabeth Purdom < [email protected]> wrote:

Thanks Diya, that is very helpful, because I wouldn't have found that since I'm not using the development version. I have pushed a patch to the develop branch of cluster experiment to skip this step if it hits an error there (this is what puts names next to the colored rectangles next to the samples). Try it and see if it at least clears without giving an error.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/epurdom/clusterExperiment/issues/235#issuecomment-361106698, or mute the thread https://github.com/notifications/unsubscribe-auth/APdpML6m8rO6viaYWT1eHHFup2NF0sz7ks5tPQOSgaJpZM4Ru7hi .

[rufletch]

Hi Elizabeth,

Well - I emailed before inspecting the heatmaps carefully. It works, but now it plots the color rectangles for which there are no samples. I've attached a screen shot of a heatmap where there are seven clusters. Let me know if I should create another issue.

thanks, Russell

On Mon, Jan 29, 2018 at 12:41 PM, Russell Fletcher [email protected] wrote:

Works! Thank you.

On Mon, Jan 29, 2018 at 9:24 AM, Russell Fletcher [email protected] wrote:

Thanks, Elizabeth. I'll update to the latest clusterExperiment and give it a shot later today.

Best, Russell

On Sun, Jan 28, 2018 at 3:32 PM, Elizabeth Purdom < [email protected]> wrote:

Thanks Diya, that is very helpful, because I wouldn't have found that since I'm not using the development version. I have pushed a patch to the develop branch of cluster experiment to skip this step if it hits an error there (this is what puts names next to the colored rectangles next to the samples). Try it and see if it at least clears without giving an error.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/epurdom/clusterExperiment/issues/235#issuecomment-361106698, or mute the thread https://github.com/notifications/unsubscribe-auth/APdpML6m8rO6viaYWT1eHHFup2NF0sz7ks5tPQOSgaJpZM4Ru7hi .

[epurdom]

Hi Russell, The image is not coming through because this is through github issues tracking and the attachments to those don't get posted. Can you go to the issue tracking online and upload the image there?

Is this in legend, you mean? Is it showing levels of the clustering factor for which there are no samples (e.g. because you subset the data)?

Thanks, Elizabeth

[rufletch]

Hi Elizabeth, This is an example of the heatmaps I'm getting - in this case there are 11 clusters. On my end I can probably solve it by subsetting the color palette for the clusters to be the same length as the number of clusters. Best, Russell

[epurdom]

Thanks Russell. I think I know why this is happening (I made a change to fix another problem…). I’ll try to look into shortly.

On Jan 30, 2018, at 9:45 PM, Russell [email protected] wrote:

https://user-images.githubusercontent.com/16214320/35590266-24bc6adc-05bb-11e8-864d-051f4f870e09.png Hi Elizabeth, This is an example of the heatmaps I'm getting - in this case there are 11 clusters. On my end I can probably solve it by subsetting the color palette for the clusters to be the same length as the number of clusters. Best, Russell

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/epurdom/clusterExperiment/issues/235#issuecomment-361728540, or mute the thread https://github.com/notifications/unsubscribe-auth/AHXGVRwAMfm_tpJQIsnmP0ymsHQi-39Zks5tP398gaJpZM4Ru7hi.

[epurdom]

Hi Russell, I'm having a hard time replicating this. Could you give me some information about the vector giving the cluster identification and the color vector that goes along with it?

In particular, if clusters is the vector giving the cluster assignment,

1. Is clusters a factor
2. If so what are the levels (levels(clusters))
3. If not, what is class(clusters)
4. What does table(clusters,useNA="ifany") look like?
5. Does clusters have NA values?

If clusterColors is the vector of colors,

1. What are the names of clusterColors (if any)

Thanks, Elizabeth

[rufletch]

Hi Elizabeth,

----Yes, clusters (scone20_cl) is a factor with 11 levels.

----levels(scone20_cl) [1] "1" "2" "3" "4" "5" "6" "7" "8" "9" "10" "11"

----table(scone20_cl) scone20_cl 1 2 3 4 5 6 7 8 9 10 11 1340 1247 773 434 359 211 142 139 136 28 16

----No, there are no NA values.

----The color vector is pal <- clusterExperiment::bigPalette

Here is an example of code for the image: plotHeatmap(log2(sconeNorm[unique(top5_scone20$IndexInOriginal),]+1), clusterSamplesData = ce@dendro_samples, clusterFeatures=T, sampleData=data.frame(clusters=scone20_cl, expt=colData(ce)[,2]), clusterLegend=list(clusters=pal, expt=cole), main=paste("oneVall5_SNNscone20PC_sconeNorm"), labCol=NA)

Hope this helps.

Thanks, Russell

On Wed, Jan 31, 2018 at 1:00 AM, Elizabeth Purdom [email protected] wrote:

Hi Russell, I'm having a hard time replicating this. Could you give me some information about the vector giving the cluster identification and the color vector that goes along with it?

In particular, if clusters is the vector giving the cluster assignment,

1. Is clusters a factor
2. If so what are the levels (levels(clusters))
3. If not, what is class(clusters)
4. What does table(clusters,useNA="ifany") look like?
5. Does clusters have NA values?

If clusterColors is the vector of colors,

1. What are the names of clusterColors (if any)

Thanks, Elizabeth

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/epurdom/clusterExperiment/issues/235#issuecomment-361866721, or mute the thread https://github.com/notifications/unsubscribe-auth/APdpMJ4elj22YSAgzGK4J0jalk-Uv3hCks5tQCuygaJpZM4Ru7hi .

[epurdom]

One last question, are you using the development version of NMF? Can you send me a copy of your sessionInfo()?

I’ve tried to create similar code to yours with random data. This code works fine for me (i.e. doesn’t have the problem with legend). Can you try it and tell me if it replicates your problem on your end? I’m wondering if this has something to do with changes in the development version, but I would like to know that I have a replicable example before I dig into the development version of NMF.

set.seed(478931) pal <- clusterExperiment::bigPalette clFac<-sample(factor(rep(1:11,times=c(1340, 1247 ,773 ,434 ,359, 211 ,142 ,139, 136, 28 ,16)))) n<-length(clFac) mat<-matrix(rnorm(n=5*n), ncol=n) dendro<-as.dendrogram(hclust(dist(t(mat)))) plotHeatmap(mat, clusterSamplesData = dendro, clusterFeatures=T, sampleData=data.frame(clusters=clFac, expt=clFac), clusterLegend=list(clusters=pal, expt=pal[1:11]), main=paste("test"), labCol=NA)

On Feb 1, 2018, at 9:06 PM, Russell [email protected] wrote:

Hi Elizabeth,

----Yes, clusters (scone20_cl) is a factor with 11 levels.

----levels(scone20_cl) [1] "1" "2" "3" "4" "5" "6" "7" "8" "9" "10" "11"

----table(scone20_cl) scone20_cl 1 2 3 4 5 6 7 8 9 10 11 1340 1247 773 434 359 211 142 139 136 28 16

----No, there are no NA values.

----The color vector is pal <- clusterExperiment::bigPalette

Here is an example of code for the image: plotHeatmap(log2(sconeNorm[unique(top5_scone20$IndexInOriginal),]+1), clusterSamplesData = ce@dendro_samples, clusterFeatures=T, sampleData=data.frame(clusters=scone20_cl, expt=colData(ce)[,2]), clusterLegend=list(clusters=pal, expt=cole), main=paste("oneVall5_SNNscone20PC_sconeNorm"), labCol=NA)

Hope this helps.

Thanks, Russell

On Wed, Jan 31, 2018 at 1:00 AM, Elizabeth Purdom [email protected] wrote:

Hi Russell, I'm having a hard time replicating this. Could you give me some information about the vector giving the cluster identification and the color vector that goes along with it?

In particular, if clusters is the vector giving the cluster assignment,

1. Is clusters a factor
2. If so what are the levels (levels(clusters))
3. If not, what is class(clusters)
4. What does table(clusters,useNA="ifany") look like?
5. Does clusters have NA values?

If clusterColors is the vector of colors,

1. What are the names of clusterColors (if any)

Thanks, Elizabeth

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/epurdom/clusterExperiment/issues/235#issuecomment-361866721, or mute the thread https://github.com/notifications/unsubscribe-auth/APdpMJ4elj22YSAgzGK4J0jalk-Uv3hCks5tQCuygaJpZM4Ru7hi .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/epurdom/clusterExperiment/issues/235#issuecomment-362386110, or mute the thread https://github.com/notifications/unsubscribe-auth/AHXGVYb3PmfaPBinNDTx6azri2xoM4BSks5tQhlegaJpZM4Ru7hi.

[drisso]

If some changes in the development version of NMF caused this, is this a good time to switch to pheatmap?

I see at least two advantages:

• In my experience pheatmap is more stable than aheatmap (e.g., it can be displayed in RStudio)
• Depending on the NMF package seems an overkill since we're only using its heat map function.

[rufletch]

Elizabeth,

I bet that's it. I'm using NMF_0.23.6.

A few weeks back, I think I changed to the development version of NMF, hoping to be able to plot in RStudio. It didn't solve that problem, but I never switched back.

Best, Russell

Here's the sessionInfo():

sessionInfo() R version 3.4.3 (2017-11-30) Platform: x86_64-apple-darwin15.6.0 (64-bit) Running under: macOS Sierra 10.12.6

Matrix products: default BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/ A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib LAPACK: /Library/Frameworks/R.framework/Versions/3.4/ Resources/lib/libRlapack.dylib

locale: [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages: [1] parallel stats4 stats graphics grDevices utils datasets methods base

other attached packages: [1] Seurat_2.2.0 Hmisc_4.1-1 Formula_1.2-2 [4] survival_2.41-3 cellrangerRkit_2.0.0 Rmisc_1.5

[7] plyr_1.8.4 lattice_0.20-35 bit64_0.9-7

[10] bit_1.1-12 RColorBrewer_1.1-2 Matrix_1.2-12 [13] zinbwave_1.0.0 Rtsne_0.13 cowplot_0.9.2 [16] magrittr_1.5 clusterExperiment_1.5.0-9012 scone_1.2.0

[19] scater_1.6.2 SingleCellExperiment_1.0.0 ggplot2_2.2.1 [22] SummarizedExperiment_1.8.1 DelayedArray_0.4.1 matrixStats_0.53.0 [25] Biobase_2.38.0 GenomicRanges_1.30.1 GenomeInfoDb_1.14.0 [28] IRanges_2.12.0 S4Vectors_0.16.0 BiocGenerics_0.24.0 [31] BiocInstaller_1.28.0

loaded via a namespace (and not attached): [1] rtracklayer_1.38.3 ModelMetrics_1.1.0 prabclus_2.2-6

[4] R.methodsS3_1.7.1 pkgmaker_0.26.11 tidyr_0.7.2

[7] acepack_1.4.1 knitr_1.19 irlba_2.3.2

[10] aroma.light_3.8.0 R.utils_2.6.0 data.table_1.10.4-3

[13] rpart_4.1-12 hwriter_1.3.2 RCurl_1.95-4.10

[16] doParallel_1.0.11 GenomicFeatures_1.30.1 RSQLite_2.0

[19] VGAM_1.0-4 proxy_0.4-21 phylobase_0.8.4

[22] bayesm_3.1-0.1 lubridate_1.7.1 xml2_1.2.0

[25] httpuv_1.3.5 bold_0.5.0 assertthat_0.2.0

[28] viridis_0.4.1 gower_0.1.2 tximport_1.6.0

[31] DEoptimR_1.0-8 progress_1.1.2 caTools_1.17.1

[34] dendextend_1.6.0 igraph_1.1.2 DBI_0.7

[37] geneplotter_1.56.0 htmlwidgets_1.0 reshape_0.8.7

[40] tensorA_0.36 EDASeq_2.12.0 ddalpha_1.3.1

[43] rARPACK_0.11-0 purrr_0.2.4 RSpectra_0.12-0

[46] dplyr_0.7.4 backports_1.1.2 energy_1.7-2

[49] trimcluster_0.1-2 annotate_1.56.1 compositions_1.40-1

[52] gridBase_0.4-7 locfdr_1.1-8 biomaRt_2.34.2

[55] ROCR_1.0-7 caret_6.0-78 withr_2.1.1

[58] sfsmisc_1.1-1 taxize_0.9.0 robustbase_0.92-8

[61] checkmate_1.8.5 GenomicAlignments_1.14.1 prettyunits_1.0.2

[64] mnormt_1.5-5 mclust_5.4 softImpute_1.4

[67] cluster_2.0.6 gsl_1.9-10.3 ape_5.0

[70] ADGofTest_0.3 diffusionMap_1.1-0 segmented_0.5-3.0

[73] lazyeval_0.2.1 crul_0.5.0 genefilter_1.60.0

[76] recipes_0.1.2 glmnet_2.0-13 edgeR_3.20.7

[79] pkgconfig_2.0.1 nlme_3.1-131 vipor_0.4.5

[82] nnet_7.3-12 bindr_0.1 rlang_0.1.6

[85] diptest_0.75-7 registry_0.5 rngtools_1.2.4

[88] boot_1.3-20 base64enc_0.1-3 beeswarm_0.2.3

[91] ggridges_0.4.1 whisker_0.3-2 pheatmap_1.0.8

[94] viridisLite_0.2.0 rjson_0.2.15 stabledist_0.7-1

[97] bitops_1.0-6 shinydashboard_0.6.1 R.oo_1.21.0

[100] rncl_0.8.2 KernSmooth_2.23-15 Biostrings_2.46.0

[103] blob_1.1.0 DRR_0.0.3 lars_1.2

[106] stringr_1.2.0 ShortRead_1.36.0 scales_0.5.0

[109] ica_1.0-1 memoise_1.1.0 hexbin_1.27.2

[112] howmany_0.3-1 gplots_3.0.1 gdata_2.18.0

[115] zlibbioc_1.24.0 compiler_3.4.3 dimRed_0.1.0

[118] dtw_1.18-1 Rsamtools_1.30.0 ade4_1.7-10

[121] XVector_0.18.0 pbapply_1.3-4 htmlTable_1.11.2

[124] tidyselect_0.2.3 MASS_7.3-48 stringi_1.1.6

[127] copula_0.999-18 yaml_2.1.16 locfit_1.5-9.1

[130] latticeExtra_0.6-28 grid_3.4.3 tools_3.4.3

[133] rstudioapi_0.7 uuid_0.1-2 foreach_1.4.4

[136] foreign_0.8-69 RNeXML_2.0.8 gridExtra_2.3

[139] prodlim_1.6.1 scatterplot3d_0.3-40 digest_0.6.13

[142] lava_1.6 FNN_1.1 shiny_1.0.5

[145] fpc_2.1-11 bindrcpp_0.2 Rcpp_0.12.15

[148] broom_0.4.3 SDMTools_1.1-221 httr_1.3.1

[151] AnnotationDbi_1.40.0 psych_1.7.8 kernlab_0.9-25

[154] colorspace_1.3-2 ranger_0.9.0 XML_3.98-1.9

[157] CVST_0.2-1 splines_3.4.3 RcppRoll_0.2.2

[160] sn_1.5-1 flexmix_2.3-14 xtable_1.8-2

[163] jsonlite_1.5 timeDate_3042.101 ipred_0.9-6

[166] modeltools_0.2-21 tclust_1.3-1 R6_2.2.2

[169] pillar_1.1.0 htmltools_0.3.6 mime_0.5

[172] NMF_0.23.6 glue_1.2.0 pspline_1.0-18

[175] BiocParallel_1.12.0 RMySQL_0.10.13 DESeq_1.30.0

[178] class_7.3-14 RUVSeq_1.12.0 codetools_0.2-15

[181] tsne_0.1-3 pcaPP_1.9-73 mvtnorm_1.0-7

[184] tibble_1.4.2 mixtools_1.1.0 numDeriv_2016.8-1

[187] curl_3.1 ggbeeswarm_0.6.0 gtools_3.5.0

[190] limma_3.34.6 munsell_0.4.3 rhdf5_2.22.0

[193] GenomeInfoDbData_0.99.1 iterators_1.0.9 reshape2_1.4.3

[196] gtable_0.2.0

On Fri, Feb 2, 2018 at 1:36 AM, Elizabeth Purdom [email protected] wrote:

One last question, are you using the development version of NMF? Can you send me a copy of your sessionInfo()?

I’ve tried to create similar code to yours with random data. This code works fine for me (i.e. doesn’t have the problem with legend). Can you try it and tell me if it replicates your problem on your end? I’m wondering if this has something to do with changes in the development version, but I would like to know that I have a replicable example before I dig into the development version of NMF.

set.seed(478931) pal <- clusterExperiment::bigPalette clFac<-sample(factor(rep(1:11,times=c(1340, 1247 ,773 ,434 ,359, 211 ,142 ,139, 136, 28 ,16)))) n<-length(clFac) mat<-matrix(rnorm(n=5*n), ncol=n) dendro<-as.dendrogram(hclust(dist(t(mat)))) plotHeatmap(mat, clusterSamplesData = dendro, clusterFeatures=T, sampleData=data.frame(clusters=clFac, expt=clFac), clusterLegend=list(clusters=pal, expt=pal[1:11]), main=paste("test"), labCol=NA)

On Feb 1, 2018, at 9:06 PM, Russell [email protected] wrote:

Hi Elizabeth,

----Yes, clusters (scone20_cl) is a factor with 11 levels.

----levels(scone20_cl) [1] "1" "2" "3" "4" "5" "6" "7" "8" "9" "10" "11"

----table(scone20_cl) scone20_cl 1 2 3 4 5 6 7 8 9 10 11 1340 1247 773 434 359 211 142 139 136 28 16

----No, there are no NA values.

----The color vector is pal <- clusterExperiment::bigPalette

Here is an example of code for the image: plotHeatmap(log2(sconeNorm[unique(top5_scone20$IndexInOriginal),]+1), clusterSamplesData = ce@dendro_samples, clusterFeatures=T, sampleData=data.frame(clusters=scone20_cl, expt=colData(ce)[,2]), clusterLegend=list(clusters=pal, expt=cole), main=paste("oneVall5_SNNscone20PC_sconeNorm"), labCol=NA)

Hope this helps.

Thanks, Russell

On Wed, Jan 31, 2018 at 1:00 AM, Elizabeth Purdom < [email protected]> wrote:

Hi Russell, I'm having a hard time replicating this. Could you give me some information about the vector giving the cluster identification and the color vector that goes along with it?

In particular, if clusters is the vector giving the cluster assignment,

1. Is clusters a factor
2. If so what are the levels (levels(clusters))
3. If not, what is class(clusters)
4. What does table(clusters,useNA="ifany") look like?
5. Does clusters have NA values?

If clusterColors is the vector of colors,

1. What are the names of clusterColors (if any)

Thanks, Elizabeth

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <https://github.com/epurdom/clusterExperiment/issues/235# issuecomment-361866721>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ APdpMJ4elj22YSAgzGK4J0jalk-Uv3hCks5tQCuygaJpZM4Ru7hi> .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/epurdom/clusterExperiment/issues/235# issuecomment-362386110>, or mute the thread <https://github.com/ notifications/unsubscribe-auth/AHXGVYb3PmfaPBinNDTx6azri2xoM4 BSks5tQhlegaJpZM4Ru7hi>.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/epurdom/clusterExperiment/issues/235#issuecomment-362535112, or mute the thread https://github.com/notifications/unsubscribe-auth/APdpMBG3EwimlIeLOn-T65HcFElmc460ks5tQtcagaJpZM4Ru7hi .

[epurdom]

I’m interested in thinking about pheatmap, but I would note that I think fixing this is a minor issue. I know what caused it (I made a small change in how the colors are handled in plotHeatmap to handle another problem), so I can undoubtable fix it quickly if I download the development version of NMF.

About switching to pheatmap I’m going to switch to the other thread on that one.

On Feb 2, 2018, at 3:46 PM, Davide Risso [email protected] wrote:

If some changes in the development version of NMF caused this, is this a good time to switch to pheatmap?

I see at least two advantages:

In my experience pheatmap is more stable than aheatmap (e.g., it can be displayed in RStudio) Depending on the NMF package seems an overkill since we're only using its heat map function. — You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/epurdom/clusterExperiment/issues/235#issuecomment-362600424, or mute the thread https://github.com/notifications/unsubscribe-auth/AHXGVZ7ozE4P3Pyo9GCecbmVqGZVRhZMks5tQxslgaJpZM4Ru7hi.