Issue with addPeakAnnotations for Custom Deviations

[agalianese]

Attach your log file ArchR-addDeviationsMatrix-c27565ce590-Date-2021-09-16_Time-12-11-10.log

Describe the bug

I am trying to create a custom deviation (following chapter 13.2.3), but while the addPeakAnnotations step completes, the resulting matrix is unavailable. Using the command

testProj <- addPeakAnnotations(ArchRProj = testProj, regions = "mESSuperEnhancers.bed", name = "SuperAnno")

the step completes. But when I try to access the matrix using getMatrixFromProject, I get the following error

Error in h5read(ArrowFile, paste0(subGroup, "/Info/CellNames")) : 
  Object 'SuperAnnoMatrix/Info/CellNames' does not exist in this HDF5 file.

The addDeviationsMatrix step fails as well, and I have attached that log file and the resulting error. Please let me know if I'm doing anything wrong. Thanks!

> testProj <- addDeviationsMatrix(
+   ArchRProj = testProj, 
+   peakAnnotation = "SuperAnno"
+ )
Identifying Background Peaks!
ArchR logging to : ArchRLogs/ArchR-addDeviationsMatrix-c27565ce590-Date-2021-09-16_Time-12-11-10.log
If there is an issue, please report to github with logFile!
NULL
2021-09-16 12:11:15 : Batch Execution w/ safelapply!, 0 mins elapsed.
2021-09-16 12:11:17 : chromVAR deviations GFP+ (1 of 1) Schep (2017), 0.036 mins elapsed.
first element used of 'length.out' argument
************************************************************
2021-09-16 12:11:33 : ERROR Found in .computeDeviations for GFP+ (1 of 1) 
LogFile = ArchRLogs/ArchR-addDeviationsMatrix-c27565ce590-Date-2021-09-16_Time-12-11-10.log

<simpleError in seq_len(ncol(annotationsMatrix)): argument must be coercible to non-negative integer>

************************************************************

Error in .logError(e, fn = ".computeDeviations", info = prefix, errorList = errorList,  : 
  Exiting See Error Above

[rcorces]

But when I try to access the matrix using getMatrixFromProject, I get the following error

addPeakAnnotations() does not add a matrix to the project. It creates a file in the "Annotations" folder

The addDeviationsMatrix step fails as well

I dont know why this is failing but presumably something related to your annotation.

[agalianese]

It creates SuperAnno In-Peaks-Summary.rds, Matches-In-Peaks.rds, and Positions-In-Peaks.rds, so that step is working at least.

[rcorces]

You can try stepping through the relevant code: https://github.com/GreenleafLab/ArchR/blob/968e4421ce7187a8ac7ea1cf6077412126876d5f/R/MatrixDeviations.R#L27

ArrowFiles <- getSampleColData(ArchRProj)$ArrowFiles
  threads <- min(length(ArrowFiles), threads)
  allCells <- rownames(getCellColData(ArchRProj))
  outDir <- getOutputDirectory(ArchRProj)
  if(!all(file.exists(ArrowFiles))){
    stop("Error Input Arrow Files do not all exist!")
  }

  ##############################################################
  #Annotations Matrix!
  ##############################################################
  print(matches)
  if(is.null(matches)){
    anno <- getPeakAnnotation(ArchRProj, peakAnnotation)
    matches <- readRDS(anno$Matches)
    if(is.null(matrixName)){
      matrixName <- paste0(anno$Name, "Matrix")
    }
  }else{
    if(is.null(matrixName)){
      matrixName <- paste0("MotifMatrix")
    }    
  }
  annotationsMatrix <- SummarizedExperiment::assay(matches)
  rownames(annotationsMatrix) <- paste0(seqnames(matches), "_", start(matches), "_", end(matches))
  annotationsMatrix <- as(annotationsMatrix, "dgCMatrix")
  #rm(matches)
  gc()

  ##############################################################
  #Get Expectations!
  ##############################################################
  ArrowFiles <- getArrowFiles(ArchRProj)
  useMatrix <- "PeakMatrix"
  availableChr <- .availableSeqnames(ArrowFiles, useMatrix)
  
  rS <- suppressMessages(.getRowSums(
      ArrowFiles = ArrowFiles, 
      seqnames = availableChr,
      useMatrix = useMatrix,
      filter0 = FALSE
    ))
  rownames(rS) <- paste0(rS$seqnames,"_",rS$idx)
  rS <- rS[paste0(seqnames(ArchRProj@peakSet), "_", mcols(ArchRProj@peakSet)$idx),]
  rS$start <- start(ArchRProj@peakSet)
  rS$end <- end(ArchRProj@peakSet)
  rS$GC <- ArchRProj@peakSet$GC
  rownames(rS) <- paste0(rS$seqnames, "_", rS$start, "_", rS$end)

  annotationsMatrix <- annotationsMatrix[rownames(rS), ]

The annotationsMatrix object is then passed to a separate function that runs:

seq_len(ncol(annotationsMatrix))

which is the line that is giving an error.

[agalianese]

There doesn't seem to be an issue with the Annotations Matrix, but where can I find the code for the function .getRowSums, used here? Thanks!

rS <- suppressMessages(.getRowSums(
      ArrowFiles = ArrowFiles, 
      seqnames = availableChr,
      useMatrix = useMatrix,
      filter0 = FALSE
    ))

[rcorces]

you can pull all "hidden" functions from ArchR into your environment using:

#Load all hidden ArchR functions
fn <- unclass(lsf.str(envir = asNamespace("ArchR"), all = TRUE))
for (i in seq_along(fn)) {
  tryCatch({
    eval(parse(text = paste0(fn[i], "<-ArchR:::", fn[i])))
  }, error = function(x) {
  })
}

alternatively, anytime you come across a function with . in front of the name, you can substitute like this ArchR:::.getRowSums

[agalianese]

I found 2 issues. If you only supply one bed file, then you get an error about the annotations matrix. There also seems to be an incompatibility with chromVAR with the mouse genome (mm10). I tried adding a custom annotation using files from Encode, but it still failed at .customDeviationsSingle. I couldn't see a reason within chromVAR that it was failing though.

[rcorces]

Thanks. I'll look into the one bed file issue.

I'm not sure I follow your issue about chromVAR but feel free to add more details and I'll see if I can help.