Issues running velocyto

[shabs24]

Hi,

I am trying to run velocyto using both run and run10x command. I get the same error in each case. It looks like an issue with samtools

I am running it on cluster using 1 node and 12ppn. I am not sure how to set the parameters in veloctyto for samtools on a cluster

I tried to run samtools on passorted_bam.bam to generate cell sorted file and try to run velocyto on it and I still get same error.

See the commands and error below

Any help will be appreciated!

Many Thanks

run10x

velocyto run10x -m /cluster/projects//mm10_rmsk.gtf /cluster/projects/MFS_DCS_pool /cluster/projects//genes.gtf

run

velocyto run -b outs/filtered_gene_bc_matrices/mm10/barcodes.tsv -o ./velocyto_run -m /cluster/projects//mm10_rmsk.gtf outs/possorted_genome_bam.bam /cluster/projects//genes.gtf

run -> after sort

samtools sort -l 0 -O BAM -t CB outs/possorted_genome_bam.bam -o cellsorted_possorted_genome_bam.bam velocyto run -b outs/filtered_gene_bc_matrices/mm10/barcodes.tsv -o ./velocyto_run_cellsorted -m /cluster/projects//mm10_rmsk.gtf cellsorted_possorted_genome_bam.bam /cluster/projects//genes.gtf ############################## And I get the following error ############################## 018-12-14 15:32:12,809 - DEBUG - Read first 290 million reads 2018-12-14 15:32:31,928 - DEBUG - End of file. Reset index: start scanning from initial position. 2018-12-14 15:32:31,965 - DEBUG - 2742974 reads were skipped because no apropiate cell or umi barcode was found 2018-12-14 15:32:31,966 - INFO - Now just waiting that the bam sorting process terminates

samtools sort: couldn't allocate memory for bam_mem /cluster/tools/software/python/3.6.5/lib/python3.6/site-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type. from ._conv import register_converters as _register_converters Traceback (most recent call last): File "/cluster/tools/software/velocyto/0.17.13/bin/velocyto", line 11, in sys.exit(cli()) File "/cluster/tools/software/velocyto/0.17.13/lib/python3.6/site-packages/click/core.py", line 764, in call return self.main(*args, **kwargs) File "/cluster/tools/software/velocyto/0.17.13/lib/python3.6/site-packages/click/core.py", line 717, in main rv = self.invoke(ctx) File "/cluster/tools/software/velocyto/0.17.13/lib/python3.6/site-packages/click/core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/cluster/tools/software/velocyto/0.17.13/lib/python3.6/site-packages/click/core.py", line 956, in invoke return ctx.invoke(self.callback, **ctx.params) File "/cluster/tools/software/velocyto/0.17.13/lib/python3.6/site-packages/click/core.py", line 555, in invoke return callback(*args, **kwargs) File "/cluster/tools/software/velocyto/0.17.13/lib/python3.6/site-packages/velocyto/commands/run10x.py", line 106, in run10x samtools_memory=samtools_memory, dump=dump, verbose=verbose, additional_ca=additional_ca) File "/cluster/tools/software/velocyto/0.17.13/lib/python3.6/site-packages/velocyto/commands/_run.py", line 225, in _run Otherwise sort manually by samtools sort -l [compression] -m [mb_to_use]M -t [tagname] -O BAM -@ [threads_to_use] -o cellsorted_[bamfile] [bamfile]") MemoryError: bam file #0 could not be sorted by cells. This is probably related to an old version of samtools, please install samtools >= 1.6. In alternative this could be a memory error, try to set the --samtools_memory option to a value compatible with your system. Otherwise sort manually by samtools sort -l [compression] -m [mb_to_use]M -t [tagname] -O BAM -@ [threads_to_use] -o cellsorted_[bamfile] [bamfile]

[idalarsson]

@shabs24 Did you manage to solve this? I receive the same error.

[YingMa0107]

@shabs24 @idalarsson Did you solve this? I also got the same error when I tried run10X, run after sort both.

[shabs24]

@shabs24 @idalarsson Did you solve this? I also got the same error when I tried run10X, run after sort both.

@idalarsson @YingMa1993 I was running on cluster and was able to solve by requesting more memory (60 GB).

[davisidarta]

@shabs24 , how much memory do you have avaiable on your cluster? I also got the same error. The file is 10x, So I suppose it's already sorted. I'm running it on a 128GB RAM system.

[shabs24]

@davisidarta, I have used 60 Gb memory on the cluster.

[ChelseaCHENX]

I had the same problem ... how many threads are you using? (want an estimation of threads ~ memory per sample)

[saeedfc]

I am using 240 GB for a 1000 cells and still hitting the same error. No idea what is going wrong. Samtools is version 1.9 ~$ velocyto run10x -m /alltracks_mask.gtf /KUL-1-1000cells/ /refdata-cellranger-mm10-3.0.0/genes/genes.gtf --samtools-memory 240000

MemoryError: bam file #0 could not be sorted by cells.
                This is probably related to an old version of samtools, please install samtools >= 1.6.                In alternative this could be a memory error, try to set the --samtools_memory option to a value compatible with your system.                 Otherwise sort manually by samtools ``sort -l [compression] -m [mb_to_use]M -t [tagname] -O BAM -@ [threads_to_use] -o cellsorted_[bamfile] [bamfile]``

[saeedfc]

If anyone is struggling with the same issue, please see this

https://github.com/velocyto-team/velocyto.py/issues/192#issuecomment-545349435

[denvercal1234GitHub]

@saeedfc @shabs24 After you cellsort the possort bam file, did you just use the same bam.bai index file that came with the possort bam from cellranger? Or, did you index the cellsorted_possorted bam? Thank you so much! --- issue #321

[linxy29]

I faced the same error, I tried requesting more memory and it didn't work. Since I used conda, so I created a new environment and installed velocyto based on the installation guide. The problem is solved.