SubPhaser icon indicating copy to clipboard operation
SubPhaser copied to clipboard

OSError: [Errno 12] Cannot allocate memory

Open emircankaaya opened this issue 1 year ago • 8 comments

(SubPhaser) eck@DESKTOP-ED16O0T:~/SubPhaser$ subphaser -i genome.fasta.gz -c sg.config 24-08-11 22:55:26 [WARNING] Grid computing is not available because DRMAA not installed: No module named 'drmaa' 24-08-11 22:55:26 [INFO] No DRMAA (see https://github.com/pygridtools/drmaa-python), Switching to local mode. 24-08-11 22:55:28 [WARNING] Grid computing is not available because DRMAA not installed: No module named 'drmaa' 24-08-11 22:55:28 [INFO] No DRMAA (see https://github.com/pygridtools/drmaa-python), Switching to local mode. 24-08-11 22:55:28 [INFO] Command: /home/eck/miniconda3/envs/SubPhaser/bin/subphaser -i genome.fasta.gz -c sg.config 24-08-11 22:55:28 [INFO] Version: 1.2.6 24-08-11 22:55:28 [INFO] Arguments: {'genomes': ['genome.fasta.gz'], 'sg_cfgs': ['sg.config'], 'labels': None, 'no_label': False, 'target': None, 'sg_assigned': None, 'sep': '|', 'custom_features': None, 'prefix': None, 'outdir': 'phase-results', 'tmpdir': 'tmp', 'colors': ['#f9c00c', '#00b9f1', '#7200da', '#f9320c', '#00b8a9', '#F4A460', '#009999', '#00C02E', '#980000', '#00ffff', '#0000ff', '#ff0000', '#4a86e8', '#ff9900', '#ffff00', '#00ff00', '#9900ff', '#ff00ff', '#20124d', '#274e13', '#000000', '#cccccc', '#7f6000', '#a64d79', '#6aa84f', '#fff2cc', '#47a952', '#3ea6b6', '#a5b805', '#8f9276', '#ca8d7c'], 'k': 15, 'min_fold': 2, 'min_freq': 200, 'baseline': 1, 'ratio': 1, 'lower_count': 3, 'min_prop': None, 'max_freq': 1000000000.0, 'max_prop': None, 'low_mem': None, 'by_count': False, 're_filter': False, 'nsg': None, 'replicates': 1000, 'jackknife': 50, 'max_pval': 0.05, 'test_method': 'ttest_ind', 'figfmt': 'pdf', 'heatmap_colors': ('green', 'black', 'red'), 'heatmap_options': "Rowv=T,Colv=T,scale='col',dendrogram='row',labCol=F,trace='none',key=T,key.title=NA,density.info='density',main=NA,xlab='Differential kmers',margins=c(2.5,12)", 'just_core': False, 'disable_ltr': False, 'ltr_detectors': ['ltr_harvest'], 'ltr_finder_options': '-w 2 -D 15000 -d 1000 -L 7000 -l 100 -p 20 -C -M 0.8', 'ltr_harvest_options': '-seqids yes -similar 80 -vic 10 -seed 20 -minlenltr 100 -maxlenltr 7000 -mintsd 4 -maxtsd 6', 'tesorter_options': '-db rexdb -dp2', 'all_ltr': False, 'intact_ltr': False, 'exclude_exchanges': False, 'non_specific': False, 'mu': 1.3e-08, 'disable_ltrtree': False, 'subsample': 1000, 'ltr_domains': ['INT', 'RT', 'RH'], 'trimal_options': '-automated1', 'tree_method': 'FastTree', 'tree_options': '', 'ggtree_options': "branch.length='none', layout='circular'", 'disable_circos': False, 'window_size': 1000000, 'disable_blocks': False, 'aligner': 'minimap2', 'aligner_options': '-x asm20 -n 10', 'min_block': 100000, 'alt_cfgs': None, 'chr_ordered': None, 'ncpu': 4, 'max_memory': '11.5G', 'cleanup': False, 'overwrite': False} 24-08-11 22:55:28 [INFO] Target chromosomes: ['chr01a', 'chr01b', 'chr01c', 'chr01d', 'chr02a', 'chr02b', 'chr02c', 'chr02d', 'chr03a', 'chr03b', 'chr03c', 'chr03d', 'chr04a', 'chr04b', 'chr04c', 'chr04d', 'chr05a', 'chr05b', 'chr05c', 'chr05d', 'chr06a', 'chr06b', 'chr06c', 'chr06d', 'chr07a', 'chr07b', 'chr07c', 'chr07d'] 24-08-11 22:55:28 [INFO] Splitting genomes by chromosome into /home/eck/SubPhaser/tmp/ 24-08-11 22:55:28 [INFO] Check point file: /home/eck/SubPhaser/tmp/split.ok exists; skip this step 24-08-11 22:55:29 [INFO] Chromosomes: ['chr01a', 'chr01b', 'chr01c', 'chr01d', 'chr02a', 'chr02b', 'chr02c', 'chr02d', 'chr03a', 'chr03b', 'chr03c', 'chr03d', 'chr04a', 'chr04b', 'chr04c', 'chr04d', 'chr05a', 'chr05b', 'chr05c', 'chr05d', 'chr06a', 'chr06b', 'chr06c', 'chr06d', 'chr07a', 'chr07b', 'chr07c', 'chr07d'] 24-08-11 22:55:29 [INFO] Chromosome Number: 28 24-08-11 22:55:29 [INFO] CONFIG: [[['chr01a'], ['chr01b'], ['chr01c'], ['chr01d']], [['chr02a'], ['chr02b'], ['chr02c'], ['chr02d']], [['chr03a'], ['chr03b'], ['chr03c'], ['chr03d']], [['chr04a'], ['chr04b'], ['chr04c'], ['chr04d']], [['chr05a'], ['chr05b'], ['chr05c'], ['chr05d']], [['chr06a'], ['chr06b'], ['chr06c'], ['chr06d']], [['chr07a'], ['chr07b'], ['chr07c'], ['chr07d']]] 24-08-11 22:55:29 [INFO] Genome size: 1,892,584,152 bp 24-08-11 22:55:29 [INFO] ###Step: Kmer Count 24-08-11 22:55:29 [INFO] Counting kmer by jellyfish 24-08-11 22:55:29 [INFO] Start Pool with 4 process(es) 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr01a.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr01c.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr02a.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr02c.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr01b.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr01d.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr02b.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr02d.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr03a.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr03c.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr04a.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr04c.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr03b.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr03d.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr04b.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr04d.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr05a.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr05c.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr06a.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr06c.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr05b.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr05d.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr06b.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr06d.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr07a.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr07c.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr07b.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr07d.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Loading kmer matrix from jellyfish 24-08-11 22:55:29 [INFO] Start Pool with 4 process(es) 24-08-12 00:29:31 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr01a.fasta_15.fa 24-08-12 00:30:23 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr01b.fasta_15.fa 24-08-12 00:31:03 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr01c.fasta_15.fa 24-08-12 00:31:15 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr01d.fasta_15.fa 24-08-12 00:32:18 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr02a.fasta_15.fa 24-08-12 00:34:03 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr02b.fasta_15.fa 24-08-12 00:34:12 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr02c.fasta_15.fa 24-08-12 00:35:55 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr02d.fasta_15.fa 24-08-12 00:36:03 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr03a.fasta_15.fa 24-08-12 00:36:07 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr03b.fasta_15.fa 24-08-12 00:36:11 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr03c.fasta_15.fa 24-08-12 00:38:07 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr03d.fasta_15.fa 24-08-12 00:38:10 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr04a.fasta_15.fa 24-08-12 00:38:14 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr04b.fasta_15.fa 24-08-12 00:38:17 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr04c.fasta_15.fa 24-08-12 00:38:20 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr04d.fasta_15.fa 24-08-12 00:38:23 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr05a.fasta_15.fa 24-08-12 00:38:27 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr05b.fasta_15.fa 24-08-12 00:38:43 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr05c.fasta_15.fa 24-08-12 00:38:46 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr05d.fasta_15.fa 24-08-12 00:42:05 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr06a.fasta_15.fa 24-08-12 00:42:17 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr06b.fasta_15.fa 24-08-12 00:42:34 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr06c.fasta_15.fa 24-08-12 00:42:48 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr06d.fasta_15.fa 24-08-12 00:43:12 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr07a.fasta_15.fa 24-08-12 00:43:41 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr07b.fasta_15.fa 24-08-12 00:44:07 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr07c.fasta_15.fa 24-08-12 00:44:49 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr07d.fasta_15.fa 24-08-12 00:46:21 [INFO] 28129798 kmers in total 24-08-12 00:46:21 [INFO] Filtering differential kmers Traceback (most recent call last): File "/home/eck/miniconda3/envs/SubPhaser/bin/subphaser", line 33, in sys.exit(load_entry_point('subphaser==1.2.6', 'console_scripts', 'subphaser')()) File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/site-packages/subphaser-1.2.6-py3.8.egg/subphaser/main.py", line 797, in main pipeline.run() File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/site-packages/subphaser-1.2.6-py3.8.egg/subphaser/main.py", line 422, in run d_mat = dumps.filter(d_mat, lengths, self.sgs, outfig=histfig, #d_targets=d_targets, File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/site-packages/subphaser-1.2.6-py3.8.egg/subphaser/Jellyfish.py", line 491, in filter for kmer, freqs, tot_freq in pool_func(_filter_kmer, args, self.ncpu, File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/site-packages/subphaser-1.2.6-py3.8.egg/subphaser/RunCmdsMP.py", line 336, in pool_func pool = multiprocessing.Pool(processors) File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/multiprocessing/context.py", line 119, in Pool return Pool(processes, initializer, initargs, maxtasksperchild, File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/multiprocessing/pool.py", line 212, in init self._repopulate_pool() File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/multiprocessing/pool.py", line 303, in _repopulate_pool return self._repopulate_pool_static(self._ctx, self.Process, File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/multiprocessing/pool.py", line 326, in _repopulate_pool_static w.start() File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/multiprocessing/process.py", line 121, in start self._popen = self._Popen(self) File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/multiprocessing/popen_fork.py", line 70, in _launch self.pid = os.fork() OSError: [Errno 12] Cannot allocate memory (SubPhaser) eck@DESKTOP-ED16O0T:~/SubPhaser$

emircankaaya avatar Aug 11 '24 21:08 emircankaaya

You can try -q 200 -lower_count 10 to reduce the kmer number.

2024-08-12 05:57:24"jenny315" @.***>写道:

(SubPhaser) @.:/SubPhaser$ subphaser -i genome.fasta.gz -c sg.config 24-08-11 22:55:26 [WARNING] Grid computing is not available because DRMAA not installed: No module named 'drmaa' 24-08-11 22:55:26 [INFO] No DRMAA (see https://github.com/pygridtools/drmaa-python), Switching to local mode. 24-08-11 22:55:28 [WARNING] Grid computing is not available because DRMAA not installed: No module named 'drmaa' 24-08-11 22:55:28 [INFO] No DRMAA (see https://github.com/pygridtools/drmaa-python), Switching to local mode. 24-08-11 22:55:28 [INFO] Command: /home/eck/miniconda3/envs/SubPhaser/bin/subphaser -i genome.fasta.gz -c sg.config 24-08-11 22:55:28 [INFO] Version: 1.2.6 24-08-11 22:55:28 [INFO] Arguments: {'genomes': ['genome.fasta.gz'], 'sg_cfgs': ['sg.config'], 'labels': None, 'no_label': False, 'target': None, 'sg_assigned': None, 'sep': '|', 'custom_features': None, 'prefix': None, 'outdir': 'phase-results', 'tmpdir': 'tmp', 'colors': ['#f9c00c', '#00b9f1', '#7200da', '#f9320c', '#00b8a9', '#F4A460', '#009999', '#00C02E', '#980000', '#00ffff', '#0000ff', '#ff0000', '#4a86e8', '#ff9900', '#ffff00', '#00ff00', '#9900ff', '#ff00ff', '#20124d', '#274e13', '#000000', '#cccccc', '#7f6000', '#a64d79', '#6aa84f', '#fff2cc', '#47a952', '#3ea6b6', '#a5b805', '#8f9276', '#ca8d7c'], 'k': 15, 'min_fold': 2, 'min_freq': 200, 'baseline': 1, 'ratio': 1, 'lower_count': 3, 'min_prop': None, 'max_freq': 1000000000.0, 'max_prop': None, 'low_mem': None, 'by_count': False, 're_filter': False, 'nsg': None, 'replicates': 1000, 'jackknife': 50, 'max_pval': 0.05, 'test_method': 'ttest_ind', 'figfmt': 'pdf', 'heatmap_colors': ('green', 'black', 'red'), 'heatmap_options': "Rowv=T,Colv=T,scale='col',dendrogram='row',labCol=F,trace='none',key=T,key.title=NA,density.info='density',main=NA,xlab='Differential kmers',margins=c(2.5,12)", 'just_core': False, 'disable_ltr': False, 'ltr_detectors': ['ltr_harvest'], 'ltr_finder_options': '-w 2 -D 15000 -d 1000 -L 7000 -l 100 -p 20 -C -M 0.8', 'ltr_harvest_options': '-seqids yes -similar 80 -vic 10 -seed 20 -minlenltr 100 -maxlenltr 7000 -mintsd 4 -maxtsd 6', 'tesorter_options': '-db rexdb -dp2', 'all_ltr': False, 'intact_ltr': False, 'exclude_exchanges': False, 'non_specific': False, 'mu': 1.3e-08, 'disable_ltrtree': False, 'subsample': 1000, 'ltr_domains': ['INT', 'RT', 'RH'], 'trimal_options': '-automated1', 'tree_method': 'FastTree', 'tree_options': '', 'ggtree_options': "branch.length='none', layout='circular'", 'disable_circos': False, 'window_size': 1000000, 'disable_blocks': False, 'aligner': 'minimap2', 'aligner_options': '-x asm20 -n 10', 'min_block': 100000, 'alt_cfgs': None, 'chr_ordered': None, 'ncpu': 4, 'max_memory': '11.5G', 'cleanup': False, 'overwrite': False} 24-08-11 22:55:28 [INFO] Target chromosomes: ['chr01a', 'chr01b', 'chr01c', 'chr01d', 'chr02a', 'chr02b', 'chr02c', 'chr02d', 'chr03a', 'chr03b', 'chr03c', 'chr03d', 'chr04a', 'chr04b', 'chr04c', 'chr04d', 'chr05a', 'chr05b', 'chr05c', 'chr05d', 'chr06a', 'chr06b', 'chr06c', 'chr06d', 'chr07a', 'chr07b', 'chr07c', 'chr07d'] 24-08-11 22:55:28 [INFO] Splitting genomes by chromosome into /home/eck/SubPhaser/tmp/ 24-08-11 22:55:28 [INFO] Check point file: /home/eck/SubPhaser/tmp/split.ok exists; skip this step 24-08-11 22:55:29 [INFO] Chromosomes: ['chr01a', 'chr01b', 'chr01c', 'chr01d', 'chr02a', 'chr02b', 'chr02c', 'chr02d', 'chr03a', 'chr03b', 'chr03c', 'chr03d', 'chr04a', 'chr04b', 'chr04c', 'chr04d', 'chr05a', 'chr05b', 'chr05c', 'chr05d', 'chr06a', 'chr06b', 'chr06c', 'chr06d', 'chr07a', 'chr07b', 'chr07c', 'chr07d'] 24-08-11 22:55:29 [INFO] Chromosome Number: 28 24-08-11 22:55:29 [INFO] CONFIG: [[['chr01a'], ['chr01b'], ['chr01c'], ['chr01d']], [['chr02a'], ['chr02b'], ['chr02c'], ['chr02d']], [['chr03a'], ['chr03b'], ['chr03c'], ['chr03d']], [['chr04a'], ['chr04b'], ['chr04c'], ['chr04d']], [['chr05a'], ['chr05b'], ['chr05c'], ['chr05d']], [['chr06a'], ['chr06b'], ['chr06c'], ['chr06d']], [['chr07a'], ['chr07b'], ['chr07c'], ['chr07d']]] 24-08-11 22:55:29 [INFO] Genome size: 1,892,584,152 bp 24-08-11 22:55:29 [INFO] ###Step: Kmer Count 24-08-11 22:55:29 [INFO] Counting kmer by jellyfish 24-08-11 22:55:29 [INFO] Start Pool with 4 process(es) 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr01a.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr01c.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr02a.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr02c.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr01b.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr01d.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr02b.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr02d.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr03a.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr03c.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr04a.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr04c.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr03b.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr03d.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr04b.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr04d.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr05a.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr05c.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr06a.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr06c.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr05b.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr05d.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr06b.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr06d.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr07a.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr07c.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr07b.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Check point file: /home/eck/SubPhaser/tmp/chromosomes/chr07d.fasta_15.fa.ok exists; skip this step 24-08-11 22:55:29 [INFO] Loading kmer matrix from jellyfish 24-08-11 22:55:29 [INFO] Start Pool with 4 process(es) 24-08-12 00:29:31 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr01a.fasta_15.fa 24-08-12 00:30:23 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr01b.fasta_15.fa 24-08-12 00:31:03 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr01c.fasta_15.fa 24-08-12 00:31:15 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr01d.fasta_15.fa 24-08-12 00:32:18 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr02a.fasta_15.fa 24-08-12 00:34:03 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr02b.fasta_15.fa 24-08-12 00:34:12 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr02c.fasta_15.fa 24-08-12 00:35:55 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr02d.fasta_15.fa 24-08-12 00:36:03 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr03a.fasta_15.fa 24-08-12 00:36:07 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr03b.fasta_15.fa 24-08-12 00:36:11 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr03c.fasta_15.fa 24-08-12 00:38:07 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr03d.fasta_15.fa 24-08-12 00:38:10 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr04a.fasta_15.fa 24-08-12 00:38:14 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr04b.fasta_15.fa 24-08-12 00:38:17 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr04c.fasta_15.fa 24-08-12 00:38:20 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr04d.fasta_15.fa 24-08-12 00:38:23 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr05a.fasta_15.fa 24-08-12 00:38:27 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr05b.fasta_15.fa 24-08-12 00:38:43 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr05c.fasta_15.fa 24-08-12 00:38:46 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr05d.fasta_15.fa 24-08-12 00:42:05 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr06a.fasta_15.fa 24-08-12 00:42:17 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr06b.fasta_15.fa 24-08-12 00:42:34 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr06c.fasta_15.fa 24-08-12 00:42:48 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr06d.fasta_15.fa 24-08-12 00:43:12 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr07a.fasta_15.fa 24-08-12 00:43:41 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr07b.fasta_15.fa 24-08-12 00:44:07 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr07c.fasta_15.fa 24-08-12 00:44:49 [INFO] Loading /home/eck/SubPhaser/tmp/chromosomes/chr07d.fasta_15.fa 24-08-12 00:46:21 [INFO] 28129798 kmers in total 24-08-12 00:46:21 [INFO] Filtering differential kmers Traceback (most recent call last): File "/home/eck/miniconda3/envs/SubPhaser/bin/subphaser", line 33, in sys.exit(load_entry_point('subphaser==1.2.6', 'console_scripts', 'subphaser')()) File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/site-packages/subphaser-1.2.6-py3.8.egg/subphaser/main.py", line 797, in main pipeline.run() File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/site-packages/subphaser-1.2.6-py3.8.egg/subphaser/main.py", line 422, in run d_mat = dumps.filter(d_mat, lengths, self.sgs, outfig=histfig, #d_targets=d_targets, File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/site-packages/subphaser-1.2.6-py3.8.egg/subphaser/Jellyfish.py", line 491, in filter for kmer, freqs, tot_freq in pool_func(_filter_kmer, args, self.ncpu, File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/site-packages/subphaser-1.2.6-py3.8.egg/subphaser/RunCmdsMP.py", line 336, in pool_func pool = multiprocessing.Pool(processors) File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/multiprocessing/context.py", line 119, in Pool return Pool(processes, initializer, initargs, maxtasksperchild, File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/multiprocessing/pool.py", line 212, in init self._repopulate_pool() File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/multiprocessing/pool.py", line 303, in _repopulate_pool return self._repopulate_pool_static(self._ctx, self.Process, File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/multiprocessing/pool.py", line 326, in _repopulate_pool_static w.start() File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/multiprocessing/process.py", line 121, in start self._popen = self._Popen(self) File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/home/eck/miniconda3/envs/SubPhaser/lib/python3.8/multiprocessing/popen_fork.py", line 70, in _launch self.pid = os.fork() OSError: [Errno 12] Cannot allocate memory (SubPhaser) @.:/SubPhaser$

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: @.***>

zhangrengang avatar Aug 12 '24 00:08 zhangrengang

Unfortunately, the same error was raised. Do you know any alternative way?

emircankaaya avatar Aug 12 '24 07:08 emircankaaya

If the genome is so huge, you can cut a segment from each chromosome to respresent the chromosome.

2024-08-12 15:39:11"jenny315" @.***>写道:

Unfortunately, the same error was raised. Do you know any alternative way?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

zhangrengang avatar Aug 12 '24 12:08 zhangrengang

Solved

emircankaaya avatar Aug 14 '24 12:08 emircankaaya

@emircankaaya I have the same problem how have you solved the problem ?

LikeLucasLab avatar Aug 16 '24 10:08 LikeLucasLab

You can try -q 200 -lower_count 10 to reduce the kmer number.

I tried this command line which iş suggested by Dr. Zhang if you have still some error let me know and we can try another way.

16 Ağu 2024 Cum 13:33 tarihinde Lucas Munnes @.***> şunu yazdı:

@emircankaaya https://github.com/emircankaaya I have the same problem how have you solved the problem ?

— Reply to this email directly, view it on GitHub https://github.com/zhangrengang/SubPhaser/issues/38#issuecomment-2293261931, or unsubscribe https://github.com/notifications/unsubscribe-auth/BJVTSW6MO5OYJZSZIJQ3O7TZRXIP3AVCNFSM6AAAAABMLC2UHWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJTGI3DCOJTGE . You are receiving this because you were mentioned.Message ID: @.***>

emircankaaya avatar Aug 16 '24 10:08 emircankaaya

You may check your swap and memory information with "free -h"

Then you would increase the values.

"

sudo fallocate -l ...G /swapfile -----------------you would write your free swap to "..." in the command line. E.g. "sudo fallocate -l 32G /swapfile" sudo chmod 600 /swapfile sudo mkswap /swapfile sudo swapon /swapfile

Emir Can Kaya @.***>, 16 Ağu 2024 Cum, 13:43 tarihinde şunu yazdı:

You can try -q 200 -lower_count 10 to reduce the kmer number.

I tried this command line which iş suggested by Dr. Zhang if you have still some error let me know and we can try another way.

16 Ağu 2024 Cum 13:33 tarihinde Lucas Munnes @.***> şunu yazdı:

@emircankaaya https://github.com/emircankaaya I have the same problem how have you solved the problem ?

— Reply to this email directly, view it on GitHub https://github.com/zhangrengang/SubPhaser/issues/38#issuecomment-2293261931, or unsubscribe https://github.com/notifications/unsubscribe-auth/BJVTSW6MO5OYJZSZIJQ3O7TZRXIP3AVCNFSM6AAAAABMLC2UHWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJTGI3DCOJTGE . You are receiving this because you were mentioned.Message ID: @.***>

emircankaaya avatar Aug 16 '24 10:08 emircankaaya

@emircankaaya thanks your suggestions worked now it is running

LikeLucasLab avatar Aug 17 '24 09:08 LikeLucasLab