(memory_new) Error in memory allocation: 12: Cannot allocate memory

[zhangrengang]

Hi, I have run CAFE 4.0.2 and get the following error:

---

Family information: species.ogcount.txt Log: log-run2.txt The number of families is 37995 Root Family size : 1 ~ 2399 Family size : 0 ~ 2302 P-value: 0.01 Num of Threads: 4 Num of Random: 1000 ((((((((((Gossypium-raimondii:25.9804,Durio-zibethinus:25.9804):4.29216,Theobroma-cacao:30.2726):53.3515,(Carica-papaya:75.9728,Arabidopsis-thaliana:75.9728):7.65128):3.32215,((Dimocarpus-longan:31.3362,Xanthoceras-sorbifolium:31.3362):28.0047,Citrus-grandis:59.341):27.6053):3.05376,Populus-trichocarpa:90):16.6329,((Malus-domestica:94.5268,Cucumis-melo:94.5268):6.07958,Glycine-max:100.606):6.02656):5.91262,(Punica-granatum:83.1347,Eucalyptus-grandis:83.1347):29.4108):11.4545,Vitis-vinifera:124):13.7855,Coffea-canephora:137.785):128.232,Oryza-sativa:266.018) The number of lambdas is 1 Lambda Tree: ((((((((((1,1)1,1)1,(1,1)1)1,((1,1)1,1)1)1,1)1,((1,1)1,1)1)1,(1,1)1)1,1)1,1)1,1) Empirical Prior Estimation Result: 14 Poisson lambda: 0.211068 & Score: inf WARNING: Calculated posterior probability for family OG-1 = 0 Lambda : 0.00056219243791 & Score: -inf .WARNING: Calculated posterior probability for family OG-1 = 0 Lambda : 0.00059030205981 & Score: -inf .WARNING: Calculated posterior probability for family OG-1 = 0 Lambda : 0.00053408281602 & Score: -inf .WARNING: Calculated posterior probability for family OG-1 = 0 Lambda : 0.00054813762696 & Score: -inf .WARNING: Calculated posterior probability for family OG-1 = 0 Lambda : 0.00057624724886 & Score: -inf .WARNING: Calculated posterior probability for family OG-1 = 0 Lambda : 0.00056921984339 & Score: -inf .WARNING: Calculated posterior probability for family OG-1 = 0 Lambda : 0.00055516503244 & Score: -inf .WARNING: Calculated posterior probability for family OG-1 = 0 Lambda : 0.00055867873518 & Score: -inf .WARNING: Calculated posterior probability for family OG-1 = 0 Lambda : 0.00056570614065 & Score: -inf .WARNING: Calculated posterior probability for family OG-1 = 0 Lambda : 0.00056394928928 & Score: -inf .WARNING: Calculated posterior probability for family OG-1 = 0 Lambda : 0.00056043558654 & Score: -inf .WARNING: Calculated posterior probability for family OG-1 = 0 Lambda : 0.00056131401223 & Score: -inf . Lambda Search Result: 5 Lambda : 0.00056219243791 & Score: inf DONE: Lambda Search or setting, for command: lambda -s -t ((((((((((1,1)1,1)1,(1,1)1)1,((1,1)1,1)1)1,1)1,((1,1)1,1)1)1,(1,1)1)1,1)1,1)1,1) Running Viterbi algorithm.... (memory_new) Error in memory allocation: 12: Cannot allocate memory (memory_new) Error in memory allocation: 12: Cannot allocate memory (memory_new) Error in memory allocation: 12: Cannot allocate memory /opt/gridengine/default/spool/compute-0-5/job_scripts/125237: line 1: 62302 Segmentation fault (core dumped) cafe run_cafe2.cafe

This is the file run_cafe2.cafe: #!cafe load -i species.ogcount.txt -t 4 -l log-run2.txt tree ((((((((((Gossypium-raimondii:25.980427,Durio-zibethinus:25.980427):4.292164,Theobroma-cacao:30.272591):53.351498,(Carica-papaya:75.972809,Arabidopsis-thaliana:75.972809):7.651280):3.322150,((Dimocarpus-longan:31.336222,Xanthoceras-sorbifolium:31.336222):28.004735,Citrus-grandis:59.340957):27.605282):3.053761,Populus-trichocarpa:90.000000):16.632920,((Malus-domestica:94.526782,Cucumis-melo:94.526782):6.079577,Glycine-max:100.606359):6.026561):5.912620,(Punica-granatum:83.134742,Eucalyptus-grandis:83.134742):29.410799):11.454460,Vitis-vinifera:124.000000):13.785478,Coffea-canephora:137.785478):128.232231,Oryza-sativa:266.017709) lambda -s -t ((((((((((1,1)1,1)1,(1,1)1)1,((1,1)1,1)1)1,1)1,((1,1)1,1)1)1,(1,1)1)1,1)1,1)1,1) report report-run2

The data file species.ogcount.txt looks like this: Desc Family ID Arabidopsis-thaliana Carica-papaya Citrus-grandis Coffea-canephora Cucumis-melo Dimocarpus-longan Durio-zibethinus Eucalyptus-grandis Gossypium-raimondii Malus-domestica Oryza-sativa Populus-trichocarpa Punica-granatum Theobroma-cacao Vitis-vinifera Xanthoceras-sorbifolium Glycine-max (null) OG-1 0 0 0 0 1 1919 0 0 1 0 0 0 00 0 0 1 (null) OG-2 2 7 2 270 10 196 116 54 61 130 8 115 29 70 60 51 75 (null) OG-3 5 7 79 148 11 74 88 208 75 136 96 41 39 46 22 57 21 (null) OG-4 79 11 0 13 27 32 46 173 30 208 0 170 12 18 20 4 138 (null) OG-5 27 13 35 31 20 82 36 179 46 78 27 140 34 53 33 24 83 (null) OG-6 16 15 70 80 17 69 101 148 50 48 21 66 25 35 41 19 64 (null) OG-7 13 3 2 15 6 68 61 147 64 34 22 70 40 43 9 4 47 (null) OG-8 0 0 2 0 0 644 0 0 0 0 0 1 10 0 0 12 (null) OG-9 0 0 0 0 0 0 1 0 0 0 656 0 00 0 0 0

Would you please help me? Thank you.

[zhangrengang]

The gene family counts is separated by TAB.

[benfulton]

Hi,

I ran your data through CAFE 4.2 on a powerful computer and was able to get it to complete. You might want to try version 4.2, but do note that the complexity of the calculations is related to the largest family size in your file. Filtering out the very large families might be necessary to get CAFE to run successfully on a laptop.

[YossGG]

Hi @benfulton

I'm wondering, if you can help me to solve my problem related with "Segmentation fault" only when analyzing large gene families file I have been running my analysis with the versions: 3, 4.1 and 4.2 on a powerful computer, but I could not complete successfully the analysis.

The largest file is:

Desc Family ID Ajamaicensis Clupus Drotundus Ecaballus Hsapiens Lnivalis Lyerbabuenae Mbrandtii Mdavidii Mharrisoni Mmusculus Mnatalensis Mwaterhousii Pparnelli Pvampyrus Raegyptiacus (null) 1 1204 0 0 1 0 4515 2 1 0 1758 0 01121 154 1 1 (null) 2 212 284 305 339 1604 268 122 284 310 246 593 225 242 148 1083 274 (null) 3 656 0 0 0 0 3571 1 0 0 1475 1 0399 22 0 0 (null) 4 366 0 0 0 0 1544 0 0 0 604 0 0448 22 0 0 (null) 5 19 210 127 368 106 44 53 117 99 36 411 99 44 26 177 201 (null) 6 196 5 3 5 4 851 4 3 3 413 3 3205 11 6 2 (null) 7 15 174 77 273 113 29 32 63 57 26 219 52 28 23 122 123 (null) 8 9 82 102 160 41 31 33 107 92 20 123 61 20 11 113 112 (null) 9 9 128 45 183 65 15 25 46 51 8 171 37 16 8 78 108 (null) 10 32 48 52 45 124 34 33 64 58 36 83 54 33 27 119 52 (null) 11 30 49 58 53 108 31 43 61 62 32 83 61 29 20 76 63 (null) 12 31 51 46 54 121 35 45 34 38 36 67 42 34 26 125 38 (null) 14 32 16 27 18 160 33 29 27 30 27 83 34 29 16 132 30 (null) 15 48 30 29 31 32 79 150 28 30 71 34 24 51 20 32 29 (null) 16 20 28 56 30 134 23 24 45 43 24 84 34 25 14 107 20 (null) 17 18 112 38 55 79 18 32 43 32 13 44 34 20 12 63 67 (null) 18 20 28 30 17 152 20 23 22 25 22 85 25 18 17 136 33 (null) 20 27 20 32 27 92 34 47 35 32 33 61 30 32 15 105 23 (null) 24 26 17 25 17 76 31 29 24 24 31 110 28 28 19 69 27 (null) 27 19 11 18 8 162 21 28 20 25 17 60 21 18 15 94 15 (null) 29 5 67 32 105 21 14 7 45 39 11 60 32 10 7 37 38 (null) 30 65 0 0 0 0 146 87 0 0 130 0 074 4 0 0 (null) 31 16 13 17 15 139 20 24 31 21 17 53 20 17 13 63 14 (null) 34 15 17 21 12 123 19 16 16 15 17 53 19 17 9 79 14 (null) 37 11 9 21 6 90 16 20 15 18 14 37 17 15 12 148 8 (null) 54 5 3 19 15 105 11 15 48 50 4 28 11 6 1 11 3 (null) 60 9 9 1 1 4 117 4 15 11 40 38 827 3 18 4 (null) 79 2 7 2 3 9 2 2 9 5 2 203 72 2 9 4 (null) 87 2 2 0 1 123 1 3 3 4 1 29 32 2 73 2 (null) 105 1 5 4 11 10 3 4 2 5 3 132 62 1 9 6 (null) 110 0 3 7 29 4 1 0 0 0 1 147 70 0 1 2 (null) 114 1 0 2 5 36 2 2 15 10 2 102 12 0 7 7 (null) 151 3 3 2 1 101 1 3 2 1 3 11 33 2 16 3

I did split the file and ran the calculation (cafe_run2.sh), but It's the same problem. Should I need to get more memory? Can you help to figure out a solution ?

cafe_run2.sh:

#!cafe

load -i large_filtered_cafe_input_1.txt -t 4 -l reports/log_run2.txt tree ((Hsapiens: 88.464846, (Clupus: 76.756963, (Ecaballus: 73.745437, ((Pvampyrus: 23.019102, Raegyptiacus: 23.019102): 42.097932, ((Pparnelli: 38.979498, (Mwaterhousii: 33.115163, (Drotundus: 30.683813, (Ajamaicensis: 26.275739, (Mharrisoni: 21.576576, (Lnivalis: 11.591220, Lyerbabuenae: 11.591220): 9.985356): 4.699163): 4.408074): 2.431349): 5.864335): 17.594998, (Mnatalensis: 47.881537, (Mbrandtii: 20.547808, Mdavidii: 20.547808): 27.333729): 8.692959): 8.542538): 8.628403): 3.011526): 11.707883): 15.053281, Mmusculus: 103.518127); lambda -l 0.00970873786400 -t ((1,(1,(1,((1,1)1,((1,(1,(1,(1,(1,(1,1)1)1)1)1)1)1,(1,(1,1)1)1)1)1)1)1)1,1) report reports/report_run2

[NayeliGutierrez]

Hi @YossGG,

Did you find a solution for the segmentation fault? I am having the exact same problem. I am using cafe v5 in the HPC of my Institution. I split the input file and got the segmentation fault error only with the large gene families file. Please advise.

Thanks!