"mdb_env_open: Cannot allocate memory" and "singularity version can't find the input file"

[zmz1988]

Dear Shujun,

Sorry to bother you again. I had got successful runs in June 2021 with EDTA, but now no idea why EDTA complains about RMblast engine is not installed. I thought it is probably due to the upgrading of our computer system, or repeat masker broke, as you mentioned here #168. So I deleted the previous EDTA environment and rebuilt one in conda by using the EDTA.yml file. However, the error message stays.

`(EDTA) [user@608-3 annotation]$ perl /home/EDTA/EDTA.pl --genome Sample_15122021.fasta --cds Araport11_genes.202106.cds.fasta --overwrite 1 --sensitive 1 --evaluate 1 --threads 10 --anno 1 `

########################################################
##### Extensive de-novo TE Annotator (EDTA) v2.0.0  ####
##### Shujun Ou ([email protected])             ####
########################################################

Fri Jan 14 11:32:02 GMT 2022	Dependency checking:
Error: The RMblast engine is not installed in RepeatMasker!
(EDTA) [user@608-3 annotation]$ ls
01_repeat_masking.sh             RM_42884.FriJan141132022022
02_rnaseq_mapping.sh              Sample_15122021.fasta  dummy060817.fa.437457
Araport11_genes.202106.cds.fasta  EDTA.sif            repeat_masking.log

`(EDTA) [user@608-3 annotation]$ RepeatMasker -e ncbi -q -pa 1 -no_is -norna -nolow dummy060817.fa* -lib dummy060817.fa*`
RepeatMasker version 4.1.1
Search Engine: NCBI/RMBLAST [ 2.10.0+ ]
Using Custom Repeat Library: dummy060817.fa.437457

Building general libraries in: /miniconda3/envs/EDTA/share/RepeatMasker/Libraries//general
RepeatMasker::createLib(): Error invoking/miniconda3/envs/EDTA/bin//makeblastdb on file /miniconda3/envs/EDTA/share/RepeatMasker/Libraries//general/is.lib.

I tried to find out why the is.lib file is not working, and found a file in /miniconda3/envs/EDTA/share/RepeatMasker/Libraries//general/ called rmblastdb.log, in which it contains error:

Building a new DB, current time: 01/14/2022 11:35:52
New DB name:  /miniconda3/envs/EDTA/share/RepeatMasker/Libraries/general/is.lib
New DB title: /miniconda3/envs/EDTA/share/RepeatMasker/Libraries//general/is.lib
Sequence type: Nucleotide
Deleted existing Nucleotide BLAST database named /miniconda3/envs/EDTA/share/RepeatMasker/Libraries/general/is.lib
Keep MBits: T
Maximum file size: 1000000000B

No volumes were created.

Error: mdb_env_open: Cannot allocate memory

I'm not sure what kind of memory it is complaining, but I ran the repeatmasker command on a node that has 60 GB memory to use...

I have been stuck here for days, and don't know how to solve it. Could you please help me out? Thanks a lot in advance!

BTW, I also tried using the singularity image of EDTA, but I don't know why EDTA can't find my input files at all. The EDTA.sif file is together with the genome file and the cds file. I don't know how to solve this problem either. If I could get this version of EDTA work, then would be great too!

[oushujun]

Hello @zmz1988,

You may try other conda ways to install EDTA, as listed in readme. For the singularity version, did you follow the usage in readme? Can you paste the command here? Thanks!

Shujun

[zmz1988]

Dear Shujun,

Thanks for replying me! yes, I had followed the usage instruction in readme file. Here are the commands:

(base) [user@dusty annotation]$ singularity pull EDTA.sif docker://oushujun/edta:2.0.0
INFO:    Using cached SIF image
(base) [user@dusty annotation]$ ls
01_repeat_masking.sh              RM_42884.FriJan141132022022
02_rnaseq_mapping.sh              dummy060817.fa.437457  RM_43236.FriJan141135502022
Araport11_genes.202106.cds.fasta  EDTA.sif
Sample_15122021.fasta                repeat_masking.log
(base) [user@dusty annotation]$ export LC_ALL=C
(base) [user@dusty annotation]$ singularity exec ./EDTA.sif EDTA.pl --genome Sample_15122021.fasta 

########################################################
##### Extensive de-novo TE Annotator (EDTA) v2.0.0  ####
##### Shujun Ou ([email protected])             ####
########################################################


At least 1 parameter is required:
1) Input fasta file: --genome

This is the Extensive de-novo TE Annotator that generates a high-quality
structure-based TE library. Usage:

perl EDTA.pl [options]

... (omit the option list here)

[oushujun]

Hello,

I could not reproduce your issue. I cloned EDTA with singularity 3.8.0 and tested with both 3.8.0 and 3.1.1, both worked. This is what I did:

@.*** test_011722]$ SINGULARITY_CACHEDIR=./

@.*** test_011722]$ export SINGULARITY_CACHEDIR

@.*** test_011722]$ singularity pull EDTA.sif docker://oushujun/edta:2.0.0

INFO: Converting OCI blobs to SIF format

INFO: Starting build...

Getting image source signatures

Copying blob 852e50cd189d done

Copying blob a6236801494d done

Copying blob 679c171d6942 done

Copying blob a547c2c45029 done

Copying blob 472d4a740df7 done

Copying blob 11bced570584 done

Copying blob ae93be0bde06 done

Copying blob 2210efc56b17 done

Copying blob 6ec0e15d839a done

Copying config ae72fe4b04 done

Writing manifest to image destination

Storing signatures

2022/01/17 10:07:08 info unpack layer: sha256:852e50cd189dfeb54d97680d9fa6bed21a6d7d18cfb56d6abfe2de9d7f173795

2022/01/17 10:07:09 info unpack layer: sha256:a6236801494d5ca9acfae6569427398c2942c031375b96cac887cafe1de5a09b

2022/01/17 10:07:12 info unpack layer: sha256:679c171d6942954a759f2d3a1dff911321940f23b0cdbe1d186f36bef0025124

2022/01/17 10:07:15 info unpack layer: sha256:a547c2c45029c28e7019e6c652697845a604ffd24e15a0434681a94d49ec1438

2022/01/17 10:07:15 info unpack layer: sha256:472d4a740df7df56e25c99e40be6e521648ace24680e90c5deef6431d23c0bdb

2022/01/17 10:07:15 info unpack layer: sha256:11bced570584cc3addd3d688e317b70e4b874e6ccf7dcdbb450452ca5dbf765d

2022/01/17 10:07:15 info unpack layer: sha256:ae93be0bde0627843640b882447ee91656d2131470f0effc1325625925fd3ac0

2022/01/17 10:07:16 info unpack layer: sha256:2210efc56b17214f9e16e5b2b6683462271d9fd3c877d0ba7e18822dbace3170

2022/01/17 10:07:51 info unpack layer: sha256:6ec0e15d839a3ceb1c159b18d6a2a69f65ca4dfbd99de74b0c19ac78b32b9ba1

INFO: Creating SIF file...

@.*** test_011722]$ ll

total 1536402

drwx------. 8 oushujun domain users 8 Jan 17 10:06 cache

drwxr-xr-x. 8 oushujun domain users 17 Jan 17 10:02 EDTA

-rwxr-xr-x. 1 oushujun domain users 1582268416 Jan 17 10:08 EDTA.sif

@.*** test_011722]$ singularity exec ./EDTA.sif EDTA.pl --genome EDTA/test/genome.fa

########################################################

Extensive de-novo TE Annotator (EDTA) v2.0.0

Shujun Ou @.***)

########################################################

Mon Jan 17 10:09:35 CST 2022 Dependency checking:

All passed!

Mon Jan 17 10:09:40 CST 2022 Obtain raw TE libraries using various structure-based programs:

Mon Jan 17 10:09:40 CST 2022 EDTA_raw: Check dependencies, prepare working directories.

Mon Jan 17 10:09:43 CST 2022 Start to find LTR candidates.

Mon Jan 17 10:09:43 CST 2022 Identify LTR retrotransposon candidates from scratch.

Shujun

On Mon, Jan 17, 2022 at 6:06 AM zzz @.***> wrote:

Dear Shujun,

Thanks for replying me! yes, I had followed the usage instruction in readme file. Here are the commands:

(base) @.*** annotation]$ singularity pull EDTA.sif docker://oushujun/edta:2.0.0 INFO: Using cached SIF image (base) @.*** annotation]$ ls 01_repeat_masking.sh RM_42884.FriJan141132022022 02_rnaseq_mapping.sh dummy060817.fa.437457 RM_43236.FriJan141135502022 Araport11_genes.202106.cds.fasta EDTA.sif Sample_15122021.fasta repeat_masking.log (base) @.*** annotation]$ export LC_ALL=C (base) @.*** annotation]$ singularity exec ./EDTA.sif EDTA.pl --genome Can_15122021.fasta

########################################################

Extensive de-novo TE Annotator (EDTA) v2.0.0

Shujun Ou @.***)

########################################################

At least 1 parameter is required:

1. Input fasta file: --genome

This is the Extensive de-novo TE Annotator that generates a high-quality structure-based TE library. Usage:

perl EDTA.pl [options]

... (omit the option list here)

— Reply to this email directly, view it on GitHub https://github.com/oushujun/EDTA/issues/249#issuecomment-1014397954, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNX4NCQUU7DHBMM2ZU7VPDUWPZ2LANCNFSM5L6TTXFQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you commented.Message ID: @.***>

[zmz1988]

Thanks for trying both versions for me, Shujun! Then I guess it's due to the upgrading of our computer system... I might need to talk to our cluster help! Thanks again!

[oushujun]

You may also want to look at this thread: #250

[zmz1988]

Thanks @oushujun! I gave up in trying, and used another cluster system with a newly build environment. Everything is fine now. Thanks!