database/pdb_mmcif/raw/*/*.cif': No such file or directory

[CFDavidHou]

Hi there,

I started downloading all data into my designated folder with sufficient room. While I am able to download params, bfd, mgnify, pdb70, and uniclust30, I got error message for downloading mmcif.

$ ./download_pdb70.sh /database
02/02 09:21:07 [NOTICE] Downloading 1 item(s)
[#b9fa7b 1.8GiB/19GiB(9%) CN:1 DL:2.7MiB ETA:1h47m45s]

but

$ ./download_pdb_mmcif.sh /database
Running rsync to fetch all mmCIF files (note that the rsync progress estimate might be inaccurate)...
Unzipping all mmCIF files...
Flattening all mmCIF files...
mv: cannot stat '/home5/alphafold_database/pdb_mmcif/raw/*/*.cif': No such file or directory

I also noticed that I am unable to download uniref90. Is this might be the firewall issue on my end? My university did put firewall on any ftp source.

Please help.

Thank you!

David

[spencer411]

use the file from the docker github instead: https://github.com/deepmind/alphafold/blob/main/scripts/download_pdb_mmcif.sh

[AlenGhzau]

the problem may caused by the comments code of the script download_pdb_mmcif.sh:

# rsync --recursive --links --perms --times --compress --info=progress2 --delete --port=33444 \
#   rsync.rcsb.org::ftp_data/structures/divided/mmCIF/ \
#   "${RAW_DIR}"

# rsync --recursive --links --perms --times --compress --info=progress2 --delete --port=33444 \
#   data.pdbj.org::ftp_data/structures/divided/mmCIF/ \
#   "${RAW_DIR}"

make one of them uncommentted, and then the work goes on if you want to go on the task ,you need to edit the download_all_data.sh and comment the previous code like this

`echo "Downloading AlphaFold parameters..."
#bash "${SCRIPT_DIR}/download_alphafold_params.sh" "${DOWNLOAD_DIR}"

echo "Downloading BFD..."
#bash "${SCRIPT_DIR}/download_bfd.sh" "${DOWNLOAD_DIR}"

echo "Downloading MGnify..."
#bash "${SCRIPT_DIR}/download_mgnify.sh" "${DOWNLOAD_DIR}"

echo "Downloading PDB70..."
#bash "${SCRIPT_DIR}/download_pdb70.sh" "${DOWNLOAD_DIR}"

echo "Downloading PDB mmCIF files..."
bash "${SCRIPT_DIR}/download_pdb_mmcif.sh" "${DOWNLOAD_DIR}"

echo "Downloading Uniclust30..."
bash "${SCRIPT_DIR}/download_uniclust30.sh" "${DOWNLOAD_DIR}"

echo "Downloading Uniref90..."
bash "${SCRIPT_DIR}/download_uniref90.sh" "${DOWNLOAD_DIR}"

end

[jruhym]

@kuixu thanks for this non-docker solution. Might I suggest:

1. un-commenting one of the download lines that @AlenGhzau mentioned. You can leave a comment above them stating what the difference is.
2. If download of these files is not desired to be the default behavior, you could comment out the line in the download-all script that calls the mmCIF-specific script and add a comment above stating that, if one wants these files, to uncomment this line and to look at the mmCIF-specific download script to select from which DB the user wants those files
3. Add a line in the README informing the user of these choices.

OR a cleaner way might be to implement an optional input parameter to the download-all script that is a enum of options like [-m <none|rcsb|japan>] where none is default, say.