MEMORY ERROR on old server for teaching

[fglaser]

Dear all,

I am trying to run colabfold_batch in an old server for teaching purposes, for small structures < 200aa it runs oks, but for complexes it fails with the following ERROR message:

2023-05-16 08:56:35.453801: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT 2023-05-16 08:56:36,089 Running on GPU 2023-05-16 08:56:36.097971: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1956] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform. Skipping registering GPU devices... 2023-05-16 08:56:36,548 Found 4 citations for tools or databases 2023-05-16 08:56:36,549 Query 1/1: BFL_FF1 (length 1458) COMPLETE: 100%|█████████████████████████████████████████████████████████████████████| 150/150 [elapsed: 00:01 remaining: 00:00] 2023-05-16 08:56:38,584 Setting max_seq=49, max_extra_seq=1 2023-05-16 08:57:06,288 Could not predict BFL_FF1. Not Enough GPU memory? RESOURCE_EXHAUSTED: Failed to allocate request for 1.03GiB (1105168384B) on device ordinal 0 2023-05-16 08:57:06,294 Done

I am running CUDA 11.6 linux on ubuntu Ubuntu 20.04.4 LTS (GNU/Linux 5.15.0-71-generic x86_64) within the following environment.

I know it is an old computer, but it would greatly help junior students to be able to run larger complexes.

Do you think it's possible to tweak or overcome the problem to allow larger proteins to run? (even if slower). I noticed also the tensor warning, reinstalled tensorflow but still the warning remains.

Any help will be highly appreciated.

Thanks a lot in advance,

Fabian Glaser Technion, Fabian

nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2020 NVIDIA Corporation Built on Mon_Oct_12_20:09:46_PDT_2020 Cuda compilation tools, release 11.1, V11.1.105 Build cuda_11.1.TC455_06.29190527_0

Tue May 16 14:40:59 2023
+-----------------------------------------------------------------------------+ | NVIDIA-SMI 510.108.03 Driver Version: 510.108.03 CUDA Version: 11.6 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 NVIDIA GeForce ... Off | 00000000:02:00.0 Off | N/A | | 0% 39C P8 12W / 185W | 5MiB / 4096MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ | 1 NVIDIA GeForce ... Off | 00000000:03:00.0 Off | N/A | | 0% 40C P8 12W / 185W | 5MiB / 4096MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ | 2 NVIDIA GeForce ... Off | 00000000:82:00.0 Off | N/A | | 0% 32C P8 12W / 185W | 5MiB / 4096MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ | 3 NVIDIA GeForce ... Off | 00000000:83:00.0 Off | N/A | | 0% 32C P8 12W / 185W | 5MiB / 4096MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | 0 N/A N/A 1505 G /usr/lib/xorg/Xorg 3MiB | | 1 N/A N/A 1505 G /usr/lib/xorg/Xorg 3MiB | | 2 N/A N/A 1505 G /usr/lib/xorg/Xorg 3MiB | | 3 N/A N/A 1505 G /usr/lib/xorg/Xorg 3MiB | +-----------------------------------------------------------------------------+