AIR-ASVspoof icon indicating copy to clipboard operation
AIR-ASVspoof copied to clipboard

from torch._C import * Import Error

Open SuchitReddi opened this issue 2 years ago • 6 comments

Terminal Input: python train.py --add_loss ocsoftmax -o ./models1028/ocsoftmax/train --gpu 1

Terminal Output: Traceback (most recent call last): File "train.py", line 5, in from resnet import setup_seed, ResNet File "D:\Programming\Python\Python\AIR-ASVspoof\resnet.py", line 1, in
import torch File "D:\Programming\Python\Python\venv\lib\site-packages\torch_init_.py", line 79, in <module> from torch._C import * ImportError: DLL load failed: The specified procedure could not be found.

Comments: The same repeats even for --gpu 0. Tried reinstalling torch version 1.1.0 with and without GPU.

For additional information; the same code run on python terminal gave the following output:

D:\Programming\Python\Python\venv\Scripts\python.exe C:/Users/Suchit/AppData/Roaming/JetBrains/IntelliJIdea2022.2/plugins/python/helpers/pydev/pydevconsole.py --mode=client --host=127.0.0.1 --port=62758 import sys; print('Python %s on %s' % (sys.version, sys.platform)) sys.path.extend(['D:\Programming\Python\Python']) PyDev console: starting. Python 3.6.0 (v3.6.0:41df79263a11, Dec 23 2016, 08:06:12) [MSC v.1900 64 bit (AMD64)] on win32 runfile('D:\Programming\Python\Python\AIR-ASVspoof-Suchit\train.py', wdir='D:\Programming\Python\Python\AIR-ASVspoof-Suchit') Traceback (most recent call last): File "C:\Users\Suchit\AppData\Local\Programs\Python\Python36\lib\code.py", line 91, in runcode exec(code, self.locals) File "", line 1, in File "C:\Users\Suchit\AppData\Roaming\JetBrains\IntelliJIdea2022.2\plugins\python\helpers\pydev_pydev_bundle\pydev_umd.py", line 198, in runfile pydev_imports.execfile(filename, global_vars, local_vars) # execute the script File "C:\Users\Suchit\AppData\Roaming\JetBrains\IntelliJIdea2022.2\plugins\python\helpers\pydev_pydev_imps_pydev_execfile.py", line 18, in execfile exec(compile(contents+"\n", file, 'exec'), glob, loc) File "D:\Programming\Python\Python\AIR-ASVspoof-Suchit\train.py", line 5, in from resnet import setup_seed, ResNet File "C:\Users\Suchit\AppData\Roaming\JetBrains\IntelliJIdea2022.2\plugins\python\helpers\pydev_pydev_bundle\pydev_import_hook.py", line 21, in do_import module = self._system_import(name, *args, **kwargs) File "D:\Programming\Python\Python\AIR-ASVspoof-Suchit\resnet.py", line 1, in import torch File "C:\Users\Suchit\AppData\Roaming\JetBrains\IntelliJIdea2022.2\plugins\python\helpers\pydev_pydev_bundle\pydev_import_hook.py", line 21, in do_import module = self.system_import(name, *args, **kwargs) File "D:\Programming\Python\Python\venv\lib\site-packages\torch_init.py", line 79, in from torch._C import * File "C:\Users\Suchit\AppData\Roaming\JetBrains\IntelliJIdea2022.2\plugins\python\helpers\pydev_pydev_bundle\pydev_import_hook.py", line 21, in do_import module = self._system_import(name, *args, **kwargs) ImportError: DLL load failed: The specified procedure could not be found.

Please look into this @yzyouzhang. Thank you.

SuchitReddi avatar Oct 25 '22 08:10 SuchitReddi

Did you try to reinstall the library?

yzyouzhang avatar Oct 27 '22 02:10 yzyouzhang

Can you please tell me how I can do that? I tried manually installing torch packages, but not the whole library.

SuchitReddi avatar Oct 27 '22 02:10 SuchitReddi

If you use conda to manage the environment, I suggest you to create a new environment and install packages from scratch. Then use the new environment to run the code.

yzyouzhang avatar Oct 27 '22 03:10 yzyouzhang

I am using IntelliJ with a virtual environment (venv). I tried deleting the venv folder, creating a new one and running it again. It didn't work. Is there anything related to paths in the code, which I might be getting wrong?

SuchitReddi avatar Oct 27 '22 03:10 SuchitReddi

Here is a minor update on the issue. So, I tried running train.py on another laptop, but I got lazy to downgrade the existing python version from 3.9 to 3.6 and just left it in 3.9. On this testing laptop, I used the pycharm community edition. I installed the required packages like torch(the latest version instead of 1.1.0). This time I did not get the import error. So, I will try upgrading the python version from 3.6 on my laptop too. I will update my results here shortly.

SuchitReddi avatar Oct 28 '22 10:10 SuchitReddi

image Also, just found out that pytorch is not supporting 3.6 anymore, only 3.7-3.9 are supported. That may be causing the error.

SuchitReddi avatar Oct 28 '22 10:10 SuchitReddi

My recent repo https://github.com/yzyouzhang/HBAS_chapter_voice3 uses the newest python and pytorch. Feel free to check it out.

yzyouzhang avatar Oct 29 '22 06:10 yzyouzhang

Thank you. I am looking into it. Will post here if there are any more issues.

SuchitReddi avatar Oct 30 '22 04:10 SuchitReddi

The torch error is resolved, but I keep encountering others.

So, is there any guide available anywhere on what steps I have to follow exactly to make a demonstration of this project? Like step by step guide.

Thank you @yzyouzhang .

SuchitReddi avatar Oct 31 '22 05:10 SuchitReddi

Could you let me know what issues you encountered? I think the README file is a clear guide.

yzyouzhang avatar Nov 01 '22 00:11 yzyouzhang

Terminal Input: (venv39) D:\Programming\Python\Python\HBAS_chapter_voice3>python train.py -o D:/Programming/Python/Python/AIR-ASVspoof/models1028/ocsoftmax/train -l ocsoftmax -m resnet

Terminal Output: Cuda device available: False Traceback (most recent call last): File "D:\Programming\Python\Python\HBAS_chapter_voice3\train.py", line 654, in train(args) File "D:\Programming\Python\Python\HBAS_chapter_voice3\train.py", line 207, in train
trainDataLoader = DataLoader(training_set, batch_size=args.batch_size, File "D:\Programming\Python\Python\venv39\lib\site-packages\torch\utils\data\dataloader.py", line 344, in init sampler = RandomSampler(dataset, generator=generator) # type: ignore[arg-type] File "D:\Programming\Python\Python\venv39\lib\site-packages\torch\utils\data\sampler.py", line 107, in init raise ValueError("num_samples should be a positive integer " ValueError: num_samples should be a positive integer value, but got num_samples=0

Comments: And apart from this error, I observed that dataset.py and raw_dataset.py have many classes like ASVspoof2015, ASVspoof2019LASim, ASVspoof2019Transm_aug, etc. I do not understand what paths I have to provide, like path_to_deviced in the class ASVspoof2019LASim in dataset.py (line 160). Should any specific voice recording device from the provided devices in the code (line 170) be used for the project?

I am presenting this paper as my University project. I successfully retraced your steps until the data preparation part of the project. I am encountering errors in the train and test parts. That was why I asked for a step-by-step guide to reproduce the results.

SuchitReddi avatar Nov 01 '22 14:11 SuchitReddi

If you are only reproducing my one-class paper, using ASVspoof2019LA is fine. The others are data augmented version for studying the channel robustness. From the error you presented, it seems the dataset folder is empty so the num_samples=0. If you want to reproduce the results, you may not need to run training. I have provided the pre-trained model.

yzyouzhang avatar Nov 01 '22 17:11 yzyouzhang

What do you mean by pre-trained model? So, if I don't have to run training, what about testing part?

SuchitReddi avatar Nov 01 '22 18:11 SuchitReddi

After training, you will get a trained model, right? I have provided that. In test, you can simply load it.

yzyouzhang avatar Nov 01 '22 21:11 yzyouzhang

I will look into it and let you know of any further issues. Thank you again for your help @yzyouzhang .

SuchitReddi avatar Nov 02 '22 07:11 SuchitReddi

In test.py, I have a few doubts regarding the paths of these variables:

line 64: with open(os.path.join(dir_path, 'checkpoint_cm_score_ASVspoof2019LA.txt'), 'w') as cm_score_file: What path should I give here?

line 378: model_path = os.path.join(args.model_dir, "anti-spoofing_feat_model.pt") line 379: loss_model_path = os.path.join(args.model_dir, "anti-spoofing_loss_model.pt") I could find the anti-spoofing_loss_model.pt at \AIR-ASVspoof\models1028\ocsoftmax. But I couldn't find anti-spoofing_feat_model.pt anywhere. Is it the same as anti-spoofing_lfcc_model.pt?

SuchitReddi avatar Nov 02 '22 15:11 SuchitReddi

Yes. should be lfcc_model

yzyouzhang avatar Nov 02 '22 19:11 yzyouzhang

What about the cm_score_file?

SuchitReddi avatar Nov 03 '22 05:11 SuchitReddi

It is the output. You can put it wherever you want.

yzyouzhang avatar Nov 04 '22 02:11 yzyouzhang

I think I am very close to running test.py successfully. I resolved most of the issues. I am sure that this might be the final obstacle I am facing. The error is related to wrongly configured CUDA:

Terminal Output torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 30.00 MiB (GPU 0; 2.00 GiB total capacity; 1.64 GiB already allocated; 0 bytes free; 1.72 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

I believe that the solution must either run on a CPU or configure CUDA correctly. So, please tell me how to configure CUDA correctly. Should I install a specific version of CUDA? I have a CUDA-compatible GPU NVIDIA GeForce MX250. What is the correct process for CUDA configuration for this project?

SuchitReddi avatar Nov 04 '22 18:11 SuchitReddi

You can try reduce the batch size. Maybe the batch of data is too large to fit on your GPU.

yzyouzhang avatar Nov 05 '22 02:11 yzyouzhang

I successfully ran test.py file. Thank you @yzyouzhang.

SuchitReddi avatar Nov 05 '22 09:11 SuchitReddi