DeepSegmentor icon indicating copy to clipboard operation
DeepSegmentor copied to clipboard

Training issue due to environment.yml file is not updated

Open davidvct opened this issue 1 year ago • 10 comments

The environment.yml file not updated, and has the exploding gradient issue.

I changed to python 3.8 and Pytorch 2.0.1, and the training ran smoothly. One other thing need to update is the tensor self.label needs to be converted to float before feed into self.criterionSeg, in file deepcrack_model.py.

davidvct avatar Aug 10 '23 04:08 davidvct

Yes, the environment is somehow a bit old. I am thinking about to update the project in some days.

yhlleo avatar Aug 15 '23 02:08 yhlleo

The environment.yml file not updated, and has the exploding gradient issue.

I changed to python 3.8 and Pytorch 2.0.1, and the training ran smoothly. One other thing need to update is the tensor self.label needs to be converted to float before feed into self.criterionSeg, in file deepcrack_model.py.

Hi, did you just change python and pytorch version? I changed these two but still failed to install these dependencies. I am not sure what is wrong here.

citystrawman avatar Mar 15 '24 08:03 citystrawman

The environment.yml file not updated, and has the exploding gradient issue. I changed to python 3.8 and Pytorch 2.0.1, and the training ran smoothly. One other thing need to update is the tensor self.label needs to be converted to float before feed into self.criterionSeg, in file deepcrack_model.py.

Hi, did you just change python and pytorch version? I changed these two but still failed to install these dependencies. I am not sure what is wrong here.

Yes, I only changed those for installation. Could you paste your error message so that others can better understand the error? Are you install on Mac/Win/Linux?

davidvct avatar Mar 16 '24 12:03 davidvct

The environment.yml file not updated, and has the exploding gradient issue. I changed to python 3.8 and Pytorch 2.0.1, and the training ran smoothly. One other thing need to update is the tensor self.label needs to be converted to float before feed into self.criterionSeg, in file deepcrack_model.py.

Hi, did you just change python and pytorch version? I changed these two but still failed to install these dependencies. I am not sure what is wrong here.

Yes, I only changed those for installation. Could you paste your error message so that others can better understand the error? Are you install on Mac/Win/Linux?

I just managed to make it. But I got another problem saying that result type Float can't be cast to the desired output type Long at File "D:\Program Files\anaconda3\envs\deepsegmentlh\lib\site-packages\torch\nn\functional.py", line 3165, in binary_cross_entropy_with_logits . Is that what you mentioned in your post?

citystrawman avatar Mar 17 '24 06:03 citystrawman

The environment.yml file not updated, and has the exploding gradient issue. I changed to python 3.8 and Pytorch 2.0.1, and the training ran smoothly. One other thing need to update is the tensor self.label needs to be converted to float before feed into self.criterionSeg, in file deepcrack_model.py.

Hi, did you just change python and pytorch version? I changed these two but still failed to install these dependencies. I am not sure what is wrong here.

Yes, I only changed those for installation. Could you paste your error message so that others can better understand the error? Are you install on Mac/Win/Linux?

I just managed to make it. But I got another problem saying that result type Float can't be cast to the desired output type Long at File "D:\Program Files\anaconda3\envs\deepsegmentlh\lib\site-packages\torch\nn\functional.py", line 3165, in binary_cross_entropy_with_logits . Is that what you mentioned in your post?

I think it might be, but not quite sure as it was quite some time ago. Maybe try changing the file deepcrack_model.py as mentioned in my first post, and see if it would fix your issue.

davidvct avatar Mar 18 '24 06:03 davidvct

The environment.yml file not updated, and has the exploding gradient issue. I changed to python 3.8 and Pytorch 2.0.1, and the training ran smoothly. One other thing need to update is the tensor self.label needs to be converted to float before feed into self.criterionSeg, in file deepcrack_model.py.

Hi, did you just change python and pytorch version? I changed these two but still failed to install these dependencies. I am not sure what is wrong here.

Yes, I only changed those for installation. Could you paste your error message so that others can better understand the error? Are you install on Mac/Win/Linux?

I just managed to make it. But I got another problem saying that result type Float can't be cast to the desired output type Long at File "D:\Program Files\anaconda3\envs\deepsegmentlh\lib\site-packages\torch\nn\functional.py", line 3165, in binary_cross_entropy_with_logits . Is that what you mentioned in your post?

I think it might be, but not quite sure as it was quite some time ago. Maybe try changing the file deepcrack_model.py as mentioned in my first post, and see if it would fix your issue.

Thank you. I've been able to run the program. However, I am not quite clear about the logic of testing for this program: At first I thought test_img folder should be the images for testing, while test_lab folder should be the testing results, however, when I run the program, the testing process requires not only test_img, but also the counterpart img in test_lab which is the identified cracks of test_img.

citystrawman avatar Mar 18 '24 08:03 citystrawman

The environment.yml file not updated, and has the exploding gradient issue. I changed to python 3.8 and Pytorch 2.0.1, and the training ran smoothly. One other thing need to update is the tensor self.label needs to be converted to float before feed into self.criterionSeg, in file deepcrack_model.py.

Hi, did you just change python and pytorch version? I changed these two but still failed to install these dependencies. I am not sure what is wrong here.

Yes, I only changed those for installation. Could you paste your error message so that others can better understand the error? Are you install on Mac/Win/Linux?

I just managed to make it. But I got another problem saying that result type Float can't be cast to the desired output type Long at File "D:\Program Files\anaconda3\envs\deepsegmentlh\lib\site-packages\torch\nn\functional.py", line 3165, in binary_cross_entropy_with_logits . Is that what you mentioned in your post?

I think it might be, but not quite sure as it was quite some time ago. Maybe try changing the file deepcrack_model.py as mentioned in my first post, and see if it would fix your issue.

Thank you. I've been able to run the program. However, I am not quite clear about the logic of testing for this program: At first I thought test_img folder should be the images for testing, while test_lab folder should be the testing results, however, when I run the program, the testing process requires not only test_img, but also the counterpart img in test_lab which is the identified cracks of test_img.

test_lab should be the mask for test images. Refer to this link.

davidvct avatar Mar 20 '24 02:03 davidvct

The environment.yml file not updated, and has the exploding gradient issue. I changed to python 3.8 and Pytorch 2.0.1, and the training ran smoothly. One other thing need to update is the tensor self.label needs to be converted to float before feed into self.criterionSeg, in file deepcrack_model.py.

Hi, did you just change python and pytorch version? I changed these two but still failed to install these dependencies. I am not sure what is wrong here.

Yes, I only changed those for installation. Could you paste your error message so that others can better understand the error? Are you install on Mac/Win/Linux?

I just managed to make it. But I got another problem saying that result type Float can't be cast to the desired output type Long at File "D:\Program Files\anaconda3\envs\deepsegmentlh\lib\site-packages\torch\nn\functional.py", line 3165, in binary_cross_entropy_with_logits . Is that what you mentioned in your post?

I think it might be, but not quite sure as it was quite some time ago. Maybe try changing the file deepcrack_model.py as mentioned in my first post, and see if it would fix your issue.

Thank you. I've been able to run the program. However, I am not quite clear about the logic of testing for this program: At first I thought test_img folder should be the images for testing, while test_lab folder should be the testing results, however, when I run the program, the testing process requires not only test_img, but also the counterpart img in test_lab which is the identified cracks of test_img.

test_lab should be the mask for test images. Refer to this link.

As I understand, the masks(labels) for test images are output of a test procedure, so test_lab should be empty before testing. but when I run the program, it will pop out error if test_lab is empty. Please point out if I have some misunderstanding. Thanks.

citystrawman avatar Mar 20 '24 06:03 citystrawman

yes, you stored in the image you want to predict in test_img folder, and test_lab is for groundtruth.

The predicted result should be the 'fused' file in .\results folder, after executing sh ./scripts/test_deepcrack.sh <gpu_id>

davidvct avatar Mar 20 '24 09:03 davidvct

yes, you stored in the image you want to predict in test_img folder, and test_lab is for groundtruth.

The predicted result should be the 'fused' file in .\results folder, after executing sh ./scripts/test_deepcrack.sh <gpu_id>

@davidvct Thank you for the explanation! I wonder if we want to predict for images that "do not have ground-truth", is this doable? For example, if I have a set of satellite images but do not have the segmentation of those images, can I run test script to predict the result? If yes, how should I prepare the testing dataset? Thank you!

GoroYeh-HRI avatar Apr 09 '24 22:04 GoroYeh-HRI