cross_view_transformers icon indicating copy to clipboard operation
cross_view_transformers copied to clipboard

About test dataset

Open yelin2 opened this issue 1 year ago • 9 comments

Hi, i'm training this model with Nuscenes dataset.

when I train cross-view transformer network using this code, model's best IoU for road reach 71%

but in the "Cross-view Transformers for real-time Map-view Semantic Segmentation" paper road's best IoU is 74.3%

Is the reason why my experimental results are different is that the dataset for scoring in this paper is not a validation dataset? Or is it simply because the hyperparameters to get the highest score are different?

yelin2 avatar Aug 04 '22 00:08 yelin2

image

Same here. Thank you for the great work. I simply ran trainer.validate(model_module, datamodule=data_module, ckpt_path=ckpt_path) with the checkpoint you provided (cvt_nuscenes_vehicles_50k.ckpt) but I don't see the 36.0 or 37.5 [email protected] in the paper. All I could see was 34.8. Is there a separate test set other than the validation set? Or is there a checkpoint that was learned additionally?

curie3170 avatar Aug 21 '22 13:08 curie3170

Hi, i'm training this model with Nuscenes dataset.

when I train cross-view transformer network using this code, model's best IoU for road reach 71%

but in the "Cross-view Transformers for real-time Map-view Semantic Segmentation" paper road's best IoU is 74.3%

Is the reason why my experimental results are different is that the dataset for scoring in this paper is not a validation dataset? Or is it simply because the hyperparameters to get the highest score are different?

SOrry to disturb you . I just run the command like " python3 train.py +experiment=cvt_nuscenes_vehicle data.dataset_dir=datasets/nuscenes data.labels_dir=datasets/cvt_labels_nuscenes" But it go with error like below: Global seed set to 2022 Error executing job with overrides: ['+experiment=cvt_nuscenes_vehicle', 'data.dataset_dir=datasets/nuscenes', 'data.labels_dir=datasets/cvt_labels_nuscenes'] Error locating target '../cross_view_transformer.model.encoder.Encoder', see chained exception above. full_key: model.encoder

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

How can solve that? Thanks

emilyemliyM avatar Oct 26 '22 07:10 emilyemliyM

Hi @curie3170,

I got the validation results similar to yours, do you figure out why?

Thanks.

flymin avatar May 30 '23 10:05 flymin

Hello, do you have this problem?? Error executing job with overrides: ['+experiment=cvt_nuscenes_vehicle' git.exc.InvalidGitRepositoryError: /home/leezy/disk10t/lzx/cross_view_transformers

1146065545 avatar Jan 22 '24 12:01 1146065545

Hi, i'm training this model with Nuscenes dataset. when I train cross-view transformer network using this code, model's best IoU for road reach 71% but in the "Cross-view Transformers for real-time Map-view Semantic Segmentation" paper road's best IoU is 74.3% Is the reason why my experimental results are different is that the dataset for scoring in this paper is not a validation dataset? Or is it simply because the hyperparameters to get the highest score are different?

SOrry to disturb you . I just run the command like " python3 train.py +experiment=cvt_nuscenes_vehicle data.dataset_dir=datasets/nuscenes data.labels_dir=datasets/cvt_labels_nuscenes" But it go with error like below: Global seed set to 2022 Error executing job with overrides: ['+experiment=cvt_nuscenes_vehicle', 'data.dataset_dir=datasets/nuscenes', 'data.labels_dir=datasets/cvt_labels_nuscenes'] Error locating target '../cross_view_transformer.model.encoder.Encoder', see chained exception above. full_key: model.encoder

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

How can solve that? Thanks

did you solve this ??

1146065545 avatar Jan 22 '24 12:01 1146065545

Hi @curie3170,

I got the validation results similar to yours, do you figure out why?

Thanks.

did you solve this ??

1146065545 avatar Jan 22 '24 12:01 1146065545

Have you had this problem before, and how did you solve it, thank you! Error executing job with overrides: ['+experiment=cvt_nuscenes_vehicle' git.exc.InvalidGitRepositoryError: /home/leezy/disk10t/lzx/cross_view_transformers

1146065545 avatar Jan 23 '24 06:01 1146065545

Hi @curie3170,

I got the validation results similar to yours, do you figure out why?

Thanks.

hi, i have solved this problem ,you just need to enter :git init

1146065545 avatar Jan 23 '24 08:01 1146065545

Hi, i'm training this model with Nuscenes dataset. when I train cross-view transformer network using this code, model's best IoU for road reach 71% but in the "Cross-view Transformers for real-time Map-view Semantic Segmentation" paper road's best IoU is 74.3% Is the reason why my experimental results are different is that the dataset for scoring in this paper is not a validation dataset? Or is it simply because the hyperparameters to get the highest score are different?

SOrry to disturb you . I just run the command like " python3 train.py +experiment=cvt_nuscenes_vehicle data.dataset_dir=datasets/nuscenes data.labels_dir=datasets/cvt_labels_nuscenes" But it go with error like below: Global seed set to 2022 Error executing job with overrides: ['+experiment=cvt_nuscenes_vehicle', 'data.dataset_dir=datasets/nuscenes', 'data.labels_dir=datasets/cvt_labels_nuscenes'] Error locating target '../cross_view_transformer.model.encoder.Encoder', see chained exception above. full_key: model.encoder

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

How can solve that? Thanks

hi, i have solved this problem ,you just need to enter :git init

1146065545 avatar Jan 23 '24 08:01 1146065545