Training issue with fcos_r50v1_fpn_1x.py

[dragon515]

/home/www/anaconda3n/envs/simpledet/bin/python /home/www/simpledet/detection_train.py --config config_cancer/fcos_r50v1_fpn_1x.py Traceback (most recent call last): File "_ctypes/callbacks.c", line 232, in 'calling callback function' File "/home/www/anaconda3n/envs/simpledet/lib/python3.7/site-packages/mxnet/operator.py", line 720, in creator op_prop = prop_cls(**kwargs) File "/home/www/simpledet/models/FCOS/input.py", line 90, in init self.stride = self.p.stride AttributeError: 'NoneType' object has no attribute 'stride'

Segmentation fault: 11

Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)

[RogerChern]

@kfxw

[kfxw]

Hi, @dragon515 Can you provide more details about what changes you've made? Thx

[kfxw]

@RogerChern Maybe we should design a new way to pass a config class into python custom op. Now I use 'from config.fcos_r50v1_fpn_1x import throwout_param' in custom op's. It is too risky if others modify the config file's name or paths, and asking users to change this line in input.py is also trival. Can we change the detection_train/test.py a little bit and re-use the returned result of parse_args()?

[RogerChern]

I see. Can you just pass the config fields needed as normal parameters? I don't understand why a config class need to be passed.

On Mon, Dec 9, 2019 at 4:23 PM kfxw [email protected] wrote:

@RogerChern https://github.com/RogerChern Maybe we should design a new way to pass a config class into python custom op. Now I use 'from config.fcos_r50v1_fpn_1x import throwout_param' in custom op's. It is too risky if others modify the config file's name or paths, and asking users to change this line in input.py is also trival. Can we change the detection_train/test.py a little bit and re-use the returned result of parse_args()?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/TuSimple/simpledet/issues/269?email_source=notifications&email_token=ABGODH4EFLSDAFSZAR6AKYLQXX6APA5CNFSM4JYEASG2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGIILYA#issuecomment-563119584, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABGODH2PBGU45CQLH6ODPBDQXX6APANCNFSM4JYEASGQ .

[kfxw]

@RogerChern Sometimes you may need a list or array. Such cases face the same problem as a class has.

[RogerChern]

@kfxw I see. You can use the same hack in mxnext to pass any python object across python-cxx-python boundary.

The idea is simple:

1. serialize objects to bytes
2. pass bytes to cxx interfaces
3. deserialize objects in CustomOp python code

Encode: https://github.com/RogerChern/mxnext/blob/cb0b8446fcf6cd8e05c1ef7ae41fec4964a06e2b/mxnext/debug.py#L102-L106

Decode: https://github.com/RogerChern/mxnext/blob/cb0b8446fcf6cd8e05c1ef7ae41fec4964a06e2b/mxnext/debug.py#L31-L33

[kfxw]

@RogerChern Got it. Maybe update the codes in a few days.

[dragon515]

Hi, @dragon515 Can you provide more details about what changes you've made? Thx

I changed the dataset, the class number, and I used two gpus so change the gpu idx, nothing else.

[kfxw]

@dragon515 I suppose you've also changed the path of the configuration file. If so, an immediate remedy is to modify https://github.com/TuSimple/simpledet/blob/dd4042c3e69456445ad6625d4e66b60ed4bc62a9/models/FCOS/input.py#L88 and https://github.com/TuSimple/simpledet/blob/dd4042c3e69456445ad6625d4e66b60ed4bc62a9/models/FCOS/input.py#L147 from from config.fcos_r50v1_fpn_1x import throwout_param into from config_cancer.fcos_r50v1_fpn_1x import throwout_param