tasn icon indicating copy to clipboard operation
tasn copied to clipboard
verified publisher icon researchmm

Using Attention-based Sampler (AttSampler) in TASN without the need of rebuilding MXNet

Open wkcn opened this issue 6 years ago • 17 comments

Hi, there. I wrote a project in order to use attention-based sampler of TASN without the need of rebuilding MXNet. The link of this project is https://github.com/wkcn/AttentionSampler

It is available for MXNet and PyTorch.

The result (default setting):

INFO:root:Epoch[204] Train-att_net_accuracy=1.000000
INFO:root:Epoch[204] Train-part_net_accuracy=0.979167
INFO:root:Epoch[204] Train-master_net_accuracy=0.989583
INFO:root:Epoch[204] Train-part_net_aux_accuracy=0.979167
INFO:root:Epoch[204] Train-master_net_aux_accuracy=0.989583                         
INFO:root:Epoch[204] Train-distillation_loss=4.280940                               
INFO:root:Epoch[204] Time cost=20.882
INFO:root:Epoch[204] Validation-att_net_accuracy=0.806771
INFO:root:Epoch[204] Validation-part_net_accuracy=0.849132
INFO:root:Epoch[204] Validation-master_net_accuracy=0.856944
INFO:root:Epoch[204] Validation-part_net_aux_accuracy=0.870486
INFO:root:Epoch[204] Validation-master_net_aux_accuracy=0.867361
INFO:root:Epoch[204] Validation-distillation_loss=3.713491



INFO:root:Epoch[299] Train-att_net_accuracy=1.000000
INFO:root:Epoch[299] Train-part_net_accuracy=0.984375
INFO:root:Epoch[299] Train-master_net_accuracy=0.984375
INFO:root:Epoch[299] Train-part_net_aux_accuracy=1.000000
INFO:root:Epoch[299] Train-master_net_aux_accuracy=1.000000
INFO:root:Epoch[299] Train-distillation_loss=4.100089
INFO:root:Epoch[299] Time cost=20.978
INFO:root:Saved checkpoint to "./model/tasn-0300.params"
INFO:root:Epoch[299] Validation-att_net_accuracy=0.804986
INFO:root:Epoch[299] Validation-part_net_accuracy=0.856728
INFO:root:Epoch[299] Validation-master_net_accuracy=0.860485
INFO:root:Epoch[299] Validation-part_net_aux_accuracy=0.864754
INFO:root:Epoch[299] Validation-master_net_aux_accuracy=0.869023
INFO:root:Epoch[299] Validation-distillation_loss=3.620270

Hope that it will be helpful for you!

wkcn avatar Jul 24 '19 08:07 wkcn

Hi，I tried your method,but get this error File "./AttentionSampler/attention_sampler/attention_sampler.py", line 26, in forward self.F.broadcast_minimum(threshold, attx, out=attx) File "<string>", line 48, in broadcast_minimum File "/usr/local/lib/python3.6/site-packages/mxnet-1.3.1-py3.6.egg/mxnet/_ctypes/ndarray.py", line 92, in _imperative_invoke ctypes.byref(out_stypes))) File "/usr/local/lib/python3.6/site-packages/mxnet-1.3.1-py3.6.egg/mxnet/base.py", line 253, in check_call raise MXNetError(py_str(_LIB.MXGetLastError())) mxnet.base.MXNetError: [10:27:32] src/operator/tensor/./elemwise_binary_broadcast_op.h:68: Check failed: l == 1 || r == 1 operands could not be broadcast together with shapes [12,1] [12,512,1]

tiancity-bytedance avatar Jul 25 '19 02:07 tiancity-bytedance

@tiancity-bytedance Thank you for the report. I will check it.

wkcn avatar Jul 25 '19 02:07 wkcn

Thanks ! look forward your response

tiancity-bytedance avatar Jul 25 '19 02:07 tiancity-bytedance

hello， can you tell the mean of function mobula.func.cumsum File "./AttentionSampler/attention_sampler/attention_sampler.py", line 53, in forward mobula.func.cumsum(N, attx, attxi, att_size) File "/mnt/cephfs_hl/vc/zhangtiancheng/finegrained/code/tasn-master/MobulaOP/mobula/func.py", line 184, in __call__ using_async=using_async) File "/mnt/cephfs_hl/vc/zhangtiancheng/finegrained/code/tasn-master/MobulaOP/mobula/func.py", line 89, in __call__ func = self.loader(self, arg_types, ctx, **self.loader_kwargs) File "/mnt/cephfs_hl/vc/zhangtiancheng/finegrained/code/tasn-master/MobulaOP/mobula/op/loader.py", line 539, in op_loader _build_lib(cpp_fname, code_buffer, ctx, dll_fname) File "/mnt/cephfs_hl/vc/zhangtiancheng/finegrained/code/tasn-master/MobulaOP/mobula/op/loader.py", line 274, in _build_lib build_path_ctx = os.path.join(build_path, ctx) File "/usr/local/lib/python3.6/posixpath.py", line 94, in join genericpath._check_arg_types('join', a, *p) File "/usr/local/lib/python3.6/genericpath.py", line 149, in _check_arg_types (funcname, s.__class__.__name__)) from None TypeError: join() argument must be str or bytes, not 'NoneType'

tiancity-bytedance avatar Jul 25 '19 06:07 tiancity-bytedance

@tiancity-bytedance mobula.func.cumsum is similar to np.cumsum

wkcn avatar Jul 25 '19 12:07 wkcn

@tiancity-bytedance mobula.func.cumsum is similar to np.cumsum

thanks for you respones, but np.cumsum just one array param. but what mean the four params in the func.cumsum?

tiancity-bytedance avatar Jul 26 '19 01:07 tiancity-bytedance

@tiancity-bytedance cumsum_kernel(const int N, const T* X, T* I, const int att_size)

The four parameters are batch size, input, output, the number of elements in a batch respectively.

wkcn avatar Jul 26 '19 03:07 wkcn

thanks ! it works for me ! and have you test the accuracy ?

tiancity-bytedance avatar Jul 26 '19 05:07 tiancity-bytedance

Sorry, I have not tested it. I'm busy recently. I will train it.

wkcn avatar Jul 26 '19 13:07 wkcn

Hi, @tiancity-bytedance . I have tested it and got the 86~87 accuracy on CUB-200-2011.

Setting: Number of GPUs: 4 Batch Size: 48 MobulaOP/mobula/config.yaml : USING_ASYNC_EXEC: 0

wkcn avatar Oct 01 '19 04:10 wkcn 

INFO:root:Epoch[299] Train-att_net_accuracy=1.000000
INFO:root:Epoch[299] Train-part_net_accuracy=0.984375
INFO:root:Epoch[299] Train-master_net_accuracy=0.984375
INFO:root:Epoch[299] Train-part_net_aux_accuracy=1.000000
INFO:root:Epoch[299] Train-master_net_aux_accuracy=1.000000
INFO:root:Epoch[299] Train-distillation_loss=4.100089
INFO:root:Epoch[299] Time cost=20.978
INFO:root:Saved checkpoint to "./model/tasn-0300.params"
INFO:root:Epoch[299] Validation-att_net_accuracy=0.804986
INFO:root:Epoch[299] Validation-part_net_accuracy=0.856728
INFO:root:Epoch[299] Validation-master_net_accuracy=0.860485
INFO:root:Epoch[299] Validation-part_net_aux_accuracy=0.864754
INFO:root:Epoch[299] Validation-master_net_aux_accuracy=0.869023
INFO:root:Epoch[299] Validation-distillation_loss=3.620270

wkcn avatar Jan 02 '20 05:01 wkcn

Hi, can you help me with this error?

Error in CustomOp.forward: Traceback (most recent call last): File "C:\Users\DIT\Anaconda3\envs\mxnet\lib\site-packages\mxnet\operator.py", line 987, in forward_entry aux=tensors[4]) File "d:\software\mobulaop\mobula\glue\mx.py", line 103, in forward out = self._forward(*in_data) File "./AttentionSampler/attention_sampler\attention_sampler.py", line 44, in forward attxi = F.cumsum(attx, 1) AttributeError: module 'mxnet.ndarray' has no attribute 'cumsum'

vb123er951 avatar Jun 05 '20 10:06 vb123er951

Hi @vb123er951 , the function mx.nd.cumsum is supported in the eldder version of MXNet. Please use the latest version such as MXNet 1.6 : )

wkcn avatar Jun 05 '20 11:06 wkcn

Hi @wkcn , thank you for reply, I am using Windows, but MXNet 1.6 seems not support in Windows... does there has any other solution?

vb123er951 avatar Jun 08 '20 01:06 vb123er951

Hi @vb123er951 , I have updated the code, which supports the old version of MXNet without cumsum.

wkcn avatar Jun 08 '20 04:06 wkcn

@wkcn Thank you very much! cumsum problem solved now, trying to solve other problems...

vb123er951 avatar Jun 08 '20 07:06 vb123er951

I have the problem

AttributeError: module "mobula.op" has no attribute "AttSamplerGrid"

Can anyone help me? Thanks a lot

YAOSL98 avatar Jun 30 '21 12:06 YAOSL98