any way to get text/image embedding

[xogo123]

is there any way to get text or image embedding if given text or image as input? thanks

[layumi]

Hi @xogo123 You may check the code in the test dir. You would find the code with name extract feature. I used them to extract text & image embedding.

[xogo123]

thanks a lot. there's another problem showed below when I run the code "extract_pic_feature_word2_plus_52" on my macbook

Undefined variable "dagnn" or class "dagnn.DagNN.loadobj". Error in extract_pic_feature_word2_plus_52 (line 3) net = dagnn.DagNN.loadobj(netStruct.net);

so I followed the instruction below to compile http://www.vlfeat.org/matconvnet/install/ run

mex -setup C++ cd <MatConvNet> addpath matlab vl_compilenn run <MatConvNet>/matlab/vl_setupnn

and run extract_pic_feature_word2_plus_52 again but another issue come out as below

Error using dagnn.Layer/load (line 200) No property init for a layer of type dagnn.Conv. Error in dagnn.DagNN.loadobj (line 27) block.load(struct(s.layers(l).block)) ; Error in extract_pic_feature_word2_plus_52 (line 3) net = dagnn.DagNN.loadobj(netStruct.net);

any thought? are you using matlab r2017b? matconvnet-1.0-beta25? or others? what's your other environment? (OS? compiler?)

or is there any chance to pack your "extract" code as an python app such that input : sentence or image output : sentence or image embedding after your model I think that will be very helpful for many python user

thanks

[layumi]

Hi @xogo123 Sorry for that. In fact, I write this code on two machines. Some part of code is mis-matching.

One easy way is to ignore the init in the Conv Layer. (I use it to chose msra init or other init methods.) Please comment https://github.com/layumi/Image-Text-Embedding/blob/master/matlab/%2Bdagnn/Layer.m#L205

Hope it may help you. If the bug still appears, let me know.

[jingyaa]

@layumi hello, I have the same problem, Error using dagnn.Layer/load (line 200) No property init for a layer of type dagnn.Conv.

Error in dagnn.DagNN.loadobj (line 27) block.load(struct(s.layers(l).block)) ;

Error in extract_pic_feature_word2_plus (line 4) net = dagnn.DagNN.loadobj(netStruct.net); how can I solve it?

[layumi]

Hi @jingyaa Would you like to ignore the line in your code? https://github.com/layumi/Image-Text-Embedding/blob/master/matlab/%2Bdagnn/Layer.m#L205

If the bug still appears, let me know.

[jingyaa]

Hi @layumi My Layer.m #205 is %error('No property %s for a layer of type %s.', ... But the bug still appears.

[layumi]

@jingyaa Could you show me the code in dagnn.Layer/load (line 200) as mentioned in the log? The whole function is better.

[jingyaa]

@layumi dagnn.Layer/load (line 200) is error('No property %s for a layer of type %s.', ... I ignore the line run again. No public property init exists for class dagnn.Conv.

Error in dagnn.Layer/load (line 203) obj.(fc) = s.(fc) ;

Error in dagnn.DagNN.loadobj (line 27) block.load(struct(s.layers(l).block)) ;

Error in extract_pic_feature_word2_plus (line 4) net = dagnn.DagNN.loadobj(netStruct.net);

[layumi]

@jingyaa Did you use my Matconvnet? You install two matconvnet? Actually I has included my Matconvnet in this repos.

So there are two methods.

1. Remove the matconvnet that you installed. Then install with my included matconvnet.

2. Or replace the your matconvnet Layer.m with my Layer.m. I am not sure whether other bugs will appear.

So I suggest you uninstall the matconvnet you downloaded. And install my matconvnet.

[jingyaa]

@layumi I install another Matconvnet. I will try your solution later. Thanks a lot.