support xavier in the distributed version

[mli]

hi guys,

is it possible to having xavier in the server node?

https://github.com/dmlc/cxxnet/blob/master/src/nnet/nnet_ps_server.cpp#L127

currently a server nodes can get the same args as workers, it would be like if a server node can parse the net config and get the xavier inited:

https://github.com/dmlc/cxxnet/blob/master/src/nnet/nnet_ps_server.cpp#L167

thanks mu

[tqchen]

To do this, we will need to build up information about node shapes

A quick way might be directly create a netconfig and a net in CPU by calling InitConnection, and free it immediately after this. This will create some potential memory allocation which was OK since the net was not so big for now. A better way may be add another function like https://github.com/dmlc/cxxnet/blob/master/src/layer/layer.h#L205 InitConnection, to setup output nodes's shape, without actually allocating the internal resources in connection.

@antinucleon @winstywang do any of you want to look into this?

[antinucleon]

I think a quick fix is using an extra python script to convert xavier/kaiming to normal gaussin/uniform conf file.