Considering changing std::vector<FeatureTransform*> to graph container

[poweic]

In class NNet, I use std::vector<FeatureTransform*> to store all the feature transformations. For Recurrent Neural Network (RNN) or other user-defined NN structures, std::vector is not enough. A graph container and file format for graphs are needed.

Here are some graph libraries I found on Stackoverflow:

• Graphviz
• Boost Graph Library
• Lemon
• igraph
• OGDF

Also, I'm considering to split the big single model file into small ones and save them in a directory. Something like this:

my.model.init/
├── nnet.topo.xml   # a small topology file (need not to be XML), easier to modify
├── C1.dat          # model parameters in text/binary format
├── C2.dat          # model parameters in text/binary format
├── T1.dat          # model parameters in text/binary format
└── T2.dat          # model parameters in text/binary format

We can also provide a simple Web App to edit/draw nnet.topo.xml.

[supergrover]

With limiting the number of dependencies in mind, I think it would be easier to conjure up a graph system ourselves. We don't need fancy-pantsy graph stuff, just simplicity and speed.

[poweic]

I think building our own graph library is okay, but file format would be an issue. We should follow standards such as GML. And I hate to parse file : (

[supergrover]

I like the way caffe solved this problem. They store network structures in google protobuf files. Free parser and a compact and clear format. Also, user customizable as hell.

For example:

layer {
    name: 'data_layer'
    type: SVM_DATA
    .. options, file to read from or whatever ...

    top: 'data'
}

layer {
    name: 'label_layer'
    type: SVM_DATA
    ... options, file to read from or whatever ...
    top: 'label'
}
layer {
    name: 'sigmoid_kayer'
    type: SIGMOID

    .. geometry options .. 

    bottom: 'data'
    top: 'layer-1-out'
}
layer {
    name: 'error'
    type: SOFTMAX_ERROR
    bottom: 'layer-1-out'
    bottom: 'label'
    .. options ..
}