It will be perfect that adding save and load model using c++

[zhuantouer]

Very instructive work

[theflofly]

Thank you!

I can add that, how do you see it exactly ? I save the C++ model after training and in another class I load it and use it ?

[zhuantouer]

Yes, train and save the model by using c++ and load it separatelly in another c++ class even in other languages. This is cool.

[PinkySan]

Good idea. That will help a lot. I guess that could be done with WriteBinaryProto() or WriteTextProto(). Great work so far.

[PinkySan]

@theflofly Do you have any progress within Saving and Restoring the model and the weights? As I already mentioned, saving the model might be done within WriteBinaryProto() or WriteTextProto(). But I am not able to save the manipulated weights

[theflofly]

@PinkySan I did the restoring part, so I am able to restore a graph saved in python and use it. Now I am on the saving part. For some reason I don’t have my computer anymore so I can’t really progress on that until next week. I saw that someone from Google added some code to freeze a model and maybe save it (not sure), did you see https://github.com/tensorflow/tensorflow/tree/master/tensorflow/cc/tools ?

[theflofly]

@PinkySan Any success ? I will soon work on that issue.

[PinkySan]

Sorry. Totally forgot to answer. I was also struggling with a whole lot of problems with saving and restoring the data. Within my own github-repository i was adding some basic handling tests, where i am now heading to the saving and restoring problem.

I guess you have to save the model on the one hand (WriteBinaryProto -> *.pb-File) and saving the weights and biases on the other hand (SaveV2 -> checkpointFiles).

Take a look into my branch BasicHandlingTest and give it a try. Unfortunatly i am using windows and cmake

[theflofly]

I am also working on the saving part, the loading part is okay. I read the code that I shared in a previous comment about the freeze tool, there is no saving at all so it does not solve the issue.

My goal is to read how this is done in Python and call the same protobuff methods in a C++ API. If it works well I will maybe submit a PR. You can expect some updates within a week or two.

[PinkySan]

Were you able to restore the graph with the customized weights and biases?

[theflofly]

Yes but the graph has been saved using python. I will soon update the blog post but in the meantime here is the code.

#include "tensorflow/cc/saved_model/loader.h"
#include "tensorflow/core/public/session.h"

#include "data_set.h"

using namespace tensorflow;
using namespace std;

int main() {

    // useful for the data set metadata
    DataSet data_set("normalized_car_features.csv");

    // create the tensor containing the car to infer
    Tensor to_infer(DataTypeToEnum<float>::v(), TensorShape{1, 3});
    vector<float> car{data_set.input(110000.f, Fuel::DIESEL, 7.f)};
    copy_n(car.begin(), car.size(), to_infer.flat<float>().data());

    // loading the serialized model
    SavedModelBundle bundle;
    SessionOptions session_options;
    RunOptions run_options;

    TF_CHECK_OK(LoadSavedModel(session_options, run_options,
                   "/Users/fco/Documents/personnel/projects/dnn_from_scratch/saved_model/",
                   {"dnn_from_scratch_tensorflow"}, &bundle));

    std::vector<Tensor> outputs;

    // prediction using the trained neural net
    TF_CHECK_OK(bundle.session->Run({{"input/cars:0", to_infer}}, {"layer_3/Tanh:0"}, {}, &outputs));
    cout << "DNN output: " << *outputs[0].scalar<float>().data() << endl;
    std::cout << "Price predicted " << data_set.output(*outputs[0].scalar<float>().data()) << " euros" << std::endl;

    return 0;
}

The annoying part is to find out the exact names of the nodes in order to call them.

[PinkySan]

It is possible to get the names and types of all nodes. At least, if it is a graphDef-object. That might help. I like the way of saving these freeze models. That will help a lot. If i understand it right, it will change all variables into constants.

Freeze + SavedModel:

• Complete trained model
• No further training possible
• All variables are changed into constants
• All the relevant information is encapsulated within the pb-File (???)

SaveV2 + RestoreV2:

• Further training possible
• Saved checkpoints of model
• Variables still exist
• Model needs to be loaded + Checkpoint needs to be loaded and assigned

This list summarized the saving options the way I see it. That might be a wrong.