model.save does not save anything

[FatemehEsfahani]

Hi I have tried running MnistFnnKerasFunctional.cs which trains a model on Mnist data and saves it to a file called "mnist_model". However, the model does not save anything. Also, I get error when uncommenting "keras.models.load_model". It says that the load_model does not exist. could any one help with it? Thanks

[Moranic]

I ran into this myself, AFAIK the save/load functionality just isn't in yet. I ended up making my own save-load functionality. You can get the trainable weights from each layer, and reinstantiate them using a custom KernelInitializer and BiasInitializer. Definitely not how you're supposed to do it, but it works for me.

[FatemehEsfahani]

@Moranic Is it possible to share that part of your code?

[Moranic]

You can make a custom initialiser like this:

    class LoadInitializer : IInitializer
    {
        float[] toLoad;

        public LoadInitializer(Tensor toLoad) {
            this.toLoad = toLoad.ToArray<float>();
        }

        public Tensor Apply(InitializerArgs args) {
            NDArray data = new NDArray(toLoad, args.Shape);
            //var et = new Tensor(data);
            var et = new EagerTensor(data, "from_file");
            return et;
        }
    }

The kernel initialiser and the bias initialiser both inherit from IInitializer, so you can provide this one for both. You can obtain the trainable variables from a layer using this:

    float[] Kernel = yourLayerHere.trainable_variables[0].AsTensor(TF_DataType.TF_FLOAT).ToArray<float>();
    float[] Bias = yourLayerHere.trainable_variables[1].AsTensor(TF_DataType.TF_FLOAT).ToArray<float>();

The only other things you need to save per layer is the activation type (you can convert it to your own enum for example) and the number of neurons. I'm currently only using fully connected (e.g. dense) layers, so this class works for me:

    [Serializable]
    public class SerialisableLayer
    {
        public SerialisableLayer() { }

        public SerialisableLayer(Layer output, int numClasses) {
            Neurons = numClasses;
            Kernel = output.trainable_variables[0].AsTensor(TF_DataType.TF_FLOAT).ToArray<float>();
            Bias = output.trainable_variables[1].AsTensor(TF_DataType.TF_FLOAT).ToArray<float>();
        }

        public SerialisableLayer(Layer layer, LayerArgs args) {
            Neurons = args.Neurons;
            Activation = args.Activation == keras.activations.Linear ? ActivationType.Linear :
                         args.Activation == keras.activations.Relu ? ActivationType.Relu :
                         args.Activation == keras.activations.Sigmoid ? ActivationType.Sigmoid :
                         args.Activation == keras.activations.Tanh ? ActivationType.Tanh :
                         ActivationType.Unknown;
            Kernel = layer.trainable_variables[0].AsTensor(TF_DataType.TF_FLOAT).ToArray<float>();
            Bias = layer.trainable_variables[1].AsTensor(TF_DataType.TF_FLOAT).ToArray<float>();
        }

        public LayerArgs ToLayerArgs() {
            return new LayerArgs(
                Neurons, 
                Activation == ActivationType.Linear ? keras.activations.Linear :
                Activation == ActivationType.Relu ? keras.activations.Relu :
                Activation == ActivationType.Sigmoid ? keras.activations.Sigmoid :
                Activation == ActivationType.Tanh ? keras.activations.Tanh : 
                null,
                new LoadInitializer(new Tensor(Kernel)),
                new LoadInitializer(new Tensor(Bias)));

        }

        public int Neurons { get; set; }
        public ActivationType Activation { get; set; } = ActivationType.Unknown;
        public float[] Kernel { get; set; }
        public float[] Bias { get; set; }

        public enum ActivationType { Relu, Tanh, Linear, Sigmoid, Unknown};

    }

LayerArgs is my own class which saves the arguments needed to reconstruct a layer:

    public class LayerArgs
    {
        public LayerArgs(int neurons, Activation activation, IInitializer kernelInitializer = null, IInitializer biasInitializer = null) {
            Neurons = neurons;
            Activation = activation;
            KernelInitializer = kernelInitializer;
            BiasInitializer = biasInitializer;
        }

        public int Neurons { get; set; }
        public Activation Activation { get; set; }
        public IInitializer KernelInitializer { get; set; }
        public IInitializer BiasInitializer { get; set; }

    }

You then need to convert those LayerArgs into an actual neural network, and a neural network into a bunch of LayerArgs. That code is really quite specific to my case so it won't help you, but it should also not be too difficult.

I should also note that this method really isn't a very good way of doing this, but the upside is that it does work for what I need it to do.

[FatemehEsfahani]

@Moranic Thank you so much for sharing this. I really appreciate. I am new to C# and for me it seems to be very difficult to implement a similar code like the one you have written in C#. So, I think I would better using keras.NET which allows loading the python Keras models in C#.