Training and Evaluation of Neural Networks trained on MPC?

[danijimmy19]

I have two questions related to Neural Networks (NN) trained on MPC.

1. If the Neural Network is trained on plaintext values, can this NN be used for making predictions on the secure values?

• Here, the training of the NN is performed on plaintext values, and no MPC protocol is used while performing the training. Can the model without MPC be used for making predictions on secure values over MPC

2. If we were to train the model using say torch_mnist_dense.mpc script provided in the MP-SPDZ repo, what kind of machine configurations are recommended for training the NN over MPC?

• Here, I have tried using a machine with 72-core CPU for training the model over MPC, and I kept the program running for a few hours, but not even 1 iteration or epoch was completed. This same script is working in emulate settings where no MPC is used.

[mkskeller]

1. Yes, this is what torch_mnist_dense_pretrain.mpc does.
2. It depends on the protocol. Table 5 in https://eprint.iacr.org/2022/933.pdf shows that there are several orders of magnitude between the different protocols.

[danijimmy19]

Hi @mkskeller , I have some concerns about loading the trained model. So, I trained a dense model using MNIST dataset. I loading that model in another script using following code snippet:

f = open('Player-Data/Binary-Output-P0-0')
dense_model = np.fromfile(f, "double", count=128) # where 128 is the shape of feature vector for 1 datapoint

start = 0
for var in optimizer.trainable_variables:
    start = var.read_from_file(start)
    
# making prediction on the trained model
n_correct, loss = optimizer.reveal_correctness(test_samples, test_labels, 128, running=True)
print_ln('Secure accuracy (testing): %s/%s', n_correct, len(test_samples))

When I load the model using this code, it always throws this error

mp-spdz-0.3.8/Scripts/run-common.sh: line 90: 94099 Aborted                 (core dumped) $my_prefix $SPDZROOT/$bin $i $params 2>&1
     94100 Done                    | { if test "$BENCH"; then
    if test $i = $front_player; then
        tee -a $log;
    else
        cat >> $log;
    fi;
else
    if test $i = $front_player; then
        tee $log;
    else
        cat > $log;
    fi;
fi; }

Even the machine has enough memory.

I'm using this command to execute the script:

Scripts/compile-run.py -E mascot Programs/Source/mnist_*.mpc

Can you please guide me on what seems to be the issue here?

This is the dump from log file.

terminate called after throwing an instance of 'signature_mismatch'
  what():  Signature in Persistence/Transactions-P0.data doesn't match protocol. Re-run preprocessing
=== Party 1
Using statistical security parameter 40
terminate called after throwing an instance of 'file_error'
  what():  File Error : Got to EOF when reading from disk (expecting 4096 bytes from 37).

[mkskeller]

If you use read_from_file, you have to run the right training first in the same protocol, for example torch_mnist_dense before torch_mnist_dense_test. Instead, it looks you're trying the get the weights a file from a previous cleartext output. You could try moving the file to Player-Data/Input-Binary-P0-0 and then use the input_from(0, binary=True) on the layers, but this hasn't been tested.