Low accuracy of merged model

[lejelly]

Hi, Thanks for your great work!

I attempted to reproduce the results in Table 1(b) of your paper.

From my understanding, this experiment involves the following steps:

1. Divide CIFAR100 into 50 classes each.
2. Train two Resnet20 (widthx8) models separately.
3. Validate the classification results after merging.

I believe Table 1(b) presents the average and standard deviation of these 1, 2, and 3 steps repeated four times.

In an effort to replicate this, I executed the code with minimal modifications. The modifications I made were:

• In cifar_resnet_training.py, I set model_width = 8.
• In cifar50_resnet20.py, I changed 'eval_type' to 'clip'.

As a result, the accuracy after merging decreased regardless of the merging method. Are there any hyperparameters that need to be taken into consideration?(params a and b are default setting)

I appreciate once again that you are sharing this wonderful research with me.