Tree-Transformer icon indicating copy to clipboard operation
Tree-Transformer copied to clipboard

Reproduce parsing performance

Open xgeric opened this issue 3 years ago • 0 comments

Thank you for the awesome work!

We were trying to reproduce the performance of the parsing module with four different random seeds (1, 11, 111, 1111), while the average f1 score given by the default script is 47.6, with max f1 around 50.4.

For your reference, we are controlling random seeds using the following function:

def set_random_seed(seed=0):
    random.seed(seed)
    numpy.random.seed(seed)
    torch.manual_seed(seed)
    torch.cuda.manual_seed(seed)
    torch.cuda.manual_seed_all(seed)  # if you are using multi-GPU.
    torch.backends.cudnn.enabled = False 
    torch.backends.cudnn.benchmark = False
    torch.backends.cudnn.deterministic = True

If you may, could you please kindly share the random seeds or any specific settings aside from the default parameters for the Table 1 experiments for reproduction? Thank you!

xgeric avatar May 09 '21 21:05 xgeric