SDGym icon indicating copy to clipboard operation
SDGym copied to clipboard

Benchmark with F1 score (reproduce leaderboard)

Open mnwright opened this issue 3 years ago • 0 comments

I'm trying to reproduce you CTGAN NeurIPS paper and/or the leaderboard linked here on Github and I wonder how to benchmark with the F1 score.

I'm following the code in README.md, i.e.:

import numpy as np
import pandas as pd
from sdv.tabular import GaussianCopula


def gaussian_copula(real_data, metadata):
    gc = GaussianCopula(default_distribution='gaussian')
    table_name = metadata.get_tables()[0]
    gc.fit(real_data[table_name])
    return {table_name: gc.sample()}
  
import sdgym

scores = sdgym.run(synthesizers=gaussian_copula, datasets=['adult'])

But it seems the result is accuracy and not F1? Where can I find the code to reproduce the leaderboard linked in the repo (this one)?

Thanks!

mnwright avatar Jan 03 '22 13:01 mnwright