VATr icon indicating copy to clipboard operation
VATr copied to clipboard

result issue @ fid.

Open Fyzjym opened this issue 2 years ago • 1 comments

Hi, I'm once again amazed by your great work. However, there seem to be some issues with testing the FID in the four scenarios. According to GANwriting's description,

  • first, a synthetic dataset is generated for each author wid_i based on in-vocabulary and out-of-vocabulary words,
  • and then the FID is calculated against the real dataset of each author wid_i,
  • and finally, it's averaged.

My understanding is as follows:

Two dictionaries: in-vocabulary dictionary (114 words) and out-of-vocabulary dictionary (400 words). Five hundred authors: visible style (339, train) and invisible style (161, test). Synthetic dataset: a total of 1000 samples, 339 * 2 + 161 + 2. However, the results I obtained differ significantly from the levels reported by VATr (delta over 50). Is my understanding correct, or is there something I'm missing?

Looking forward to your reply! 😄

Fyzjym avatar Sep 21 '23 03:09 Fyzjym

or ,calculated the FID for each of the four scenarios according to the code provided by GANwriting, without the step of "calculating separately and then averaging."

Fyzjym avatar Sep 21 '23 03:09 Fyzjym

Please check this out HWD

vittoriopippi avatar Mar 07 '25 10:03 vittoriopippi