Synthetic data from PARSynthesizer does not follow original data distribution

[PaudGS]

Environment details

If you are already running SDV, please indicate the following details about the environment in which you are running it:

• SDV version: 1.16.1
• Python version: 3.11.9
• Operating System: WIndows

Problem description

Trying to create synthetic numeric values using PARSynthesizer returns values very close to the mean of the original distribution, with little variance between values. The data is a simple table consisting of patient_id(sequence_id), mesure_id, measure_date_time(sequence key) and value of measurement.

The histograms of both distributions look like this:

What I already tried

I have tried different epoch values, running with a larger input dataset and the different RDT transforms. Running the same data with the GaussianCopulaSynthesizer yields much better results, but I would lose the time series aspect of the original data.

Is this the expected behaviour of the PARSynthesizer or am I doing something wrong?

[srinify]

Hi @PaudGS 👋

At the moment, our single table and multi table synthesizers are definitely a bit more mature than PARSynthesizer, our sequential synthesizer. So this difference alone might be causing the shortcoming you're experiencing unfortunately, especially if you've already experimented with different epochs and different transformers.

To rule out a few more things, it would be helpful if you could share your metadata, the column(s) you care the most about, and maybe some sample values that represent the rough distribution (e.g. you can take your original values but scale them by a factor to add a layer of fuzziness). Oh, and also some more context into your use case in general!

This way, I can try to replicate the same distributions on my end, then suggest any possible improvements, and if needed we can document the shortcomings you encountered in a new issue for the team!

[PaudGS]

Hi @srinify, Thanks for the quick response. About the use case, I am working with data of measurements (blood pressure, heart rate, respiratory rate...) on hospitalized patients. We are looking for ways to automatize the creation of synthetic patients and model their health evolution through hospital stay. For privacy concerns I have attached manually curated synthetic data of blood pressure measurements instead of the original data, but the results are similar with both datasets. metadata.json data.csv

Thanks in advance!

[srinify]

Thanks for sharing this @PaudGS I'll leave this thread open for now so we can collect more examples here from the community and hopefully invest in improving the experience over time!

[ardulat]

Hi, @srinify! I am experiencing the same issue for a numerical column in the time series sampled with PARSynthesizer. Here is the distribution plot:

I would really appreciate any help you can provide!

[srinify]

Hi @PaudGS thanks again for sharing some of your data and metadata! I was able to reproduce the issue on my end and can also confirm that changing the transformer didn't help. PAR seems to be creating a very narrow distribution around the mean value. This will be helpful for the team when we're able to work on improving the sequential synthesizer experience.