Generative AI for synthetic data and method evaluation
Every claim that a statistical method works rests on a simulation study, and every simulation study rests on a data-generating process that somebody wrote down. Those processes tend to be tidy in ways real data never is: normal errors, clean nesting, no missingness. A method that performs well there can disappoint on the data it was built for.
The alternative is to learn the data-generating process instead of assuming it. A generative model fitted to one real sample can produce as many synthetic replicates as a Monte Carlo study needs, keeping the awkward structure of the original: skewed variables, categorical mixtures, and clustering of students within schools.
That makes a different kind of evaluation possible. Across replicates, an estimator can be scored not only on bias and variance but on fairness across groups and on the utility of the policy it implies, which are the quantities that matter when the estimate will be used to make a decision.
Methods
-
tabularCTGAN and diffusion models
Conditional generative models for tabular data, and ClavaDDPM for multi-table and multilevel structure.
-
sequentialSequence generation
Generating ordered records, such as course-taking histories, where each step depends on the ones before it.
-
scaleHigh-throughput computing
Simulation campaigns run on the OSG Open Science Pool at roughly a thousand concurrent jobs, which is what makes large simulation campaigns affordable.
Papers and talks
-
Using Generative AI for Sequential Data Generation in Monte Carlo Simulation Studies
Journal of Educational and Behavioral Statistics
2025Published -
Generative AI-Based Monte Carlo Simulation for Method Evaluation Using Synthetic Multilevel Data
Under review -
AI-Based Monte Carlo Simulation for Method Evaluation Using Synthetic Multilevel Data
SREE 2026, oral presentation
2026