TLDR: We show that a diffusion transformer can do in-context Thompson sampling at inference time after pre-training, as in prior-fitted networks. We built a set of non-Gaussian priors for general black-box optimization, and our results show that pre-trained prior-fitted diffusion transformers with non-Gaussian priors can outperform traditional Gaussian-process pipelines.
Background
We are interested in a global optimization problem where we want to find the global optimum of an unknown function in a sample-efficient setting. We assume every evaluation is expensive, so we must balance exploring uncertain regions with exploiting promising observations – the exploration-exploitation tradeoff. Bayesian optimization with Gaussian processes (GPs) is a classic approach to this type of problem, but the surrogate is restricted to Gaussian priors 1 . More recently, prior-fitted networks (PFNs) have shown that transformers can approximate the posterior predictive distribution , suggesting a meta-learning scheme for optimization 2 3 4. PFNs support flexible priors while shifting the training burden to the pre-training stage, but need output discretization to represent predictive distributions and still require a decision rule after quantifying uncertainty. Meanwhile, with the huge success of diffusion models in computer vision, there has been more work using expressive generative models for black-box optimization. However, much of this work comes with bespoke training loss functions or regularizers to handle the exploration-exploitation tradeoff 5 6 7.
Prior-fitted Diffusion Transformers
In our work, we suggest a framework doing in-context Thompson sampling using pre-trained diffusion transformers 8 while keeping the pipeline simple and rooted in decision-theoretic principles.
In this framework, we sample objective functions from our prior and build datasets of input-output pairs to feed into transformers. Through cross-attention, this information is fused with a sampled diffusion time step and intermediate state . Unlike the original PFN, our target distribution is the posterior distribution over the maximizer, and we adopt diffusion models with a flow-matching loss 9 to learn this distribution.
Priors
We propose a non-Gaussian prior family for general optimization problem, called the envelope-plus-wiggles prior, where we have a smooth envelope shape around the maximizer and wiggles as you move away from the maximizer. Concretely, we propose the family
where is sampled uniformly over the domain, and are the envelope and wiggles. We propose unimodal priors for the envelope and two wiggle priors in the paper.
Experiments
We tested our method on both synthetic and empirical benchmark against diffusion-based and GP-based baselines, as shown in the below.
Conclusion
In this work, we suggest a framework that performs Thompson sampling in-context with diffusion transformers. Our results show that our method performs strong on various benchmarks when trained on the carefully designed priors. This work is particularly interesting because it suggests a new way to build and pre-train foundation models for optimization problems.
Citation
@inproceedings{park2026thompson,
title={Thompson Sampling using Prior-fitted Diffusion Transformers},
author={Park, Sihwa and Zhu, Jingsen and Jain, Vinamr and Chou, Sheng-Yen and Terenin, Alexander},
booktitle={Advances in Neural Information Processing Systems},
year={2026}
}References
R. Garnett. Bayesian Optimization. Cambridge University Press, 2023. ↩
S. Müller, N. Hollmann, S. Pineda Arango, J. Grabocka, and F. Hutter. Transformers Can Do Bayesian Inference. International Conference on Learning Representations, 2022 ↩
S. Müller, M. Feurer, N. Hollmann, and F. Hutter. PFNs4BO: In-Context Learning for Bayesian Optimization. International Conference on Machine Learning, 2023 ↩
N. Hollmann, S. Müller, K. Eggensperger, and F. Hutter. TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second. International Conference on Learning Representations, 2023 ↩
S. Krishnamoorthy, S. M. Mashkaria, and A. Grover. Diffusion Models for Black-Box Optimization. International Conference on Machine Learning, 2023. ↩
T. Yun, K. Om, J. Lee, S. Yun, and J. Park. Posterior Inference with Diffusion Models for High-Dimensional Black-Box Optimization. International Conference on Machine Learning, 2025 ↩
D. Wu, N. L. Kuang, R. Niu, Y.-A. Wang, and R. Yu. Diff-BBO: Diffusion-Based Inverse Modeling for Online Black-Box Optimization ↩
W. Peebles and S. Xie. Scalable Diffusion Models with Transformers. IEEE/CVF International Conference on Computer Vision, 2023 ↩
Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le. Flow Matching for Generative Modeling. International Conference on Learning Representations, 2023 ↩