Conditions for reliable transfer in synthetic‑task in‑context learning

Determine the conditions on the distribution of synthetic tasks and training procedures under which in‑context learning models trained on synthetic tasks produce reliable and transferable prediction performance on real‑world datasets.

Background

In-context learning approaches train models to perform prediction by conditioning on entire training sets, often using large collections of synthetically generated tasks. This strategy aims to enable zero‑shot adaptation to new datasets without fine‑tuning by learning how to select appropriate prediction strategies from context.

The paper notes that success depends critically on the realism and diversity of the synthetic task distribution, and that formal guarantees are limited. Establishing when synthetic task generation yields reliable transfer to real problems is explicitly identified as an open problem.