Samplable PAC Learning & Evasive Sets

• Samplable PAC Learning is a variant of the PAC framework that restricts attention to efficiently samplable data distributions, reducing theoretical sample complexity.
• The approach uses explicit evasive sets to achieve exponential separations from standard PAC learning even when the VC-dimension is high.
• Integrating cryptographic assumptions and bounded adversaries, the framework enhances both computational and online learning efficiency.

Samplable PAC learning is a refinement of the classical Probably Approximately Correct (PAC) learning framework, in which the requirement to learn under all possible data distributions is relaxed to learning under distributions that are efficiently samplable. This modification, first explicitly formalized by Blum, Furst, Kearns, and Lipton (1993), has significant implications for both the statistical and computational complexity of learning, leading to new separations and open questions regarding the true nature of learnability when data-generation processes are algorithmically constrained (Blanc et al., 1 Dec 2025).

1. Formal Definitions and Framework

In the standard PAC (Valiant, 1984), a concept class $C \subseteq \{c: X \rightarrow \{0,1\}\}$ is PAC-learned if a learner %%%%1%%%%, given $m$ labeled examples drawn i.i.d. from any distribution $D$ over the instance space $X$ and any target $c \in C$, outputs a hypothesis $h$ such that

$$\Pr_{S \sim (D, c)^m} [\Pr_{x \sim D}[h(x) \neq c(x)] \leq \epsilon] \geq 1 - \delta.$$

for all $c \in C$ and all $D$.

The samplable PAC model retains this structure but only requires $A$ to succeed when $D$ is efficiently samplable: that is, there exists a Boolean circuit $G$ of size at most $s$ such that $G(\text{Uniform on } \{0,1\}^\ell)$ induces $D$ for some $\ell$. The learner must succeed for all samplable $D$ of size at most $\text{poly}(n)$. The formal error metric is $err_D(h) = \Pr_{x \sim D}[h(x) \neq c(x)]$ (Blanc et al., 1 Dec 2025).

2. Statistical Separations from Standard PAC Learning

A principal finding is the existence of concept classes where samplable PAC learning is exponentially more powerful than standard PAC. Specifically, there exist classes $C$ over $\{0,1\}^n$ such that:

• $\text{VCdim}(C) = 2^{\Omega(n)}$ implies that standard PAC sample complexity is $2^{\Omega(n)}$.
• The same class is learnable in samplable PAC with polynomial sample complexity.

This is realized through the construction of an $(\epsilon, k)$-evasive set $H \subset \{0,1\}^n$, defined so every size-$s$ samplable distribution $D$ “misses” $H$ outside of its $k=O((s \log s)/\epsilon)$ heaviest points, meaning $D(H \setminus H^*) < \epsilon$ for some $H^* \subset H$, $|H^*| = k$. The corresponding concept class $C_H$ is defined by $$C_H = \{f_H: f: \{0,1\}^n \to \{0,1\}, f_H(x) = f(x)$$ if $x \in H$, $0$ otherwise$\}$. While its VC-dimension remains large, the evasiveness property ensures that, against samplable $D$, memorization-based learners cover nearly all of $D$ using a small sample $S$ (Blanc et al., 1 Dec 2025).

3. Explicit Evasive Sets and Efficient Learnability

Explicit evasive sets are sets $H_n \subset \{0,1\}^n$ characterized by:

• Membership in $H_n$ is decidable by a circuit of size $\text{poly}(n)$.
• $|H_n|$ is super-polynomial in $n$.
• For all $s = \text{poly}(n)$, every size-$s$ sampler $D$ $(1/s, s^c)$-misses $H_n$ for some constant $c$.

Such sets, if constructed, provide both hardness and efficient recognizability, crucial for sharp separations between standard and samplable PAC learnability. Their existence is connected to core complexity-theoretic hypotheses: if $P = NP$, existential sampling makes $H$ non-evasive. In the random oracle model, explicit $H^O$ can be produced that are evasive for all polynomial-size oracle samplers, with $|H^O| = 2^{\Omega(n)}$ (Blanc et al., 1 Dec 2025).

4. Computational Separations and Reductions

The study extends beyond sample complexity to computational complexity:

• Let $F$ be a pseudorandom function family secure against $\text{poly}(n)$ circuits; define $C = \{f_H: f \in F\}$, where $f_H(x) = f(x)$ if $x \in H$, $0$ otherwise.
• Learning $C$ with respect to the uniform distribution on $H$ requires breaking the PRF, which is infeasible in $\text{poly}(n)$ time (under standard cryptographic assumptions).
• However, by the evasiveness of $H$, $C$ becomes learnable in $\text{poly}(n)$ time for all size-$s$ samplable distributions, as almost all of the probability mass falls on a small, easily memorizable subset.

This separation persists relative to a random oracle, demonstrating unconditional distinctions between standard and samplable PAC in oracle models (Blanc et al., 1 Dec 2025).

5. Online Learning and Adversarial Efficiency

The samplable PAC principle translates to online learning models. In the classic online framework (Littlestone 1988), an adversary provides instances; with no computational restriction, the adversary can force exponentially many mistakes (corresponding to the Littlestone dimension $2^{\Omega(n)}$ for some $C$). If the adversary is limited to producing instances by polynomial-size circuits (efficient adversary), for the same concept classes the best online learner's number of mistakes drops to $O(s \log s)$. A “default-zero” memorization learner achieves low mistake bounds because an efficient adversary can only produce a bounded number of distinct “hard” points in an evasive set $H$ (Blanc et al., 1 Dec 2025).

Computationally, the same construction methods (PRFs + explicit $H$) yield classes where online learning is hard against unbounded adversaries, yet efficient against bounded adversaries.

6. Connections to Classical PAC Learning Bounds

Classical results in PAC learning show that for a concept class $C$ of finite VC-dimension $d$, the sample complexity in the realizable PAC setting is $\Theta((1/\epsilon)(d + \log(1/\delta)))$ (Hanneke, 2015). These optimal bounds hold for learning under arbitrary distributions. The samplable PAC findings indicate that, if distributions are restricted to samplable ones, these bounds may not fully capture the true sample or computational complexity: samplable PAC learning can circumvent high sample complexity by exploiting evasiveness in the domain, even when the VC-dimension is exponential. This underscores a fundamental distinction between distributional assumptions in learning theory (Blanc et al., 1 Dec 2025).

7. Open Problems and Future Directions

A central open problem is to characterize samplable PAC sample complexity in terms of both the complexities of the concept class and the samplability of the distribution, paralleling the role of VC-dimension in standard PAC settings. Current characterizations do not yield a combinatorial or analytical measure analogous to VC for samplable PAC. Further lines of investigation include developing measures of distributional complexity beyond samplability and constructing explicit evasive sets unconditionally (that is, outside random oracle or cryptographic assumptions), with the aim of better understanding and harnessing the expanded power of efficient learning under realistic data-generation constraints (Blanc et al., 1 Dec 2025).