Benefit of LimiX-style random feature identifiers in TabICLv2

Determine whether incorporating random feature identifiers, as used in LimiX, into the row-wise attention stage of TabICLv2 yields measurable performance benefits on tabular prediction tasks, and quantify any gains across dataset regimes.

Background

The paper introduces TabICLv2, a tabular foundation model that performs in-context learning with architectural changes over prior models. In the row-wise interaction stage, the authors considered design variants inspired by other TFMs.

They briefly experimented with using random feature identifiers in the manner of LimiX but did not reach a definitive conclusion on their utility, explicitly noting the uncertainty. Establishing whether this mechanism is beneficial would clarify design choices for future iterations and cross-model architectural transfer.