Stack-sorting with Stacks Avoiding Vincular Patterns

Abstract: We introduce the stack-sorting map $\text{SC}\sigma$ that sorts, in a right-greedy manner, an input permutation through a stack that avoids some vincular pattern $\sigma$. The stack-sorting maps of Cerbai et al. in which the stack avoids a pattern classically and Defant and Zheng in which the stack avoids a pattern consecutively follow as special cases. We first characterize and enumerate the sorting class $\text{Sort}(\text{SC}\sigma)$, the set of permutations sorted by $s\circ\text{SC}\sigma$, for seven length $3$ patterns $\sigma$. We also decide when $\text{Sort}(\text{SC}\sigma)$ is a permutation class. Next, we compute $\max_{\pi\in \mathfrak S_n}|\text{SC}\sigma^{-1}(\pi)|$ and characterize the periodic points of $\text{SC}\sigma$ for several length $3$ patterns $\sigma$. We end with several conjectures and open problems.