Constructed tableaux and a new shifted Littlewood-Richardson rule

Abstract: We give a new Littlewood-Richardson rule for the Schubert structure coefficients of isotropic Grassmannians, equivalently for the multiplication of $P$-Schur functions. Serrano (2010) previously gave a formula in terms of classes in his shifted plactic monoid. However, this formula is challenging to use because of the difficulty of characterizing shifted plactic classes. We give the first algebraic proof of this formula. We then use it to obtain a new rule that is easy to implement. Our rule is based on identifying a subtle analogue of Yamanouchi tableaux, which we characterize. We show that for some families of structure coefficients, our rule leads to an algorithm with exponentially better time complexity than the original rule of Stembridge (1989).