Beyond Cosine Similarity

Abstract: Cosine similarity, the standard metric for measuring semantic similarity in vector spaces, is mathematically grounded in the Cauchy-Schwarz inequality, which inherently limits it to capturing linear relationships--a constraint that fails to model the complex, nonlinear structures of real-world semantic spaces. We advance this theoretical underpinning by deriving a tighter upper bound for the dot product than the classical Cauchy-Schwarz bound. This new bound leads directly to recos, a similarity metric that normalizes the dot product by the sorted vector components. recos relaxes the condition for perfect similarity from strict linear dependence to ordinal concordance, thereby capturing a broader class of relationships. Extensive experiments across 11 embedding models--spanning static, contextualized, and universal types--demonstrate that recos consistently outperforms traditional cosine similarity, achieving higher correlation with human judgments on standard Semantic Textual Similarity (STS) benchmarks. Our work establishes recos as a mathematically principled and empirically superior alternative, offering enhanced accuracy for semantic analysis in complex embedding spaces.

• The paper introduces recos, a similarity metric derived from a tighter rearrangement inequality that captures order-concordant relationships beyond linear dependence.
• Empirical evaluations across 77 model-dataset settings show recos outperforms cosine similarity in 92.2% of cases with statistically significant improvements.
• The study validates recos for embedding-based retrieval and ranking, offering enhanced semantic analysis in complex, nonlinear vector spaces.

Theoretical and Empirical Advancements Beyond Cosine Similarity

Introduction

Cosine similarity, predicated on the Cauchy-Schwarz inequality, has long been the canonical metric for semantic similarity in high-dimensional vector spaces and is entrenched in applications ranging from document retrieval to embedding-based semantic analysis. Its broad acceptance arises from its computational efficiency, scale invariance, and its geometric interpretation as an angular measure. However, this linear-alignment bias is increasingly at odds with the observed structure of modern embedding spaces, particularly those induced by deep pre-trained models. The paper "Beyond Cosine Similarity" (2602.05266) interrogates the mathematical foundation of cosine similarity and introduces recos, a metric derived from a strictly tighter rearrangement-inequality-based upper bound on the dot product. This essay systematically presents the theoretical construction, empirical validation, and implications of recos as a principled and superior alternative for semantic similarity measurement.

Revisiting Similarity: Limitations of Cosine and the Rationale for recos

The cosine metric's reliance on the Cauchy-Schwarz bound ultimately constrains it to expressing linear dependencies—that is, maximal similarity is only attained for proportional vectors. This is a marked limitation when semantic similarity, particularly as judged by humans or used in modern LLM-driven systems, is better captured by more flexible structures, like monotonic or order-preserving relations. The paper formalizes similarity based not on strict metric alignment, but on the ordinal concordance of vector elements—a condition termed "similar vectors." This definition extends beyond metric proximity to capture vector pairs with consistent ordering, aligning more closely with practical notions of semantic relatedness.

Hierarchy of Inequalities and Novel Normalization Metrics

The key contribution is the derivation of a new chain of inequalities for the dot product, ordered in increasing strength:

• Rearrangement Inequality Bound: tightest possible, saturating for monotonic (order-concordant) vectors,
• Cauchy-Schwarz Bound: intermediate, saturating for linear dependency,
• Arithmetic-Mean/Quadratic-Mean Bound ("decos"): loosest, saturates only for exact identity or anti-identity.

Each upper bound yields a different normalization:

• $\mathrm{recos}$ uses the rearrangement-based denominator and captures ordinal alignment.
• Standard $\cos$ uses the Cauchy-Schwarz normalization and reflects angular relationships.
• $\mathrm{decos}$ is tied to near-identity and is closely related to the Tanimoto coefficient.

The differences in the strictness of saturation conditions directly correspond to their practical "capture range," i.e., the diversity of meaningful relationships each metric can recognize. This theoretical hierarchy is illustrated in the following schematic.

Figure 1: Relative "capture range" of similarity metrics arising from progressively tighter dot product bounds.

recos: Mathematical Properties and Interpretability

Formally, for vectors $\mathbf{u}, \mathbf{v} \in \mathbb{R}^d$,

$$\mathrm{recos}(\mathbf{u}, \mathbf{v}) = \frac{\mathbf{u} \cdot \mathbf{v}}{\left|\mathbf{u}^\uparrow \cdot \mathbf{v}^{\updownarrow}\right|}$$

where $\mathbf{u}^\uparrow$ is $\mathbf{u}$ sorted ascending, and $\mathbf{v}^{\updownarrow}$ matches the order with $\mathbf{u}$. Critically, $\mathrm{recos} = 1$ if and only if the vectors are order-concordant (not necessarily linearly dependent). Unlike cosine, this metric does not conflate order-preserving nonlinear effects with dissimilarity.

Through corollaries proved in the paper, it is established that for normalized vectors, decos and cosine are identical. However, recos always remains distinct as its denominator encodes the rank structure, not just the global norm.

Empirical Validation and Performance Analysis

A comprehensive empirical study evaluates $\mathrm{recos}$, cosine, and decos across 11 pre-trained embedding models—including static, contextualized, and universal text representations—on seven standard STS benchmarks. Performance is measured by Spearman's $\rho$ with human-annotated similarity judgments.

Key numerical findings include:

• Across 77 model-dataset settings, recos outperforms cosine in 71 cases (92.2% win rate; median gain ≈ 0.16 points), and improvement is statistically significant (Wilcoxon $V=2581$, $p<0.001$, effect size $r=0.835$).
• Gains are amplified for specialized or universal embeddings (e.g., CLIP-ViT, SPECTER, DPR), where recos provides absolute performance improvements up to +1.36 points on individual datasets.
• For unit-norm representations (BGE, E5), absolute gains are small, corroborating the theoretical equivalence between cosine and decos under normalization.

The consistent empirical lift, especially for models whose vector geometries deviate from strict linear semantics, validates the broader inductive bias captured by recos.

Theoretical and Practical Implications

By constructing a chain of similarity metrics grounded in successively more permissive dot-product bounds, the recos metric advances the toolkit for semantic analysis, particularly in complex or nonlinear vector spaces. Its main theoretical implication is a shift towards understanding similarity as order-concordance rather than metric alignment. This aligns with empirical evidence that deep distributed representations often encode semantic meaning nonlinearly.

Practically, recos is directly applicable to retrieval, ranking, and clustering tasks involving embeddings, especially where traditional angular similarity underestimates nuanced semantic matches. Its $O(d \log d)$ sorting complexity is a manageable trade-off versus $O(d)$ for cosine when improved alignment with human judgment is required, although large-scale deployment may motivate approximate or subsampled ranking schemes.

By demonstrating both statistical and effect-size significance, the paper opens the way for reconsideration of similarity metrics in contrastive learning, embedding evaluation, and zero-shot transfer setups.

Future Directions

Research directions include:

• Optimizing recos for large-scale applications (e.g., via efficient partial sorting or quantization),
• Integrating recos into contrastive or supervised training objectives to directly leverage ordinal concordance,
• Probing its performance for embeddings aligned to non-textual modalities (vision, cross-modal, multi-modal),
• Theorizing the conditions under which ordinal agreement catalyzes mutual information in semantic representation, versus purely angular approaches.

Conclusion

The recos metric, founded on the rearrangement inequality, represents a principled and empirically validated evolution beyond cosine similarity for semantic embedding comparison. It captures a spectrum of vector relationships aligned with practical notions of semantic similarity, especially for state-of-the-art embedding models. Its statistical superiority and theoretical generalization over legacy metrics underscore the importance of broadening the inductive bias of similarity computation in AI.