Balanced Position Calibration (BPC) Overview

• BPC in BPM calibration systematically averages voltage readings across a scan grid to fit per-electrode gains and coupling errors, restoring beam position accuracy.
• In LLM evaluation, BPC employs dual ordering and Monte Carlo sampling to neutralize positional bias, achieving outcomes closely aligned with human judgments.
• The method leverages symmetry and least-squares fitting in BPMs alongside mean aggregation in LLMs to achieve calibration accuracies with only a few-percent error.

Balanced Position Calibration (BPC) refers to two independent, domain-specific calibration methodologies developed for distinct technical challenges: compensating position-dependent gain errors in beam position monitors (BPMs) with orthogonal stripline electrodes (Zou et al., 2013), and mitigating positional bias in LLM evaluation protocols for pairwise response comparisons (Wang et al., 2023). Both instances share a unifying strategy: systematic averaging or fitting across all relevant candidate positions to nullify systematic positional dependencies or biases.

1. BPC in Beam Position Monitor Calibration

The Balanced Position Calibration technique for BPMs with orthogonally symmetric electrodes was developed to address electronic gain variation and machining tolerances that introduce crosstalk and scaling errors, thereby corrupting extracted beam positions (Zou et al., 2013). In typical four-electrode BPMs (electrodes at 0°, 90°, 180°, 270°), even small differences in electronic gain or mechanical alignment couple the nominally independent transverse (horizontal and vertical) position signals, leading to systematic errors in position and scale.

The BPC protocol involves bench-top scanning of a reference source (hot tungsten filament) across a grid in the BPM aperture. At each grid point, the voltages from all four electrodes $(V_R, V_T, V_L, V_B)$ are recorded. Those signals are linearly combined to define normalized coordinate observables: $$\Sigma_m \equiv \frac{V_R - V_L}{V_R + V_L},\; \Sigma_n \equiv \frac{V_T - V_B}{V_T + V_B},\; \Sigma_{mn} \equiv \frac{(V_R + V_L) - (V_T + V_B)}{V_R + V_L + V_T + V_B}$$ A beam-charge-independent second-order “mn-relation” is established: $$\Sigma_{mn} = k_{mn} \cdot \Sigma_m \Sigma_n,\quad k_{mn} = \frac{1}{4\tan(\phi/2)}$$ with $\phi = 45^\circ$ for the HLS II BPM design.

Incorporation of electrode coupling (cross-talk, with coefficients $K_1,K_2$) and unknown relative per-electrode gains $(g_R, g_T, g_L, g_B)$ is achieved by reformulating all observables in terms of the physically measured and gain-scaled voltages. The practical BPC algorithm fits these gain factors via least-squares minimization, enforcing the mn-relation over the full scan data set. This corrects for gain asymmetry and recovers physical position scale and offsets to within the few-percent calibration accuracy dictated by electronic front-ends. Across the 19 injector BPMs, fitted gain factors cluster within $[0.9,1.1]$ (standard deviation $\simeq5\%$), and geometric coefficients after BPC converge to theoretical design values.

2. BPC for Evaluation Bias in LLMs

Balanced Position Calibration in LLM evaluation addresses the systematic order bias observed when LLMs are tasked with scoring or ranking candidate responses based on a prompt-presented ordering (Wang et al., 2023). Off-the-shelf LLMs such as GPT-4 and ChatGPT exhibit strong slot biases—preferring whichever response is presented in a particular position. For example, GPT-4 prefers slot 1, while ChatGPT prefers slot 2, with positional conflict rates reaching up to 46% (GPT-4) or 82% (ChatGPT) on close-quality instance pairs.

BPC, as introduced in "LLMs are not Fair Evaluators," averages scores for each candidate response across all possible positions. For a question $q$ and candidate responses $r_1$, $r_2$:

• Both orderings $(r_1, r_2)$ and $(r_2, r_1)$ are evaluated.
• For each ordering, $k$ Monte-Carlo samples (with $T>0$) are drawn, recording score pairs $(S_{r_1}^i, S_{r_2}^{\prime i})$ and $(S_{r_2}^i, S_{r_1}^{\prime i})$.
• For each response, a calibrated score is computed: $$CS_{r_1} = \frac{1}{2k} \left(\sum_{i=1}^k S_{r_1}^i + \sum_{i=1}^k S_{r_1}^{\prime i}\right),\qquad CS_{r_2} = \frac{1}{2k} \left(\sum_{i=1}^k S_{r_2}^i + \sum_{i=1}^k S_{r_2}^{\prime i}\right)$$ The process ensures that both $r_1$ and $r_2$ appear equally in both response slots, and the final outcome depends on the mean calibrated score.

BPC can be combined with Multiple Evidence Calibration (MEC), where each ordering is sampled $k$ times to reduce stochasticity. Experiments using Vicuna and ChatGPT show that BPC in tandem with MEC (with $k=3$) increases agreement with human judgments by 3.8% (GPT-4) and 5.5% (ChatGPT), and reduces conflict rates (judgment reversals under slot swap) to 0% (by construction), compared to 82.5% for uncalibrated evaluations (Wang et al., 2023).

3. Mathematical and Algorithmic Foundations

The core feature of both BPC frameworks is systematic exploitation of symmetry by sampling all possible slot configurations and estimating position-invariant aggregate values:

• In BPMs, normalized voltage observables and their theoretical mn-relation are enforced across the scan, and unknown gain and coupling parameters are fit to minimize the sum of squared deviations.
• In LLM evaluation, for each candidate, aggregate scoring is performed over both relative positions and multiple evidence samples, yielding unbiased point estimates.

A generic pseudocode for BPC in LLM evaluation is:

Input: query q, candidate responses r1, r2, sample count k Initialize lists L1 ← [], L2 ← [] for i in 1…k do # original order (score1, score2) ← LLM_chain_of_thought(T_EC(q,r1,r2)) L1.append(score1) L2.append(score2) # swapped order (sw_score1, sw_score2) ← LLM_chain_of_thought(T_EC(q,r2,r1)) L1.append(sw_score2) # S_r1'^i L2.append(sw_score1) # S_r2^i end for CS_r1 ← mean(L1) CS_r2 ← mean(L2) if CS_r1 > CS_r2: outcome = "Assistant1 wins" elif CS_r2 > CS_r1: outcome = "Assistant2 wins" else: outcome = "tie" return CS_r1, CS_r2, outcome

This aggregation is essential to neutralize slot bias in algorithmic scoring.

4. Practical Implementation and Performance

Implementation details for the respective domains are as follows:

Beam Position Monitors (Zou et al., 2013):

• Filament scanned over ±2.5 mm × ±2.5 mm grid, typical step size 0.5 mm.
• Front-end signals acquired using commercial BPM electronics.
• BPC fitting uses least-squares minimization for per-electrode gains and coupling-adjusted $k_{mn}$.
• After calibration, BPMs exhibited reduced zero-point offsets and geometric coefficients converging on theoretical design ($b \cdot \tan(\phi/2) = 7.55$ mm), with fitted gains exhibiting 5% standard deviation.

LLM Evaluation (Wang et al., 2023):

• For each question, both candidate orderings evaluated $k$ times (empirically $k=3$ is a tradeoff for cost and variability).
• LLM invoked via a chain-of-thought “evidence-first” prompt.
• Final verdicts computed using mean scores across all slots and samples.
• In experiments, BPC combined with MEC delivered $\sim62.5\%$ human-aligned accuracy for GPT-4 (Cohen's $\kappa=0.37$) versus $52.7\%$ for vanilla GPT-4.

Domain	Source/Instrument	BPC Mechanism	Calibration Targets
Accelerator BPM	4 stripline electrodes	Scan grid, fit gains/coupling, mn-relation	Per-electrode gain, coupling
LLM Response Evaluation	LLM (GPT-4, ChatGPT, etc.)	Dual ordering, sample averaging	Mutual slot positional bias

5. Limitations and Contextual Notes

Intrinsic limitations and boundary conditions for BPC are documented in both application domains:

• Beam Position Monitors: Validity of the quadratic expansion (second order) is limited to beam displacements $\lesssim 5$ mm; third-order terms may be necessary for larger displacements. Coupling coefficients require re-estimation if physical cabling or geometry is changed. Although the bench-top scan uses a filament, the calibration is fully beam-charge independent, and may be easily adapted to on-beam operation. Gains can drift, but re-calibration is automatable.
• LLM Evaluation: The number of LLM calls is doubled ($2k$ per comparison), and combined with $k$ for MEC, scales to six calls per instance when $k=3$. Definition is strictly pairwise; extension to $n$-way comparisons would require $n!$ permutations or other designs. BPC specifically addresses positional bias—it does not directly account for other biases such as prompt format, verbosity, or lexical overlap. LLM judgments might be sensitive to different aggregation strategies; only mean is reported. Adaptive sampling is proposed but not implemented.

6. Impact and Future Directions

Balanced Position Calibration has demonstrably enhanced the reliability of both hardware instrumentation and LLM-based evaluation methodologies.

In BPM calibration, the method yields robust, beam-charge-independent gain normalization, directly reducing systematic error in position readouts, and ensuring hardware-limited sensitivity. For LLM evaluation, BPC achieves near-complete elimination of positional order bias, yielding adjudications that are statistically aligned with randomized or human-labeled ground truth, as shown by increased agreement rates and reduced conflict rates (Wang et al., 2023).

Open directions include adaptive or robust aggregation measures (median or trimmed mean), extensions to non-pairwise (multi-candidate) setups, and dynamic sampling strategies to minimize resource consumption. In both settings, BPC can be further integrated with complementary calibration and bias-mitigation frameworks as new use-cases and requirements emerge.

7. Summary Table: BPC Applications

Application	Calibration Objective	Key Methodological Elements
BPMs (Zou et al., 2013)	Remove gain/coupling errors in beam position	Grid scan, mn-relation least-squares fit, per-electrode gain normalization
LLM Evaulation (Wang et al., 2023)	Eliminate slot/position bias in scoring	Dual-order prompt sampling, mean aggregation across all slots and samples

Both methodologies leverage systematic permutation of candidate positions and statistical aggregation to enforce invariance against positional bias or gain asymmetry, achieving high-precision calibration in their respective problem domains.