Situated Brushing and Linking in Analytics

• Situated brushing and linking is a visual analytics paradigm that couples virtual data selection with the real-world highlighting of physical objects.
• The approach leverages formal models such as bidirectional slicing and Galois connections to ensure precise, invertible mappings between virtual views and physical referents.
• Empirical studies show that techniques like solid color fills, outlines, and 3D spatial links improve user performance in AR/VR setups while addressing challenges like visual clutter and scalability.

Situated brushing and linking is an emerging paradigm in visual analytics that extends traditional brushing and linking from purely virtual, 2D coordinated views to analytics tasks situated in physical or mixed-reality environments. In this paradigm, the act of selecting (brushing) data in a virtual visualization is tightly coupled to the highlighting (linking) of corresponding physical or embedded objects in the real world, presenting new perceptual and interactional challenges not encountered in conventional desktop systems (Doerr et al., 2024, Quijano-Chavez et al., 2 Feb 2026).

1. Foundations and Definitions

Brushing and linking refers to the coordinated selection of data across multiple visual representations. In classical desktop environments, brushing a subset of data in one view (e.g., points in a scatterplot) automatically links, or highlights, corresponding entities in another view (e.g., rows in a table, elements in a map) (Perera et al., 2021). Situated brushing and linking generalizes this paradigm to contexts where virtual data visualizations are “situated” within a spatial environment and linked to physical referents—objects, locations, or embedded displays. The core challenge becomes how to effectively guide user attention from virtual selections to real-world targets, often over non-trivial distances and within cluttered or occluded settings (Doerr et al., 2024, Quijano-Chavez et al., 2 Feb 2026).

2. Formal Models and Algorithms for Linking

The computational foundation for brushing and linking across multiple views is formalized via bidirectional program slicing and Galois connection theory (Perera et al., 2021). For two views $V_1$ and $V_2$ defined over shared data $D$, bidirectional slicing constructs monotone mappings between selection lattices $\text{Sel}(V_1)$, $\text{Sel}(V_2)$, and $\text{Sel}(D)$. A user selection $s_1$ in $V_1$ yields a backward slice $g_1(s_1) \in \text{Sel}(D)$, tracing which data records are implicated.

To propagate the brush to $V_2$, the De Morgan dual of the forward function $f_2^\bullet(d) \equiv \neg f_2(\neg d)$ computes the minimal set of elements in $V_2$ necessarily dependent on $d$. The composed map $g_2^\bullet \circ f_1$ thus realizes principled, invertible brushing and linking. This architecture supports robust view coordination, selection negation, and deterministic over-/under-approximation guarantees via Galois-connection round-tripping.

This model permits adjusting selection semantics (e.g., sparse vs. composite selection lattices), efficient incremental propagation, and is agnostic to whether the linked outputs are virtual or physically-embedded (Perera et al., 2021).

3. Highlighting Techniques in Situated Contexts

Bringing classic brushing and linking into situated analytics introduces the perceptual challenge of making linked physical referents salient within a mixed-reality environment. Several visual highlighting techniques have been empirically studied (Doerr et al., 2024, Quijano-Chavez et al., 2 Feb 2026):

• Solid color fill (“Color Cheating”): Brushed physical objects are rendered with a solid, high-contrast color (e.g., solid yellow via alpha blending $$C_{out} = \alpha C_{highlight} + (1-\alpha) C_{orig}$$, $\alpha=1$ for max saliency). This approach produces maximal visual pop-out but occludes native object details and labels.
• Outline and silhouette: A screen-space contour is rendered around each brushed object's silhouette (e.g., 2 px yellow outline, or “fat” green/orange animated edge via normal-based masking and mask dilation). This maintains legibility of interior object features while providing a less disruptive signal.
• 3D link or beam: An explicit spatial link (e.g., yellow Catmull–Rom spline or Bézier curve) connects each brushed mark on the virtual plot to its corresponding physical object, visually guiding attention across the environment. Thickness and curvature are tuned to enhance perceptibility.
• Animated arrows: 3D vectors “fly” from the viewpoint or tablet toward targets at fixed velocity, offering explicit directionality for off-screen or occluded referents via streaming motion cues.

Advanced techniques for AR/VR further include animated outlines (hue-shifting contours) to avoid packaging color conflicts, and combinations of outlines with spatial links for enhanced attention guidance (Quijano-Chavez et al., 2 Feb 2026).

4. Empirical Evaluation and Metrics

Situated brushing and linking has been systematically studied in both virtual (VR) and physical/augmented reality (AR) supermarket environments (Doerr et al., 2024, Quijano-Chavez et al., 2 Feb 2026). Typical experimental workflows involve a situated tablet (virtual or physical) displaying a scatterplot; brushing selects subsets of products, which are then linked to their physical locations.

Key metrics for assessment include:

• Completion time ($T_{total}$): Time from task onset to final selection.
• Linking time ($T_{link}$): Time required to map the virtual selection to physical referents, discounting plot brushing/fitering delay.
• Error rate ($E$): Number or proportion of incorrect object selections.
• Euclidean error distance ($D$): Mismatch distance between selected and true referents.
• Subjective workload (NASA-TLX), usability (UMUX-Lite/SUS), and presence (IPQ) scores.

A standardized methodology is the use of 95% bootstrap confidence intervals (CIs) for all pairwise differences (“new statistics” approach), with the absence of p-values or traditional ANOVA due to the exploratory nature of the studies (Doerr et al., 2024, Quijano-Chavez et al., 2 Feb 2026).

Empirical results indicate that solid color and explicit 3D link techniques consistently yield lowest linking times and errors, especially in VR. Animated arrow techniques introduce clutter and delays. Outlines are effective in static, safety-critical contexts where object identity and label legibility matter, but underperform in out-of-view or highly cluttered arrangements. In AR, real-world texture and lighting afford more efficient target localization with all tested overlays (Quijano-Chavez et al., 2 Feb 2026).

5. Comparative Insights: VR vs. AR, Task, and Layout

Cross-context findings reveal that AR consistently outperforms VR in both accuracy and linking time (e.g., AR: 8.73 s vs. VR: 10.89 s for linking, and AR: 10.42% vs. VR: 21.25% error rate), a result ascribed to stable, familiar environmental cues in the real world. In VR, animated links (L) substantially accelerate linking in out-of-view or multi-aisle layouts but at a cost of greater error rates when the field of view is saturated. Animated outlines (A) minimize error but increase completion time by approximately two seconds due to search slow-down induced by continuous color alternation.

Error analysis shows that color and outline errors stem from gross mislocalization (distances often >1 m), while link and arrow errors manifest as confusion among visually similar items within proximity (<0.5 m), indicating that different mechanisms underlie user failure depending on feedback modality (Doerr et al., 2024).

Under multi-selection and spatial judgment tasks, the technique-dependent differences attenuate, though link overlays remain advantageous for out-of-field guidance. Subjective feedback indicates high frustration and visual clutter with arrow cues and high appreciation for link overlays, particularly in AR contexts (Quijano-Chavez et al., 2 Feb 2026).

6. Design Recommendations for Situated Analytics

A set of high-level operational guidelines for system designers has been distilled from both VR and AR studies (Doerr et al., 2024, Quijano-Chavez et al., 2 Feb 2026):

1. High-contrast static color fills should be deployed when rapid, error-minimal selection is required and occlusion of original labels is acceptable.
2. Outline contours are suited for safety-critical applications, with increased width or dynamic hues to maximize perceptibility against diverse backgrounds.
3. 3D spatial links are critical for out-of-view or occluded targets, with thickness modulated as $t(d) \propto d^{0.5}$ to improve peripheral detectability.
4. Animated arrows are only appropriate for single-target scenarios, due to cumulative visual clutter; suppression of re-spawning can mitigate overload in dense selections.
5. Hybrid approaches—such as combining static outlines with on-demand links triggered by gaze or additional input—facilitate scalability in multi-target settings.
6. Technique adaptivity: Systems should dynamically adjust highlighting modes according to referent count, target distance, and user workload, e.g., switching from outline to color when the number of active highlights exceeds three.
7. Link recentering: Explicit mechanisms for reorienting links or arrows ensure that target guidance is not occluded by user body or interface elements.

Additional AR-specific guidance includes tuning hues for perceptual distinctiveness against environmental backgrounds, reducing field-of-view clutter, and leveraging real-world spatial cues for natural searching (Quijano-Chavez et al., 2 Feb 2026).

7. Limitations, Open Issues, and Theoretical Connections

Current implementations are limited by hardware rendering constraints in mixed-reality contexts (e.g., synthetic lighting in VR degrades overlay pop-out, while AR benefits from familiar textures). The formal Galois-slicing model supports only first-order data and does not capture dynamic branching or mutable state, restricting its explanatory power for complex view interdependencies (Perera et al., 2021).

Challenges persist in scaling to high-density referent fields, minimizing attentional tunneling, and retaining original object affordances under strong visual highlighting. Theoretical advances are also needed for seamless extension of formal view-coordination models to situations involving physical referents, mutable environments, or real-time user feedback.

The situated brushing and linking paradigm catalyzes a rethinking of how data selectors are coupled to spatial attention in analytic workflows, motivating further research into hybrid AR/VR-physical analytics systems, adaptive overlay strategies, and principled metrics for perceptual effectiveness (Doerr et al., 2024, Perera et al., 2021, Quijano-Chavez et al., 2 Feb 2026).