LaySPA: RL for Spatial Layout Design

• LaySPA is a reinforcement learning-based framework that augments LLMs with explicit spatial reasoning for designing coherent, content-aware graphic layouts.
• It models layout design as an episodic Markov decision process, integrating geometric validity, structural fidelity, and visual quality through a hybrid reward function.
• The framework employs iterative self-exploration and interpretable '<think>' reasoning traces to optimize layout structures and ensure precise alignment and spacing.

LaySPA is a reinforcement learning-based framework designed to augment LLM agents with explicit spatial reasoning capabilities for layout design. It addresses the spatial cognition deficiency observed in standard LLMs when tasked with structuring content-aware graphic layouts, where precise placement, alignment, and organization of multiple elements are crucial within constrained visual spaces (Li, 21 Sep 2025).

1. Foundations of Spatial Reasoning in Layout Design

Content-aware graphic layout design demands modeling multi-object relationships—such as alignment, non-overlap, and hierarchical structure—while respecting canvas constraints (boundaries, saliency regions) and established design principles (spacing rhythm, visual balance). Standard LLMs, though effective at textual reasoning and instruction-following, lack native geometric understanding. Attempts to use text-based LLMs for spatial design frequently result in structurally invalid layouts with misaligned or overlapping elements and violations of critical boundaries. This deficit precludes the reliable production of visually coherent layouts by LLMs unaided by explicit spatial reasoning mechanisms.

2. Mathematical Formulation and Reward Structure

LaySPA reframes layout generation as an episodic Markov decision process, solved using policy-gradient reinforcement learning (RL). The state at each time step is a JSON-style encoding of the canvas, comprising saliency boxes for key regions and element descriptors with masked position and size. The action space consists of outputting bounding box coordinates for an element and providing a structured alignment/placement rationale ("> " trace).

The transition dynamics are deterministic: once placed, each element’s box remains fixed. LaySPA employs a hybrid reward function, aggregating geometric validity, structural fidelity, and visual quality into a scalar objective:

$$R(L) = \lambda_1 R_\text{geo}(L) + \lambda_2 R_\text{struct}(L) + \lambda_3 R_\text{vis}(L)$$

Reward terms include:

• Format correctness ($R_\text{format}$): Validity of JSON and rationale schema.
• Layout quality ($R_\text{quality}$): Geometric and visual organization.
• IoU matching ($R_\text{IoU}$): Overlap fidelity to human reference layouts.

Explicit metrics include inverse collision rate ($R_\text{icr}$), alignment score ($R_\text{al}$), distribution score ($R_\text{dis}$), spacing consistency ($R_\text{sp}$), underlay-text pairing ($R_\text{ut}$), and IoU matching. Experimental λ-weights are $$\lambda_\text{format}=0.1, \lambda_\text{quality}=0.8, \lambda_\text{IoU}=0.1$$.

Policy optimization uses Group Relative Policy Optimization (GRPO) with a KL-divergence penalty to a reference policy $\pi_\text{ref}$, maximizing expected hybrid rewards subject to constrained policy divergence.

3. Training Paradigm and Interpretability

LaySPA is initialized with a pretrained, instruction-tuned LLM (e.g., Qwen-2.5). Training is executed via iterative self-exploration, generating candidate layouts sampled from the current policy, and evaluating each with the hybrid reward. The group advantage for each rollout is computed as $A_i = r_i - \text{mean}_j r_j$, guiding the subsequent GRPO gradient steps under LoRA-based adapters.

A distinguishing feature of LaySPA is its interpretability: each agent rollout generates a "<think>" chain-of-thought trace that details the rationale behind spatial decisions (e.g., alignment, avoidance of saliency regions), supporting practitioner audit and debugging of the reasoning process.

4. System Architecture and Workflow

The LaySPA workflow comprises:

1. Preprocessing: Saliency boxes are detected on the background, and the canvas plus elements are encoded in JSON with masked coordinates.
2. Rollout Loop: For $G$ candidate layouts per input, the LLM agent emits a "<think>" trace and JSON with predicted boxes; each layout is scored by the hybrid reward model.
3. Policy Update: GRPO steps adjust model parameters to improve group-relative rewards, constrained by KL regularization.
4. Inference: At test time, several rollouts are performed; the highest-scoring layout and its reasoning trace are selected for output.

5. Quantitative Evaluation: Datasets, Baselines, and Metrics

LaySPA has been empirically validated on two datasets:

• CGL: 48.4k train, 6.06k test samples; includes text, logo, underlay, and embellishment elements.
• PKU: 7.97k train, 0.997k test samples; includes text, logo, underlay.

Baselines for comparison include DS-GAN (specialized generative model), PosterLlama (LLM-to-HTML), Qwen-3B/7B (zero-shot), and GPT-4o (zero-shot).

Metrics span structural/aesthetic scores (format correctness, collision rate, alignment, spacing consistency, distribution) and graphic/content scores (overlay, underlay effectiveness, occlusion).

Key results: On CGL, LaySPA-tuned Qwen-7B models achieve significant improvements over the base model:

Metric	Base Qwen-7B	Qwen-7B + LaySPA
Format	0.873	0.998
Collision rate	0.692	0.431
Alignment	0.319	0.597
Spacing consistency	0.326	0.569
Distribution	0.253	0.317

Compared to SOTA models: PosterLlama achieves the best overlap and underlay scores, while Qwen-7B+LaySPA demonstrates the lowest occlusion among LLMs. On PKU, LaySPA-tuned models outperform base LLMs and are competitive with specialized generators.

6. Qualitative Insights and Limitations

DS-GAN produces evenly packed layouts but is prone to misalignment and saliency overlap errors. PosterLlama demonstrates visually balanced layouts by leveraging visual feature encoders and specialized priors. GPT-4o, despite its scale, yields inconsistent spatial organization, misalignments, and overcrowding.

LaySPA-tuned Qwen models consistently generate interpretable reasoning traces and layouts that respect saliency, reduce collisions, enforce rhythmic spacing, and maintain grid coverage. Notable limitations for LaySPA include dependence on pre-computed saliency maps (no end-to-end visual processing) and single-turn layout generation. Future directions may encompass integrated vision semantics and multi-turn RL refinements.

7. Summary and Significance

LaySPA establishes that reframing layout design as a hybrid reward-driven RL problem, directly modeling geometric, structural, and aesthetic criteria, enables LLM agents to acquire authentic 2D spatial reasoning capabilities. The framework delivers structured, visually appealing layouts that rival both base and specialized models, thus substantially extending the domain of high-fidelity, interpretable spatial decision-making in LLMs (Li, 21 Sep 2025).