PaperJury: Due-Process Review for Bounded LaTeX Revision
Abstract: Pre-submission hardening of human-authored LaTeX computer science papers differs from drafting assistance because it requires adversarial whole-paper review, explicit no-fix outcomes, and bounded artifact-safe revision. Existing writing assistants, critique generators, and judge-centered loops lack durable issue identity across rounds, deterministic routing from critique to adjudication, and manuscript control that can reject invalid concerns or defer author-dependent ones. We present PaperJury, a closed-loop review-verdict-revise-verify system built on a deterministic-versus-semantic split: deterministic orchestration manages decomposition, a frozen claim spine, a durable ledger, routing, stopping, and exact-once patch application, while semantic agents are limited to bounded review, judgment, and repair. PaperJury combines bounded holistic review, contestability-based routing, a due-process trial, and risk-proportional guard chains for anchor-bounded edits, yielding terminal outcomes of invalid-drop, valid-fixable, and author-required. In a two-arm expert-review evaluation on held-out Vision, natural language processing, and machine learning papers against four baselines, we assess issue quality, verdict and routing quality, edit safety, convergence behavior, and cost, supporting the thesis that load-bearing safety and completion logic should reside in deterministic orchestration rather than model discretion. PaperJury is available at https://github.com/u7079256/paperjury.
Summary
- The paper presents PaperJury, a deterministic system that replaces open-ended critiques with a due-process review for safe, bounded LaTeX revisions.
- It employs a deterministic-versus-semantic split, confining LLMs to bounded tasks while ensuring edit safety via immutable claim spines and guard chains.
- Experimental results show improved F1, verdict precision, and safety metrics, demonstrating its robust performance for pre-submission manuscript hardening.
PaperJury: Deterministic Orchestration for Reliable Bounded LaTeX Revision
Overview and Motivation
PaperJury addresses the pre-submission hardening of LaTeX-based computer science manuscriptsโa task distinct from conventional drafting assistance or local style critique. The core challenge is not simply generating critiques or refining prose but orchestrating a closed adversarial loop: identifying substantive manuscript flaws, governing which critiques lead to action, and safely applying bounded revisions, all under constraints of precision, recall, and computational cost. Existing automated writing assistants or critique generators lack several essential properties for this high-stakes context: persistent issue identity across review rounds, deterministic routing from critique to verdict, and artifact-safe, bounded revision primitives.
System Architecture: Deterministic vs. Semantic Split
PaperJuryโs design is based on a deterministic-versus-semantic pipeline separation. Deterministic orchestration manages all critical state transitions, decomposition, routing, and application/reversion of edits, ensuring reproducibility and consistency. Semantic agents (e.g., LLMs) are confined to well-bounded subtasks such as issue generation, judgment, and drafting revisions. This split ensures that safety, completion, and ledger state are governed by code rather than subject to LLM variability.
Figure 1: Overview of the PaperJury review-verdict-revise-verify pipeline.
Figure 2: Deterministic-versus-semantic split with ledger-backed control flow.
Manuscript decomposition () produces sections, passage anchors, and cross-reference targets, while a frozen claim spine () records central assertions. This claim spine, established before review and revision, is immutable under automated edits, mitigating risks of claim drift or silent semantic mutation.
Issues are generated by a small set of bounded holistic reviewers, each providing evidence-grounded weaknesses. These issues are assembled into a durable ledger, retaining provenance and enabling exact-once tracking through multiple review rounds.
Deterministic Routing, Due-Process Trial, and Adjudication
A defining feature is deterministic contestability-based routing. Not all issues are handled identically; only contestable, substantive-major issues proceed to expensive adjudication. Others follow lightweight paths. Contestable issues undergo a due-process trial: a whole-paper defense argues the case, while a decorrelated local-context jury provides localized evaluation.
Figure 3: Deterministic routing and due-process adjudication for candidate issues.
Final verdicts for each issue are computed under explicit majority and quorum thresholdsโnever by a single semantic agent. Each issue attains one of three terminal classes:
- invalid-drop: reject the issue,
- valid-fixable: accept and machine-edit,
- author-required: defer action to the author where revision is unsafe or outside automated scope.
This strict isolation between issue validity and machine editability, paired with deterministic resolution, is instrumental in preserving both artifact safety and reliable process closure.
Anchored, Guarded Revision and Edit Safety
Edits are only drafted and applied for issues labelled valid-fixable. Each draft patch is subject to a guard chain before application, including anchor-bounded diffs, reference and compile checks, and selective semantic audits depending on a risk categorization (). Edits violating the frozen claim spine, cross-reference integrity, or compilation constraints are blocked. All edit actions are exactly-once and journaling ensures reversibility.
Experimental Protocol
The empirical evaluation adopts a two-arm expert-review protocol. One arm benchmarks issue discovery against expert panel ground truth; the second arm audits terminal class assignment accuracy, routing correctness, edit safety (edit-safety violation rate, ESVR), round convergence, and efficiency.
Figure 4: Two-arm expert-review evaluation protocol for PaperJury.
Comparison is made against four baselines:
- Forward-only Rewriter (no critique/issue ledger, direct revision),
- LLM Critic Only (critique without enforced verdict or safe revision),
- LLM-as-Judge Loop (judge-centered loop, relying on model for verdict/stopping),
- Naive Unbounded Generator (exhaustive high-recall critique generation).
Cost, measured as tokens per paper and wall-clock time, is evaluated to highlight practical deployment considerations.
Key Results
PaperJury achieves the best panel-relative F1 (0.656, macro-averaged per paper), the highest audited precision ($0.847$ on expert-audited issues), and the lowest edit-safety violation rate among editing systems (ESVR 0.025). Terminal verdict accuracy (expert agreement) is $0.887$, routing accuracy is $0.913$, and typical completion occurs in rounds, with no executions hitting the five-round cap. Compared to the judge-centered loop, PaperJury achieves statistically significant improvements in F1, verdict precision, and ESVR while operating at moderate cost ($2.47$ hours, $6.76$ million tokens per paper).
Figure 5: Quality-cost trade-off across issue-producing systems; PaperJury is positioned on the frontier, balancing F1 and efficiency.
Per-domain performance across vision, NLP, and ML papers confirms stability of outcomes (variation <0.03 in key metrics). Pairwise bootstrap comparisons at the paper level show consistent dominance over all baselines.
Ablation studies further analyze system design:
- Removing bounded review, routing, trial, claim spine, or guard chain demonstrably degrades F1, verdict accuracy, and edit safety in accordance with the function of each component.
Implications and Future Directions
PaperJury shifts the paradigm for trustworthy pre-submission revision from open-ended critique generation to closed-loop, governed orchestration. Practice-oriented implications include stricter control over edit traceability, deterministic convergence, and artifact safetyโprerequisites for reliable unattended scientific writing assistance. The explicit three-way terminal verdict structure aligns well with real-world editorial workflows, where not all critiques should lead to revision and some issues require qualified author involvement.
Theoretical implications include a new template for agentic scientific workflows: one prioritizing deterministic orchestration, explicit state and verdict tracking, and bounded semantic agent applications. This work suggests that for high-stakes document revision and review, especially in scientific domains, load-bearing decisions and process safety must reside outside the discretion of generative models.
Extensions may target even broader classes of document artifacts, integration with submission or versioning platforms, transfer to multimodal (e.g., visual/textual) workflows, or co-design with further adversarial or human-in-the-loop verification modules. Addressing efficiency/fidelity trade-offs at even larger scale and under more adversarial distribution shifts is an open research challenge.
Conclusion
PaperJury demonstrates that placing critical process authority in deterministic orchestration, with semantic agents confined to bounded, auditable subtasks, produces measurable gains in both manuscript review quality and artifact safety for LaTeX scientific papers. The result is a robust, scalable approach to pre-submission hardening, with broad implications for the design of accountable AI-assisted scientific authoring tools and agentic scientific workflows.
Reference: "PaperJury: Due-Process Review for Bounded LaTeX Revision" (2606.16322)
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Explain it Like I'm 14
Overview: What this paper is about
This paper introduces PaperJury, a tool that helps authors proofโcheck and safely improve their LaTeX research papers before they submit them to a conference or journal. Instead of just fixing grammar or making suggestions, PaperJury acts like a careful referee: it looks for serious problems across the whole paper (like claims that arenโt fully supported by the experiments), decides what to do about each problem, and then makes only safe, limited editsโwithout accidentally changing the main messages of the paper.
Key goals and questions
The authors ask:
- How can we find real, important problems in a paper without wasting time on fake ones?
- How can we decide which problems to fix automatically, which to drop, and which should be left for the human author?
- How can we make sure any automatic edits are safeโmeaning they donโt break references, change key claims, or introduce new mistakes?
- Is it better to let AI make the big decisions, or should strict, ruleโbased code control the process?
How PaperJury works (in simple terms)
Think of PaperJury as a โmini court systemโ for your paper, guided by two parts:
- Ruleโbased controller (always consistent)
- This is code that follows exact rules and behaves the same way every time.
- It breaks the paper into parts, keeps a stable โlogbookโ of issues, decides which issues need a full โtrial,โ and applies edits exactly once.
- It also decides when the system should stop.
- AI readers/writers (smart, but limited)
- These are LLMs that read the paper, spot issues, and suggest fixes.
- They donโt control the workflow; they only do bounded tasks (reviewing, judging with set rules, drafting small edits).
Key ideas and safety checks:
- Frozen claim spine: PaperJury extracts the paperโs main claims (its โbackboneโ) and locks them. Edits can improve wording or support but canโt quietly change what the paper is claiming.
- Dueโprocess trial: For tricky, important issues, PaperJury runs a structured โmini court.โ A โdefenseโ uses wholeโpaper context; a separate โjuryโ looks at local evidence. A verdict is reached by rules, not by one modelโs opinion.
- Three possible verdicts:
- Invalid-drop: the complaint is wrongโdo nothing.
- Valid-fixable: the issue is real and can be safely autoโfixed.
- Author-required: the issue is real, but needs author action (like new experiments)โdonโt autoโedit.
- Anchorโbounded edits: Fixes are tightly limited to the intended spot (like swapping one Lego block without touching the rest).
- Guard chain (safety checks): Before applying any change, PaperJury runs checks for references, meaning drift, LaTeX compiling, and whether a fix is touching too much. Riskier edits get stricter checks.
- Exactโonce patching with a ledger: Every change is recorded and applied only once, so itโs easy to track or revert.
Main findings: What the tests showed
The authors tested PaperJury on real LaTeX papers from computer vision, natural language processing, and machine learning, and compared it to four other approaches (like simple critics, freeโwriting rewriters, and โLLM-as-judgeโ loops).
In plain language, PaperJury:
- Found more real, important problems while avoiding many false alarms.
- Made fewer unsafe edits (it was the safest among systems that actually edit).
- Chose better verdicts (experts agreed with its decisions more often).
- Stopped reliably in about three rounds, without getting stuck going in circles.
- Used time and compute reasonably (not the cheapest, but far from the most expensive).
The authors also removed pieces of PaperJury to see what mattered:
- Without the โmini courtโ (trial), verdict quality dropped.
- Without the โclaim spineโ or safety guards, unsafe edits jumped a lot.
- Without bounded review and smart routing, cost went up and overall quality slipped.
Bottom line: putting the โbig, loadโbearingโ decisions (what to accept, what to fix, when to stop) in the ruleโbased controllerโand keeping AI focused on reading and draftingโmade the system both safer and more reliable.
Why this matters
Good research papers can still hide serious issues that arenโt just typosโcontradictions across sections, claims that arenโt backed by results, or edits that quietly change the meaning. PaperJury shows a practical way to:
- Strengthen a paper before submission.
- Avoid risky automatic edits that change key claims or break LaTeX.
- Save authors time by catching real problems and safely fixing the ones that can be fixed.
More broadly, this โdueโprocess + safetyโfirstโ design could help other AI tools that work with important documents (like legal or medical forms). It suggests a general rule: let strict, predictable code control the workflow and safety, and use AI where itโs bestโunderstanding text and drafting small, checked changes.
The code is available for others to try and build on at: https://github.com/u7079256/paperjury
Knowledge Gaps
Knowledge gaps, limitations, and open questions
Below is a single consolidated list of concrete gaps, limitations, and open questions left unresolved by the paper that future researchers could act on:
- External validity across domains and formats: Generalization beyond LaTeX computer-science manuscripts (e.g., theory-heavy math, systems/HCI, biomedical, social sciences), other authoring formats (Word/Markdown/PDF), multi-file projects with custom macros, and non-English manuscripts is untested.
- Claim spine fidelity: The method for extracting the โfrozen claim spineโ is not specified or evaluated for precision/recall; its ability to capture implicit claims, mathematical statements, and distributed claims across sections remains unclear.
- Safe handling of numerical content: It is not demonstrated how anchor-bounded edits and guards detect or prevent unsafe changes to numbers in tables/figures or result summaries (without recomputation), and how โresultโ mutation is programmatically detected.
- Figures, equations, and LaTeX complexity: Guarding and edit-bounding around equations, algorithm floats, complex cross-references, custom environments, and macro-generated text are not characterized; robustness to compile-time idiosyncrasies (BibTeX/biber, glossaries, subfiles) is unknown.
- Risk classification accuracy: The criteria and accuracy for categorizing patches as LOW vs RISKY are unspecified; there is no audit of misclassification rates or their impact on safety, coverage, and cost.
- Guard-chain false positives/negatives: While a 17% guard-block rate is reported, the paper does not quantify how often guards block safe patches (over-conservative) or miss unsafe ones (under-conservative), nor how different guard configurations shift ESVR vs coverage.
- Identity and deduplication in the ledger: Algorithms for durable issue identity, deduplication, and merging semantically similar issues are not described or evaluated; collision and fragmentation rates and their effect on routing and stopping are unknown.
- Deterministic stopping predicate ฯ: The exact definition, thresholds, and sensitivity of ฯ are not provided; there is no analysis of premature stopping, oscillations, or failure modes under different ฯ settings or paper complexities.
- Jury decorrelation mechanism: How โdecorrelated local-context juryโ independence is enforced (prompting, context isolation, agent diversity) is unspecified; leakage and framing effects between defense and jury are not measured.
- Verdict boundary calibration: The difficult boundary between valid-fixable and author-required drives most errors; no concrete criteria, calibration procedure, or decision aids are provided to improve consistency on this boundary.
- Adversarial robustness: The systemโs susceptibility to adversarially written text (e.g., obfuscation, rhetorical misdirection, misleading cross-references) and to strategic โdefenseโ arguments is untested.
- Scalability and cost optimization: Token/runtime costs (โ6.8M tokens and โ2.5 hours per paper) may be prohibitive at scale; strategies for adaptive budget allocation, early-exit policies, reviewer count tuning, or selective deepening are not explored.
- Model dependence and reproducibility: The paper omits model identities, inference settings (temperature, seeds), and variance across model families; stability across reruns and portability to open-source vs proprietary models are not established.
- Panel-ground truth uncertainty: The expert issue panels as reference standards are not fully specified (panel size, expertise, inter-rater reliability); sensitivity of P_panel/R to alternate panels and the reproducibility of โgoldโ issues remain open.
- Missing human-in-the-loop evaluation: There is no user study on author experience, trust, cognitive load, or acceptance of verdicts and edits; impacts on writing time, revision quality, and eventual acceptance outcomes are unmeasured.
- Downstream efficacy: The systemโs effect on real submission outcomes (e.g., reviewer scores, acceptance rates) and on post-revision error rates is not assessed.
- Baseline coverage: Direct head-to-head comparisons with the most competitive contemporary systems cited in Related Work (e.g., XtraGPT, SWIF2T, TreeReview, multi-agent peer-review frameworks) under matched evaluation harnesses are missing.
- Long-document limits: The bounded holistic review (2โ4 reviewers, single pass) may under-cover very long manuscripts; the paper does not test length scaling, memory constraints, or multi-pass/sectional strategies for long-form works.
- Supplementary artifacts: The system does not handle code, data, appendices, or external artifacts; cross-artifact consistency checks (e.g., verifying reported numbers against code logs) remain unaddressed.
- Ethical, privacy, and IP concerns: Risks of uploading confidential drafts to LLMs, ledger data retention, and compliance with double-blind policies and institutional confidentiality requirements are not analyzed.
- Collaboration and concurrency: Exact-once patching and ledger semantics under multi-user concurrent edits (e.g., Git/Overleaf workflows) are not discussed; conflict resolution and merge policies are unspecified.
- Allowing legitimate claim revisions: The frozen claim spine prevents โsilentโ claim drift but may block warranted reframing; a governed mode for claim-level changes with explicit author consent and stronger review is not provided.
- Contestability routing transparency: The deterministic rules for contestability labeling and routing thresholds are not fully specified; their sensitivity and fairness across issue types/domains are unknown.
- Formal analysis of the trilemma: The precisionโrecallโcost trilemma is described empirically but lacks a formal framework to optimize budgets, routing thresholds, and guard strengths for target objectives.
- Convergence characterization: Beyond average rounds, there is no analysis of issue birth/closure dynamics, new-issue introduction rates after edits, or conditions guaranteeing convergence vs cycling.
- Generalization across CS subfields: The 12-paper evaluation spans vision/NLP/ML only; applicability to subfields like security, PL, databases, theory, and systems (with different evidence standards) is unknown.
- Non-textual integrity checks: The approach validates compilation and references but does not verify semantic coherence across sections beyond localized audits; whole-paper coherence metrics post-edit are not reported.
- Calibration of q and m (quorum/majority): No sensitivity studies show how jury size, quorum, and majority thresholds affect verdict accuracy, cost, and error profiles.
- Failure taxonomy and diagnostics: The paper reports aggregate metrics (F1, Acc_v, ESVR) but lacks a taxonomy of failure modes (e.g., missed cross-section contradiction, over-zealous surface edits) and diagnostic tools to guide system debugging.
- Public release completeness: The repository is mentioned, but it is unclear whether full prompts, orchestration scripts, guard configurations, and evaluation harnesses (incl. expert audit protocols) are available for exact replication.
- Multilingual and style variation: Robustness to diverse writing styles, heavy use of abbreviations, or non-native English phrasingโand its effect on issue discovery and adjudicationโhas not been studied.
- Policy compliance and integrity checks: The system does not address plagiarism detection, ethics/IRB compliance, dataset licensing statements, or broader integrity checks often critical in pre-submission hardening.
Practical Applications
Immediate Applications
Below are concrete, deployable use cases that directly leverage PaperJuryโs deterministic orchestration, durable ledger, dueโprocess adjudication, claim spine, anchorโbounded edits, and guard chains.
Academia and Publishing
- Preโsubmission hardening for LaTeX CS papers (lab and individual author workflows)
- What: Run a CLI/CI pipeline that produces a durable issue ledger, threeโway verdicts (invalidโdrop, validโfixable, authorโrequired), and exactโonce, guardโverified patches prior to arXiv/journal submission.
- Sector/tools/products: Software/publishing; โPaperJury CLIโ as a GitHub Action or Overleaf plugโin; โClaim Spine Viewโ panel for editors.
- Dependencies/assumptions: Access to competent LLM reviewers; LaTeX projects must compile; authors agree to anchorโbounded edits; manuscript privacy/compliance guardrails.
- Editorial deskโcheck assist for workshops/conferences
- What: Triage incoming LaTeX submissions to separate noโaction items, safe autoโfixes, and authorโrequired issues, with a ledgered audit trail for editorial accountability.
- Sector/tools/products: Publishing; integration with OpenReview/HotCRP; โDueโProcess Adjudicatorโ microservice.
- Dependencies/assumptions: Program chairs willing to adopt machineโassisted preโchecks; reviewer policy alignment on machineโgenerated fixes.
Software and DevOps Documentation
- Guarded documentation revision in code repos
- What: Apply anchorโbounded edits to API docs/release notes with a โdoc claim spineโ (e.g., API guarantees) and exactโonce patching in CI to prevent accidental contract drift.
- Sector/tools/products: Software; โGuardedDoc CIโ for Markdown/LaTeX; โAPI Claim Spine Extractor.โ
- Dependencies/assumptions: Docs stored with source; deterministic decomposition of sections/anchors; CI secrets for LLM access; acceptance of threeโway outcomes (some issues escalated).
Education
- Thesis/dissertation and capstone paper hardening
- What: Wholeโdocument, bounded holistic review for crossโsection contradictions and unsupported claims; safe, reversible edits with a ledger for committee review.
- Sector/tools/products: Education; campusโhosted โPaperJury@Universityโ with privacy safeguards.
- Dependencies/assumptions: Institutional approval for AIโassisted editing; LaTeX template compatibility; compute budget control.
Healthcare and Life Sciences (research artifacts)
- Preprint and model card consistency checks (nonโregulated)
- What: Validate that key claims in ML for health manuscripts/model cards are consistent across sections and figures; permit only guardโpassed edits, elevate authorโrequired items.
- Sector/tools/products: Healthcare research; โModelโCard Guardโ workflow using claim spines for reported metrics.
- Dependencies/assumptions: Not a replacement for regulatory validation; protected health information must be excluded; LaTeX/Markdown structure.
Finance and Compliance (nonโfiling drafts)
- Whitepapers and technical disclosures (internal drafts)
- What: Enforce consistency in risk disclosures and metrics through a durable ledger and guardโverified edits before external release.
- Sector/tools/products: Finance; โDisclosure Ledgerโ for internal review boards.
- Dependencies/assumptions: Not for official filings without legal signโoff; secure onโprem deployment; robust roleโbased access.
Policy and Governance (internal memos/guidelines)
- Internal policy draft governance
- What: Apply dueโprocess triage to policy drafts so that contestable issues are adjudicated and only safely fixable items are autoโedited.
- Sector/tools/products: Government/NGO; โPolicy DueโProcess Assistant.โ
- Dependencies/assumptions: Document formats conform to structured markup (LaTeX/Markdown); clear escalation rules for authorโrequired decisions.
Tools and Platforms
- Overleaf/VS Code extensions exposing safeโedit workflows
- What: Surface claim spines, anchored diffs, and guard outcomes inโeditor; allow oneโclick apply/revert of vetted patches.
- Sector/tools/products: Software tools; โClaim Spine Sidebar,โ โGuard Chain Dashboard.โ
- Dependencies/assumptions: Editor plugโin APIs; stable LaTeX AST parsing; user consent for external model calls.
LongโTerm Applications
The following use cases extend PaperJuryโs principles beyond LaTeX CS papers and/or require additional research, scaling, or ecosystem change.
CrossโFormat and CrossโLanguage Expansion
- Word/Google Docs/HTML support with robust claim extraction
- What: Generalize deterministic decomposition and claimโspine extraction to richโtext and web formats; preserve exactโonce patching.
- Sector/tools/products: Office productivity; โGuardedDocs for Word/Docs.โ
- Dependencies/assumptions: Reliable crossโformat anchoring; model accuracy on varied layouts/languages; versioning APIs.
- Multilingual scholarly and policy documents
- What: Apply dueโprocess review in nonโEnglish manuscripts.
- Sector/tools/products: Publishing/policy; localized juror/defense agents.
- Dependencies/assumptions: Highโquality multilingual LLMs; localeโspecific style and legal standards.
Regulated Documentation and Submissions
- Regulatory submission prep (eCTD sections, audit trails)
- What: Use threeโway verdicts to gate which changes are automated vs escalated; maintain ledgered provenance for audits in FDA/EMA workflows.
- Sector/tools/products: Healthcare; โRegDoc Guardโ tying guard chains to CFR/ICH checklists.
- Dependencies/assumptions: Formal validation and compliance approval; domain ontologies; human final signโoff; secure, onโprem deployment.
- Financial filings and earnings reports
- What: Anchorโbounded edits with claim spines mapping to material statements; deterministic stopping with executive/legal review.
- Sector/tools/products: Finance; โ10โK/10โQ Guardrail Suite.โ
- Dependencies/assumptions: Regulatory acceptance; integration with disclosure controls; strict changeโcontrol governance.
MultiโArtifact and SystemโofโRecord Governance
- Crossโdocument consistency across paper, supplement, code, and website
- What: Maintain a shared claim spine spanning multiple artifacts; verify edits donโt silently desynchronize claims/results.
- Sector/tools/products: Software/science; โCrossโArtifact Ledger.โ
- Dependencies/assumptions: Robust crossโrepo linking; consent to centralized governance; tooling to parse varied artifacts.
- Knowledgeโbase and encyclopedia maintenance
- What: Treat each fact as a claim with a spine; anchorโbounded updates with dueโprocess trials for contested facts.
- Sector/tools/products: Education/reference; โClaimโCentric KB Maintainer.โ
- Dependencies/assumptions: Provenance metadata, citation retrieval; community arbitration models.
CodeโAware Documentation and API Governance
- Coupled code+docs updates with contractโaware spines
- What: Use API signatures/tests as a โcode claim spineโ; allow doc edits that pass contract checks, escalate risky changes.
- Sector/tools/products: Software/DevOps; โAPI Contract Guard.โ
- Dependencies/assumptions: Stable AST/contract extraction; CI integration with test suites; synchronized versioning.
SafetyโCritical Operational Manuals
- Robotics/energy SOP updates with hazardโaware guards
- What: Extend guard chains to include safety analysis (e.g., checklists, references to risk assessments) before applying edits.
- Sector/tools/products: Robotics/energy; โSafety Manual Guard.โ
- Dependencies/assumptions: Formal hazard models; domain checkers; human safety officer signโoff; liability frameworks.
Education Platforms and Assessment
- Dueโprocess writing tutor with escalations to instructors
- What: Provide students with bounded critiques, safe local edits, and a ledger of authorโrequired items requiring human feedback.
- Sector/tools/products: EdTech/LMS; โDueโProcess Tutor.โ
- Dependencies/assumptions: Pedagogical acceptance; antiโplagiarism integration; tiered privacy controls.
Civic Law and Governance
- Legislative drafting with contestability and auditability
- What: Apply defense/jury adjudication to contested clauses; anchorโbounded changes with immutable audit trails.
- Sector/tools/products: Civic tech; โCivic Draft Guard.โ
- Dependencies/assumptions: Legal/ethical frameworks; public transparency requirements; multiโstakeholder jury design.
Enterprise AI Orchestration
- Deterministic control for agentic workflows beyond documents
- What: Adopt the ledger, deterministic routing, threeโway verdicts, and exactโonce application to marketing copy, customer comms, or policy updates.
- Sector/tools/products: Crossโindustry; โAgent Orchestrator with DueโProcess.โ
- Dependencies/assumptions: Tooling that surfaces intermediate traces; alignment with internal changeโcontrol processes; scalability and observability.
Notes on Core Assumptions Across Applications
- Deterministic decomposition and stable anchoring are feasible for the target format(s).
- Claimโspine extraction accurately captures material commitments; teams accept a norm that claimโlevel changes require explicit escalation.
- LLMs provide reliable, bounded holistic review within cost constraints; domainโexpert supervision remains available for โauthorโrequiredโ items.
- Privacy, IP, and compliance requirements are satisfied (e.g., onโprem deployment, data retention policies).
- Organizations adopt threeโway outcomes and deterministic stopping instead of freeโform, modelโdriven looping.
Glossary
- ablation table: a results table that reports performance when components are removed or varied to analyze their contribution. "An ablation table announces the best result on a benchmark,"
- adjudication: the decision-making process that determines which issues to accept, reject, or escalate. "The first is adjudication: which complaints to honor, and when to stop."
- agentic workflows: multi-step, tool-using AI processes that autonomously plan and act, emphasizing control over intermediate steps. "the dominant safety surface in agentic workflows lies in the models' intermediate execution traces"
- anchor-bounded diff checks: safeguards that restrict and verify changes to only the targeted anchored text spans. "The guard chain includes anchor-bounded diff checks, cross-reference checks, semantic meaning or edit audits when triggered, compile checks, and application journaling."
- anchor-bounded edits: constrained edits localized to predefined anchors to limit unintended document changes. "risk-proportional guard chains for anchor-bounded edits,"
- anchor drift: movement or misalignment of previously defined text anchors, causing references or guards to mis-target content. "structural failures such as reference breakage, anchor drift, or silent claim mutation remain possible."
- anti-skim checks: automated tests that detect superficial review and prompt deeper rereads for coverage and grounding. "invokes targeted re-read only when deterministic anti-skim checks indicate missed coverage or weak grounding,"
- artifact-safe LaTeX revision: editing procedures that ensure changes do not corrupt the documentโs structure, references, or claims. "artifact-safe LaTeX revision with a frozen claim spine, anchor-bounded edits, or orchestrator-controlled apply and revert semantics"
- blast radius: the scope of unintended side effects caused by an edit. "no reliable way to bound its own blast radius,"
- bounded holistic review: whole-paper assessment with explicit limits on reviewers and scope to balance quality, cost, and recall. "PaperJury then runs bounded holistic review with a small set of whole-paper reviewers."
- clean re-review: a fresh, full reread of the updated manuscript that ignores previous model outputs to surface new issues. "After guarded edits, PaperJury performs clean re-review on the current manuscript state"
- closed-loop review-verdict-revise-verify system: an iterative pipeline that cycles through critique, judgment, revision, and validation with controlled stopping. "We present PaperJury, a closed-loop review-verdict-revise-verify system"
- compile-aware validation: checks that ensure LaTeX edits still compile successfully as part of safety gating. "guarded variants use compile-aware validation under orchestrator-controlled apply and revert semantics."
- contestability label: a tag indicating how disputable an issue is, used to route it to the appropriate adjudication path. "assigns each issue a deterministic contestability label and routes it accordingly."
- contestability-based routing: directing issues to different processes based on how contestable or consequential they are. "contestability-based routing,"
- decorrelated local-context jury: a panel that evaluates issues using localized evidence while being separated from prior reviewer framing. "a decorrelated local-context jury evaluates it"
- deterministic decomposition: a reproducible partition of the manuscript into structured units (sections, passages, anchors) before semantic processing. "The pipeline begins with deterministic decomposition through "
- deterministic orchestration: control logic implemented in code that governs state, routing, stopping, and patching consistently across runs. "deterministic orchestration manages decomposition, a frozen claim spine, a durable ledger, routing, stopping, and exact-once patch application,"
- deterministic outer-loop convergence: a guaranteed, rule-based stopping condition for iterative review cycles. "deterministic outer-loop convergence"
- deterministic routing: rule-based, reproducible assignment of issues to adjudication paths rather than relying on model judgment. "deterministic routing from critique to adjudication"
- deterministic-versus-semantic split: architectural separation where deterministic code handles control and safety, while models handle reading and drafting. "built on a deterministic-versus-semantic split:"
- due-process adjudication: a structured, two-sided evaluation process that provides defense and impartial judgment for contested issues. "durable state, due-process adjudication, and deterministic stopping"
- due-process trial: a formal, procedural evaluation of contestable issues with defense and jury components. "contestable issues enter a due-process trial"
- durable issue identity: persistent identifiers that track the same issue across review rounds. "lack durable issue identity across rounds"
- durable ledger: a long-lived record of issues, evidence, verdicts, and edits maintained across iterations. "A durable ledger gives each issue a stable identity across rounds,"
- exact-once patch application: a guarantee that approved edits are applied a single time and not duplicated across runs. "exact-once patch application,"
- execution cap: an upper limit on the number of review rounds to bound cost and runtime. "subject to a five-round execution cap."
- frozen claim spine: an extracted, protected set of core claims that revisions must not alter. "the system first decomposes the manuscript and extracts a frozen claim spine"
- guard chain: a sequenced set of structural and semantic checks that a proposed edit must pass before application. "screened by a risk-proportional guard chain"
- Guardrailed Agent Workflows: workflows for AI agents with safeguards to validate intermediate steps and outputs. "Guardrailed Agent Workflows"
- ledger-query predicate: a deterministic rule evaluated over the ledger state to decide when to stop iterating. "stops when a deterministic ledger-query predicate"
- LLM-as-a-Judge: the use of LLMs to score or decide outcomes, often with known biases. "Studies of LLM-as-a-Judge report systematic scoring and position biases,"
- panel-relative F1: F1 computed relative to an expert panelโs issues, combining precision and recall under that reference. "attains the best panel-relative F1 (0.656, vs.\ 0.519 for the judge-centered loop)"
- paired-bootstrap: a resampling method that pairs observations to estimate confidence bounds on performance differences. "paper-cluster paired-bootstrap 95\% lower bounds"
- precision-recall-cost trilemma: the three-way trade-off between finding more real issues, avoiding false ones, and controlling budget. "under a precision-recall-cost trilemma:"
- risk-proportional guard chains: variable-strength validation pipelines that scale with the assessed risk of a proposed edit. "risk-proportional guard chains for anchor-bounded edits,"
- schema-validated semantic agents: model components whose inputs/outputs conform to predefined schemas to ensure structural correctness. "deterministic orchestration over schema-validated semantic agents"
- silent claim mutation: unintended changes to the meaning or commitments of a claim introduced by edits. "silent claim mutation"
- stopping predicate: a deterministic condition that triggers termination of the review loop without model self-assessment. "deterministic stopping predicate for unattended execution"
- two-arm expert-review evaluation: an experimental design with two complementary expert-based assessment procedures. "We use a two-arm expert-review evaluation on a held-out corpus of LaTeX computer science papers"
- typed, validated workflow: a structured process with explicit types and validations that constrain and check each generation step. "placing generation inside a typed, validated workflow rather than leaving structure and state identity to unconstrained model behavior"
Collections
Sign up for free to add this paper to one or more collections.
