Owl-AuraID 1.0: An Intelligent System for Autonomous Scientific Instrumentation and Scientific Data Analysis

Abstract: Scientific discovery increasingly depends on high-throughput characterization, yet automation is hindered by proprietary GUIs and the limited generalizability of existing API-based systems. We present Owl-AuraID, a software-hardware collaborative embodied agent system that adopts a GUI-native paradigm to operate instruments through the same interfaces as human experts. Its skill-centric framework integrates Type-1 (GUI operation) and Type-2 (data analysis) skills into end-to-end workflows, connecting physical sample handling with scientific interpretation. Owl-AuraID demonstrates broad coverage across ten categories of precision instruments and diverse workflows, including multimodal spectral analysis, microscopic imaging, and crystallographic analysis, supporting modalities such as FTIR, NMR, AFM, and TGA. Overall, Owl-AuraID provides a practical, extensible foundation for autonomous laboratories and illustrates a path toward evolving laboratory intelligence through reusable operational and analytical skills. The code are available at https://github.com/OpenOwlab/AuraID.

• The paper introduces a GUI-native agent system that automates instrument control and data analysis in heterogeneous laboratory environments.
• It leverages modular Type-1 and Type-2 skills to replicate expert workflows for spectral and imaging characterization.
• Results demonstrate scalable automation across multi-modal instruments, enhancing laboratory efficiency and data integrity.

Owl-AuraID 1.0: Autonomous Scientific Instrumentation and Data Analysis via GUI-Native Agents

Introduction and Motivation

Owl-AuraID 1.0 presents an agentic system designed to achieve end-to-end automation of scientific characterization workflows in laboratory environments characterized by heterogeneous instrumentation and widely varying, vendor-specific software GUIs (2603.29828). Laboratory automation has been thoroughly explored in synthesis and robotic handling, but scientific characterization remains a persistent bottleneck due to closed-source software, lack of standardized APIs, and knowledge transfer primarily occurring at the level of visual, GUI-driven procedures rather than programmatically-encoded logic.

Existing autonomous platforms typically depend on tightly integrated APIs, CLI schemas, or highly specific middleware, which do not generalize to real-world heterogeneous laboratories. Owl-AuraID addresses this by adopting a GUI-native computer-use paradigm, allowing machine agents to interact with instrument control software as human experts do—through the GUI—while capturing both the physical and analytical procedural knowledge in a reusable, extensible fashion. The system is architected as a skill-centric agent runtime, abstracting laboratory procedures into "Type-1" GUI operational skills and "Type-2" analytical script skills, thus bridging embodied manipulation, GUI-based instrument operation, and scientific data analysis.

Figure 1: Overview of the Owl-AuraID 1.0 architecture for autonomous scientific characterization, connecting user command translation, embodied agent operation, and multi-modal analytics across instruments.

Architectural Overview and Agent Runtime

Owl-AuraID is constructed as a collaborative software–hardware system built atop the InnoClaw agent-runtime, inheriting multi-turn planning, workspace access, execution of commands/scripts, and autonomous task decomposition. However, the interaction interface is reoriented to a conversation-first model, favoring natural-language instructions and feedback for experimental scientists over direct code manipulation. The agent loop is responsible for decomposing high-level objectives into sequential actionable steps, which may involve direct file I/O, command-line execution, GUI automation, or invocation of stored skills.

Critically, the platform is modular, supporting the importation of external skills and analytical routines from shared repositories, promoting extensibility across diverse laboratory environments without requiring model retraining or system redeployment. This design enables incremental accumulation and broad reuse of both operational knowledge and analytical logic.

Skill-Centric Capability Accumulation

A fundamental contribution of Owl-AuraID is the adoption of skills as first-class knowledge artifacts—divided into:

• Type-1 (GUI Operational Skills): Encapsulate expert-demonstrated procedures for instrument GUI operation. They capture not just click/keypress sequences but parameterizable logic, conditional branching, state verification, and tacit heuristics observed in expert practice. Their execution is mediated by a computer-use agent that visually grounds itself in the current UI, replicating complex procedures like CT reconstruction, SEM setup, or multistage spectral acquisition.
• Type-2 (Analytical Script Skills): Capture laboratory-specific data processing logic expressed in natural language, translated by the agent into standardized, parameterized scripts using libraries such as NumPy, SciPy, or domain-specific tools. These skills automate baseline correction, peak fitting, structure assignment, and statistical quantification, and can be immediately validated and iteratively refined before being packaged for repeatable use.

This skill taxonomy enables a synergy where Type-1 skills automate physical experimentation and data generation, while Type-2 skills process and interpret resulting data. The framework supports closed-loop experimentation, where analytical outcomes drive further operational choices.

Figure 2: Multi-modal characterization enabled by platform integration of spectroscopy (UV-Vis, PL), imaging (SEM, Micro-CT), and elemental analysis (EDS).

End-to-End Autonomous Characterization Workflows

Spectral Characterization

Owl-AuraID delivers full workflow automation for multi-modal spectral analysis, combining robotic manipulation for precise sample positioning with GUI-native control of instrument software. Agents autonomously initiate measurements, configure parameters, monitor visual feedback, and dynamically adapt procedures—mirroring the decision-making heuristics of expert human operators.

Figure 3: AuraID workflow for spectral characterization, exemplifying both physical manipulation and dynamic, agent-driven instrument software interaction for UV-vis and PL.

Microscopic Imaging Characterization

For imaging modalities such as SEM, EDS, and Micro-CT, the system realizes closed-loop automation: from robotic sample loading and chamber evacuation, to real-time parameter tuning and data collection. The embodied agent adapts critical imaging parameters (e.g., accelerating voltage, focus, field-of-view) based on live visual output, minimizing the need for manual intervention and supporting robust, reproducible batch processing across samples and modalities.

Figure 4: AuraID workflow for integrated microscopic imaging characterization, spanning SEM, Micro-CT, and EDS.

Intelligent Scientific Data Analysis

The analytical component leverages Type-2 skills to transform raw experimental output into structured scientific knowledge. Owl-AuraID provides ready-to-use routines for:

• Spectral Data Processing: Including baseline correction, automated peak identification, functional group assignment in FTIR via database cross-referencing, NMR structural elucidation through resonance integration and chemical shift mapping, and comprehensive TGA thermodynamic parameter extraction.

  Figure 5: Agent-driven spectral analysis workflow, covering baseline correction, peak extraction, and advanced interpretation tasks for multiple spectroscopic modalities.

• Imaging Data Analysis: Encompassing automatic quantification of structural features in SEM images, compositional analysis in EDS, nanoscale surface metrology for AFM topographs, and crystallographic quantification in EBSD. Batch-processing and statistical analysis pipelines are generated and executed autonomously.

  Figure 6: Automated imaging analysis workflow, demonstrating batch quantification and statistical feature extraction for electron and force microscopy data.

Implications and Prospects

Owl-AuraID advances the state of laboratory autonomy beyond synthesis/robotics into the most fragmented domain of scientific research—characterization. Notably, the GUI-native agent approach bypasses the need for standardized APIs or open-source drivers, rapidly enabling automation across commercially diverse instrument ecosystems and facilitating continual growth of laboratory intelligence via reusable procedural and analytical skills.

The system's multi-modal coverage—demonstrated across at least ten categories of precision scientific instruments—validates the extensibility and generality of the approach. This architecture not only accelerates data acquisition and interpretation but also systematically generates high-quality datasets critical for the training and deployment of the next generation of AI scientific foundation models.

Current limitations include reliance on human-expert demonstration for skill acquisition and residual challenges in the physical transfer of delicate samples across instruments, suggesting opportunities for self-supervised skill learning and advanced multi-robot orchestration. Moreover, the broader adoption of GUI-native AI agents will require foundational advances in robust vision-language grounding and adaptive reasoning under highly irregular, stateful software environments.

Conclusion

Owl-AuraID 1.0 establishes a robust foundation for autonomous scientific characterization in laboratories with heterogeneous, closed-source instrumentation. By formalizing GUI-native agentic operation and centering on a skill-centric framework, the system bridges physical sample manipulation, complex software interaction, and richly parametrized data analysis. This paradigm provides a pathway for scalable and generalizable laboratory intelligence, with lasting implications for high-throughput experimentation, scientific data integrity, and the development of foundation models that integrate deep physical understanding with empirical proficiency. As programmable skills accumulate, systems like Owl-AuraID are poised to become core infrastructure for AI-driven scientific discovery.