NeuroMesh: A Unified Neural Inference Framework for Decentralized Multi-Robot Collaboration

Abstract: Deploying learned multi-robot models on heterogeneous robots remains challenging due to hardware heterogeneity, communication constraints, and the lack of a unified execution stack. This paper presents NeuroMesh, a multi-domain, cross-platform, and modular decentralized neural inference framework that standardizes observation encoding, message passing, aggregation, and task decoding in a unified pipeline. NeuroMesh combines a dual-aggregation paradigm for reduction- and broadcast-based information fusion with a parallelized architecture that decouples cycle time from end-to-end latency. Our high-performance C++ implementation leverages Zenoh for inter-robot communication and supports hybrid GPU/CPU inference. We validate NeuroMesh on a heterogeneous team of aerial and ground robots across collaborative perception, decentralized control, and task assignment, demonstrating robust operation across diverse task structures and payload sizes. We plan to release NeuroMesh as an open-source framework to the community.

• The paper presents a novel decentralized inference framework that unifies multi-robot collaboration through a four-stage pipeline and dual-aggregation paradigm.
• The paper details a modular, parallel processing strategy implemented in ROS2 with hardware-agnostic plug-and-play modules for real-time operation.
• The paper validates NeuroMesh across applications like collaborative perception and decentralized control, demonstrating enhanced accuracy and robustness.

NeuroMesh: A Unified Neural Inference Framework for Decentralized Multi-Robot Collaboration

Motivation and Problem Statement

Learning-based multi-robot systems increasingly demand scalable, robust frameworks capable of on-the-fly deployment in heterogeneous settings—spanning dense perception, decentralized control, and resource-constrained task assignment. Existing frameworks either lack generality across task domains, are limited to homogeneous hardware, or do not decouple task definition from a performant execution stack. NeuroMesh addresses this gap by offering a unified, high-performance neural inference framework built for decentralized multi-robot collaboration, supporting heterogeneous computation and communication, with explicit support for modular model composition and real-time multi-robot operation.

Methodological Formulation

NeuroMesh organizes decentralized multi-robot inference into a four-stage process: observation encoding, message passing, feature aggregation, and task output decoding. Each robot encodes its sensory signals into a feature tensor, exchanges features with neighbors, aggregates these using either reduction or broadcast paradigms, then decodes an actionable output or estimate. This process is not constrained to a specific neural network family or communication model, supporting Graph Neural Network-based, transformer-based, or classical analytical stacks, and agnostic to transport (centralized or mesh-decentralized).

Figure 1: The four-stage pipeline models the generic decentralized inference computation from local encoding, message passing/aggregation, to task decoding.

A central technical innovation is the dual-aggregation paradigm:

• Reduction aggregation: Classical for consensus or synchronization, information from neighbors is fused via operations like sum, mean, or max.
• Broadcast aggregation: Designed for dense perception tasks, self-features are paired with every neighbor's features and concatenated, preserving pairwise information for relational reasoning without architectural changes.

Figure 2: Illustration of the reduction aggregation paradigm, fusing self and neighbor features into a single representation.

The computation pipeline is parallelized at the encoder, aggregator, and decoder stages, each asynchronously progressing on separate data cycles. This design decouples throughput from maximum per-stage latency, optimizing for real-time constraints in heterogeneous robot teams.

Figure 3: NeuroMesh’s parallel pipeline enables concurrent execution of encoding, aggregation, and decoding for different data cycles, thereby limiting system cycle time to the longest active stage rather than the sum of all.

System Implementation and Architecture

NeuroMesh is implemented in C++, designed for the ROS2 middleware, with inference engines selectable per architecture and platform (TensorRT for NVIDIA GPUs, ONNX Runtime for CPU/non-NVIDIA). Each pipeline module operates as a microservice, with standardized tensor serialization and communication wrappers. The inter-robot transport leverages Zenoh in peer mode for mesh topologies, shown to achieve substantially lower latency, jitter, and packet loss in high-throughput scenarios compared to DDS-based transports.

Figure 4: Pipeline instantiation illustrating plug-and-play insertion of encoders, aggregators, and decoders, each supporting diverse task-specific neural models.

Critical to cross-platform support, all stages admit custom networks—including deep vision models, compact MLPs for control/state, or analytic decision algorithms—inserted as plug-ins. Communication protocols are engineered to degrade gracefully under packet loss, supporting both blocking and best-effort aggregation semantics.

Experimental Validation

Collaborative Perception

NeuroMesh enabled the deployment of decentralized vision-based geometric reasoning and semantic segmentation for aerial-ground teams. For collaborative depth estimation (DUSt3R), self and neighbor visual features were fused, yielding consistent 3D reconstructions with lower uncertainty and improved per-pixel accuracy over single-robot baselines. Performance parity ($5.29$ ARE, $61.32\%$ inlier ratio) was maintained between centralized and decentralized inference under NeuroMesh, demonstrating architectural equivalence.

Figure 5: Multi-robot collaborative perception cases, including depth estimation and semantic segmentation, quantitatively and qualitatively outperforming single-robot inferencing.

Decentralized Control

Adapting a GNN-based RL policy for ground robots with non-holonomic constraints, NeuroMesh managed real-time deployment of decentralized planners. Despite sim-to-real challenges (especially in angular dynamics), the system accomplished point navigation without collisions with up to $60\%$ real-world task success rate (compared to $80\%$ in sim). Communication was restricted to compact state vectors, validating the efficacy of NeuroMesh for high-frequency, low-latency policy execution.

Figure 6: Demonstration of collaborative navigation and control, showcasing synchronous goal seeking and collision avoidance.

Task Assignment

NeuroMesh also handled GNN-based goal assignment with discrete matching constraints. Five-robot outdoor deployments yielded nearly optimal assignment costs (within $0.03\%$ of the centralized Hungarian algorithm), with a $93.9\%$ success rate for $128$-byte messages. Increasing message sizes mitigated assignment failure, quantifying the trade-off between communication payload and decision fidelity.

Performance and Scalability

Extensive ablation studies reveal that Zenoh peer mode is uniquely effective for both small (control) and large (perception) payloads under mesh networking, maintaining low-latency throughput even as robot counts scale to $50$. Per-module computation costs illustrate that large encoders (e.g., for vision tasks) benefit from GPU offloading, while compact models for control and task assignment preferentially run on the CPU to avoid data transfer overhead.

Figure 7: Throughput analysis for both small and large message payloads, highlighting scalability and protocol selection strategies.

The parallelized processing pipeline further provides measurable improvements: overall system throughput is increased, and computation is bounded by the slowest single stage, rather than the cumulative latency of sequential pipelines.

Practical and Theoretical Implications

NeuroMesh’s modular, domain-agnostic, and high-performance design supports the rapid deployment and benchmarking of decentralized learning algorithms in real-world robotic collectives. The dual-aggregation paradigm facilitates seamless generalization from compact control state fusion to dense perception and relational reasoning, enabling unified analysis across task families. The architecture agnosticism with respect to neural backbone and transport ensures system longevity and reusability.

Theoretically, NeuroMesh abstracts decentralized inference into a compositional process, providing insight into how interaction topology, aggregation schemes, and model structure influence the expressivity and robustness of multi-robot behavior. Practically, it offers a reference platform for scalable, distributed robotics—spanning collaborative exploration, autonomous task allocation, and swarm control under communication constraints.

Future Directions

The NeuroMesh framework is poised for extension to larger robot collectives, adaptation to variable RF environments, and augmentation with non-visual modality encodings. The support for both analytical and learned planners, under a consistent abstraction, lays groundwork for hybrid symbolic-connectionist approaches. Open-sourcing will encourage broad adoption and facilitate community-driven improvements—accelerating reproducibility and standardized benchmarking in decentralized robotic inference.

Conclusion

NeuroMesh establishes a robust foundation for modular, multi-domain neural inference in decentralized multi-robot systems. Its architectural innovations—the dual-aggregation paradigm, parallel pipeline, and hardware-agnostic plug-and-play implementation—directly address the deployment gap in collaborative robotics. Empirical results underline its generality, efficiency, and extensibility, with practical utility in perception, control, and decision-making. The platform is well-positioned to serve as a community standard for scalable, real-world decentralized multi-robot collaboration and serves as a theoretical anchor point for further investigations in decentralized neural inference systems.