Smaller, Smarter, Closer: The Edge of Collaborative Generative AI

Abstract: The rapid adoption of generative AI (GenAI), particularly LLMs, has exposed critical limitations of cloud-centric deployments, including latency, cost, and privacy concerns. Meanwhile, Small LLMs (SLMs) are emerging as viable alternatives for resource-constrained edge environments, though they often lack the capabilities of their larger counterparts. This article explores the potential of collaborative inference systems that leverage both edge and cloud resources to address these challenges. By presenting distinct cooperation strategies alongside practical design principles and experimental insights, we offer actionable guidance for deploying GenAI across the computing continuum.