Automatic Circuit Discovery (ACDC)

• ACDC is an algorithmic framework that identifies minimal, functionally sufficient subnetworks (circuits) within broad systems for specific tasks.
• It spans applications in mechanistic neural network interpretability, analog circuit design, quantum synthesis, and synthetic biochemistry through techniques like activation patching and differentiable masking.
• The methodologies emphasize faithfulness, sparsity, and scalability while tackling challenges such as NP-hard complexity and trade-offs between resolution and computational efficiency.

Automatic Circuit Discovery (ACDC) refers to algorithmic and data-driven methodologies for inferring, generating, or engineering “circuits”—task-specific, functionally coherent subnetworks—within broader systems. The term is context-dependent, appearing prominently in mechanistic neural network interpretability, analog and quantum circuit synthesis, and even DNA strand-displacement catalytic networks. These approaches target the automation of subgraph or topology discovery that would otherwise require significant domain expertise or labor-intensive empirical investigation.

1. Principles and Definitions

Automatic Circuit Discovery (ACDC) formalizes the search for subnetworks—termed “circuits”—that are minimal and functionally sufficient to produce a desired behavior within a model or physical system. In transformer-based language and vision models, a circuit is a sparse subgraph of the overall computation graph (nodes: e.g., attention heads, MLPs, neurons; edges: data flow between nodes), whose activation flow implements a specific cognitive or algorithmic function, such as syntactic binding or visual concept detection (Conmy et al., 2023, Rajaram et al., 2024, Kwon et al., 3 Aug 2025). In analog circuit design, ACDC denotes the automated synthesis of circuit topologies at the schematic level, selecting devices and netlists that fulfill target specifications (Gao et al., 28 Feb 2025, Li et al., 20 Jul 2025). In synthetic biochemistry, ACDC refers to systematic construction and verification of catalytic DNA networks, ensuring that dynamic, out-of-equilibrium behavior maps precisely onto an abstract reaction network (Lankinen et al., 2020).

Key properties in all domains include:

• Sufficiency: The subcircuit performs the target function in isolation.
• Minimality/Sparsity: The subcircuit retains as few components as possible.
• Faithfulness and Completeness: The circuit alone preserves function, and its complement is functionally deficient (Yu et al., 2024).
• Scalability: Methodologies must mitigate prohibitive complexity as models or schematics grow.

2. Algorithmic Methodologies

Multiple algorithmic strategies have been developed for ACDC across its application domains:

Mechanistic Interpretability (NLP and Vision):

• Activation Patching: Intervene on edge activations (e.g., substitute with corrupted or zeroed values), assess output impact, and greedily prune weakly contributing edges (Conmy et al., 2023). The ACDC algorithm prunes edges with negligible KL-divergence effect, proceeding in reverse-topological order, supporting circuit sizes orders of magnitude smaller than the full graph.
• Edge Attribution Patching (EAP): A linear Taylor expansion approximation enables fast scoring of edge importance using only two forward and one backward pass per dataset batch, bypassing the quadratic cost of direct patching (Syed et al., 2023, Haklay et al., 7 Feb 2025).
• Contextual Decomposition (CD-T): Carries out a two-part additive decomposition of activations through transformer modules, propagating "relevant" and "irrelevant" contributions. CD-T enables efficient, fine-grained construction of functionally faithful circuits and produces high ROC AUC circuit recovery (Hsu et al., 2024).
• Differentiable Masking: Learn binary or continuous masks over edges and/or weights via hard-concrete relaxation, jointly optimizing faithfulness, completeness, and sparsity. This enables gradient-based learning of minimal circuits in a single phase (Yu et al., 2024, Haider et al., 11 Dec 2025).
• Position-aware Circuit Discovery: Introduces position-differentiated attributions and token-span schemas for variable-length tasks, discovering circuits that capture cross-positional dynamics (Haklay et al., 7 Feb 2025).
• Hierarchical and Multi-Granular Pruning: Simultaneously masks at multiple structural resolutions (block, head, neuron), reducing memory footprint and enabling neuron-level interpretability (Haider et al., 11 Dec 2025).

Analog Circuit Topology Discovery:

• Generative Modeling: Transformer-based graph-to-sequence models, such as AnalogGenie, encode netlists via permutation-preserving Eulerian traversals, enabling scalable and unambiguous representation of the design space (Gao et al., 28 Feb 2025).
• Federated Learning: Circuit topology generators can be trained using decentralized data (e.g., different institutes’ proprietary libraries), aggregating via FedAvg and leveraging token embeddings to ensure data homogeneity and privacy (Li et al., 20 Jul 2025).

Synthetic Biochemistry:

• Compiler-based Synthesis: Automated generation of strand-displacement circuit components and precise domain assignments, using graph motif detection and sequence-level optimization (e.g. NUPACK) for arbitrary reaction networks (Lankinen et al., 2020).

Quantum Circuit Synthesis:

• Evolutionary Multi-Objective Search: Genetic optimization over gate sequences, with explicit objectives for fidelity, gate count, depth, and implementability, produces Pareto-surfaces trading off accuracy vs. circuit resources (Potoček et al., 2018).

3. Evaluation Metrics and Benchmarks

Metrics for ACDC are highly dependent on domain but share several themes:

• Faithfulness: Circuit’s output matches the full system on the target task (Yu et al., 2024, Kwon et al., 3 Aug 2025).
• Completeness: Removing the circuit from the system destroys task performance (Yu et al., 2024).
• Sparsity: Fraction of edges, nodes, or weights retained (Conmy et al., 2023, Haider et al., 11 Dec 2025).
• Novelty: Discovery of topologies or subgraphs not present in the training set (especially in analog design) (Gao et al., 28 Feb 2025, Li et al., 20 Jul 2025).
• Versatility and Scalability: Ability to capture multiple functional or concept classes, and support for increasing subcircuit sizes (Gao et al., 28 Feb 2025, Kwon et al., 3 Aug 2025).
• Efficiency: Computation time, memory footprint, and number of passes required (especially acute for large transformer models) (Wang et al., 27 Oct 2025, Syed et al., 2023).

Task-specific benchmarks include Indirect Object Identification (IOI), Greater-Than (number comparison), and Docstring completion in NLP; logit-drop and deletion/insertion curves in vision; and electrical figures-of-merit (e.g., GBW × phase margin for Op-Amps) in analog design.

4. Computational Complexity and Scalability

ACDC procedures face intrinsic complexity-theoretic barriers:

• Hardness Results: The problem of finding a minimal, sufficient circuit in multilayer perceptrons is typically NP-complete or $\Sigma_2^p$-complete even for local queries, and W[1]-hard for small circuit sizes (Adolfi et al., 2024).
• Inapproximability: No polynomial-time approximation schemes exist unless P=NP, barring restricted relaxations such as quasi-minimal circuits or bounded local queries.
• Tractable Islands: Quasi-minimal circuit identification and gnostic neuron detection (threshold-based scanning for activation patterns) are PTIME. Brute-force enumeration is fixed-parameter tractable in small regions.
• Solver Transformations: Practical circuit queries can be encoded as SAT or QBF instances, enabling scalable ACDC via well-engineered constraint solvers (Adolfi et al., 2024).

Algorithmic advances such as mixed-precision patching (PAHQ) exploit the one-intervention-at-a-time nature of circuit search for runtime/memory gains or leverage fine-grained masking to reduce O($N^2$) memory requirements to a single fine-tuning pass (Wang et al., 27 Oct 2025, Haider et al., 11 Dec 2025).

5. Domain Extensions: Analog, Quantum, and Biochemical ACDC

Analog Circuit Discovery:

• Sequence-based graph encodings (Eulerian walks over device pins) coupled to GPT-like decoders permit domain-constraint satisfaction (Kirchhoff's node law by construction), expansion to multiple device classes, and generation of novel, simulation-valid topologies (Gao et al., 28 Feb 2025).
• Federated learning approaches extend generative ACDC to collaborative, privacy-sensitive design across institutions, maintaining high validity and novelty with minor degradation as the number of clients increases (Li et al., 20 Jul 2025).

Quantum Circuit Synthesis:

• Multi-objective evolutionary ACDC discovers exact or near-exact textbook implementations (QFT, Grover) and novel trade-off circuits balancing depth, width, and gate count, using a hybrid of population-based search and parameter tuning (Potoček et al., 2018).

Synthetic Biochemistry:

• Active Circuits of Duplex Catalysts (ACDC) provide automated design of catalytic networks with explicit kinetic models and leak suppression via targeted sequence mismatches, automating previously labor-intensive strand-displacement network engineering (Lankinen et al., 2020).

6. Limitations, Challenges, and Future Directions

Current ACDC methodologies face the following:

• Scale and Resolution Trade-offs: Edge-centric methods risk overestimating circuit size at block granularity and underestimating at node or neuron levels; fine-grained masking is memory-intensive and introduces hierarchy enforcement requirements (Haider et al., 11 Dec 2025).
• Faithfulness-Grounded Evaluation: Differentiable pruning approaches require new faithfulness/completeness criteria as standard patching can overestimate subcircuit sufficiency (Yu et al., 2024).
• Algorithmic Bottlenecks: Large-LM circuits remain challenging for activation patching and even gradient-based approximations; mixed-precision and federated learning strategies open new avenues for scaling (Wang et al., 27 Oct 2025, Li et al., 20 Jul 2025).
• Model-centric Bias: Automated methods, especially in vision, are sensitive to internal network bias and threshold choices, potentially capturing non-causal correlations (Kwon et al., 3 Aug 2025).
• Extension Beyond Topology: For analog and synthetic biology, current ACDC stops at topology; end-to-end integration of performance- or feedback-guided generation and device-level optimization is an open frontier (Gao et al., 28 Feb 2025).
• Non-polynomial Hardness: Fundamental complexity results indicate no generic polynomial-time solution for minimal circuit search; future work focuses on special cases, efficient approximations, and leveraging domain knowledge (Adolfi et al., 2024).

Future progress in ACDC is anticipated in hybrid methods (e.g., combining pruning with causal mediation), privacy-preserving and cross-institutional generative models, fine-grained causal attribution, and integration with real-world experimental pipelines.

7. Representative Papers and Comparative Summary

Domain	Key Paper(s)	Methodological Innovations
Transformer interpretability	(Conmy et al., 2023, Syed et al., 2023, Haklay et al., 7 Feb 2025, Hsu et al., 2024, Haider et al., 11 Dec 2025)	Edge-patching, EAP, position-aware, CD-T, multi-granular masking
Analog IC topology synthesis	(Gao et al., 28 Feb 2025, Li et al., 20 Jul 2025)	Sequence-based graph encoding, federated generative models
Vision/concept circuits	(Kwon et al., 3 Aug 2025, Rajaram et al., 2024)	Fine-grained DAGs, cross-layer attributions
Synthetic biochemistry	(Lankinen et al., 2020)	Automated compiler, leak-suppressed design
Quantum circuit synthesis	(Potoček et al., 2018)	Multi-objective genetic programming, hybrid optimization
Computational complexity	(Adolfi et al., 2024)	Complexity theory of circuit queries, SAT/QBF transformation
Differentiable pruning	(Yu et al., 2024, Haider et al., 11 Dec 2025)	Joint weight and edge masking, functional faithfulness

These works collectively advance the field by automating, formalizing, and scaling discovery of circuits across diverse scientific and engineering disciplines, providing theory-grounded, practical methodologies for extracting mechanistic insight from complex systems.