Bridging Distant Ideas: the Impact of AI on R&D and Recombinant Innovation

Abstract: We study how AI affects firms' incentives to pursue incremental versus radical knowledge recombinations. We develop a model of recombinant innovation embedded in a Schumpeterian quality-ladder framework, in which innovation arises from recombining ideas across varying distances in a knowledge space. R&D consists of multiple tasks, a fraction of which can be performed by AI. AI facilitates access to distant knowledge domains, but at the same time it also increases the aggregate rate of creative destruction, shortening the monopoly duration that rewards radical innovations. Moreover, excessive reliance on AI may reduce the originality of research and lead to duplication of research efforts. We obtain three main results. First, higher AI productivity encourages more distant recombinations, if the direct facilitation effect is stronger than the indirect effect due to intensified competition from rivals. Second, the effect of increasing the share of AI-automated R&D tasks is non-monotonic: firms initially target more radical innovations, but beyond a threshold of human-AI complementarity, they shift the focus toward incremental innovations. Third, in the limiting case of full automation, the model predicts that optimal recombination distance collapses to zero, suggesting that fully AI-driven research would undermine the very knowledge creation that it seeks to accelerate.

• The paper introduces a formal framework analyzing how AI alters R&D incentives by affecting the cognitive distance between knowledge domains.
• It demonstrates that moderate AI automation fosters radical innovation, whereas excessive automation leads to negative externalities like duplication and reduced originality.
• The analysis reveals a non-monotonic relationship between AI task share and recombinant innovation, stressing the need for balanced human-AI collaboration.

The Impact of AI on Recombinant Innovation and R&D Directionality

Introduction

The paper "Bridging Distant Ideas: the Impact of AI on R&D and Recombinant Innovation" (2604.02189) presents a rigorous formal framework examining how AI influences firms’ incentives to engage in radical versus incremental innovation via knowledge recombination in R&D. By embedding the recombinant innovation process within a Schumpeterian quality-ladder model, the authors introduce a multi-channel analysis of AI’s effect in a competitive setting, characterizing both facilitative and inhibitory impacts of AI on the innovation process. The model incorporates key negative externalities arising from AI adoption, such as the streetlight effect and duplication of research (“stepping-on-toes”), and it explicitly models the creative destruction dynamic resulting from increased innovation rivalry.

Theoretical Framework and Model Structure

The knowledge production process is conceptualized as a graph-based knowledge space, where nodes denote distinct knowledge domains and edge weights encode cognitive distances between fields.

Figure 1: Schematic illustration of the knowledge space, including the introduction of a new field bridging previously distant domains and expanding local recombination possibilities.

AI is modeled as affecting both the probability of successful cross-domain recombinations and the cost structure of R&D. The framework’s task-based production function allocates a fraction $\alpha$ of R&D tasks to AI, characterized by productivity $m$ and complementarity parameter $\kappa$. The “AI power” index $\lambda_{\text{AI}}$ captures the efficiency gains from AI automation, but incorporates potential non-monotonicities resulting from excessive automation and loss of human-AI complementarity.

The innovation arrival process follows a Poisson formulation, with success probabilities decaying exponentially with recombinant distance but increasing in $\lambda_{\text{AI}}$. Successful distant recombinations produce radical innovations, with value functions increasing (but at diminishing marginal rates) in distance. The quality-ladder extension embeds this recombinant search process in a competitive environment where successful innovators enjoy transient monopolies before being displaced by subsequent innovations, aligning the analysis with Schumpeterian creative destruction.

AI as a Double-Edged Instrument in R&D

AI affects innovation incentives through four primary channels:

1. Direct facilitation: AI directly raises the probability of successful distant recombinations, lowering “cognitive distance” and enabling firms to explore more radical innovation spaces.
2. GPT-effect (Indirect expansion): Radical recombinations enabled by AI generate new nodes and links in the knowledge space, creating future opportunities and raising aggregate innovation potential.
3. Acceleration of creative destruction: As AI is available to all firms, the innovation arrival rate increases economy-wide, intensifying competition and reducing expected monopoly durations, which dampens the incentive for high-risk, long-distance innovation.
4. Erosion of originality (streetlight and stepping-on-toes effects): High $\alpha$ leads to more uniform, AI-directed search, concentrating research efforts on already well-explored areas and increasing the probability of duplicative, less novel output, ultimately decreasing the effective innovative power of AI and diminishing the returns to distance.

These forces result in ambiguous and non-monotonic equilibrium comparative statics, which are a core focus of the analysis.

Main Analytical Results

The analysis yields three central results:

1. Impact of AI Productivity: Higher AI productivity ($m$) unambiguously lowers costs and increases the entry of R&D firms. However, its effect on optimal recombinant distance ($d^*$) is ambiguous; while direct gains from facilitation push $d^*$ upward, an increase in the aggregate innovation arrival rate due to new entrants shortens the expected benefit from radical innovations, counteracting this force. When AI capability is moderate, the direct effect dominates.
2. Impact of AI Task Share ($\alpha$): The relationship between the fraction of R&D tasks automated by AI and the optimal target distance is non-monotonic. For low $m$0, increasing automation complements human creativity, facilitating longer-distance, high-value recombinant innovation. Once $m$1 surpasses a threshold ($m$2), further automation leads to reduced complementarity, triggering the streetlight and duplication effects, and causes firms to revert to shorter, safer, incremental recombinations.

Figure 2: Simulated trajectory of the knowledge stock growth rate under different AI price evolutions, illustrating the feedback between automation, innovation intensity, and economic growth.

Figure 3: Comparative statics with respect to $m$3 (fraction of AI-performed tasks), demonstrating the non-monotonic relationship with optimal recombinant distance and number of R&D firms.

1. Full Automation Regime: As $m$4, representing total task automation, and with a sufficiently strong streetlight/stepping-on-toes effect, optimal target distance collapses to zero. This implies that a fully automated, AI-driven research process becomes incapable of generating knowledge expansion through radical recombinant innovation; knowledge growth halts as originality and diversity vanish.

Equilibrium, Growth, and Policy Implications

The paper establishes conditions for existence and uniqueness of a balanced growth path (BGP) in the presence of AI-driven R&D, analyzing the interplay between AI costs, innovation monopoly durations, and labor allocation. Notably, the BGP critically depends on the trajectory of AI costs relative to the knowledge stock: scenarios of falling AI costs expand entry but ultimately dilute monopoly rents and disincentivize radical search, while rapidly rising costs suppress entry, prolong monopolies, and encourage more radical innovations. These findings have policy implications for AI pricing, public provision of generic AI tools, and intellectual property regimes.

Practically, the results indicate the necessity of maintaining intermediate levels of AI automation in R&D—high enough to capture the benefits of distant recombination, but not so high as to suppress originality and exacerbate competitive crowding. The model highlights that policies over-promoting AI adoption without safeguards for human-AI synergy may inadvertently reduce the rate of knowledge expansion.

Theoretically, the paper advances the literature by jointly modeling the competitive displacement effects of AI and their feedback on the incentive for innovation novelty—an aspect absent from prior, non-competitive or partial equilibrium models. The formalization of the knowledge space as a graph structure, while focusing on distances for tractability, opens avenues for further analysis leveraging network-theoretic tools and empirical operationalization via citation and semantic similarity data.

Directions for Future Work

Endogenization of AI Supply: The model assumes competitive AI supply and exogenous price trajectories; introducing oligopolistic or monopolistic AI provision would enable analysis of strategic pricing, platform access control, and their systemic effects on R&D directionality.

Richness of Knowledge Space Topology: The current model simplifies the knowledge graph to pairwise distances; extensions could explore the impact of clustering, network centrality, and dynamic path formation on aggregate innovation rates and structural transformation of scientific fields.

Empirical Validation: The framework motivates measurement programs using patent citation networks, NLP-based knowledge proximity metrics, and co-authorship networks to test model predictions regarding AI’s effects on the distribution of innovation distances.

Conclusion

The paper articulates a nuanced account of AI’s role in shaping the topology and directionality of recombinant innovation within a competitive Schumpeterian framework (2604.02189). AI’s facilitation of distant idea bridging, while powerful, is subject to diminishing returns, externalities, and risks of creativity collapse with excessive automation. The analysis demonstrates the existence of an intermediate optimum for AI integration in R&D, warning against unalloyed enthusiasm for full automation. The results directly inform both the economics of innovation policy and the strategic deployment of AI tools within research-intensive organizations. Future research directions include deepening the relationship between knowledge network structure and innovation dynamics, endogenizing AI provision, and empirically quantifying the thresholds and non-linearities identified by the model.