Towards the Terminator Economy: Assessing Job Exposure to AI through LLMs

Abstract: AI and related technologies are reshaping jobs and tasks, either by automating or augmenting human skills in the workplace. Many researchers have been working on estimating if and to what extent jobs and tasks are exposed to the risk of being automatized by AI-related technologies. Our work tackles this issue through a data-driven approach by: (i) developing a reproducible framework that uses cutting-edge open-source LLMs to assess the current capabilities of AI and robotics in performing job-related tasks; (ii) formalizing and computing a measure of AI exposure by occupation, the Task Exposure to AI (TEAI) index, and a measure of Task Replacement by AI (TRAI), both validated through a human user evaluation and compared with the state of the art. Our results show that the TEAI index is positively correlated with cognitive, problem-solving and management skills, while it is negatively correlated with social skills. Applying the index to the US, we obtain that about one-third of US employment is highly exposed to AI, primarily in high-skill jobs requiring a graduate or postgraduate level of education. We also find that AI exposure is positively associated with both employment and wage growth in 2003-2023, suggesting that AI has an overall positive effect on productivity. Considering specifically the TRAI index, we find that even in high-skill occupations, AI exhibits high variability in task substitution, suggesting that AI and humans complement each other within the same occupation, while the allocation of tasks within occupations is likely to change. All results, models, and code are freely available online to allow the community to reproduce our results, compare outcomes, and use our work as a benchmark to monitor AI's progress over time.

• The paper introduces the TEAI index, a novel method using LLMs to measure the automation potential of job tasks.
• It finds that roughly one-third of US jobs, particularly high-skill positions, face significant AI exposure.
• Reproducible findings support policy and industry strategies to balance AI innovation with workforce development.

Assessing Job Exposure to AI through LLMs

The paper "Towards the Terminator Economy: Assessing Job Exposure to AI through LLMs" offers a structured framework for evaluating the risk that AI poses to various occupations. This evaluation focuses on the extent to which current AI and robotic technologies can automate job-related tasks. At its core, this work formulates the TEAI (Task Exposure to AI) index and employs a systematic approach using LLMs to analyze job-task exposure data comprehensively.

The authors innovatively develop an internal assessment methodology by using LLMs to analyze O*NET's extensive database of occupational tasks. Traditionally, AI exposure has been assessed through expert judgments or patent analysis, but this methodology relies directly on the performance evaluations by the LLMs themselves. The study concludes that about one-third of U.S. jobs are highly exposed to AI, focusing especially on high-skill occupations.

Key Methodological Contributions

1. Internal Assessment via LLMs: The paper introduces an approach wherein open-source LLMs evaluate tasks based on their ability to automate them. LLMs provide a task rating, and the evaluation involves a consensus system to mitigate model variance or hallucinations—a known issue with LLMs.
2. Granular Task Analysis: The authors create the TEAI index by normalizing the AI's ability to perform specific tasks across occupations. This measure is aligned with task relevance, importance, and frequency derived from the O*NET database.
3. Transparency and Reproducibility: A significant contribution lies in the reproducibility of the results, made possible by publicly available codes and data sets. This facilitates benchmarking advancements in AI over time.

Analysis of Findings

The TEAI index contrasts with existing measures, showing high AI exposure among occupations traditionally seen as less routine. The authors find that high-skill, cognitive-focused jobs bear the greatest exposure to AI, diverging from earlier works that emphasized more routine tasks at risk. Moreover, occupations rich in cognitive, problem-solving, and management skills have higher AI exposure indices, whereas roles needing social skills are less exposed.

Surprisingly, the study reveals a positive correlation between AI exposure and employment and wage growth from 2003 to 2023. This contradicts the notion that AI necessarily displaces human labor, suggesting instead that AI may complement human work, driving productivity gains and wage increments.

Implications

The implications of these findings are manifold:

• For Researchers: The methodology can be further refined to separate substitutive effects from complementary effects in AI-human task performance. This distinction is crucial for accurately predicting labor market transformations driven by AI advancements.
• For Policymakers: Understanding AI’s differential impacts across skill demographics informs workforce development policies. Strategies to leverage AI for productivity while mitigating labor disruptions can be crafted more effectively.
• For Industry: The analysis indicates sectors where AI adoption could yield productivity benefits. Industries could use this insight to target skill development initiatives aimed at maximizing the symbiotic potential of AI in human tasks.

Future Research Directions

Looking ahead, future research could build upon this work by incorporating more nuanced AI capabilities assessments, potentially using advanced LLMs to track progress continually. Furthermore, a deeper investigation into the socio-economic impacts of AI in high-exposure jobs could offer richer insights into how AI-driven task automation may shape the labor market dynamics dynamically over longer periods.

The methodology and findings presented in this paper offer a robust framework for assessing AI's current and expected job market impacts, providing both a benchmark and a research toolset for investigating future AI capabilities and their economic implications.