From Transcripts to Insights: Uncovering Corporate Risks Using Generative AI

Abstract: We explore the value of generative AI tools, such as ChatGPT, in helping investors uncover dimensions of corporate risk. We develop and validate firm-level measures of risk exposure to political, climate, and AI-related risks. Using the GPT 3.5 model to generate risk summaries and assessments from the context provided by earnings call transcripts, we show that GPT-based measures possess significant information content and outperform the existing risk measures in predicting (abnormal) firm-level volatility and firms' choices such as investment and innovation. Importantly, information in risk assessments dominates that in risk summaries, establishing the value of general AI knowledge. We also find that generative AI is effective at detecting emerging risks, such as AI risk, which has soared in recent quarters. Our measures perform well both within and outside the GPT's training window and are priced in equity markets. Taken together, an AI-based approach to risk measurement provides useful insights to users of corporate disclosures at a low cost.

• The paper introduces a GPT-3.5 Turbo approach to analyze earnings call transcripts, quantifying political, climate, and AI risks at the firm level.
• It demonstrates that AI-generated risk assessments outperform traditional methods in predicting stock volatility and influencing investment decisions.
• Moreover, the study reveals that distinct risk measures are linked to proactive firm responses such as lobbying and patent filings, impacting market pricing.

Corporate Risk Assessment via Generative AI

This essay discusses the paper "From Transcripts to Insights: Uncovering Corporate Risks Using Generative AI" (2310.17721), which explores the application of generative AI, specifically LLMs like ChatGPT, in identifying and quantifying corporate risks. The authors develop firm-level measures of political, climate, and AI-related risks by analyzing earnings call transcripts with GPT-3.5 Turbo. The study demonstrates that these AI-driven risk measures contain significant information and outperform traditional methods in predicting firm volatility and strategic choices.

LLMs for Risk Assessment

The paper posits that LLMs offer unique advantages over traditional textual analysis methods, such as dictionary-based approaches, for corporate risk assessment. LLMs can understand complex relationships within text, incorporate context, and make inferences, which is critical for nuanced risk analysis. Moreover, their general AI nature allows them to leverage knowledge beyond the immediate text, incorporating broader economic and regulatory contexts. The ability of LLMs to synthesize information into coherent narratives provides both quantitative assessments and supporting explanations.

The authors highlight the transformer architecture [Vaswani et al., 2017] as crucial for LLMs' efficacy. This architecture enables the model to focus on relevant words and sentences across the entire text, unlike dictionary-based algorithms that rely on a limited "attention" window around specific keywords. The pre-training of LLMs on vast datasets equips them with extensive general knowledge, which acts as a "prior" when interpreting new text and assessing risks.

Methodology and Implementation

The methodology involves using OpenAI's GPT3.5-Turbo to analyze earnings call transcripts from January 2018 to March 2023. Given the token limits of GPT, the transcripts are divided into chunks, preserving speaker and question-answer coherence. Two types of outputs are generated: risk summaries, which focus solely on document content, and risk assessments, which integrate context with general knowledge to make judgments.

The risk exposures, $RiskSum$ for summaries and $RiskAssess$ for assessments, are calculated as:

$$RiskSum_{it} = \frac{\sum_{l=1}^{K_{it}} len(S(c_{it}^l))}{len(c_{it})}$$

$$RiskAssess_{it} = \frac{\sum_{l=1}^{K_{it}} len(A(c_{it}^l))}{len(c_{it})}$$

where $c_{it}$ is the transcript for company $i$ in quarter $t$, divided into $K_{it}$ chunks, $S(\cdot)$ is the GPT-based risk summary function, and $A(\cdot)$ is the risk assessment function. The temperature parameter is set to zero to ensure replicability.

Figure 1: Time Trend in SK Telecom's Political Risk Exposure.

Separate prompts are designed to capture political, climate, and AI-related risks. Political risk assessments include explanations of regulatory uncertainties; climate risk assessments encompass the impact of extreme weather and environmental policies; and AI risk assessments discuss the potential replacement or augmentation of firm operations by AI.

Data and Variables

The study uses quarterly earnings call transcripts from Capital IQ S{content}P Global Transcript database for US firms. The sample is split into a baseline period (2018-2021) and a post-GPT-training period (2022-2023). Capital market variables include implied volatility from OptionMetrics and abnormal volatility, measured using root mean squared errors (RMSE) from market model residuals following \citet{loughran2014measuring}. Economic variables include capital investments, lobbying activity from the Center for Responsive Politics, and green and AI-related patents from the USPTO.

Descriptive Statistics and Validation

Descriptive statistics reveal that risk exposure assessments are longer than summaries, indicating more detailed content. The tobacco industry shows the highest political risk, coal mining the highest climate risk, and business services the highest AI risk. Time series analysis indicates that political and climate risks spiked in 2020, coinciding with the COVID-19 pandemic and the US Presidential Election. AI-related risks have increased sharply in recent years, reflecting the emergence of AI technologies.

Figure 2: Measuring Risks with Generative AI.

Variance decomposition analysis reveals that firm-specific factors account for the bulk of variation in risk exposures, emphasizing the importance of firm-level risk measurement. The paper also presents examples of GPT-generated summaries and assessments, demonstrating their readability and relevance.

Capital Market Consequences

The study examines the relationship between GPT-based risk measures and stock price volatility, estimating the following OLS regression:

$$Volatility_{it+1} = \beta Risk_{it} + \gamma X_{it} + \delta_x + \varepsilon_{it}$$

where $Volatility_{it+1}$ is a volatility proxy, $Risk_{it}$ is the risk exposure measure, $X_{it}$ is a vector of controls, and $\delta_x$ represents fixed effects.

Results indicate a strong positive association between GPT-based political and climate risk measures and stock price volatility. Risk assessments consistently outperform risk summaries, highlighting the value of AI-generated insights. Even in the out-of-sample period (2022-2023), GPT-based risk measures remain predictive of volatility. AI-related risk becomes significant in explaining volatility in the most recent years, aligning with the increasing prominence of AI technologies.

Risk Exposure and Firm Decisions

The paper explores how risk exposures influence firms' economic decisions, focusing on capital investments and risk mitigation activities. Theoretical considerations suggest that higher risk exposure increases the cost of capital, leading to decreased investments. The study finds a negative association between political and climate risk exposures and investment, while AI risk exhibits a positive association in the recent period, reflecting investments in AI-related technologies.

Firms also proactively mitigate risks through lobbying, green patent filings, and AI-related patent filings. A positive association is found between political risk and lobbying, climate risk and green patents, and AI risk and AI patents, supporting the notion that firms respond to risks by taking mitigating actions.

Figure 3A: Industry Averages of PRiskAssess.

Figure 4C: Time Trend of AIRiskAssess.

Equity Market Pricing

The asset pricing implications of GPT-based risk measures are examined using \citet{fama1973risk} regressions and portfolio analysis. Results suggest that environmental and AI risks command significant equity risk premia. The study constructs quintile portfolios based on annualized firm-level risk exposure measures and finds that high-minus-low portfolio alphas are positive and statistically significant for climate and AI risks, indicating that these risks are priced in equity markets.

Figure 5C: Word Cloud of AIRiskAssess.

Figure 6B: Regressions with All Three Risks.

Conclusion

The study concludes that generative AI technology offers a valuable tool for assessing corporate risks, providing insights that outperform traditional methods. The findings highlight the economic usefulness of LLMs in distilling disclosures and extracting information about diverse risk categories. The authors acknowledge limitations, such as the sensitivity of GPT outcomes to prompt quality and the potential for hallucinations, recommending that users reconcile GPT's answers with source documents. Despite these limitations, the study demonstrates the potential of generative AI in enhancing risk assessment and informing investment decisions.