8 AI for financial research

This chapter argues for a pragmatic, evidence-first way to use AI in investing: treat models as tools that help you reason, not oracles that predict the future. It contrasts discriminative machine learning, which returns structured, quantitative outputs, with generative AI, which synthesizes language and ideas, and stresses that both become valuable when the problem scope is narrow, assumptions are explicit, and results are independently verified. The overarching takeaway is to combine automation with human judgment: let ML structure noisy markets and let LLMs accelerate research, while you maintain rigorous due diligence and risk awareness.

The first half examines ML in finance through practical workflows and cautions. Unsupervised learning can cluster stocks by return and volatility to surface outliers and candidates for deeper review; supervised approaches then label outcomes and search for feature–signal relationships across time frames. A simple price-forecasting demo underscores common pitfalls: efficient markets, random walks, shifting regimes, seasonality, leadership and accounting changes, and bubbles all degrade naïve models. On the technical side, volatility, feature selection (over/underfitting), data scale, sector heterogeneity, and MLDevOps overhead make robust pipelines hard to sustain. The guidance is to narrow objectives (e.g., industry-specific metrics or business-model levers), train multiple models, and favor interpretable, maintainable systems that complement—not replace—fundamental analysis.

The second half pivots to generative AI as a research accelerator with clear guardrails. LLMs can hallucinate, drift, or rely on outdated data, and simple experiments show non-deterministic, sometimes questionable stock suggestions; therefore, use them for tasks like document summarization, code generation, feature ideation, and high-level non-financial risk scans—not blind stock picking. The chapter demonstrates API-based integration with major providers and open-source models, weighing cost, latency, privacy, and hardware trade-offs. It then introduces prompt engineering as an edge: set durable context via an investor profile (goals, risk tolerance, experience, tax and ethical constraints), ask specific, verifiable questions (e.g., management quality or moat drivers), and translate answers into a scorecard you can audit. Used together—ML for structured signals, GenAI for faster insight generation—you can automate parts of the research loop while keeping decisions disciplined and explainable.

Shows one-year price forecasts for Nvidia based on data collected via Yahoo Finance, divided into high, low, and mean.

Elbow curve indicating the number of clusters for one year of data until May 13th, 2025.

A plot with four clusters, hovering with the mouse over the single dots would reveal each ticker.

A labeled dataset for which we need to select features. Each feature would represent one column.

Illustrates the difference between actual and forecasted prices, showing that in some cases, the predictions are more accurate than others.

Price development of Cosmos (ATOM) between 2021 and 2023.

The MLDevOps lifecycle, including preparation and deployment

Shows a highly simplified model reduced to the most central figures that Netflix is profitable if revenues from paying subscribers exceed production and licensing costs. While, of course, many more numbers exist that influence accounting sheets, such illustrations help us to extract these numbers that can be most decisive about a specific company’s long-term success.

• We can integrate machine learning algorithms as code snippets into web notebooks to aid us in financial research, such as forecasting stock price developments, including uptrends.
• Unsupervised learning may help us identify outliers, which, in the investment scenario, could be companies that perform differently from the rest and present an investment opportunity.
• We can use unsupervised learning to find potential labels for supervised learning algorithms.
• Supervised learning can help us forecast a company's future success by analyzing its historical data.
• Successfully applying supervised learning algorithms can require a high level of technical expertise, encompassing everything from collecting data from multiple sources to processing algorithms at scale.
• As the market is affected by many factors (fundamental data, technical data, or other data unrelated to a specific company), most models risk being either over- or underfitted.
• The efficient market theory suggests that for most assets, the market price accurately reflects the asset's intrinsic value.
• The random market theory and the allegory of Mr. Market suggest that a market is unpredictable. This makes machine learning models challenging.
• Technological and social changes affect investment strategies. For example, ESG regulations are more important to investors today than they were in the past.
• Seasonal and economic trends impact the performance of stocks and must be considered in machine learning models.
• Different accounting practices within companies make it more difficult to compare companies.
• To integrate machine learning, we must invest considerable effort in identifying specific industry patterns and retraining models. Only a fraction of hedge funds run largely successful machine learning use cases.
• LLMs are game-changers in natural language processing. They employ a new paradigm called the attention mechanism, which focuses solely on the relationship between keywords. This makes it possible to create models trained on billions of parameters and serve as chatbots for users to address generic topics.
• LLMs can be trained to answer specific questions related to topics, including finance and investment.
• LLMs are also excellent for providing an overview of how specific non-financial risks can impact individual companies. Just be aware that you might have to review the output due to possible hallucinations.
• Users can choose from various LLMs, and an API for sending queries to an LLM is easy to integrate into Python code. In most cases, all users require an API key to authenticate against the provider and obtain authorization to process LLM inference on the LLM provider's infrastructure.
• Many LLMs come in versions that differ in complexity. Some may require a paid subscription, while others are designed for environments with fewer computing resources, such as mobile phones.
• We can classify LLMs as open-source or proprietary. We can download models of open-source LLMs to a compute node or local workstation to run the inference there. Proprietary LLMs are executed on the provider's infrastructure.
• AI ethics professionals are concerned about proprietary large language models (LLMs), as providers could misuse the information provided to them through users' questions. Additionally, the transparency of how these providers collect information is a significant ethical concern.
• Whether an LLM is open-source or proprietary, it can provide us with investment ideas. Betting on LLM investment advice without further investigation is highly speculative.
• LLMs are great for tasks that require them to act like research assistants. When investors require less time for research, they can devote more attention to decision-making processes.
• The code generation capabilities of LLMs enable programmers to write more efficient code and analyze investments in various scenarios using programming languages like Python.
• Although we can use LLMs to generate investment ideas, their core asset is helping us with due diligence on existing ideas, specifically when we need to find details on possible investments to create a personal scorecard for informed purchase decisions.
• Although LLMs help us speed up finance research tasks, they occasionally make mistakes and provide false information, often called hallucinations.
• The question is not whether to use discriminative AI(traditional machine learning) or generative AI; it is rather how to pragmatically use what is there for specific scenarios and combine the possibilities of each technology.
• Prompt engineering is a discipline that enhances the output of LLMs by enabling users to ask more effective questions. It provides ways to pass more details, reference examples, and additional context for a request to an LLM.
• Prompt engineering cannot replace common wisdom, as it depends on us to clarify what we want to know to avoid generic questions in an LLM.
• Creating an investor profile that includes information such as goals, risk tolerance, and tax residency can help an LLM provide more refined answers.
• Be as specific as possible and give as much context as possible when submitting prompts. It helps to provide reference examples.

FAQ