11 Building the qualitative engine with news analysis and LLMs

This chapter introduces a complementary qualitative engine designed to turn financial news into structured, auditable signals. It argues that markets are driven not only by prices and volumes, but also by narratives, policy shifts, and geopolitical developments that LLMs can help interpret when they are grounded in reliable retrieved context. The core idea is to use large language models as probabilistic research assistants, not as oracle-like fact engines, and to measure how market language evolves over time rather than merely summarizing it.

The chapter emphasizes a pragmatic workflow built around Retrieval-Augmented Generation. It shows how to gather news from multiple public sources with different strengths: a rich, recent feed, a specialized industry source, a headline-focused backfill source, and a deep historical archive. These sources are cleaned, unified, deduplicated, and enriched with metadata so the system can preserve traceability and support time-based filtering. The resulting text is then embedded into vector representations and stored in a vector database, creating a searchable knowledge base that balances current detail with historical breadth.

On top of this retrieval layer, the chapter builds an “LLM Analyst” that analyzes targeted news context and outputs structured JSON signals such as sentiment, supply risk, policy tone, conviction, and key events. A two-step retrieval strategy is used: one pass scans recent headlines for trend context, while a second pass focuses on the most recent content-rich articles for deeper assessment. The chapter closes by stressing that this is best viewed as a disciplined information-processing system whose outputs can later be fused with quantitative features, while noting the practical production concerns of stability, latency, cost, and fallback design.

The Qualitative Engine architecture: end-to-end pipeline from raw news to structured signals. Raw text from APIs and RSS feeds enters the RAG feature engineering pipeline (data cleaning, embedding, vector database storage). When a user query arrives, the system retrieves relevant documents and constructs a structured prompt for the LLM, which outputs quantifiable signals (sentiment, supply risk, policy tone) in JSON format.

Sample outputs from our four-source collection pipeline, showcasing the varying levels of content richness. Panels (NewsAPI) and (World Nuclear News) display content-rich records, providing both a title and a substantive description or summary field. These ~1,600 articles form the semantic core for our deep LLM analysis. In stark contrast, Panels (Google News RSS) and (GDELT) are content-sparse, offering title-only records. This heterogeneity is not a flaw, but a deliberate engineering choice that defines the "information-constrained environment" our subsequent processing must handle intelligently.

RAG pipeline verification results for the query "uranium supply disruption." The output demonstrates that the system correctly retrieves topically relevant articles about supply challenges and shortages, ranked by semantic similarity.

A sample JSON output from the LLM Analyst, demonstrating structured signal extraction. The model has synthesized multiple news snippets into a coherent, machine-readable assessment.

• Qualitative signals extracted from news narratives reveal the why behind market behavior—context that quantitative data alone cannot capture.
• General-purpose LLMs lack institutional context and premium data access, making RAG an essential foundation for grounding analysis in verifiable facts.
• A multi-source pipeline blends public, freely accessible feeds: content-rich APIs (NewsAPI, WNN) for near-term context and headline-only archives (GDELT, Google RSS) for historical trends.
• The RAG feature pipeline standardizes schemas, deduplicates overlapping feeds, embeds text into vectors, and stores them with metadata for transparent, auditable retrieval.
• Because our news snippets are short, skipping chunking preserves context and reduces unnecessary complexity.
• Structured prompt engineering defines a repeatable reasoning process—through explicit roles, boundaries, and JSON schema—to convert qualitative judgment into deterministic signals.
• The “LLM Analyst” produces structured datasets of quantifiable features (Policy Tone, Supply Risk, Novelty), each with clear evidence and reasoning for full auditability.
• The architecture is ensemble-ready, allowing multiple foundation models to act as independent analysts whose outputs can be aggregated for robustness.
• These qualitative signals form a new feature class bridging language and alpha, setting the stage for Chapter 12’s hybrid integration with quantitative features.
• Evals, guardrails, and human oversight lower these risks—they do not remove them.

FAQ