Overview

Appendix C. Dataset and feature dictionaries

The appendix brings together the data foundations behind the book’s AI systems, emphasizing that each financial use case depends on a different kind of source and schema. It explains that credit risk models rely on large anonymized application data, fraud detection uses high-velocity transaction records, market intelligence combines pricing data with news and policy text, and personalization models are built from synthetic behavioral logs designed to preserve privacy while still capturing realistic user patterns.

For credit risk, the dataset is the AMEX Default Prediction collection, sampled down for efficiency, with features centered on customer identifiers, the binary default target, aggregated payment and delinquency measures, and anonymized categorical account-state variables. For fraud detection, the core schema includes the class label, transaction time, amount, and PCA-transformed variables that protect sensitive details while supporting supervised anomaly and fraud modeling. Together, these dictionaries show how the book structures predictive features for classification, stability, and risk scoring tasks.

The market and synthetic banking sections extend the same logic to more dynamic and behavior-driven settings. Market features blend ticker identity, adjusted prices, volatility, news source metadata, and LLM-derived sentiment or supply-risk scores to support hybrid quantitative and qualitative investing engines. The synthetic retail banking dataset uses behavioral ratios, wealth and volatility measures, clustering labels, and churn probabilities to power personalization, segmentation, and retention workflows, showing how engineered features can translate raw activity into actionable financial intelligence.

FAQ

What is the purpose of Appendix C in Financial AI in Practice?Appendix C provides the data dictionaries and schemas for the core features and engineered variables used across the book’s modeling pipelines, helping readers understand the underlying data structures behind each AI system.
Why did the book use different datasets for credit risk, fraud detection, market intelligence, and personalization?Each problem domain required a different kind of data: Kaggle application data for credit risk, high-velocity transaction data for fraud detection, public market and news APIs for investment models, and a custom synthetic dataset for personalization and retention use cases.
What dataset was used for the credit risk models in Chapters 4 through 6?The credit risk pipelines used the AMEX Default Prediction dataset from Kaggle, and because the original dataset was very large, the book sampled 100,000 rows for memory-efficient modeling.
What do the core credit risk features like customer_ID and target represent?customer_ID is an anonymized unique identifier used for entity resolution and data joining, while target is the binary label indicating default (1) or non-default (0).
What are examples of engineered credit risk features in Table C.1?Examples include aggregated numeric variables such as P_2_mean and D_39_max, which summarize payment, spend, and delinquency history, along with categorical account-state features such as B_30, D_114, and D_116.
What dataset was used for fraud detection in Chapters 7 through 9?The fraud models used the Credit Card Fraud Detection dataset from Kaggle, which contains 284,807 transactions and only 492 fraud cases, making it a severely imbalanced classification problem.
Why are the fraud detection features V1 to V28 important?V1 through V28 are PCA-transformed float features that anonymize sensitive transaction details such as merchants or locations, while still supporting supervised fraud modeling.
What market data and news sources were used for the investment engine in Chapters 10 through 12?The investment engine combined structured ETF price data with unstructured news narratives, using public APIs such as Yahoo Finance, NewsAPI, and GDELT.
What do features like adj_close, volatility_30d, article_source, and policy_tone_score mean?adj_close is the split- and dividend-adjusted daily closing price, volatility_30d is the rolling 30-day return standard deviation, article_source identifies the news publisher, and policy_tone_score captures LLM-generated hawkish or dovish sentiment.
Why did the book generate a synthetic retail banking dataset for personalization and retention?Public datasets usually lack detailed sequential interaction logs needed for behavioral modeling, so the authors created a synthetic 100,000-user dataset to simulate app behavior, compute “Financial DNA” vectors, identify tribes, and support privacy-safe retention modeling.

pro $24.99 per month

  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose one free eBook per month to keep
  • exclusive 50% discount on all purchases
  • renews monthly, pause or cancel renewal anytime

lite $19.99 per month

  • access to all Manning books, including MEAPs!

team

5, 10 or 20 seats+ for your team - learn more


choose your plan

team

monthly
annual
$49.99
$499.99
only $41.67 per month
  • five seats for your team
  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose another free product every time you renew
  • choose twelve free products per year
  • exclusive 50% discount on all purchases
  • renews monthly, pause or cancel renewal anytime
  • renews annually, pause or cancel renewal anytime
  • Financial AI in Practice ebook for free
choose your plan

team

monthly
annual
$49.99
$499.99
only $41.67 per month
  • five seats for your team
  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose another free product every time you renew
  • choose twelve free products per year
  • exclusive 50% discount on all purchases
  • renews monthly, pause or cancel renewal anytime
  • renews annually, pause or cancel renewal anytime
  • Financial AI in Practice ebook for free