1 Peeking inside the black box

This chapter introduces generative AI as a new kind of application that feels familiar to use but is unusual to build. Unlike traditional software, GenAI systems rely on large language models that accept natural-language instructions and produce outputs probabilistically, which means developers must learn to handle uncertainty rather than fixed, deterministic logic. The chapter also frames the book’s approach: focus on practical building blocks, not deep mathematical detail, and use low-code tools to make experimentation and iteration easier.

The main lesson is that real GenAI applications are more than a direct call to a model. Because LLMs are stateless and pre-trained, developers must add surrounding components that provide conversation history, relevant external knowledge, and other context so each model call has enough information to answer well. Examples like remembering a user’s name or answering with up-to-date office hours show how memory and retrieval make an application feel intelligent, while also motivating later topics such as prompt engineering, tools, agents, and retrieval-augmented generation.

The chapter then opens the black box of LLMs with an intuitive explanation of how models like GPT work. It describes text being broken into tokens, turned into embeddings, processed through large neural networks, and used to predict the next token one step at a time. It also explains the training pipeline: unsupervised learning to learn language patterns, supervised fine-tuning for dialogue, and reinforcement learning from human feedback to improve usefulness. The larger point is that while the underlying model is powerful, it has fixed knowledge and heavy computational requirements, so practical GenAI systems must combine the model with architecture, context, and updating mechanisms to stay useful in the real world.