Nicole Koenigstein

Transformers in Action guides you through the design and operation of transformers and transformer-based models. You’ll dive immediately into LLM architecture, with even the most complex concepts explained clearly through easy-to-understand examples and clever analogies. Because transformers are based in mathematics, author Nicole Koenigstein carefully guides you through the foundational formulas and concepts one step at a time. You’ll also appreciate the extensive code repository that lets you instantly start exploring LLMs hands-on.

As you go, you learn how and when to use different model architectures such as decoder-only, encoder-only, and encoder-decoder. You’ll also discover when small language models make sense for specific tasks like classification, in resource-constrained environments, or when privacy is paramount. You’ll push transformers further with tasks like refining text generation with reinforcement learning, developing multimodal models including building multimodal RAG pipelines, and fine-tuning. You’ll even learn how to optimize LLMs to maximize efficiency and minimize cost.

Put on your data scientist hat for this series of liveProjects, where you’ll work at Finative, an analytics company that uses environmental, social, and governance (ESG) factors to measure companies’ sustainability, a brand new, eco-focused trend that's changing the way businesses think about investing. In each liveProject, you’ll focus on different machine learning (ML) and deep learning (DL) mathematical approaches—including Bayes' theorem, principal component analysis (PCA), cosine similarity, latent semantic analysis, and backpropagation—as you help Finative accomplish its goal of increasing its own sustainability.

You’ll develop a method to reduce the runtime of ML models, and you’ll save digital storage space by finding relevant keywords in order to determine whether documents should be discarded or saved. To increase efficiency, you’ll save training time by using a pre-trained language model to classify a sustainability report. Then, you’ll analyze the sentiment of tweets in order to detect greenwashing, the practice of spreading disinformation about a company’s sustainability. When you’re finished with these liveProjects, you’ll have a solid understanding of the mathematical basics of machine learning, strong programming and data science skills, and familiarity with sustainability.

Finative, the environmental, social, and governance (ESG) analytics company you work for, analyzes a high volume of data using advanced natural language processing (NLP) techniques to provide its clients with valuable insights about their sustainability. Your CEO has concerns that some of the companies Finative analyzes may be greenwashing: spreading disinformation about their sustainability in order to appear more environmentally conscious than they actually are.

As a data scientist for Finative, your task is to validate your sustainability reports by creating and analyzing them. You’ll compute conditional probability with Bayes’ Theorem, by hand, to better understand your model’s performance through metrics such as recall and precision. You’ll learn an efficient way to prepare your data from different sources and merge it into one dataset, which you’ll use to prepare tweets. To successfully classify the tweets, you’ll use a pre-trained large language model and fine-tune it using the Hugging Face ecosystem as well as hyperopt and Ray Tune. You’ll use TensorBoard and Weights & Biases to analyze and track your experiments, and you’ll analyze the tweets to determine whether enough negative sentiment exists to indicate that the company you analyzed has been greenwashing its data.

You’re a data scientist at Finative, an environmental, social, and governance (ESG) analytics company that analyzes a high volume of data using advanced natural language processing (NLP) techniques in order to provide its clients insights for sustainable investing. Recently, your CEO has decided that Finative should increase its own financial sustainability. Your task is to classify sustainability reports of a publicly traded company in an efficient and sustainable way.

You’ll learn the fundamental mathematics—including backpropagation, matrix multiplication, and attention mechanisms—of Transformers, empowering you to optimize your model’s performance, improve its efficiency, and handle undesirable model predictions. You’ll use Python’s pdfplumber library to extract text from a sustainability report for quick delivery to your CEO. To further increase efficiency, you’ll save training time by using a language model that’s been pre-trained with ESG data to build a pipeline for the model and classify the sustainability report.

At Finative, an ESG analytics company, you’re a data scientist who helps measure the sustainability of publicly traded companies by analyzing environmental, social, and governance (ESG) factors so Finative can report back to its clients. Recently, the CEO has decided that Finative should increase its own sustainability. You’ve been assigned the task of saving digital storage space by storing only relevant data. You’ll test different methods—including keyword retrieval with TD-IDF, computing cosine similarity, and latent semantic analysis—to find relevant keywords in documents and determine whether the documents should be discarded or saved for use in training your ML models.

Step into the role of data scientist at Finative, an analytics company that uses environmental, social, and governance (ESG) factors to measure companies’ sustainability, a brand new, eco-focused trend that's changing the way businesses think about investing. To provide its clients with the valuable insights they need in order to develop their investment strategies, Finative analyzes a high volume of data using advanced natural language processing (NLP) techniques.

Recently, your CEO has decided that Finative should increase its own sustainability. Your task is to develop a method to optimize the runtime for the company’s machine learning models. You’ll apply principal component analysis (PCA) to the data in order to speed up the ML models. To classify handwritten digits and prove your theory that PCA speeds up ML algorithms, you’ll implement logistic regression with scikit-learn. You’ll use the explained variance ratio to gain an understanding of the trade-offs between speed and accuracy. When you’re done, you’ll be able to present your CEO with proof of PCA’s efficiency in optimizing runtime.