3 Using Hugging Face Transformers and Pipelines for NLP Tasks
This chapter offers a practical tour from the core ideas of Transformers to hands‑on usage for NLP. It first builds intuition for the Transformer architecture—why self‑attention matters, how encoder and decoder stacks work, and when to use encoder‑only, decoder‑only, or encoder‑decoder designs. Foundational concepts such as tokenization strategies (word, subword, character), token embeddings, positional encodings, and the role of residual connections, layer normalization, and the final Softmax are introduced to explain how modern models process sequences efficiently and capture long‑range dependencies without recurrence.
With that grounding, the focus shifts to the Hugging Face Transformers ecosystem. The chapter explains pre‑trained models (BERT, GPT, RoBERTa, DistilBERT, T5, and others) and shows two ways to use them: directly with Auto classes for fine‑grained control, or via the high‑level pipeline API for simplicity and rapid prototyping. Readers learn how pipelines abstract tokenization, inference, and post‑processing, how to pick explicit models for reproducibility, how caching works behind the scenes, and how to target CPU or GPU devices. A sentiment analysis walkthrough with DistilBERT illustrates the contrast between manual model invocation (reading logits and mapping labels) and the streamlined pipeline approach.
The chapter culminates in a survey of common NLP tasks implemented with pipelines and representative models: text classification beyond sentiment (question detection, language identification, spam detection), text generation with GPT‑2 (controlling length and number of outputs), summarization with BART (contrasting deterministic vs sampled generation), and translation using T5 and M2M100 with task naming conventions for language pairs. It also demonstrates zero‑shot classification for text (XNLI‑based models with candidate labels and optional multi‑label scoring) and images (CLIP), and closes with extractive question answering using RoBERTa fine‑tuned on SQuAD 2.0. Throughout, the emphasis is on concise, reproducible code patterns and model selection tips that let you move quickly from idea to working NLP solutions.
The Transformer Architecture (source: Attention Is All You Need https://arxiv.org/pdf/1706.03762.pdf)
A transformer model contains of an Encoder and a Decoder
How a sentence is tokenized into three tokens using the word tokenization technique
How a sentence is tokenized into three tokens using the subword tokenization technique
How a sentence is tokenized into three tokens using the character-level tokenization technique
The visualization of token embeddings for the various words in a paragraph in two-dimensional space
Using Softmax function to generate a set of probabilities based on the values of the output layer of a neural network
The role of the Hugging Face Transformers library
Pipelines are high-level APIs that utilize the transformer models provided by the transformers library
Learning how to use the model using the Transformers library
Two ways to use the pre-trained model – through the pipeline, or use the model directly
The directory containing the downloaded model
The confidence score for each class
Finding the type of task a model falls under
Finding the type of task a model falls under
The result of the zero-shot classification
Results of the one-shot classification for two paragraphs
An image of an airplane (source: https://en.wikipedia.org/wiki/Aircraft#/media/File:Emirates_Airbus_A380-861_A6-EER_MUC_2015_04.jpg)
Result of the zero-shot image classification
The result showing that the meaning of Singapura is lion city
Summary
- At a very high level, the transformer architecture consists of two main blocks – Encoder and Decoder.
- The self-attention mechanism is a crucial component of the transformer architecture. It enables the model to weigh the importance of different words in a sequence when encoding a particular word, allowing the model to capture dependencies and relationships between words irrespective of their distance in the sequence.
- A "token" is a chunk of text that a model processes as a single unit.
- Token embeddings convert tokens into numerical vectors. These embeddings capture semantic and syntactic information about the tokens, enabling machine learning models to understand the underlying meaning and relationships between words in natural language text.
- Positional encodings provide information about the positions of words within a sequence. It allows models to understand the order of words in a sentence.
- Softmax is a mathematical function that converts a vector of numbers into a probability distribution, where the probability of each element is proportional to the exponentiation of that element's value relative to the sum of all the exponentiated values in the vector.
- The Hugging Face Transformers is an open-source library and platform developed by Hugging Face. The aim of this library is to provide an easy-to-access interface for working with state-of-the-art transformer-based models.
- The pipeline function provides a wrapper on top of the transformers library, providing convenient access to the various pre-trained models.
- Zero-shot classification is the task of classifying previously unseen classes during training of a model, while one-shot classification refers to the technique where a model is trained to recognize classes with only one example (or just a few examples) per class during training.
FAQ
What’s the difference between “Transformers” and “Transformer”?
“Transformers” refers to Hugging Face’s open‑source library that provides pre‑trained transformer models and easy‑to‑use tools. “Transformer” is the neural network architecture introduced in “Attention Is All You Need” (Vaswani et al., 2017) that underpins many modern NLP models.Which tasks fit encoder‑only, decoder‑only, and encoder–decoder transformer models?
- Encoder‑only (e.g., BERT): understanding tasks like text/sentence classification, NER, feature extraction- Decoder‑only (e.g., GPT): generation tasks like text generation/auto‑regressive modeling
- Encoder–decoder (e.g., T5, BART): sequence‑to‑sequence tasks like summarization and translation
What is self‑attention and why is it important?
Self‑attention lets each token attend to every other token in the sequence, learning dependencies regardless of distance. It enables parallel processing (faster than RNNs/LSTMs), captures long‑range relationships, and builds rich, context‑aware token representations. Multi‑head attention does this in multiple subspaces for better expressiveness.What is tokenization, and why do BERT tokens sometimes start with “##”?
Tokenization splits text into model‑friendly units (tokens). Common strategies:- Word tokenization: splits by spaces/punctuation
- Subword tokenization: splits words into meaningful pieces (e.g., prefixes/suffixes); balances vocabulary size and coverage
- Character tokenization: splits into characters; helpful for complex scripts
In BERT’s WordPiece, a “##” prefix marks a subword continuation (e.g., “un”, “##hap”, “##pi”, “##ness”). On decoding, these merge back into full words.
What are token embeddings and positional encodings, and why do we need both?
- Token embeddings map tokens to dense vectors that capture meaning and usage patterns (semantics/syntax).- Positional encodings inject order information, since self‑attention itself is permutation‑invariant. They’re added to token embeddings before the transformer layers. Many models (e.g., BERT) include learned positional embeddings by default.
What does the Hugging Face pipeline() do, and when should I use it vs. using a model directly?
The pipeline() function is a high‑level API that hides tokenization, model loading, and post‑processing, so you can run tasks with a few lines of code. Use it for simplicity and rapid prototyping. Use models directly when you need fine‑grained control over preprocessing, inference, hyperparameters, or custom post‑processing.How do I run a quick sentiment analysis with a pipeline and what output should I expect?
Create a text‑classification pipeline with a sentiment model, then pass text:-
pipeline("text-classification", model="distilbert/distilbert-base-uncased-finetuned-sst-2-english")("I loved the movie!")Typical output:
[{"label": "POSITIVE", "score": 0.9998}] indicating the predicted label and confidence score.How do I choose supported pipeline tasks and pick models (and use a GPU)?
- Discover tasks/models on the Hub or by attemptingpipeline("dummy") to see the supported task list in the error message.- Specify the task (e.g.,
"summarization", "translation_XX_to_YY", "question-answering") and a model for reproducibility (don’t rely on defaults).- Run on GPU with
pipeline(..., device="cuda"); default is CPU (device="cpu").- One‑liners for common tasks:
• Classification:
pipeline("text-classification", model=...)• Generation:
pipeline("text-generation", model=...)• Summarization:
pipeline("summarization", model="facebook/bart-large-cnn")• Translation:
pipeline("translation_en_to_fr", model="google-t5/t5-base")• QA:
pipeline("question-answering", model="deepset/roberta-base-squad2")What is zero‑shot classification and when should I use it?
Zero‑shot classification lets you assign labels that the model never saw during training. You provide candidate labels at inference time (e.g., “technology”, “politics”, “romance”) and the model scores them for your text. It’s useful when classes change often or labeled data is unavailable. Many zero‑shot text models are trained via NLI. (Contrast: one‑shot learns a class from a single example.)How do logits become probabilities, and how do I read a two‑class output?
A model’s raw outputs (logits) are converted to probabilities with Softmax, which exponentiates and normalizes them so they sum to 1. For binary classification with logits like[-4.34, 4.70], Softmax assigns a near‑zero probability to the first class and a high probability to the second. You can pick the class with argmax and interpret the Softmax score as confidence.
Hugging Face in Action ebook for free
