12 Terraform Providers
This chapter presents a practical, end-to-end guide to building Terraform providers with the Terraform Plugin Framework (Protocol v6) in Go. It explains when creating a provider is worthwhile, the importance of upfront interface design (provider, data sources, resources, and functions), and how to prioritize user experience, stability, and compatibility. Readers are walked through bootstrapping with the official scaffolding template, configuring a local development setup and dev overrides, and adopting consistent patterns for schemas, diagnostics, and debugging. Clear testing guidance distinguishes unit from acceptance tests, and the chapter closes with how to document, sign, and publish a provider so it can be consumed from registries.
The implementation centers on a concrete Mastodon example that demonstrates the core provider interfaces and patterns. You define a provider model and schema, accept configuration via both blocks and environment variables, handle unknown and null values, and use framework diagnostics and tflog for helpful errors and secure logging. The provider’s Configure function builds and authenticates a reusable client, shared with data sources and resources. A data source uses a simple Schema/Read flow to look up an account and populate state. A resource implements full CRUD for posts, including plan and state handling, plan modifiers, and careful normalization (for example, stripping server-added HTML) to avoid drift. The chapter also shows how to add a lightweight function, including its definition, parameters, return type, and a minimal Run implementation.
Development best practices and tooling are emphasized throughout. The scaffolding template accelerates setup while encouraging consistent structure, code generation, and documentation derived from schemas. Local testing is enabled via developer overrides, and the testing stack illustrates provider factories, prechecks, single-step data source tests, multi-step resource tests (create, import, update), and unit tests for functions, including null and unknown cases. To publish, you generate and manage a GPG key, register signing keys, and use automated release workflows to build, sign, and distribute multi-architecture artifacts. Altogether, the chapter equips you to design provider interfaces responsibly, implement them with robust diagnostics and logging, validate them with reliable tests, and deliver them with reproducible documentation and releases.
Terraform Mastodon Provider Flow
Summary
- Creating a provider is not something that most Terraform developers will need to do.
- Terraform providers are written in Go and require that you have a developer environment setup to support that language.
- Data Sources, Resources, and Functions can be created with the Terraform Plugin Framework.
- Rather than start from scratch you can use the Terraform Provider Scaffolding template available from Hashicorp.
- Building a provider is a process of implementing interfaces in Go, and can be surprisingly simple.
- Once finished you can publish your provider to a registry for others to use.
FAQ
What is a Terraform provider and when should I build one?
Providers are plugins that let Terraform manage external systems via resources, data sources, and functions. Build a provider when: - Your platform (internal or external) lacks a provider but exposes an API you want to manage via Terraform. - You want to offer a first-class IaC interface for a platform your team owns. - You need reusable, custom functions or logic that go beyond core Terraform capabilities. Most users can use existing providers; write one when you have a clear, reusable integration need.How do I bootstrap a new provider with the HashiCorp scaffolding template?
Steps: - Clone hashicorp/terraform-provider-scaffolding-framework and follow its README for current Go and Terraform versions. - Rename the module: rungo mod edit -module github.com/YOUR_ORG/terraform-provider-YOUR_SERVICE, then go mod tidy.
- Clean up template files (e.g., CODEOWNERS, CODE_OF_CONDUCT, .copywrite.hcl) and update README.
- Rename ScaffoldingProvider to your provider type (e.g., MastodonProvider), and set Metadata TypeName to your provider name (e.g., mastodon).
- Remove example resources/data sources/functions from the registries in Resources, DataSources, and Functions.What is the Terraform Plugin Framework and why Protocol v6?
The Terraform Plugin Framework is the modern SDK for building providers. It: - Implements the gRPC protocol (Protocol v6) so you don’t need to manage the wire details. - Provides interfaces for Provider, Resource, Data Source, and Function, unified logging (tflog), diagnostics, schemas, plan/state handling, and testing helpers.
Use it to simplify provider development and stay current with Terraform’s evolving internals.How do schemas and models work in providers, data sources, and resources?
Each component defines: - Schema: Declares attributes/parameters, types, and metadata (required/optional/computed, defaults, sensitivity, descriptions). - Model (Go struct): Mirrors schema withtfsdk tags for reading/writing config and state.
Tips:
- Use Computed for values returned by the API.
- Mark secrets as Sensitive.
- Provide sensible defaults with stringdefault/booldefault.
- Document fields with MarkdownDescription for auto-generated docs.What are best practices for error handling and logging?
Use the framework’s tools: - Diagnostics: Accumulate user-facing errors withresp.Diagnostics.AddAttributeError/AddError. Validate all inputs first; then return if HasError().
- Logging: Use tflog (Debug, Error, Trace), mask sensitive fields with tflog.MaskFieldValuesWithFieldKeys.
- Never call log.Fatal or exit the process—return diagnostics instead to avoid corrupting state.How do I run a locally built provider with Terraform during development?
Set up a dev override: - Build/install your provider:go install (binary goes to $(go env GOBIN) or $HOME/go/bin).
- Configure ~/.terraformrc:
- Use provider_installation { dev_overrides { "namespace/name" = "/path/to/GOBIN/" } direct {} }.
- Reference your provider in Terraform (required_providers with the same source string). Terraform will load your local binary.How should I design the provider Configure() logic and credentials handling?
Pattern: - Read config into a provider model and validate: - If a value isUnknown, add a diagnostic explaining how to produce it first.
- Load defaults from environment variables, then override with explicit config values if provided.
- Validate combinations (e.g., require token OR user/password).
- On success, create the API client and attach it via resp.DataSourceData and resp.ResourceData for reuse. Do not attach clients for functions (they should be pure logic).How do I implement data sources and resources, and avoid state drift?
Data sources: - ImplementMetadata, Schema, Configure (pull client from provider), and Read (look up data and set state).
Resources:
- Implement Metadata, Schema, Configure, and CRUD: Create, Read, Update, Delete (+ ImportState).
- Normalize API responses (e.g., strip server-added HTML) before saving to state to prevent perpetual diffs.
Registration:
- Add constructors to Provider.Resources()/DataSources().
Testing:
- Create, import, update, and delete in multi-step tests to exercise full lifecycle.How do I add and test custom functions in a provider?
Functions are pure (no external calls): - ImplementMetadata (name), Definition (parameters, return type), and Run (read args, transform, set result).
- Version gating: Functions require Terraform ≥ 1.8; older Terraform ignores them. In tests, use tfversion.SkipBelow(1.8.0).
- Testing: Use resource.UnitTest, not acceptance tests. Cover happy path and invalid inputs (e.g., null, unknown) with ExpectError and output checks.What’s the difference between unit and acceptance tests, and how do I run them?
- Unit tests: Isolated; no external services. Use for functions and internal logic. Run withgo test.
- Acceptance tests: Exercise real APIs for resources/data sources. Guard with TF_ACC=1 (set by make testacc in the template).
- Precheck: Verify required environment variables (e.g., host, client ID/secret, token) before running acceptance tests; fail early with a clear message.How do I publish a provider (docs, signing, registry, releases)?
- Docs: Rungo generate to use tfplugindocs; keep examples under examples/ to include in generated docs.
- Signing: Create a 4096-bit RSA GPG key, export public/private keys; store private key and passphrase as GitHub Actions secrets.
- Registry:
- Terraform: Add public key in Registry user settings; register the provider repo via “Publish”.
- OpenTofu: Submit signing key and provider via their Registry GitHub issues.
- Release: Tag a semantic version in GitHub Releases. The provided workflow (goreleaser) builds, signs, and attaches binaries; users fetch them with terraform init.
Terraform in Depth ebook for free
