12 Using the filesystem
This chapter introduces Python’s cross-platform tools for working with the filesystem—separate from basic file I/O—so scripts can handle paths and directories reliably on Linux, Windows, and macOS. It contrasts the traditional os and os.path functions with the more modern, object-oriented pathlib, showing how both approaches represent and manipulate paths. Core concepts include how filesystems are tree-structured, how path syntax differs by platform (separators, drive roots, case sensitivity), and how absolute and relative paths relate to the current working directory. The chapter walks through building and decomposing paths (join, split, basename, dirname, splitext), platform and environment detection (os.name, sys.platform, os.environ), and the analogous pathlib capabilities (Path objects, joinpath and the “/” operator, parts, name, parent, suffix, glob, and iterdir). The net effect is a toolkit for writing portable code without hardcoding OS-specific syntax.
To query the filesystem, Python provides a rich set of predicates and metadata utilities. With os.path you can check for existence and type (exists, isfile, isdir, islink, ismount, samefile, isabs) and obtain metadata (getsize, getmtime, getatime). Pathlib mirrors these as methods on Path objects, keeping logic close to the path being examined. For efficient directory inspection, os.scandir yields DirEntry objects that expose attributes (is_file, is_dir, is_symlink, etc.) and integrate cleanly with context managers for resource safety. These tools allow scripts to quickly filter and characterize files and directories before taking action.
The chapter then covers everyday operations: listing (os.listdir), wildcard expansion (glob), renaming and moving (os.rename or Path.rename), deleting files (os.remove or Path.unlink), creating and removing directories (os.mkdir/makedirs, os.rmdir, Path.mkdir, Path.rmdir), and handling whole trees (shutil.rmtree, shutil.copytree). For recursive processing, it compares os.walk—configurable via topdown, onerror, and followlinks—with pathlib’s rglob for concise pattern-driven traversal. Practical examples show how to total sizes, skip symlinks, and move matched files (for example, *.test) into a backup folder. The overarching guidance: prefer pathlib for clarity and expressiveness while remaining fluent in os/os.path for compatibility; always compose paths with join-style helpers; anchor relative paths thoughtfully; and be mindful of platform-specific differences and symlink behavior.
FAQ
What’s the difference between os/os.path and pathlib, and when should I use each?
The os and os.path modules are the traditional, function-based APIs for filesystem paths and operations; they work with strings and are widely used. pathlib (added in Python 3.5) provides an object-oriented API where Path objects represent paths and expose methods for path manipulation and file operations. Use pathlib for more concise, readable, and cross-platform code; keep os/os.path in mind for legacy code and where specific functions (for example, os.walk, os.scandir) are handy.How do I get and change the current working directory?
- With os: get usingos.getcwd(), change using os.chdir(path).- With pathlib: get using
Path.cwd(). pathlib doesn’t change the process working directory; create new Path objects instead to work in other locations.Use
os.curdir (.) and os.pardir (..) for portable references.What’s the difference between absolute and relative paths across operating systems?
- Absolute paths fully specify a location (e.g., Linux/macOS:/usr/bin; Windows: C:\Windows).- Relative paths are interpreted from a context (the current working directory).
- Separators differ: Linux/macOS use
/; Windows uses \ and drive letters. Build paths with os.path.join or Path(...)/'child' to stay portable. Be mindful of case-sensitivity: Linux/macOS are typically case-sensitive; Windows is not.How do I build and decompose paths portably?
- Build:os.path.join('dir', 'file.txt') or Path('dir') / 'file.txt'.- Split:
os.path.split(p) → head, tail; Path(p).parent, Path(p).name.- Basename/dirname:
os.path.basename(p), os.path.dirname(p); pathlib: Path.name, Path.parent.- Extension:
os.path.splitext(p); pathlib: Path.suffix.- Components: pathlib:
Path.parts.How can I list directory contents and match filenames with wildcards?
- List contents:os.listdir(path); pathlib: Path(path).iterdir() (iterator of Path objects).- Wildcards:
glob.glob(pattern) (strings) or Path(path).glob(pattern)/rglob(pattern) (Path objects). Patterns include * (any chars), ? (single char), and ranges like [0-9].How do I check whether a path exists and what type it is, and get file size or times?
- Existence/type:os.path.exists, os.path.isfile, os.path.isdir, os.path.islink, os.path.ismount.- pathlib equivalents:
Path.exists(), is_file(), is_dir(), is_symlink().- Size/timestamps:
os.path.getsize, os.path.getmtime, os.path.getatime; pathlib: Path.stat().st_size, .st_mtime, .st_atime.- Efficient per-entry checks: see
os.scandir.What is os.scandir and why use it?
os.scandir(path) returns an iterator of os.DirEntry objects that expose attributes and type checks (for example, entry.is_file()) without extra system calls in many cases. It’s typically faster and more memory-efficient than combining os.listdir with separate os.path checks. Use it as a context manager to ensure resources are released: with os.scandir(path) as it: ...How do I create, rename/move, and delete files and directories?
- Create dir:os.mkdir(path), os.makedirs(path, exist_ok=True); pathlib: Path(path).mkdir(parents=True, exist_ok=True).- Rename/move:
os.rename(src, dst); pathlib: Path(src).rename(dst).- Delete file:
os.remove(path); pathlib: Path(path).unlink().- Remove empty dir:
os.rmdir(path); pathlib: Path(path).rmdir().- Remove nonempty dir:
shutil.rmtree(path).How can I walk an entire directory tree?
Useos.walk(top, topdown=True, onerror=None, followlinks=False). It yields (root, dirs, files) for each directory. If topdown is True, you may modify the dirs list in place to control recursion (for example, skip a folder). For glob-style recursive matching with pathlib, use Path(top).rglob('pattern').How do I write cross-platform path code and detect the platform?
- Always join withos.path.join or Path(...) / 'child' (don’t hardcode separators).- Expand user/env:
os.path.expanduser('~'), os.path.expandvars('$HOME/tmp').- Platform info:
os.name returns 'posix', 'nt', or 'java'; sys.platform is more specific (for example, 'linux', 'darwin', 'win32').- Be mindful of case sensitivity (Linux/macOS often case-sensitive; Windows traditionally case-insensitive).
Why do backslashes in Windows paths look doubled in Python strings?
Backslash is an escape character in string literals (for example,\n). To avoid surprises, either use raw strings (for example, r'C:\path\to\file'), replace backslashes with forward slashes (Python accepts them on Windows), or build paths with os.path.join or pathlib so you don’t hardcode separators.
The Quick Python Book, Fourth Edition ebook for free
