1 Getting Started with Local AI
This chapter introduces the book’s main goal: building a voice-enabled AI chat application that runs entirely on your Mac, keeping your voice, prompts, and responses local for privacy, offline use, and full control. It frames the project as both a practical assistant and a learning experience, showing how speech recognition, language model inference, and a web interface can be combined into one working pipeline. The tools you will use are centered on the Mac ecosystem and include Python, VS Code, Streamlit, Ollama, and MLX Whisper.
The chapter also explains why local AI is worth learning instead of relying only on cloud services. Local AI keeps data on your machine, avoids subscriptions, works without internet access, and gives you ownership over the system you build, while cloud AI can still be better for the newest models and the most demanding reasoning tasks. To prepare you for the build, the chapter introduces large language models at a conceptual level, including how they generate text by predicting likely next words and why they can sometimes hallucinate, as well as basic terms like prompts and APIs.
Finally, the chapter gets you comfortable with the terminal, which is essential for the rest of the book. It contrasts GUI and CLI workflows, explains why the command line matters for speed, precision, automation, and modern AI tooling, and walks through opening Terminal on macOS. You learn a small set of safe but powerful navigation commands: where you are, what files are there, how to move between folders, and how to create a project directory where all later work will live. The chapter closes by reassuring beginners that these commands are simple, reversible, and enough to start building confidently.
Cloud AI (right) sends your prompt across the internet to a hosted inference service, and your data may be stored on the company's servers. Local AI (left) keeps the prompt, the model, and the response inside your machine -- no network hop, no external server, no data leaving your control.
Overview of the voice chat application. Your Mac runs three pieces locally: a Streamlit web app in the browser captures your voice and renders the chat, MLX Whisper transcribes your speech to text, and Ollama runs the LLM that generates the streaming reply. The dashed boundary marks "your machine" -- nothing crosses it.
The GUI and the CLI offer different features, but both communicate with the computer's operating system. Clicking in the GUI and typing in the CLI ultimately reach the same destination.
Exercises
- Navigate your home directory. Open the terminal and use pwd to confirm you are in your home directory. Use ls to see all your folders. Navigate into Documents using cd Documents, list its contents, and return home using cd ~.
- Create a project structure. Starting from your home directory, create the following folder structure using only mkdir and cd:
- After creating each folder, use pwd to verify your location and ls to confirm the folder was created.
- Explore hidden files. Run ls -la in your home directory. Count how many hidden files and folders (those starting with .) you see. Pick one and guess what it might be for.
- Speed comparison. Time yourself performing these tasks, first using Finder, then using the terminal:
- Navigate to your Documents folder
- Create a new folder called test_folder
- Go inside the new folder
- Go back to your home directory
- Practice `clear` and `cd`. Navigate to three different folders (Desktop, Documents, Downloads), run ls in each one, then use clear to clean the screen. Finally, return home with cd ~ and run pwd to confirm.
- Reflect on your AI use. Think about the last three times you used a cloud AI service (ChatGPT, Gemini, Claude, or similar). For each interaction, consider: Did it contain private information? Did you need an internet connection? Could a local model have handled the task? Write down your answers. You will revisit this reflection after completing Chapter 4.
FAQ
What is the main project you will build in Chapter 1?
You will build a voice-enabled local AI chat application on your Mac. It lets you speak into a microphone, transcribes your speech to text, and returns a streaming AI response entirely on your own machine.
Why does this book focus on running AI locally instead of using cloud AI?
Local AI keeps your data and voice on your own Mac, which improves privacy and lets you work offline. It also gives you more control, avoids subscription dependence, and helps you understand how AI works under the hood.
What tools will you use to build the app in this book?
The main tools are Ollama for running local language models, MLX Whisper for local speech recognition, VS Code for editing code, Python for programming, and Streamlit for building the web app interface.
What hardware and software do you need to follow along?
You need a recent Mac running macOS Ventura 13 or later, at least 8 GB of RAM, and about 20 GB of free disk space. A stable internet connection is needed for setup and downloads, but the finished AI app runs offline.
What is a large language model (LLM)?
An LLM is an AI model trained on huge amounts of text. It learns patterns in language and generates responses by predicting the next word one piece at a time.
What is an AI hallucination?
A hallucination is when an LLM gives a confident-sounding but incorrect answer. It happens because the model predicts text from patterns rather than truly understanding facts like a human would.
What is the difference between cloud AI and local AI?
Cloud AI sends your prompts to a company’s server over the internet, while local AI keeps everything on your machine. Cloud AI may be more current and powerful, but local AI is better for privacy, offline use, and control.
When should you use local AI instead of cloud AI?
Use local AI when you are handling sensitive information, want to work offline, want full control over behavior, want to learn how AI works, or want to avoid subscriptions.
Why is the terminal important in this book?
The terminal is the command-line interface you will use to install tools, run Python scripts, start Ollama, and launch the web app. It is essential because all the development tools in the book are controlled from it.
What are the four essential terminal commands introduced in Chapter 1?
The four essential commands are pwd to show your current location, ls to list files and folders, cd to change directories, and mkdir to create a new folder.
Build Applications with Local AI Models on a Mac ebook for free
