1 Getting Started with Local AI

This chapter introduces the project you will build: a voice-enabled local AI chat app that runs entirely on your Mac. You will speak into your microphone, see your speech transcribed, and receive a streaming response from a large language model without sending your data to the internet. The book frames this as both a practical tool and a learning experience, showing how speech recognition, model inference, and a web interface can work together in one private local workflow.

It also explains why local AI is worth learning. Compared with cloud-based AI, local systems offer stronger privacy, offline use, full control, predictable ownership, and no subscription costs, while cloud services may still be better for the most demanding or up-to-date tasks. To make the local approach work, the chapter introduces the main tools you will use on macOS: Ollama for running models, MLX Whisper for local transcription, Python for the application logic, Streamlit for the interface, and VS Code for editing and organizing the code.

Finally, the chapter gets you ready to build by covering the Mac hardware and software requirements, giving a beginner-friendly explanation of what an LLM is, and teaching the terminal basics you will rely on throughout the book. You learn the difference between GUI and CLI, why command-line skills matter for development and AI agents, and how to use four safe core commands: pwd, ls, cd, and mkdir. By the end, you should have a working terminal, a new project folder, and enough confidence to begin building locally.

Cloud AI (right) sends your prompt across the internet to a hosted inference service, and your data may be stored on the company's servers. Local AI (left) keeps the prompt, the model, and the response inside your machine -- no network hop, no external server, no data leaving your control.

Overview of the voice chat application. Your Mac runs three pieces locally: a Streamlit web app in the browser captures your voice and renders the chat, MLX Whisper transcribes your speech to text, and Ollama runs the LLM that generates the streaming reply. The dashed boundary marks "your machine" -- nothing crosses it.

The GUI and the CLI offer different features, but both communicate with the computer's operating system. Clicking in the GUI and typing in the CLI ultimately reach the same destination.

1. Navigate your home directory. Open the terminal and use pwd to confirm you are in your home directory. Use ls to see all your folders. Navigate into Documents using cd Documents, list its contents, and return home using cd ~.
2. Create a project structure. Starting from your home directory, create the following folder structure using only mkdir and cd:

• After creating each folder, use pwd to verify your location and ls to confirm the folder was created.

3. Explore hidden files. Run ls -la in your home directory. Count how many hidden files and folders (those starting with .) you see. Pick one and guess what it might be for.
4. Speed comparison. Time yourself performing these tasks, first using Finder, then using the terminal:
• Navigate to your Documents folder
• Create a new folder called test_folder
• Go inside the new folder
• Go back to your home directory
Which method was faster? Write down your observations.
5. Practice `clear` and `cd`. Navigate to three different folders (Desktop, Documents, Downloads), run ls in each one, then use clear to clean the screen. Finally, return home with cd ~ and run pwd to confirm.
6. Reflect on your AI use. Think about the last three times you used a cloud AI service (ChatGPT, Gemini, Claude, or similar). For each interaction, consider: Did it contain private information? Did you need an internet connection? Could a local model have handled the task? Write down your answers. You will revisit this reflection after completing Chapter 4.

FAQ