4 Downloading an LLM and having your first conversation
In this chapter, you move from having Ollama installed to actually using a local AI model on your Mac. The first step is to download a small, dependable model so you have a practical baseline for learning and comparison. The chapter uses Gemma 3 with 4 billion parameters as the starting point because it is light enough for modest hardware yet capable enough to show what a local LLM can do.
Once the model is downloaded with ollama pull, you can start an interactive chat session with ollama run and talk to the model directly from the terminal. The chapter explains that this happens entirely on your own machine, without requiring internet access, and shows how prompts and responses work in real time. It also emphasizes that model outputs can vary from run to run because language models are nondeterministic, so different wording is normal even when the overall meaning stays similar.
The chapter then builds intuition for model size, memory use, and tradeoffs between speed and capability. It explains parameters, quantization, and why larger models usually need more RAM but may produce better results, while smaller models are faster and easier to run. It also introduces major model families such as Gemma, Llama, Qwen, and Mistral, demonstrates comparing different models side by side, and closes with essential management commands like listing installed models, checking what is running, viewing model details, and removing models when you need to free space.
Summary
ollamapulldownloads a model’s weights to your local machine. The model persists on disk until you delete it.ollamarunstarts an interactive chat session. You can also pass a question in quotes for a one-shot response.- Model sizes (2B, 7B, 13B, 70B) represent the number of parameters. More parameters mean better quality but they require more RAM.
- Different model families (Gemma, Llama, Qwen, Mistral, Phi) each have unique strengths and response styles.
- Model management commands (
list,ps,show,rm) help you keep track of what is on your machine. - Everything runs locally: no internet connection is needed once the model is downloaded.
FAQ
What do I need to have running before I download an AI model with Ollama?
Theollama serve process should already be running in a terminal window. Keep that terminal open, and use a second terminal for commands like ollama pull and ollama run.What is the first model recommended for this chapter, and why?
The chapter starts withgemma3:4b, a small and reliable model. It is recommended because it keeps the first download manageable and provides a good baseline for learning on modest Mac hardware.What does the command ollama pull gemma3:4b do?
It downloads the Gemma 3 4B model to your machine, similar to how brew install downloads software. Once downloaded, the model stays on your computer unless you remove it.What does “4B” mean in a model name like gemma3:4b?
4B means the model has about 4 billion parameters. Parameters are the adjustable numerical values the model learned during training, and more parameters usually mean better capability but higher RAM use.How do I start an interactive chat with a local model?
Runollama run gemma3:4b. Your terminal prompt changes to >>>, which means you are in interactive chat mode and can type prompts directly to the model.How do I exit interactive chat mode in Ollama?
Type/bye or press Ctrl-D. Both methods return you to the normal terminal prompt.What does the chapter say about model size, RAM, and performance?
Smaller models are faster and use less RAM, while larger models can handle more complex tasks but need more memory and may respond more slowly. For example, 1B–3B models often need 2–4 GB of RAM, while 7B–9B models usually need 5–8 GB.What is quantization, and why does it matter?
Quantization compresses a model so it uses less memory while keeping most of its quality. Aq4 model typically uses about half the memory of a q8 model, but may lose a little answer quality.How can I compare different model families in Ollama?
You can download another model, such asqwen3.5:4b, and ask both models the same question using ollama run <model> "question". The chapter compares Gemma and Qwen and notes that they can differ in style, structure, and personality.What are the most important Ollama model management commands from this chapter?
The key commands areollama pull to download a model, ollama run to chat or ask a one-shot question, ollama list to see installed models, ollama ps to see running models, ollama show to view model details, and ollama rm to delete a model.
Build Applications with Local AI Models on a Mac ebook for free
