Run a Local Chatbot With Gemma 4

you own this product
prerequisites
Intermediate command line • Basic Linux administration (Linux path only) • Basics of LLMs
skills learned
Survey open weights models and estimate hardware needs • install Docker Model Runner • pull and chat with a local model • launch an Open WebUI chatbot against a local model
Michael Yuan and Tony Yuan
1 week · 4-6 hours per week · BEGINNER
Included with a Manning Online subscription

pro $24.99 per month

  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose one free eBook per month to keep
  • exclusive 50% discount on all purchases
  • renews monthly, pause or cancel renewal anytime

lite $19.99 per month

  • access to all Manning books, including MEAPs!

team

5, 10 or 20 seats+ for your team - learn more

Look inside

Socratic liveProjects pair you with an AI tutor to help you learn by doing, in real time.

In this liveProject, you’ll take on the role of an IT engineer helping a midsize company reduce its dependence on costly cloud AI services and keep sensitive data in-house. Your goal is to build a proof-of-concept private alternative to ChatGPT using open-weight models running on local hardware. You’ll survey models and hardware options, set up the cross-platform Docker Model Runner, and run Google’s Gemma 4 E4B model on your own computer. You’ll first interact with the model from the command line, then deploy it through Open WebUI to create a browser-based chatbot that can be demonstrated to management and made available to non-technical colleagues.

This project is a part of the series Open LLMs on Your Own Computer.
This project is designed for learning purposes and is not a complete, production-ready application or solution.

project authors

Tony Yuan
Tony Yuan is a student at Liberal Arts and Science Academy in Austin, Texas, who loves technology and AI. He serves as the VP for the Student Council and represents the school in national Quiz Bowl and History Bowl competitions. He leads the Map Wars club to teach geopolitics to high school students. He enjoys listening to rock music and reading history books when he's not programming or studying.
Michael Yuan
Dr. Michael Yuan is the maintainer of the open-source WasmEdge project, the most popular WebAssembly (Wasm) sandboxes for edge applications and AI agents. He is a Docker Captain and the creator of the VibeKeys series of agentic devices. Dr. Yuan holds a PhD in Astrophysics from the University of Texas at Austin.

prerequisites

This liveProject is for software developers and IT professionals who are interested in building LLM applications in their own domains. To begin this liveProject, you will need to know the following:

TOOLS
  • Basics of Linux
  • Basics of GitHub
TECHNIQUES
  • Basics of CLI usage
  • Basics of LLMs

features

Self-paced
You choose the schedule and decide how much time to invest as you build your project.
Project roadmap
Each project is divided into several achievable steps.
Get Help
While within the liveProject platform, get help from fellow participants and even more help with paid sessions with our expert mentors.
Compare with others
For each step, compare your deliverable to the solutions by the author and other participants.
books included as resources
Get full access to select books for 90 days. Permanent access to excerpts from Manning products are also included, as well as references to other resources.
choose your plan

pro

monthly
annual
$24.99
$249.99
only $20.83 per month
  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose another free product every time you renew
  • choose twelve free products per year
  • exclusive 50% discount on all purchases
  • renews monthly, pause or cancel renewal anytime
  • renews annually, pause or cancel renewal anytime
  • Run a Local Chatbot With Gemma 4 project for free

team

monthly
annual
$49.99
$499.99
only $41.67 per month
  • five seats for your team
  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose another free product every time you renew
  • choose twelve free products per year
  • exclusive 50% discount on all purchases
  • renews monthly, pause or cancel renewal anytime
  • renews annually, pause or cancel renewal anytime
  • Run a Local Chatbot With Gemma 4 project for free