5Building a ChatGPT clone with Aspire, Ollama, and Semantic Kernel
This chapter introduces the idea that modern large language models can be embedded into ordinary .NET applications to create domain-specific assistants, not just generic public chatbots. It explains that developers can combine Aspire, Ollama, and Semantic Kernel to host an LLM locally, connect it to application code, and build a ChatGPT-like experience with a custom user interface. The focus is on turning AI capabilities into a practical distributed-systems feature that can answer questions about a business domain and support richer enterprise workflows.
To do that, the chapter first shows how Ollama makes it possible to run models locally or in controlled environments, avoiding dependence on external AI APIs while improving privacy, lowering latency, and eliminating recurring usage costs. Aspire is used to provision and manage Ollama as part of the distributed application, including persistent storage, model download, and an optional web UI. The example uses a small model so the setup remains manageable, and the application waits for the model to be ready before the frontend starts, emphasizing reliable orchestration in a distributed environment.
After the model hosting is in place, Semantic Kernel is added as the abstraction layer that connects the application to the LLM. The chapter walks through building chat completion logic with conversation history, a SignalR hub for streaming responses, and a browser-based client that can send prompts, display partial output as it arrives, cancel requests, clear the screen, and copy results. It concludes by wiring the chat page into the app’s navigation and dynamically loading the JavaScript client, leaving the reader with a working, Aspire-hosted AI chat application that can serve as a foundation for more advanced AI agents.
Hosting Ollama in Aspire
Ollama user interface
Custom chat interface
Summary
Popular AI applications used for conversations, such as ChatGPT, rely on LLM technology.
Ollama allows anyone to host LLMs anywhere, free of charge. It relies on Docker containers and can host a wide range of LLMs, including Meta’s Llama.
Aspire has a community toolkit for hosting Ollama. The Ollama community toolkit for Aspire comes with its own frontend, which is similar to ChatGPT and serves the same purpose.
Semantic Kernel is a Microsoft framework for building AI agents that allows any .NET code to interact with LLMs. It can stream the LLM response letter by letter.
The official name for the LLM functionality used in chats is chat completion.
You can use SignalR to build a custom chat interface that supports answer streaming.
FAQ
What is the main goal of this chapter?The chapter shows how to build a ChatGPT-like app inside an Aspire distributed application by hosting an LLM with Ollama, connecting it through Semantic Kernel, and creating a custom chat UI.Why use Ollama for hosting the language model?Ollama lets you run LLMs locally or in controlled environments without relying on cloud AI APIs. That gives you more privacy, lower latency, no recurring API costs, and more control over model behavior.Which models can be hosted with Ollama?Ollama supports a wide range of open source models, including LLaMA, DeepSeek, Mistral, Gemma, and phi3.5. You can choose the model that best fits your project and hardware.How is Ollama integrated into an Aspire app?You add the CommunityToolkit.Aspire.Hosting.Ollama NuGet package to the Aspire host project, then configure the Ollama container with methods like AddOllama(), WithDataVolume(), WithLifetime(ContainerLifetime.Persistent), and AddModel().Why does the chapter use phi3.5 as the model?phi3.5 is a relatively small model compared with very large alternatives, so it is more practical for a local development setup. Even so, it still needs significant resources and enough free RAM.What hardware is recommended for running the Ollama setup?The chapter warns that even a small model like phi3.5 requires substantial resources. A machine with at least 16 GB of RAM is recommended, and the model itself needs about 4 GB of free RAM.Why is Semantic Kernel added to the web frontend project?Semantic Kernel acts as the LLM client and provides a .NET-friendly way to interact with chat-completion models. In this chapter, it is installed in the web frontend project because that is where the chat UI and chat logic live.How does the app get the Ollama endpoint at runtime?The model reference is passed into the app by Aspire as a connection string. The code reads it with builder.Configuration.GetConnectionString("phi35"), extracts the Endpoint value, and uses that URI when registering Semantic Kernel.Why does the chat backend use SignalR?SignalR provides a real-time connection between the browser and the server, which is ideal for streaming chat responses back to the user incrementally instead of waiting for a full response.How does the custom chat UI get the JavaScript client loaded?The page uses Blazor JavaScript interop to dynamically import ./js/chat.js during first render, then calls its init() function. This keeps the script loaded only on the chat page where it is needed.
pro $24.99 per month
access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
Building Distributed Applications that Work ebook for free
