11 Managing session state and chat history

Long chat sessions can make a local AI assistant seem forgetful or inconsistent, but the real issue is that language models do not retain memory between calls. Each request to the model is independent, so the application must resend the relevant conversation history every time. In a Streamlit chatbot, that visible history is stored in session state, which acts as application memory rather than model memory.

Streamlit’s session state is useful because the app reruns from top to bottom after every interaction, which would normally erase ordinary Python variables. By storing data in st.session_state, values such as counters, settings, and chat messages survive reruns until the browser tab is closed, the page is refreshed, or the server restarts. The chapter shows how to initialize values only once, how session state can hold many kinds of Python objects, and how this mechanism supports separate, independent conversations for different users.

The chapter also explains how conversation history interacts with an LLM’s context window, the maximum amount of text the model can process at once. As more messages accumulate, responses can slow down, degrade in quality, or lose earlier context when the window is exceeded. To avoid this, the app can trim the history sent to the model while still displaying the full conversation in the interface, and it can add a safer reset flow with confirmation so users do not accidentally clear their chat. Additional patterns include tracking conversation duration and detecting model changes, all of which help make the chatbot more robust and user-friendly.

How Streamlit session state preserves data across reruns. The counter value persists in session state while the script re-executes from top to bottom.

What happens when conversation history exceeds the context window: the model loses access to earlier messages, and the response quality degrades.

How confirm_clear state transitions enable the two-step confirmation flow

• st.session_state is a dictionary that persists across Streamlit’s rerun cycle. It is the mechanism that gives your app memory. Without it, all variables will reset every time the user interacts with the page.
• Large language models do not remember previous conversations. The illusion of memory is created by sending the full conversation history with every request.
• Every LLM has a context window with a limited size. When the conversation history exceeds this limit, the response quality degrades because the model cannot see the full context.
• By trimming the messages sent to the model, while displaying the full history to the user, you prevent context-window overflow and maintain consistent response quality.
• You can use session state to implement clear buttons with confirmation dialogs that prevent accidental data loss.
• In a chat application, data flows from user input through session state to Ollama, through the LLM, and back to the browser with streaming.

FAQ