iwantcoding.com
🔥 Daily 👥 Rooms 🏆 Top Log in Sign up

Memory

By · Reviewed & last updated · Part of LangChain Tutorial

Memory makes a chain stateful between calls — remembering what the user said earlier. The modern pattern is RunnableWithMessageHistory: wrap your chain, plug in a store, get a session-scoped history.

In-memory and a real store

EXAMPLE
from langchain_openai      import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.chat_history       import BaseChatMessageHistory, InMemoryChatMessageHistory
from langchain_core.runnables.history  import RunnableWithMessageHistory

llm = ChatOpenAI(model='gpt-4o-mini')

prompt = ChatPromptTemplate.from_messages([
    ('system', 'You are a friendly tutor. Keep replies concise.'),
    MessagesPlaceholder(variable_name='history'),
    ('human', '{input}'),
])

chain = prompt | llm

# 1) In-memory store — perfect for dev / testing
store: dict[str, BaseChatMessageHistory] = {}
def get_history(session_id: str) -> BaseChatMessageHistory:
    if session_id not in store:
        store[session_id] = InMemoryChatMessageHistory()
    return store[session_id]

with_memory = RunnableWithMessageHistory(
    chain,
    get_history,
    input_messages_key='input',
    history_messages_key='history',
)

# 2) Use — pass session_id in config
print(with_memory.invoke(
    {'input': 'What is a CTE in SQL?'},
    config={'configurable': {'session_id': 'alice'}},
).content)

print(with_memory.invoke(
    {'input': 'Give an example.'},   # remembers we were on CTEs
    config={'configurable': {'session_id': 'alice'}},
).content)

# 3) Production — back it with Redis (or any persistent store)
from langchain_redis import RedisChatMessageHistory

def get_history(session_id: str) -> BaseChatMessageHistory:
    return RedisChatMessageHistory(session_id, redis_url='redis://localhost:6379')

# 4) Trim long histories so cost stays bounded
from langchain_core.messages import trim_messages

trimmer = trim_messages(strategy='last', max_tokens=4000, token_counter=llm)

trimmed_chain = (
    RunnablePassthrough.assign(history=lambda x: trimmer.invoke(x['history']))
    | prompt | llm
)

Why it matters

Always trim. Without it, every turn carries every previous message; cost grows quadratically in turns and you eventually hit the model’s context limit mid-conversation.

Tip: Tweak the snippet with Try it Yourself », then sit the quiz at the bottom of the page.

Example

Example
from langchain_core.chat_history import InMemoryChatMessageHistory
from langchain_core.runnables.history import RunnableWithMessageHistory
history = InMemoryChatMessageHistory()
chat_chain = RunnableWithMessageHistory(
    chain, lambda _: history,
    input_messages_key='question', history_messages_key='history',
)
Try it Yourself »

Exercise

Wrap a chain so it carries chat history.

from langchain_core.runnables.history import

Discussion

Loading…