iwantcoding.com
🔥 Daily 👥 Rooms 🏆 Top Log in Sign up

Streaming

By · Reviewed & last updated · Part of LangChain Tutorial

Every Runnable supports .stream() and .astream() — LLM tokens, parsed chunks, retriever hits all stream through. Streaming changes UX: users see output start in <200ms.

Stream sync / async / token-by-token

EXAMPLE
from langchain_openai      import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

llm    = ChatOpenAI(model='gpt-4o-mini', streaming=True)
prompt = ChatPromptTemplate.from_template('Write a haiku about {topic}')
chain  = prompt | llm | StrOutputParser()

# 1) Synchronous streaming
for chunk in chain.stream({'topic': 'PostgreSQL'}):
    print(chunk, end='', flush=True)

# 2) Async streaming — what you'd use in a real web server
import asyncio

async def main():
    async for chunk in chain.astream({'topic': 'Rust'}):
        print(chunk, end='', flush=True)

asyncio.run(main())

# 3) Server-sent events (FastAPI)
from fastapi             import FastAPI
from fastapi.responses   import StreamingResponse

app = FastAPI()

@app.get('/haiku')
async def haiku(topic: str):
    async def gen():
        async for chunk in chain.astream({'topic': topic}):
            yield f'data: {chunk}\n\n'
    return StreamingResponse(gen(), media_type='text/event-stream')

# 4) astream_events — get every internal event, not just final tokens
async for event in chain.astream_events({'topic': 'CSS'}, version='v2'):
    if event['event'] == 'on_chat_model_stream':
        print(event['data']['chunk'].content, end='', flush=True)
    elif event['event'] == 'on_chain_end':
        print('\n(done)')

Why it matters

astream_events is the underrated power tool. You can render different parts of an agent run (retrieval, tool calls, final answer) in real time — the difference between “LLM app” and “polished product”.

Tip: Tweak the snippet with Try it Yourself », then sit the quiz at the bottom of the page.

Example

Example
for chunk in chain.stream({'question': 'Write a haiku about debugging'}):
    print(chunk, end='', flush=True)
Try it Yourself »

Discussion

Loading…