The previous implementation used asyncio.to_thread(llm.complete) to avoid
blocking the event loop, but Chainlit's contextvars context is not reliably
propagated across thread boundaries, causing the session to reset and clear
the message history on each user submission.
Changes:
- Add llm.acomplete() using litellm.acompletion() (native coroutine)
- Replace asyncio.to_thread() in on_message with await llm.acomplete()
- Store the welcome message in state.messages so it is replayed on reconnect
- Persist state to cl.user_session immediately after appending the user's
message (before the LLM call) so reconnect detection has the latest history
- Add pytest-asyncio dev dependency and asyncio_mode = "auto" config
- Add 6 async tests for acomplete() in tests/test_llm.py
https://claude.ai/code/session_01SUWzMzFvSfWiHXA2p6rPg9
Two related issues caused the screen to clear after each answer:
1. Blocking event loop: the synchronous llm.stream_complete() for-loop
was running directly in the async on_message handler, blocking the
event loop for the full LLM response duration. This caused the
WebSocket to time out and Chainlit to reconnect after each message.
Fix: replace stream_complete() with asyncio.to_thread(llm.complete)
so the network-bound LLM call runs in a thread pool and the event
loop (and WebSocket) stay alive throughout.
2. Reconnect resets history: on_chat_start always created a fresh empty
state and sent the welcome message, even on WebSocket reconnections
where cl.user_session still held the existing conversation.
Fix: if cl.user_session["state"] is already present, replay the
stored message history into the new thread instead of starting fresh.
https://claude.ai/code/session_01SUWzMzFvSfWiHXA2p6rPg9
The LLM returns a structured JSON object. Previously, raw tokens were
streamed directly to the user via msg.stream_token(), causing the full
JSON blob to appear in the chat.
Fix: collect all chunks silently, parse the JSON, then send only the
reply field with cl.Message(content=reply_text).send(). The JSON fields
(updates, next_step, registration_complete, language, intent) are still
processed in the background — parents never see them.
https://claude.ai/code/session_01SUWzMzFvSfWiHXA2p6rPg9
Claude