Add change spec: email-based conversation matching
Replace thread-ID-based conversation matching with email-address-based matching for more reliable conversation continuity. Key changes: - One conversation per email address (simpler model) - No data expiration (conversations persist indefinitely) - Post-completion support (questions and registration updates) - Versioned storage for registration updates (audit trail) - Admin notifications for registration changes This addresses the gap where parents sending new emails (instead of replying) would lose their registration progress. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 4.5
@@ -0,0 +1,2 @@
|
|||||||
|
schema: spec-driven
|
||||||
|
created: 2026-02-20
|
||||||
@@ -0,0 +1,104 @@
|
|||||||
|
## Context
|
||||||
|
|
||||||
|
The current implementation on branch `claude/email-agent-multi-model-c3ShZ` uses email threading headers to identify conversations. This is fragile—parents often send new emails instead of replying, breaking the thread association.
|
||||||
|
|
||||||
|
**Current behavior:**
|
||||||
|
```
|
||||||
|
Email 1 (new): "I want to register" → Thread ID: <abc@gmail.com> → New conversation
|
||||||
|
Email 2 (new): "Her name is Emma" → Thread ID: <xyz@gmail.com> → NEW conversation (context lost!)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Desired behavior:**
|
||||||
|
```
|
||||||
|
Email 1: parent@example.com → Conversation for parent@example.com (new)
|
||||||
|
Email 2: parent@example.com → Conversation for parent@example.com (continue)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Goals / Non-Goals
|
||||||
|
|
||||||
|
**Goals:**
|
||||||
|
- Reliable conversation continuity regardless of email threading behavior
|
||||||
|
- Simple mental model: one email address = one conversation
|
||||||
|
- Support post-completion interactions (questions and updates)
|
||||||
|
- Audit trail for registration changes
|
||||||
|
|
||||||
|
**Non-Goals:**
|
||||||
|
- Supporting multiple registrations per email address (one parent, multiple children handled in single conversation)
|
||||||
|
- Anonymous/guest conversations (email address is the identity)
|
||||||
|
- Complex merge logic for duplicate conversations
|
||||||
|
|
||||||
|
## Decisions
|
||||||
|
|
||||||
|
### 1. Conversation Key: Email Address
|
||||||
|
|
||||||
|
**Decision**: Use normalized sender email address as the conversation key.
|
||||||
|
|
||||||
|
**Rationale**: Email address is the only reliable identifier across email threads. Parents may use different devices, email clients, or simply compose new messages.
|
||||||
|
|
||||||
|
**Normalization**: Lowercase, trim whitespace. Consider: `maria@Example.com` = `maria@example.com`
|
||||||
|
|
||||||
|
**Trade-off**: A parent using multiple email addresses would have multiple conversations. This is acceptable—different address = different identity from the system's perspective.
|
||||||
|
|
||||||
|
### 2. Thread ID Usage
|
||||||
|
|
||||||
|
**Decision**: Store thread IDs for reply headers only, not for conversation matching.
|
||||||
|
|
||||||
|
**Rationale**: Thread IDs (`Message-ID`, `In-Reply-To`, `References`) are still needed for proper email client threading (so replies appear in the same thread in Gmail/Outlook). But matching uses email address.
|
||||||
|
|
||||||
|
**Implementation**: When sending a reply, use the most recent inbound message's ID for `In-Reply-To`.
|
||||||
|
|
||||||
|
### 3. No Data Expiration
|
||||||
|
|
||||||
|
**Decision**: Remove the 30-day retention limit for email conversations.
|
||||||
|
|
||||||
|
**Rationale**: With email-address-based matching, the conversation is a permanent record. There's no reason to delete it—if the parent returns in 6 months, their data should still be there.
|
||||||
|
|
||||||
|
**Privacy consideration**: If GDPR deletion is requested, admin can manually remove the conversation file.
|
||||||
|
|
||||||
|
### 4. Post-Completion Intent Detection
|
||||||
|
|
||||||
|
**Decision**: When a completed registration receives a new message, use the LLM to detect intent.
|
||||||
|
|
||||||
|
**Intent categories:**
|
||||||
|
- **Question**: Parent asking about fees, schedule, policies → Answer from knowledge base
|
||||||
|
- **Update request**: Parent wants to change registration data → Collect updates, version storage, notify admin
|
||||||
|
- **New registration**: Parent wants to register another child → Continue in same conversation, add to booking
|
||||||
|
|
||||||
|
**Implementation**: Add prompt guidance for post-completion state; LLM returns `intent` field.
|
||||||
|
|
||||||
|
### 5. Versioned Registration Storage
|
||||||
|
|
||||||
|
**Decision**: Store registration updates as versions, not overwrites.
|
||||||
|
|
||||||
|
**Structure:**
|
||||||
|
```
|
||||||
|
data/registrations/
|
||||||
|
parent_at_example.com/
|
||||||
|
v1_2024-09-15.json # Original registration
|
||||||
|
v2_2024-10-03.json # Updated (changed phone number)
|
||||||
|
current.json # Symlink or copy of latest
|
||||||
|
```
|
||||||
|
|
||||||
|
**Rationale**: Admin needs audit trail to see what changed and when. Original data preserved for compliance.
|
||||||
|
|
||||||
|
### 6. Admin Update Notifications
|
||||||
|
|
||||||
|
**Decision**: Send notification when registration is updated, including diff.
|
||||||
|
|
||||||
|
**Email subject**: "Registration Updated: [Child Name]"
|
||||||
|
**Body includes**: What changed (old → new), when, conversation excerpt
|
||||||
|
|
||||||
|
## Risks / Trade-offs
|
||||||
|
|
||||||
|
**Multiple children per family** → Single conversation handles this; booking can include multiple children. If needed later, extend the data model.
|
||||||
|
|
||||||
|
**Parent changes email address** → Creates new conversation. Admin would need to manually merge if needed. Acceptable for MVP.
|
||||||
|
|
||||||
|
**Storage growth** → Without expiration, conversations accumulate. Monitor disk usage; consider archival strategy later.
|
||||||
|
|
||||||
|
**LLM intent detection accuracy** → May misclassify. Err on the side of asking for clarification rather than making assumptions.
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
- Should the system support explicit "delete my data" requests via email? (GDPR)
|
||||||
|
- Should reminders stop after a certain count, or continue indefinitely for incomplete registrations?
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
## Why
|
||||||
|
|
||||||
|
The current email agent implementation uses email thread IDs (from `Message-ID`, `In-Reply-To`, `References` headers) to match conversations. This breaks when a parent sends a new email instead of replying to the existing thread—they start a fresh conversation and lose all previously collected registration data.
|
||||||
|
|
||||||
|
Parents don't always use "Reply"—they may compose a new email, use a different device, or their email client may not preserve threading headers. The system should recognize them by their email address, not by email client threading behavior.
|
||||||
|
|
||||||
|
## What Changes
|
||||||
|
|
||||||
|
- **Match conversations by sender email address** instead of thread ID
|
||||||
|
- **One conversation per email address** — simple, permanent association
|
||||||
|
- **Remove data expiration** — no 30-day retention limit; conversations persist indefinitely
|
||||||
|
- **Handle post-completion interactions** — if registration is complete, detect whether the parent is asking a question or requesting updates to their registration
|
||||||
|
- **Version registration updates** — store changes alongside original data for admin audit trail
|
||||||
|
- **Notify admin of updates** — when a completed registration is modified, notify admin with change details
|
||||||
|
|
||||||
|
### Removed Features
|
||||||
|
- ~~1-month data retention for email conversations~~
|
||||||
|
- ~~Day 30 data clearing~~
|
||||||
|
- ~~"Your registration will expire" warning~~
|
||||||
|
|
||||||
|
### Retained Features
|
||||||
|
- Email reminders for incomplete registrations (Day 3, 10, 25) — still useful to nudge parents
|
||||||
|
|
||||||
|
## Capabilities
|
||||||
|
|
||||||
|
### Modified Capabilities
|
||||||
|
|
||||||
|
- `email-channel`: Change conversation matching from thread ID to sender email address; remove data expiration
|
||||||
|
- `registration-data-store`: Add versioned storage for registration updates; key conversations by email address
|
||||||
|
- `registration-notifications`: Add notification type for registration updates (not just new registrations)
|
||||||
|
|
||||||
|
### New Capabilities
|
||||||
|
|
||||||
|
*None — this modifies existing capabilities*
|
||||||
|
|
||||||
|
## Impact
|
||||||
|
|
||||||
|
- **Email channel**: Simpler matching logic; more reliable conversation continuity
|
||||||
|
- **Storage**: Conversations keyed by email address instead of thread ID; registration updates stored as versions
|
||||||
|
- **Admin workflow**: Admin sees change history when registrations are updated
|
||||||
|
- **Data retention**: No automatic deletion; conversations persist until manually removed
|
||||||
|
- **Spec updates**: `conversation-flow.md` timeout/retention section needs updating
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
## MODIFIED Requirements
|
||||||
|
|
||||||
|
### Requirement: System identifies conversations by sender email address
|
||||||
|
The system SHALL identify conversations by the sender's email address, not by email threading headers. Each unique email address corresponds to exactly one conversation.
|
||||||
|
|
||||||
|
#### Scenario: New email from unknown address
|
||||||
|
- **WHEN** an email arrives from an address with no existing conversation
|
||||||
|
- **THEN** the system SHALL create a new conversation keyed by that email address
|
||||||
|
|
||||||
|
#### Scenario: New email from known address (any thread)
|
||||||
|
- **WHEN** an email arrives from an address with an existing conversation
|
||||||
|
- **THEN** the system SHALL continue that existing conversation regardless of email threading headers
|
||||||
|
|
||||||
|
#### Scenario: Email address normalization
|
||||||
|
- **WHEN** comparing email addresses for matching
|
||||||
|
- **THEN** the system SHALL normalize addresses (lowercase, trim whitespace) so that `Maria@Example.com` matches `maria@example.com`
|
||||||
|
|
||||||
|
### Requirement: Thread headers used for reply threading only
|
||||||
|
The system SHALL use email threading headers (`In-Reply-To`, `References`) for outbound replies to maintain proper email client threading, but SHALL NOT use them for conversation matching.
|
||||||
|
|
||||||
|
#### Scenario: Reply includes threading headers
|
||||||
|
- **WHEN** the agent sends a reply email
|
||||||
|
- **THEN** the reply SHALL include `In-Reply-To` referencing the most recent inbound message ID
|
||||||
|
- **AND** the reply SHALL include `References` header for the email thread chain
|
||||||
|
|
||||||
|
#### Scenario: Threading headers ignored for matching
|
||||||
|
- **WHEN** an inbound email has threading headers pointing to a different conversation
|
||||||
|
- **THEN** the system SHALL ignore those headers and match by sender email address only
|
||||||
|
|
||||||
|
## REMOVED Requirements
|
||||||
|
|
||||||
|
### Requirement: Email data retention and expiration
|
||||||
|
**Reason**: With email-address-based matching, conversations are permanent records. No automatic expiration needed.
|
||||||
|
**Migration**: Remove any scheduled cleanup jobs; existing conversations remain accessible indefinitely.
|
||||||
|
|
||||||
|
### Requirement: Day 30 data clearing
|
||||||
|
**Reason**: No longer applicable; data persists indefinitely.
|
||||||
|
**Migration**: None required.
|
||||||
|
|
||||||
|
### Requirement: "Registration will expire" warning
|
||||||
|
**Reason**: No expiration means no warning needed.
|
||||||
|
**Migration**: Remove from reminder sequence.
|
||||||
+52
@@ -0,0 +1,52 @@
|
|||||||
|
## MODIFIED Requirements
|
||||||
|
|
||||||
|
### Requirement: Conversations keyed by email address
|
||||||
|
The system SHALL store conversations using the sender's normalized email address as the unique key, replacing thread-ID-based storage.
|
||||||
|
|
||||||
|
#### Scenario: Conversation file naming
|
||||||
|
- **WHEN** storing a conversation for `parent@example.com`
|
||||||
|
- **THEN** the system SHALL use a filename derived from the email address (e.g., `parent_at_example.com.json`)
|
||||||
|
|
||||||
|
#### Scenario: Conversation lookup
|
||||||
|
- **WHEN** loading a conversation for an incoming email
|
||||||
|
- **THEN** the system SHALL lookup by normalized sender email address
|
||||||
|
|
||||||
|
### Requirement: Registration updates stored as versions
|
||||||
|
The system SHALL store registration updates as separate versions, preserving the original and all subsequent changes for audit purposes.
|
||||||
|
|
||||||
|
#### Scenario: Initial registration stored
|
||||||
|
- **WHEN** a registration is completed for the first time
|
||||||
|
- **THEN** the system SHALL store it as version 1 with timestamp
|
||||||
|
|
||||||
|
#### Scenario: Registration update creates new version
|
||||||
|
- **WHEN** a parent requests changes to a completed registration
|
||||||
|
- **THEN** the system SHALL store the updated data as a new version
|
||||||
|
- **AND** the system SHALL preserve all previous versions
|
||||||
|
|
||||||
|
#### Scenario: Version metadata
|
||||||
|
- **WHEN** storing a registration version
|
||||||
|
- **THEN** the version SHALL include: version number, timestamp, and change summary (which fields changed)
|
||||||
|
|
||||||
|
### Requirement: Current registration accessible
|
||||||
|
The system SHALL provide easy access to the current (latest) registration data while preserving version history.
|
||||||
|
|
||||||
|
#### Scenario: Retrieve current registration
|
||||||
|
- **WHEN** the admin or system requests the current registration for an email address
|
||||||
|
- **THEN** the system SHALL return the most recent version
|
||||||
|
|
||||||
|
#### Scenario: Retrieve version history
|
||||||
|
- **WHEN** the admin requests registration history for an email address
|
||||||
|
- **THEN** the system SHALL return all versions in chronological order
|
||||||
|
|
||||||
|
## ADDED Requirements
|
||||||
|
|
||||||
|
### Requirement: Post-completion conversation state
|
||||||
|
The system SHALL support a "completed" conversation state that allows continued interaction for questions and updates.
|
||||||
|
|
||||||
|
#### Scenario: Conversation continues after completion
|
||||||
|
- **WHEN** a parent sends an email after their registration is complete
|
||||||
|
- **THEN** the system SHALL load the existing conversation and process the message
|
||||||
|
|
||||||
|
#### Scenario: Intent detection for post-completion messages
|
||||||
|
- **WHEN** processing a message in a completed conversation
|
||||||
|
- **THEN** the system SHALL detect intent: question, update request, or new child registration
|
||||||
+48
@@ -0,0 +1,48 @@
|
|||||||
|
## ADDED Requirements
|
||||||
|
|
||||||
|
### Requirement: Notify admin on registration updates
|
||||||
|
The system SHALL send an email notification to the admin when an existing registration is updated, including details of what changed.
|
||||||
|
|
||||||
|
#### Scenario: Update notification sent
|
||||||
|
- **WHEN** a parent updates their completed registration
|
||||||
|
- **THEN** the admin SHALL receive an email notification
|
||||||
|
|
||||||
|
#### Scenario: Update notification content
|
||||||
|
- **WHEN** sending an update notification
|
||||||
|
- **THEN** the notification SHALL include:
|
||||||
|
- Child name and registration ID
|
||||||
|
- What changed (field name, old value → new value)
|
||||||
|
- When the change was made
|
||||||
|
- Version number (e.g., "Version 2 of 2")
|
||||||
|
|
||||||
|
#### Scenario: Update notification routing
|
||||||
|
- **WHEN** sending an update notification
|
||||||
|
- **THEN** the notification SHALL be routed to the same recipients as the original registration (based on playgroup type)
|
||||||
|
|
||||||
|
### Requirement: Distinguish new vs update notifications
|
||||||
|
The system SHALL clearly distinguish between new registration notifications and update notifications in the email subject and content.
|
||||||
|
|
||||||
|
#### Scenario: New registration subject
|
||||||
|
- **WHEN** sending a notification for a new registration
|
||||||
|
- **THEN** the subject SHALL be "New Registration: [Child Name] for [Playgroup Type]"
|
||||||
|
|
||||||
|
#### Scenario: Update notification subject
|
||||||
|
- **WHEN** sending a notification for a registration update
|
||||||
|
- **THEN** the subject SHALL be "Registration Updated: [Child Name]"
|
||||||
|
|
||||||
|
## MODIFIED Requirements
|
||||||
|
|
||||||
|
### Requirement: Email reminders for incomplete registrations
|
||||||
|
The system SHALL send reminder emails for incomplete registrations, but SHALL NOT threaten data deletion since data no longer expires.
|
||||||
|
|
||||||
|
#### Scenario: Reminder content without expiration warning
|
||||||
|
- **WHEN** sending a reminder for an incomplete registration
|
||||||
|
- **THEN** the reminder SHALL encourage completion but SHALL NOT mention data expiration or deletion
|
||||||
|
|
||||||
|
#### Scenario: Reminder schedule unchanged
|
||||||
|
- **WHEN** an incomplete registration exists
|
||||||
|
- **THEN** reminders SHALL be sent at Day 3, Day 10, and Day 25 after last activity
|
||||||
|
|
||||||
|
#### Scenario: Reminders stop after completion
|
||||||
|
- **WHEN** a registration is completed
|
||||||
|
- **THEN** no further reminders SHALL be sent for that conversation
|
||||||
@@ -0,0 +1,56 @@
|
|||||||
|
## 1. Update Conversation Storage
|
||||||
|
|
||||||
|
- [ ] 1.1 Modify `ConversationStore` to key conversations by normalized email address
|
||||||
|
- [ ] 1.2 Add `normalize_email()` helper function (lowercase, trim)
|
||||||
|
- [ ] 1.3 Update `_conversation_path()` to use email-based filename
|
||||||
|
- [ ] 1.4 Add `find_by_email()` method to replace thread-ID-based lookup
|
||||||
|
|
||||||
|
## 2. Update Email Channel
|
||||||
|
|
||||||
|
- [ ] 2.1 Remove `_resolve_thread_id()` from conversation matching logic
|
||||||
|
- [ ] 2.2 Pass sender email to agent instead of thread ID for conversation lookup
|
||||||
|
- [ ] 2.3 Keep thread ID handling for outbound reply headers (`In-Reply-To`, `References`)
|
||||||
|
- [ ] 2.4 Store most recent inbound message ID for reply threading
|
||||||
|
|
||||||
|
## 3. Update Agent Core
|
||||||
|
|
||||||
|
- [ ] 3.1 Modify `process_message()` to lookup conversation by email address
|
||||||
|
- [ ] 3.2 Add post-completion intent detection (question vs. update vs. new child)
|
||||||
|
- [ ] 3.3 Handle registration updates in completed conversations
|
||||||
|
- [ ] 3.4 Update prompts to guide LLM for post-completion states
|
||||||
|
|
||||||
|
## 4. Implement Versioned Registration Storage
|
||||||
|
|
||||||
|
- [ ] 4.1 Create versioned storage structure for registrations
|
||||||
|
- [ ] 4.2 Implement `save_registration_version()` method
|
||||||
|
- [ ] 4.3 Implement `get_registration_history()` method
|
||||||
|
- [ ] 4.4 Track change summary (which fields changed) between versions
|
||||||
|
- [ ] 4.5 Update `save_registration()` to use versioning for updates
|
||||||
|
|
||||||
|
## 5. Update Admin Notifications
|
||||||
|
|
||||||
|
- [ ] 5.1 Add `notify_registration_update()` method to `AdminNotifier`
|
||||||
|
- [ ] 5.2 Create email template for update notifications (include diff)
|
||||||
|
- [ ] 5.3 Distinguish "New Registration" vs "Registration Updated" subjects
|
||||||
|
- [ ] 5.4 Include version number in update notifications
|
||||||
|
|
||||||
|
## 6. Update Reminders
|
||||||
|
|
||||||
|
- [ ] 6.1 Remove expiration warnings from reminder templates
|
||||||
|
- [ ] 6.2 Update reminder messages to encourage completion without deletion threat
|
||||||
|
- [ ] 6.3 Remove any scheduled data cleanup jobs (if present)
|
||||||
|
|
||||||
|
## 7. Update Specs and Documentation
|
||||||
|
|
||||||
|
- [ ] 7.1 Update `conversation-flow.md` to remove expiration language
|
||||||
|
- [ ] 7.2 Update `channel-config.md` state management section
|
||||||
|
- [ ] 7.3 Update sample responses to remove expiration references
|
||||||
|
- [ ] 7.4 Update CLAUDE.md with new conversation matching behavior
|
||||||
|
|
||||||
|
## 8. Testing
|
||||||
|
|
||||||
|
- [ ] 8.1 Test: New email creates new conversation
|
||||||
|
- [ ] 8.2 Test: Follow-up email (same address, different thread) continues conversation
|
||||||
|
- [ ] 8.3 Test: Post-completion question is answered correctly
|
||||||
|
- [ ] 8.4 Test: Post-completion update creates new version and notifies admin
|
||||||
|
- [ ] 8.5 Test: Email address normalization works correctly
|
||||||
Reference in New Issue
Block a user