Turn History Data Loss Investigation

Details

Raw Data

{
  "id": "3edbe481-d56a-484e-a839-180acea59970",
  "type": "general",
  "status": "completed",
  "title": "Turn History Data Loss Investigation",
  "description": "Analysis of why completed turn history shows less detail than live streaming",
  "context": {
    "output": "<div style=\"background:#1e1e1e;color:#e0e0e0;padding:20px;border-radius:8px;font-family:'Consolas','Monaco',monospace;\">\n<h2 style=\"font-family:sans-serif;color:#fff;margin:0 0 16px 0;\">🔍 Turn History Data Loss - Root Cause Analysis</h2>\n\n<h3 style=\"color:#61afef;margin-top:20px;\">The Problem</h3>\n<p style=\"color:#e0e0e0;line-height:1.6;\">When a turn is <strong>running</strong>, you see rich detail (Claude's voice, file diffs, bash commands, TODO progress). When the turn is <strong>completed</strong> and you revisit the history, you only see a collapsed summary with tool icons.</p>\n\n<h3 style=\"color:#61afef;margin-top:20px;\">Root Cause Found</h3>\n<p style=\"color:#e0e0e0;line-height:1.6;\">The data is being stored correctly, but the <strong>history restoration code only renders summaries</strong> for completed turns.</p>\n\n<div style=\"background:#2d2d2d;padding:16px;border-radius:6px;margin:16px 0;\">\n<h4 style=\"color:#98c379;margin:0 0 8px 0;\">📁 Data Flow Overview</h4>\n<pre style=\"color:#abb2bf;font-size:0.9rem;line-height:1.8;\">\n<span style=\"color:#7f848e;\">LIVE STREAMING (rich detail):</span>\n┌─────────────────────────────────────────────────────────────────┐\n│ Claude → <span style=\"color:#e5c07b;\">processClaudeEvent()</span> → SSE → WebSocket → Browser     │\n│                                                                 │\n│ Events streamed: text, tool_use, tool_result, diff, bash...    │\n│ Browser renders: full diff boxes, bash output, Claude voice    │\n└─────────────────────────────────────────────────────────────────┘\n\n<span style=\"color:#7f848e;\">HISTORY RESTORATION (summary only):</span>\n┌─────────────────────────────────────────────────────────────────┐\n│ kontexts/{sessionId}.json → /sessions/:id/history → Browser    │\n│                                                                 │\n│ <span style=\"color:#c678dd;\">workingHistory</span> contains: type, tool, <span style=\"color:#e06c75;\">summary only</span>, timestamp │\n│ Browser renders: \"📖 Reading file.ts\", \"💻 cmd...\"             │\n└─────────────────────────────────────────────────────────────────┘\n</pre>\n</div>\n\n<h3 style=\"color:#61afef;margin-top:20px;\">What's Lost</h3>\n<table style=\"width:100%;border-collapse:collapse;margin:12px 0;\">\n<tr style=\"background:#2d2d2d;\">\n  <th style=\"padding:8px;text-align:left;border-bottom:1px solid #444;color:#98c379;\">Data Type</th>\n  <th style=\"padding:8px;text-align:left;border-bottom:1px solid #444;color:#98c379;\">Live</th>\n  <th style=\"padding:8px;text-align:left;border-bottom:1px solid #444;color:#98c379;\">History</th>\n</tr>\n<tr>\n  <td style=\"padding:8px;border-bottom:1px solid #333;\">Claude's voice (bullet points)</td>\n  <td style=\"padding:8px;border-bottom:1px solid #333;\">✅ Full text</td>\n  <td style=\"padding:8px;border-bottom:1px solid #333;\">❌ Truncated to 500 chars, not rendered</td>\n</tr>\n<tr>\n  <td style=\"padding:8px;border-bottom:1px solid #333;\">Edit diffs (old/new string)</td>\n  <td style=\"padding:8px;border-bottom:1px solid #333;\">✅ Full diff box</td>\n  <td style=\"padding:8px;border-bottom:1px solid #333;\">❌ Only \"Editing file.ts\"</td>\n</tr>\n<tr>\n  <td style=\"padding:8px;border-bottom:1px solid #333;\">Bash command output</td>\n  <td style=\"padding:8px;border-bottom:1px solid #333;\">✅ Syntax highlighted</td>\n  <td style=\"padding:8px;border-bottom:1px solid #333;\">❌ Only command truncated to 80 chars</td>\n</tr>\n<tr>\n  <td style=\"padding:8px;border-bottom:1px solid #333;\">Tool result content</td>\n  <td style=\"padding:8px;border-bottom:1px solid #333;\">✅ Expandable blocks</td>\n  <td style=\"padding:8px;border-bottom:1px solid #333;\">❌ Only summary line</td>\n</tr>\n<tr>\n  <td style=\"padding:8px;border-bottom:1px solid #333;\">TODO progress</td>\n  <td style=\"padding:8px;border-bottom:1px solid #333;\">✅ Visual checklist</td>\n  <td style=\"padding:8px;border-bottom:1px solid #333;\">⚠️ In details but not rendered</td>\n</tr>\n</table>\n\n<h3 style=\"color:#61afef;margin-top:20px;\">Code Evidence</h3>\n\n<div style=\"background:#2d2d2d;padding:12px;border-radius:6px;margin:8px 0;\">\n<p style=\"color:#7f848e;margin:0 0 8px 0;\">📍 <span style=\"color:#e5c07b;\">konsole/src/stream.ts:738-743</span> - Text is truncated when stored:</p>\n<pre style=\"color:#abb2bf;font-size:0.85rem;overflow-x:auto;\"><span style=\"color:#7f848e;\">// Add to working history for session state (truncate to avoid bloat)</span>\naddToWorkingHistory(sessionId, {\n  type: <span style=\"color:#98c379;\">\"text\"</span>,\n  timestamp: <span style=\"color:#c678dd;\">new Date</span>().toISOString(),\n  text: <span style=\"color:#e06c75;\">text.slice(0, 500)</span>,  <span style=\"color:#7f848e;\">// ← Truncated!</span>\n});</pre>\n</div>\n\n<div style=\"background:#2d2d2d;padding:12px;border-radius:6px;margin:8px 0;\">\n<p style=\"color:#7f848e;margin:0 0 8px 0;\">📍 <span style=\"color:#e5c07b;\">konui/src/console-history.ts:145-170</span> - History only renders summaries:</p>\n<pre style=\"color:#abb2bf;font-size:0.85rem;overflow-x:auto;\"><span style=\"color:#7f848e;\">// History restoration loop</span>\n<span style=\"color:#c678dd;\">for</span> (<span style=\"color:#c678dd;\">const</span> item <span style=\"color:#c678dd;\">of</span> (kontext.workingHistory || [])) {\n  <span style=\"color:#c678dd;\">if</span> (item.type === <span style=\"color:#98c379;\">'tool_use'</span>) {\n    line.textContent = <span style=\"color:#98c379;\">'📖 '</span> + <span style=\"color:#e06c75;\">summary</span>;  <span style=\"color:#7f848e;\">// Only summary!</span>\n  } <span style=\"color:#c678dd;\">else if</span> (item.type === <span style=\"color:#98c379;\">'tool_result'</span>) {\n    line.textContent = <span style=\"color:#98c379;\">'   → '</span> + <span style=\"color:#e06c75;\">summary</span>;  <span style=\"color:#7f848e;\">// Only summary!</span>\n  } <span style=\"color:#c678dd;\">else</span> {\n    <span style=\"color:#c678dd;\">continue</span>;  <span style=\"color:#7f848e;\">// ← type:\"text\" is SKIPPED entirely!</span>\n  }\n}</pre>\n</div>\n\n<h3 style=\"color:#61afef;margin-top:20px;\">Fix Options</h3>\n\n<div style=\"background:#2d2d2d;padding:16px;border-radius:6px;margin:12px 0;\">\n<h4 style=\"color:#98c379;margin:0 0 8px 0;\">Option A: Store Rich Details (Recommended)</h4>\n<ul style=\"color:#abb2bf;line-height:1.8;margin:0;padding-left:20px;\">\n  <li>Modify <code style=\"background:#3e3e3e;padding:2px 6px;border-radius:3px;\">addToWorkingHistory()</code> to include <code>input.details</code> for Edit/Bash tools</li>\n  <li>Don't truncate text content (or store full text separately)</li>\n  <li>Update history renderer to display details when available</li>\n  <li><span style=\"color:#e5c07b;\">Impact:</span> Larger kontexts.json files (~2-5KB per turn → ~20-50KB)</li>\n</ul>\n</div>\n\n<div style=\"background:#2d2d2d;padding:16px;border-radius:6px;margin:12px 0;\">\n<h4 style=\"color:#98c379;margin:0 0 8px 0;\">Option B: Fetch On-Demand from Kontext</h4>\n<ul style=\"color:#abb2bf;line-height:1.8;margin:0;padding-left:20px;\">\n  <li>Keep summary storage as-is</li>\n  <li>Add \"expand\" button on history items</li>\n  <li>Fetch full kontext data when user clicks expand</li>\n  <li><span style=\"color:#e5c07b;\">Impact:</span> Requires new API endpoint, more complex UX</li>\n</ul>\n</div>\n\n<div style=\"background:#2d2d2d;padding:16px;border-radius:6px;margin:12px 0;\">\n<h4 style=\"color:#98c379;margin:0 0 8px 0;\">Option C: Stream Events Log (Separate File)</h4>\n<ul style=\"color:#abb2bf;line-height:1.8;margin:0;padding-left:20px;\">\n  <li>Write all SSE events to a separate <code>events/{requestId}.jsonl</code> file</li>\n  <li>History UI replays events from log</li>\n  <li><span style=\"color:#e5c07b;\">Impact:</span> Best fidelity, but ~100KB+ per turn</li>\n</ul>\n</div>\n\n<h3 style=\"color:#61afef;margin-top:20px;\">Files to Modify</h3>\n<pre style=\"background:#2d2d2d;padding:12px;border-radius:6px;color:#abb2bf;line-height:1.8;\">\n<span style=\"color:#56b6c2;font-weight:bold;\">konsole/src/</span>\n├── <span style=\"color:#e5c07b;\">stream.ts</span>           <span style=\"color:#7f848e;\"># Store full details in addToWorkingHistory()</span>\n├── <span style=\"color:#e5c07b;\">session.ts</span>          <span style=\"color:#7f848e;\"># WorkingHistoryItem storage</span>\n└── <span style=\"color:#e5c07b;\">types.ts</span>            <span style=\"color:#7f848e;\"># Add richDetails field to WorkingHistoryItem</span>\n\n<span style=\"color:#56b6c2;font-weight:bold;\">konui/src/</span>\n└── <span style=\"color:#e5c07b;\">console-history.ts</span>  <span style=\"color:#7f848e;\"># Render details + text items</span>\n</pre>\n\n<p style=\"color:#98c379;margin-top:20px;font-weight:bold;\">Recommendation: Start with Option A - it provides the best balance of fidelity and simplicity.</p>\n</div>",
    "requestedAt": "2026-01-10T00:00:00.000Z",
    "requestId": "93fdce30-85f2-4026-a928-b90051472d39",
    "choices": [
      {
        "label": "Implement Option A",
        "value": "Implement Option A: Store rich details in workingHistory. Modify stream.ts to include full input.details for Edit/Bash tools, don't truncate text content, and update console-history.ts to render the details.",
        "primary": true
      },
      {
        "label": "Implement Option B",
        "value": "Implement Option B: Add on-demand fetching. Keep current storage, add expand buttons to history items that fetch full kontext data when clicked."
      },
      {
        "label": "Create VIBE item",
        "value": "Add this to the VIBE.md backlog as a future enhancement rather than implementing now."
      }
    ],
    "turnTiming": {
      "totalMs": 162227,
      "ttfeMs": 131,
      "thinkingMs": 162095,
      "toolExecutionMs": 0,
      "toolCallCount": 0,
      "thinkingPct": 100,
      "toolsPct": 0,
      "ttfePct": 0
    }
  },
  "createdBy": "claude",
  "createdAt": "2026-01-09T21:29:42.391Z",
  "updatedAt": "2026-01-09T21:29:55.996Z",
  "requestId": "93fdce30-85f2-4026-a928-b90051472d39",
  "scope": "vibetools",
  "tags": [
    "console",
    "history",
    "debugging"
  ],
  "sessionId": "ses_df705a8d-ea1",
  "flowId": "flow_01KEHQ5A7ZX7YDB8KG1NP2GE68",
  "flowLinks": [
    {
      "flowId": "flow_01KEHQ5A7ZX7YDB8KG1NP2GE68",
      "edgeType": "evidence",
      "createdAt": "2026-01-09T21:29:42.391Z"
    }
  ]
}