Vibe Baseline Tests - Results

Details

Raw Data

{
  "id": "eeb48528-224e-40b6-a9ea-9720745b7b4d",
  "type": "general",
  "status": "completed",
  "title": "Vibe Baseline Tests - Results",
  "description": "6 standalone tests completed with comparison to dashboard",
  "context": {
    "output": "<div style=\"background:#1e1e1e;color:#e0e0e0;padding:20px;border-radius:8px;font-family:'Consolas','Monaco',monospace;\"><h2 style=\"font-family:sans-serif;color:#fff;margin:0 0 16px 0;\">📊 Baseline Test Results</h2><div style=\"background:#2d2d2d;padding:16px;border-radius:6px;margin-bottom:16px;\"><h3 style=\"margin:0 0 12px 0;color:#ffa726;\">⚠️ Previous Turn Incomplete</h3><p style=\"color:#e0e0e0;margin:0;\">The last turn started background tests but timed out before creating a kontask. I've now collected 6 standalone baseline tests.</p></div><div style=\"background:#2d2d2d;padding:16px;border-radius:6px;margin-bottom:16px;\"><h3 style=\"margin:0 0 12px 0;color:#61afef;\">Standalone Results (6 tests)</h3><table style=\"width:100%;border-collapse:collapse;\"><tr style=\"border-bottom:1px solid #3d3d3d;\"><th style=\"padding:8px;text-align:left;color:#9ca3af;\">Prompt</th><th style=\"padding:8px;text-align:right;color:#9ca3af;\">TTFE</th><th style=\"padding:8px;text-align:right;color:#9ca3af;\">Total</th></tr><tr><td style=\"padding:6px;color:#e0e0e0;\">What is 2 + 2?</td><td style=\"padding:6px;text-align:right;color:#e0e0e0;\">6.3s</td><td style=\"padding:6px;text-align:right;color:#9ca3af;\">7.2s</td></tr><tr><td style=\"padding:6px;color:#e0e0e0;\">List 3 colors</td><td style=\"padding:6px;text-align:right;color:#e0e0e0;\">5.9s</td><td style=\"padding:6px;text-align:right;color:#9ca3af;\">6.8s</td></tr><tr><td style=\"padding:6px;color:#e0e0e0;\">Explain TypeScript...</td><td style=\"padding:6px;text-align:right;color:#e0e0e0;\">6.8s</td><td style=\"padding:6px;text-align:right;color:#9ca3af;\">7.4s</td></tr><tr><td style=\"padding:6px;color:#e0e0e0;\">What's the time in Sydney?</td><td style=\"padding:6px;text-align:right;color:#e0e0e0;\">6.3s</td><td style=\"padding:6px;text-align:right;color:#9ca3af;\">7.1s</td></tr><tr><td style=\"padding:6px;color:#e0e0e0;\">What's the current time?</td><td style=\"padding:6px;text-align:right;color:#e0e0e0;\">6.8s</td><td style=\"padding:6px;text-align:right;color:#9ca3af;\">7.7s</td></tr><tr><td style=\"padding:6px;color:#e0e0e0;\">Show me recent files</td><td style=\"padding:6px;text-align:right;color:#fbbf24;\">13.1s</td><td style=\"padding:6px;text-align:right;color:#9ca3af;\">13.8s</td></tr><tr style=\"border-top:2px solid #3d3d3d;background:rgba(201,162,39,0.1);\"><td style=\"padding:8px;color:#c9a227;font-weight:bold;\">Average</td><td style=\"padding:8px;text-align:right;color:#c9a227;font-weight:bold;\">7.5s</td><td style=\"padding:8px;text-align:right;color:#c9a227;font-weight:bold;\">8.3s</td></tr></table></div><div style=\"background:#2d2d2d;padding:16px;border-radius:6px;margin-bottom:16px;\"><h3 style=\"margin:0 0 12px 0;color:#4caf50;\">Dashboard vs Standalone Comparison</h3><table style=\"width:100%;border-collapse:collapse;\"><tr><th style=\"padding:8px;text-align:left;color:#9ca3af;\"></th><th style=\"padding:8px;text-align:center;color:#818cf8;\">Dashboard</th><th style=\"padding:8px;text-align:center;color:#4ade80;\">Standalone</th><th style=\"padding:8px;text-align:center;color:#fbbf24;\">Delta</th></tr><tr><td style=\"padding:8px;color:#e0e0e0;\">Avg TTFE</td><td style=\"padding:8px;text-align:center;color:#818cf8;font-weight:bold;\">3.3s</td><td style=\"padding:8px;text-align:center;color:#4ade80;font-weight:bold;\">7.5s</td><td style=\"padding:8px;text-align:center;color:#4caf50;font-weight:bold;\">-4.2s ✅</td></tr><tr><td style=\"padding:8px;color:#e0e0e0;\">Avg Total</td><td style=\"padding:8px;text-align:center;color:#818cf8;\">54s*</td><td style=\"padding:8px;text-align:center;color:#4ade80;\">8.3s</td><td style=\"padding:8px;text-align:center;color:#9ca3af;\">varies</td></tr></table><p style=\"color:#7f848e;margin:8px 0 0 0;font-size:0.8em;\">* Dashboard total is higher because vibe prompts do real work (tool calls, kontask creation)</p></div><div style=\"background:linear-gradient(135deg,#1a3d1a,#2d2d2d);padding:16px;border-radius:6px;border:1px solid #4caf50;\"><h3 style=\"margin:0 0 8px 0;color:#4caf50;\">🎯 Key Finding</h3><p style=\"color:#e0e0e0;margin:0;line-height:1.6;\"><strong>Dashboard is 4.2s faster for TTFE</strong> because konsole maintains a warm Claude session. Standalone pays ~6-7s startup cost every time. The konui/konsole architecture provides a significant latency improvement, not overhead.</p></div></div>",
    "requestId": "2eb120de-2bfa-4bda-b454-970e70193d86",
    "choices": [
      {
        "label": "View telemetry",
        "value": "Open the telemetry dashboard to see the full visualization",
        "primary": true
      },
      {
        "label": "More tests",
        "value": "Run additional baseline tests with different prompt types"
      },
      {
        "label": "Analyze outlier",
        "value": "The 13.1s 'Show me recent files' test was slow - investigate why"
      }
    ]
  },
  "createdBy": "claude",
  "createdAt": "2026-01-03T08:20:58.053Z",
  "updatedAt": "2026-01-03T08:20:58.477Z",
  "requestId": "2eb120de-2bfa-4bda-b454-970e70193d86",
  "scope": "vibetools",
  "tags": [
    "telemetry",
    "baseline",
    "performance"
  ],
  "targetUser": "claude"
}