What we can diagnose

The signals available when a run goes wrong: structured error payloads, per-mode status tools, mode-switch diffs, on-disk run records, analytics events, and the redacted issue-report bundle.

A
Written By Atakan KayaLast updated 3 months ago

What we can diagnose

LEX AI is instrumented so that when something goes wrong there is enough signal to understand it — live, through structured tool responses, and after the fact, through run artifacts, analytics events and diagnostic bundles.

Live signal: structured tool responses

Every tool returns a structured result, and every payload carries an ok key so the halt rule always has a field to evaluate. On failure the result is ok: false with a troubleshooting block written for you rather than for the model: a summary, what happened, the error detail, the steps to resolve it, and what to do afterwards. There are only two error categories — AUTO_FIXABLE, which returns a minimal non-halting payload, and USER_ACTIONABLE, which always halts.

Two cases carry extra fields:

  • stale_tool_call: true — with requested_tool, active_mode, suggested_mode and tool_surface_epoch. The IDE cached an old tool list after a mode switch.
  • Git failures — the redacted git context travels with the error, so the cause is visible without exposing a token.

This is what lets the assistant tell "you need to act" apart from "retry after a refresh" and put a precise message in front of you.

Progress and status

Every mode exposes a status tool that reports where the run stands, so an interrupted session can be diagnosed without reading the manifest by hand.

Mode

Status tool

Forward, MVP Completion

get_workflow_status()

Brief

get_brief_status()

Backward

get_reverse_status()

Edit

get_edit_status()

Review

get_review_status()

Test

get_test_status()

Input

get_input_change_status()

Deploy

get_deployment_status()

MVP Generator

get_mvp_status()

Forward and MVP Completion add inspect_github_repository(...), which dumps repository metadata, visibility, the default branch, the branch list and whether the default-branch ref exists when a delivery run misbehaves.

What a mode switch reports

switch_to_mode does not change the surface silently. Its response names which tools appeared, which disappeared, which are unchanged, and the new tool-surface epoch — so a disagreement between what the IDE lists and what the server exposes shows up immediately, rather than as a confusing failure three calls later.

Run artifacts

Each mode writes its own record as it goes, which means a finished or abandoned run can be read after the fact by someone who was not there: a session manifest under .lex-workflow/, .lex-brief/, .lex-test/, .lex-deploy/ and the rest, a per-unit report, an auto-maintained _index.md, and a consolidated report such as test-runs/TEST-REPORT.md or deployment/DEPLOYMENT-REPORT.md. State lives in files, not in the chat transcript.

Analytics events

The server records structured analytics events locally as JSONL at ~/.lex-mcp/analytics.jsonl, and optionally forwards them to the remote MCP. Every event carries the same envelope — event id and type, UTC timestamp, session id, run token and run label — over a catalogue that covers run lifecycle, step requests and completions, mode switches, preflight checks, git auto-repair attempts, remote calls, and any tool that returned ok: false. Emission is deliberately non-blocking: telemetry never breaks a tool call.

Diagnostic bundles

lex ai-issue-report collects the configuration and state that support needs — the project .env, the Copilot MCP config and state store, VS Code workspace storage and .vscode — into a timestamped ZIP under .lex-ai-reports/.

Credential values are masked on the way in, not on the way out, because the finished bundle can be submitted to the remote report_bug_ticket tool: an unredacted archive would not sit forgotten in a directory, it would leave the machine. manifest.json carries a redaction block naming every file touched and how many values were masked in each, and the CLI prints the total, so support can tell a masked credential from one that was never configured. Files too large to scan in memory are dropped rather than shipped unscanned. The ZIP is always written first; ticket submission failing never blocks it.

Health checks

lex ai-verify makes the on-disk assets match what the installed packages ship: it restores anything missing or drifted and prunes stale mode assets that would otherwise leave a mixed tool surface. It is also the pre-flight, run before every tool call, which is why it must print nothing on success — stdout would corrupt the JSON-RPC stream. Write failures are always reported.

What this adds up to

Between structured errors, per-mode status tools, on-disk run records, analytics events and the issue-report bundle, most failures can be pinned to a specific mode, step and cause — without guessing, and without a secret leaving your machine.

Was this helpful?

Your feedback shapes what we write next.