MCP Server (failure_mcp)

Exposes FailureAnalyzer capabilities as callable tools over the Model Context Protocol (MCP) stdio transport. Compatible with Claude Desktop, Cursor, VS Code Copilot Chat, and any other MCP-compatible AI client.

Server

MCP server entry point for mdb_api_layer.

Registers all tools from failure_mcp.tools.analyze_test_failure and starts the server on the stdio transport (default for local MCP clients).

The server exposes the following tools to any MCP-compatible AI client (Claude Desktop, Cursor, VS Code Copilot Chat, etc.):

  • analyze_failure — run LLM diagnosis on a single test failure.

  • get_results — return accumulated results filtered by minimum confidence.

  • save_results — flush results to failure_analysis.json.

Usage:

poetry run python -m failure_mcp.server

MCP client config example (~/.cursor/mcp.json or Claude Desktop settings):

{
  "mcpServers": {
    "failure-analyzer": {
      "command": "poetry",
      "args": ["run", "python", "-m", "failure_mcp.server"],
      "cwd": "/path/to/mdb_api_layer",
      "env": {
        "AI_ANALYSIS_ENABLED": "true",
        "GROQ_API_KEY": "<your-key>"
      }
    }
  }
}
failure_mcp.server.app = 'failure-analyzer'

The mcp.server.Server instance named "failure-analyzer". Decorated handler functions below are registered against this instance.

async failure_mcp.server.call_tool(name: str, arguments: dict)

Receive and forward a tool invocation from the MCP client. A pure call handling router.

Acts as a thin router: every tool call that arrives over the stdio transport is immediately delegated to handle_call(), which contains the actual dispatch logic and calls into the FailureAnalyzer singleton.

Parameters:
  • name – Tool name as sent by the MCP client. Expected values are "analyze_failure", "get_results", or "save_results".

  • arguments – Dict of arguments validated by the client against the JSON Schema advertised in list_tools().

Returns:

List of mcp.types.TextContent with the JSON result, as returned by handle_call(). Example return:

[TextContent(type="text", text='{"saved": 3, "path": "ai_analysis/failure_analysis.json"}')]

Raises:

ValueError – Propagated from handle_call if name is unknown.

async failure_mcp.server.list_tools()

Advertise all available tools to the connecting MCP client.

Called automatically by the MCP protocol during the handshake phase. Returns the static TOOLS list defined in the tools module so that tool definitions are maintained in one place.

Returns:

List of mcp.types.Tool objects — one entry per exposed capability (analyze_failure, get_results, save_results). Example return:

[
    Tool(name="analyze_failure", description="Send a test failure...", inputSchema={...}),
    Tool(name="get_results", description="Return all diagnosis results...", inputSchema={...}),
    Tool(name="save_results", description="Flush all accumulated results...", inputSchema={...}),
]

Tools

MCP tool definitions for AI-powered test failure analysis.

Exposes three tools:

  • analyze_failure — send a failure context to the LLM and get a structured diagnosis.

  • get_results — retrieve accumulated diagnosis results (with optional confidence filter).

  • save_results — flush all results to <output_dir>/failure_analysis.json.

This module is imported by failure_mcp/server.py, which registers the handlers on the mcp.server.Server instance.

Tool Schemas

Each entry in TOOLS is a mcp.types.Tool that carries:

  • name — identifier used by the MCP client when calling the tool.

  • description — human-readable summary shown in the client UI.

  • inputSchema — JSON Schema object describing required and optional arguments.

failure_mcp.tools.analyze_test_failure.TOOLS: list[mcp.types.Tool] = [mcp.types.Tool, mcp.types.Tool, mcp.types.Tool]

List of mcp.types.Tool objects advertised to MCP clients.

Contains three tools:

analyze_failure

Required args: test_name, error_message. Optional args: test_file, traceback, api_url, status_code, response_body.

get_results

Optional arg: min_confidence (int 0–100, default 0).

save_results

Optional arg: output_dir (str, default "ai_analysis").

failure_mcp.tools.analyze_test_failure._confidence_tier(score: int) → str[source]

A pure helper def: Map a 0–100 confidence score the LLM returns to a human-readable tier label.

Used to attach a confidence_tier field alongside the raw numeric confidence value in the analyze_failure response, so MCP clients can display or filter results without implementing their own thresholds. This will enrich the diagnosis before returning it to the client without modifying tests.helpers.failure_analyzer.FailureAnalyzer.

Thresholds:

Range

Tier

>= 80

high

50 – 79

medium

< 50

low

Parameters:

score – Integer confidence score (0–100) returned by the LLM.

Returns:

One of "high", "medium", or "low".

Example:

>>> _confidence_tier(92)
'high'
>>> _confidence_tier(65)
'medium'
>>> _confidence_tier(30)
'low'
async failure_mcp.tools.analyze_test_failure.handle_call(name: str, arguments: dict[str, Any]) → list[mcp.types.TextContent][source]

Dispatch an MCP tool call to the appropriate FailureAnalyzer method.

Called by failure_mcp/server.py’s call_tool hook for every tool invocation arriving from the MCP client. Delegates to the analyzer singleton without modifying failure_analyzer.py.

Tool behavior:

analyze_failure

Calls analyze() with the supplied arguments dict as the failure_context. On success, enriches the diagnosis with a confidence_tier field. On failure or when analysis is disabled, returns an error and hint dict instead.

get_results

Reads _results and filters entries whose confidence is >= min_confidence (defaults to 0, i.e. return everything).

save_results

Calls save_results() with the given output_dir (defaults to "ai_analysis"), then returns the count of results written and the output file path.

Parameters:
  • name – Tool name requested by the MCP client. Must be one of "analyze_failure", "get_results", or "save_results".

  • arguments – Key/value arguments supplied by the MCP client, validated against the JSON Schema in TOOLS.

Returns:

A single-element list of mcp.types.TextContent whose text field contains the JSON-serialized result (2-space indent).

Raises:

ValueError – If name does not match any registered tool.

Example response for analyze_failure:

{
  "root_cause": "...",
  "category": "auth_error",
  "suggested_fix": "...",
  "confidence": 92,
  "explanation": "...",
  "evidence": ["HTTP 401", "Response: Invalid API key"],
  "test_name": "test_get_movie_details",
  "model": "qwen/qwen3.8-27b",
  "confidence_tier": "high"
}

When analysis is disabled or fails, analyze_failure instead returns:

{"error": "Analysis disabled or failed.", "hint": "Set AI_ANALYSIS_ENABLED=true and GROQ_API_KEY in the environment."}

get_results returns a JSON array of the stored diagnoses (without the confidence_tier field, which only analyze_failure adds):

[
  {
    "root_cause": "...",
    "category": "auth_error",
    "suggested_fix": "...",
    "confidence": 92,
    "explanation": "...",
    "evidence": ["HTTP 401"],
    "test_name": "test_get_movie_details",
    "model": "qwen/qwen3.8-27b"
  }
]

save_results returns the count written and the output path:

{"saved": 3, "path": "ai_analysis/failure_analysis.json"}