Log Mcp

The log-mcp MCP Server provides log file analysis capabilities for Large Language Models. It processes massive log files locally and extracts semantically interesting lines.

The MCP server uses a Rust TF-IDF classifier that scans at 1.3 million lines per second to separate semantically interesting lines from routine noise. An optional BERT-mini stage refines the classification on GPU for higher precision.

log-mcp currently provides 7 tools that handle error deduplication, pattern comparison, time-range extraction, and anomaly detection across formats, including JSON, syslog, Spark/Log4j, and tab-delimited logs.

Features

• Scans log files at 1.3M lines/sec using a Rust TF-IDF classifier to identify interesting lines.
• Refines classification with an optional BERT-mini model that runs at 2K lines/sec on Metal GPU.
• Auto-detects log formats including JSON, standard text timestamps, syslog, Spark/Log4j, and tab/pipe-delimited formats.
• Normalizes variable parts (UUIDs, hex IDs, IPs, numbers) so messages differ only in IDs or timestamps group as the same pattern.
• Falls back to regex heuristics (fatal:, Permission denied, ##[error]) when log files lack standard log levels.
• Distinguishes patterns from different job steps in CI logs using prefix-aware comparison.
• Works without the Rust classifier by falling back to Python log parsing.
• Groups errors by fingerprint, counts frequencies, and extracts stack traces.

Use Cases

• Compress a 67-megabyte Spark executor log into five distinct error groups with extracted stack traces.
• Find hidden anomalies in high-performance computing logs lacking explicit error levels.
• Compare two continuous integration log files to isolate unique failure patterns in the broken run.
• Extract specific log segments based on exact time ranges or line numbers.
• Generate volume histograms and level breakdowns for system health overviews.
• Identify rapid hardware faults like instruction cache parity errors.

How To Use It

1. The MCP server requires Python with uv and Rust. On a fresh Mac, install the prerequisites:

brew install python uv
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

2. Install the MCP server

Claude Code

Install https://github.com/ascii766164696D/log-mcp as an MCP server and build the Rust classifier too

Manual Installation

git clone https://github.com/ascii766164696D/log-mcp.git
cd log-mcp
# Register the MCP server
claude mcp add log-mcp -- uv run --directory $(pwd) log-mcp
# Build the Rust classifier (optional — tools fall back to Python without it)
uv pip install -e rust/classifier

Manual Configuration

Add to your Claude project settings under mcpServers:

{
  "mcpServers": {
    "log-mcp": {
      "command": "uv",
      "args": ["run", "--directory", "/path/to/log-mcp", "log-mcp"]
    }
  }
}

Claude Desktop Configuration

Open Settings > Developer > Edit Config and add to claude_desktop_config.json:

{
  "mcpServers": {
    "log-mcp": {
      "command": "uv",
      "args": ["run", "--directory", "/path/to/log-mcp", "log-mcp"]
    }
  }
}

Available Tools

FAQS

Q: What happens if I do not install the Rust classifier?
A: The tools fall back to standard Python log parsing. The system processes all lines sequentially. The classify_lines tool throws an error.

Q: Does the classifier require standard log levels to find errors?
A: The classifier identifies semantic anomalies independent of log levels. It detects DNS failures, DHCP lease errors, and monitoring data collisions. These events often lack explicit ERROR tags.

Q: Can the tool analyze small log files?
A: You should paste small files directly into the LLM context window. The MCP server adds unnecessary indirection for files under a few hundred lines.

Q: How does the system handle domain-specific state machine reasoning?
A: The classifier evaluates each line independently. It does not track state transitions across time. A Zookeeper expert must manually identify rapid cycling between states.

Q: What hardware do I need for the BERT-mini model?
A: The BERT stage requires a Metal-capable GPU on Apple Silicon. The TF-IDF stage runs on standard CPU architecture.