CLI Reference

Run evaluations headlessly from the command line or in CI pipelines.

Installation

uv sync # or: pip install -e .

Tip: activate the virtualenv to drop the uv run prefix.

Commands

validate

Validate a tools YAML file. Exit code 0 = pass, 1 = fail.

        # Basic validation

        uv run mcp-lens validate tools.yaml

        # With LLM checks

        uv run mcp-lens validate tools.yaml --llm

report

Generate a full evaluation report.

        # JSON output to file

        uv run mcp-lens report tools.yaml --format json --output report.json

        # Plain text output

        uv run mcp-lens report tools.yaml --format text

        # With LLM checks

        uv run mcp-lens report tools.yaml --llm

compare

Compare two reports for regressions.

uv run mcp-lens compare baseline.json current.json

security

Security-only analysis.

uv run mcp-lens security tools.yaml

CI Integration

GitHub Actions example:

        # .github/workflows/mcp-eval.yml

        - name: Install dependencies

          run: uv sync

        - name: Validate MCP tools

          run: uv run mcp-lens validate tools/my-server.yaml

        - name: Generate report

          run: uv run mcp-lens report tools/my-server.yaml --format json --output report.json

        - uses: actions/upload-artifact@v4

          with:

            name: mcp-eval-report

            path: report.json