LLM Evaluation Harness MCP Server

Item: LLM Evaluation Harness MCP Server
Rating: 69
Author: M8ven

Evaluates RAG outputs on faithfulness, answer relevancy, and context precision using an LLM-as-a-Judge backend. Exposes tools for running evaluations, scoring individual samples, and checking thresholds, enabling CI gating and on-demand assessment via MCP.

→ SaiArja/llm-eval-mcp · listed on glama

Is this your MCP?

Claim it to get a verified publisher badge, a free copy of our full audit findings, and direct contact for any high-priority issues we find.

Install from

Glama

M8ven verifies MCPs across every public registry — install directly from whichever one you prefer.

// key findings

✅

No credential exfiltration, no sensitive file access, no obfuscation

Static analysis found nothing flowing your secrets to unexpected places.

✅

Open source with a license and README

Anyone can audit the code, the license is declared, and the publisher documents what it does.

🔐

You'll be asked for 1 credential: GEMINI_API_KEY

These are read from process.env at runtime. Make sure you trust where they’ll be sent.

// required environment variables

This server reads these from process.env. You'll be asked to provide them before it can run.

🔐 secretGEMINI_API_KEY— set =... (https://aistudio.google.com/app/apikey)

configJUDGE_MODEL

configMIN_FAITHFULNESS

configMIN_ANSWER_RELEVANCY

configMIN_CONTEXT_PRECISION

configUSE_RAGAS— RAGAS/DeepEval (set =true).

// full audit trail

The full breakdown of what we checked, the deductions that landed, the network hosts, the dependency advisories, and concrete fix guidance is available to verified publishers.

// improvement guidance — verified publishers only

We have 5 concrete improvements we can share with the publisher of this MCP. Each comes with specific guidance to raise the trust score.

// embed badge in your README

[![M8ven Score](https://m8ven.ai/badge/mcp/saiarja-llm-eval-mcp-gmbg43)](https://m8ven.ai/mcp/saiarja-llm-eval-mcp-gmbg43)

commit: 0e50dbb9fa244f83a4c14e2e4c27d54fb70e8e9b

code hash: e26dc2183c442e1f224454fa3e04b8ebdcf2cf717b575deaa56321abd18f9edb

verified: 7/13/2026, 9:50:39 AM

view raw JSON →