Respan MCP Server

List all experiments in your organization.

Retrieve detailed information about a specific experiment by its ID.

Create and run an experiment. Processes a dataset's inputs through a workflow chain (prompt / model / passthrough) and scores results with evaluator pipelines. REQUIRED: dataset_id, workflow, evaluator_workflow_ids. WORKFLOW TYPES (these are how each dataset row produces an output): - "prompt": Us…

List all spans (execution traces) for a specific experiment.

Retrieve detailed information about a specific span within an experiment.

Permanently delete an experiment and its spans. This action cannot be undone.

Compute average score per evaluator for an experiment by walking the spans client-side. Use this when the backend summary/histogram endpoints return empty score aggregates (known issue on some experiments). Returns avg, min, max, and count per evaluator. Pages through up to max_spans (default 500).

List all prompts in your Respan organization. Returns a paginated list of all prompts you have created in Respan. RESPONSE FIELDS (per prompt): - id: Unique prompt identifier (use this for other prompt operations) - name: Prompt name/title - description: Prompt description - created_at: Creation t…

Retrieve detailed information about a specific prompt. Returns complete prompt data including: - id: Unique prompt identifier - name: Prompt name/title - description: Prompt description - messages: The prompt template messages (array of role/content objects) - model: Default model for this prompt -…

List all versions of a specific prompt. Returns all versions of a prompt, allowing you to track changes over time. RESPONSE FIELDS (per version): - id: Version identifier - version: Version number (integer, starts at 1) - prompt_id: Parent prompt identifier - messages: The prompt template for this…

Retrieve detailed information about a specific version of a prompt. Returns complete version data including: - id: Version identifier - version: Version number - prompt_id: Parent prompt identifier - messages: Full prompt template messages array - Each message has: role (system/user/assistant), c…

Create a new prompt template. Only sets name and description. Use create_prompt_version to add content.

Update a prompt's name and/or description.

Create a new version of a prompt. The version is always created as NOT deployed.

Update an existing prompt version. Always keeps deploy: false.

Deploy a specific prompt version, making it the active version that experiments (and other workflows) will use. Background: when you create a prompt version, it starts as a draft (not deployed). The platform requires at least one DEPLOYED version before a prompt can be referenced by version number …

List all workflows (automations, monitors, evaluator pipelines) in your organization.

Filter workflows by type and other fields. Use the filters parameter to scope by type: - { "type": { "value": ["automations"], "operator": "eq" } } - { "type": { "value": ["monitors"], "operator": "eq" } } - { "type": { "value": ["evaluators"], "operator": "eq" } }

Retrieve detailed information about a workflow including its task definitions.

Create a new workflow. Workflows are event-driven pipelines with chained tasks. TYPES: - "monitors": Aggregation + threshold monitoring with notifications (default, visible on Monitors page) - "automations": Triggered actions on log/trace events (Automations page) TRIGGER EVENT TYPES: - "request_l…

Update a workflow's configuration, tasks, or metadata.

List all versions of a workflow.

Retrieve a specific version of a workflow.

Commit the current draft of a workflow/pipeline, locking it as a read-only version that can be deployed. REQUIRED before deploy_workflow. The deploy endpoint rejects calls if no committed version exists. Calls POST /api/workflows/{id}/commits/ (the correct platform endpoint — different from the SDK…

Deploy a committed workflow/pipeline version as the active (live) version. Calls POST /api/workflows/{id}/deployments/ (the correct platform endpoint — different from the SDK's deployWorkflow). If version is omitted, deploys the latest committed version. REQUIREMENT: must call commit_workflow firs…

Undeploy a workflow, stopping it from processing events.

Validate a workflow by running it against a sample log. Returns validation results and any errors.

List all datasets in your organization.

Retrieve detailed information about a specific dataset.

Create a new dataset. MODES: - Empty dataset: pass is_empty=true. No time range needed. - Sampled from logs: pass start_time, end_time, and optionally sampling (1-100) and initial_log_filters. - Duplicate existing: pass source_dataset_id to copy logs from another dataset.

Update a dataset's name and/or description.

List all logs (data points) in a dataset with pagination and filtering.

Retrieve a specific log from a dataset by its unique ID.

Import existing logs into a dataset by time range and filters. Runs in the background.

Permanently delete a dataset and all its logs. This action cannot be undone.

Replace (full overwrite) a log in a dataset. Updates input, output, expected_output, and/or metadata fields.

Remove one or more logs from a dataset by filter. To delete a single log, pass filter { unique_id: { operator: "eq", value: "<log_id>" } }. Pass is_deleting_all_logs=true to wipe the dataset contents.

Get aggregated summary statistics for logs in a dataset. Pass filters to scope the summary; omit filters to summarize all logs.

Create one or more logs in a dataset. Pass a single-item array to insert one log. Each log can include input, output, expected_output, metadata, and metrics.

List evaluation run results for a dataset. Shows past eval runs with status and results.

List all evaluators in your organization with pagination.

Retrieve detailed information about a specific evaluator including its config.

Create a new evaluator (grader). Evaluators score LLM outputs. REQUIRED: name, type, score_value_type. TYPES: - "llm": LLM-based evaluation. Requires llm_config with model + evaluator_definition. - "code": Code-based evaluation. Requires code_config with eval_code_snippet. - "human": Manual human …

Test-run a grader with sample inputs to verify it scores correctly BEFORE committing. Required keys in inputs: at least "input" and "output". Optional: "expected_output", "metrics", "metadata". Example: { "evaluator_id": "abc123", "inputs": { "input": "What is 2+2?", "output": "4", "expected_o…

Commit the current draft of a grader, creating a new read-only version. IMPORTANT: Only commit AFTER a successful test_evaluator run. After committing, use create_evaluation_pipeline to wrap the grader in a V2 pipeline that renders in the UI.

List all versions (commits) of an evaluator.

Update an existing evaluator's configuration.

Permanently delete an evaluator. This action cannot be undone.

Run an evaluator on a single log/span to verify it works. This is for quick verification of one record (e.g. confirm an evaluator scores as expected before running broader experiments). For scoring many records, create an experiment instead. Returns the actual score (boolean_value / numerical_valu…

Create an evaluator pipeline (V2 — Blockly visual editor compatible) that renders in the Evaluators page UI. Pipelines wrap committed graders into a workflow. Use this AFTER creating + committing a grader with create_evaluator + commit_evaluator. PATTERNS: - Single grader: steps=[{grader_id:…

List evaluator pipelines (V2). These are the items shown on the Evaluation Pipelines page in the UI.

Get an evaluator pipeline by ID. Accepts both the family workflow_id and the version PK.

Update an evaluator pipeline. Provide the FULL updated structure (steps, combine, weights). Existing graders are replaced. Tasks are rebuilt automatically.

List and filter LLM request logs. Supports pagination, sorting, time range, and server-side filtering. IMPORTANT: Use the "filters" parameter to filter results server-side. Do NOT fetch all logs and filter client-side. PARAMETERS: - page_size: Number of logs per page (1-50, default 20) - page: Pag…

Retrieve complete details of a single log via GET /api/request-logs/{id}/. Returns full information including: - Full input/output content (input and output fields) - Type-specific fields based on log_type (chat, embedding, workflow, etc.) - Credit and budget check results (limit_info) - Evaluation…

Retrieve aggregated summary statistics for log spans. Returns total_count, total_cost, total_tokens, avg_latency etc. Useful for getting quick insights into your LLM usage without fetching all individual spans. PARAMETERS: - start_time: Start time in ISO 8601 format (required) - end_time: End time…

List and filter traces with sorting, pagination, and server-side filtering. A trace represents a complete workflow execution containing multiple spans (individual operations). IMPORTANT: Use the "filters" parameter to filter results server-side. Do NOT fetch all traces and filter client-side. PAR…

Retrieve the complete hierarchical span tree of a single trace. Returns detailed trace information with the full span_tree structure showing: - All spans in the trace with parent-child relationships - Full input/output for each span - Timing and performance metrics per span - Model and token usage …

List customers/users with pagination and sorting. Retrieves a paginated list of customers who have made API requests through Respan. QUERY PARAMETERS: - page_size: Number of customers per page (max 50 for MCP, API supports up to 1000) - page: Page number (default 1) - sort_by: Sort field. Prefix w…

Retrieve detailed information about a specific customer including budget usage. Returns customer profile and budget data: IDENTIFICATION: - id: Internal customer ID - customer_identifier: Your unique identifier for this customer - email: Customer email (if provided) - name: Customer name (if provi…