s18
Introducing s18 — an open-source agent runtime & orchestration framework for real AI systems. ⚡ Multi-agent workflows ⚡ Real-time streaming state ⚡ Scheduling + automation ⚡ MCP tool integrations ⚡ Local-first or cloud models ⚡ Observability built in
Caution. Specific findings reduced this grade. They are listed on the page. Grades reflect the full trust pyramid: code, verification depth, and reputation. New projects cap at C until adoption is earned.
How we verified
Verified is a snapshot. Live keeps it current, and builds your track record.
⚡ Connect GitHub → continuous verification on every pushwhy connect →Who stands behind it
riteshverma
Source: github_code
Claim it to get a verified publisher badge, a free copy of our full audit findings, and direct contact for any high-priority issues we find. Or connect your repo for our deepest verification, Live Monitored: read-only, revoke anytime. What we access →
Install from
The grade above is for the source repository. Registries can serve a different version, so we mark the ones we were not able to read.
These names and descriptions are the publisher's own, read from the source code. We print them as written. Our assessment is the findings above, not this list.
dynamic_tool_378web_searchSearch the web using multiple engines (DuckDuckGo, Bing, Ecosia, etc.) and return a list of relevant result URLs
web_extract_textExtract readable text from a webpage using robust methods (Playwright/Trafilatura).
search_web_with_text_contentSearch web and return URLs with extracted text content. Gets both URLs and readable text from top search results. Ideal for exhaustive research.
fetch_search_urlsGet top website URLs for your search query. Just gets the URLs not the contents.
webpage_url_to_raw_textExtract readable text from a webpage.
browser_use_actionExecute a complex browser task using Vision and generic reasoning. Use this for: Logging in, filling forms, navigating complex sites, or when text search fails. WARNING: Slow and expensive.
get_patient_recordsRetrieve patient demographic/meta record with runtime cache, privacy, and delta handling. Returns "No record found" for 404/empty responses.
search_labsRetrieve patient labs with query/time filtering plus privacy and delta tracking. Returns "No record found" for 404/empty responses.
preview_documentPreview a document using the AI-enhanced extraction logic used for indexing.
ask_documentAsk a question about a specific document. Incorporates chat history, relevant document extracts, and optional image input.
search_stored_documents_ragSearch stored documents using HYBRID search (BM25 + vector + entity gating). Returns relevant chunks with source information. Optionally provide doc_path to search within a specific document only.
keyword_searchSearch for exact keyword matches across all indexed document chunks. Returns a list of document paths that contain the matching text.
advanced_ripgrep_searchPowerful regex/keyword search using Ripgrep. Returns structured results with file, line number, and match content. Set regex=True for pattern matching (e.g. r"error:.*"). Optionally provide target_dir to search within a specific subdirectory of data/.
reindex_documentsTrigger a manual re-index of the RAG documents. Optionally provide a target_path (relative to data/ folder) to index a specific file. If force is True, it wipes the index and performs a full fresh scan.
get_indexing_statusGet the current indexing progress status as JSON.
index_imagesBackground Worker: Scans for un-captioned images, captions them using Vision Model, and updates the index.
run_python_scriptExecute Python code in a secure sandbox. Use this for math, data processing, and logic. Returns the stdout and result of the execution.
read_workspace_fileRead a UTF-8 text file from the ClawBench task workspace. Paths may be relative to the workspace root or absolute within it.
write_workspace_fileWrite UTF-8 text to a file inside the ClawBench task workspace. Creates parent directories as needed. Use for notes, patches, and deliverables.
GEMINI_API_KEYAZURE_OPENAI_ENDPOINTOPENAI_API_VERSIONAZURE_OPENAI_API_KEYLLAMA_CPP_API_KEYEXTERNAL_MOCKEHR_BASE_URLWISE_MOCKEHR_BASE_URLMOCKEHR_PRIMARY_SOURCE_LABELMOCKEHR_HTTP_TIMEOUT_SECONDSAZURE_OPENAI_CHAT_DEPLOYMENTS18_SESSION_SUMMARY_SYNCS18_WORKSPACE_ROOTS18_PROFILEOLLAMA_BASE_URLOLLAMA_TIMEOUTLLAMA_CPP_BASE_URLLLAMA_CPP_TIMEOUTRUN_POLL_TIMEOUT_SECONDSMCP_TOOL_TIMEOUT_SECONDSMCP_MODEMCP_STARTUP_TIMEOUT_SECONDSMCP_STDIO_CONNECT_TIMEOUT_SECONDSMCP_REQUIRED_SERVERSSCHEDULER_TIMEZONEAZURE_OPENAI_EMBEDDING_DEPLOYMENTAUTH_ENABLEDSUPABASE_JWT_AUDIENCESUPABASE_LOGGING_ENABLEDTENANCY_DEFAULT_TENANT_IDTENANCY_DEFAULT_TIERTENANCY_DEFAULT_DATA_REGIONTENANCY_GROWTH_ROUTING_ENABLEDS18_MODEL_PROVIDERAGENT_MODEL_PROVIDERS18_FORCE_GEMINICELERY_BROKER_URLCELERY_RESULT_BACKENDS18_CELERY_RUNS_QUEUES18_CELERY_INGEST_QUEUES18_CELERY_TIMEZONES18_CELERY_PREFETCH_MULTIPLIERAWS_REGIONBEDROCK_EMBEDDING_MODEL_IDGOOGLE_CLOUD_PROJECTVERTEX_AI_LOCATIONVERTEX_AI_EMBEDDING_MODELVERTEX_AI_EMBEDDING_DIMENSIONGEMINI_EMBEDDING_MODELGEMINI_EMBEDDING_DIMENSIONS18_FAISS_GPU_DEVICES18_FAISS_GPU_MIN_VECTORSS18_RUN_EXECUTORS18_HARNESS_STATE_DIRGEMINI_STATUS_EXPOSE_KEY_PREVIEWGOOGLE_API_KEYCLAWBENCH_USE_OLLAMACLAWBENCH_S18_MAX_STEPSCLAWBENCH_S18_MAX_LIFELINESCLAWBENCH_S18_MCP_MODECLAWBENCH_OLLAMA_WAIT_SECONDSCLAWBENCH_OLLAMA_WARMUP_ATTEMPTSCLAWBENCH_KEEP_WORKSPACESAZURE_STORAGE_ACCOUNTAZURE_STORAGE_CONTAINERS3_BUCKETS3_KMS_KEY_IDGCS_BUCKETAZURE_SEARCH_ENDPOINTOPENSEARCH_ENDPOINTVERTEX_AI_INDEX_ENDPOINT_IDVERTEX_AI_DEPLOYED_INDEX_IDVERTEX_AI_INDEX_IDSUPABASE_ANON_KEYSUPABASE_SERVICE_ROLE_KEYSUPABASE_URLTool annotations
No tools have read-only/destructive annotations
Add readOnlyHint or destructiveHint annotations to every tool so hosts can warn users before invoking.
All four hints declared on every tool
20/20 tools missing one or more hints — dynamic_tool_378 (missing: readOnlyHint, destructiveHint, idempotentHint, openWorldHint); web_search (missing: readOnlyHint, destructiveHint, idempotentHint, openWorldHint); web_extract_text (missing: readOnlyHint, destructiveHint, idempotentHint, openWorldHint), +17 more. OpenAI's directory rejects tools where any of the four hints are missing or non-boolean.
For every tool, set all four hints (readOnlyHint, destructiveHint, idempotentHint, openWorldHint) to explicit true/false values that match the handler’s actual behaviour.
Destructive tools are labelled
1 tool perform destructive updates without destructiveHint — reindex_documents deletes at line 2164 (path.unlink())
Add destructiveHint:true to any tool whose handler calls .delete(), .upsert(), .update(), unlink, rm, DELETE, DROP, REPLACE INTO, or any operation that overwrites existing data.
Descriptions match behaviour
1 tool describes read intent but its handler mutates — keyword_search (line 1096: result = subprocess.run(cmd, capture_output=True, text=True))
Rename the tool, rewrite the description, or move the side-effect into a separate clearly-named tool.
Tool test coverage
6/20 tools referenced in tests (30%)
Write tests that reference each tool by name so every tool has at least one test.
Secrets stay with their owner
3 secrets sent to a request target we could not resolve (AZURE_OPENAI_API_KEY → dynamic, AZURE_OPENAI_API_KEY → dynamic) — often a configured endpoint, not necessarily third-party
Audit where credentials are sent. A NOTION_TOKEN should only reach api.notion.com — never a third-party host.
Tool description accuracy
keyword_search: description implies read-only but handler writes/deletes/executes
Update tool descriptions to accurately reflect all capabilities — especially write, delete, or execute operations.
Claim the listing to review these findings one by one and send us a correction where you disagree, straight to the team. Claiming also means we tell you when the grade moves, and reach you first if we find anything urgent.
[](https://m8ven.ai/mcp/riteshverma/s18)?variant=verified from the URL.Vetting this one by hand? Tool Check is an MCP that scores other MCPs. Add it once and ask Claude, ChatGPT, or any MCP client to grade a server, surface CVEs, check the publisher, and suggest safer alternatives — before you install.
https://m8ven.ai/api/mcp/tool-check