tpu-devops

Runs a deep logic check with latency reporting.

Saves a Hugging Face API token to GCP Secret Manager as secret 'hf-token'. Note: the token passes through the conversation; for maximum privacy the user can instead run `echo -n <token> | gcloud secrets versions add hf-token --data-file=-` themselves (after creating the secret once).

Generates the gcloud command for a single-host TPU v6e vLLM deployment.

Safely deletes a Queued Resource and its node. Zone defaults to the server's current zone (which `find_tpu` updates when it secures capacity elsewhere).

Ensures the primary Queued Resource exists — and DELETES every other queued resource in the zone (plus the primary itself if FAILED/SUSPENDED, so it can be recreated). Destructive by design; confirm with the user before running it in a zone that may hold resources they want kept. To create without a…

Creates a TPU Queued Resource (Flex-start or reserved) with the vLLM startup script. Non-destructive: unlike `manage_queued_resource`, it never deletes other resources — if one with this ID already exists it just reports its state.

Creates a flex-start TPU VM as a GCE instance (recommended path for v6e/v5p) and auto-starts vLLM via the startup script. Boot disk defaults to 200GB because the image default (10GB) cannot hold the vLLM TPU image.

Lists GCE TPU VM instances (ct6e/ct5p machine types) across all zones, or one zone.

Deletes a GCE TPU VM instance. Flex-start bills until deletion — confirm with the user before destroying anything they may still need.

Tails the serial-console output of a GCE TPU VM. SSH to TPU VMs is often blocked by firewall policy, so this is the primary way to watch startup-script/vLLM boot progress. Success marker: 'vLLM application startup complete.'

Returns the vLLM endpoint URLs of a GCE TPU VM and probes their health. Port 8000 is frequently unreachable from outside the VPC (firewall) even when serving is healthy — if both probes fail, check `get_tpu_vm_serial_log` for the startup-complete marker.

Polls a GCE flex-start TPU VM every 30s until vLLM serving is ready, checking the health endpoint and the serial-console startup marker. One call replaces manual serial-log polling; model load typically takes ~10 min. Also fails fast if the startup script logs an ERROR (e.g. Secret Manager access de…

Retrieves the GCP zones that have a non-zero quota limit for a specific metric.

Sweeps every zone with available quota, creating the queued resource and polling until one reaches ACTIVE. Side effects: deletes the created resource in each zone that fails or times out; on success switches the server's default zone to the winning zone; records failed zones in ~/.cache/tpu-devops/t…

GCE counterpart of `find_tpu`: tries flex-start TPU VM creation across zones until one grants capacity. TPUS_PER_TPU_FAMILY quota is only discoverable by attempting creation, so quota failures fail fast and the sweep moves on; each attempt's request expires after `per_zone_wait` so no pending reques…

Manages the vLLM Docker container on the serving host. Targets the queued resource's node by default, or a GCE flex-start TPU VM when instance_name is given.

Lists all Queued Resources in a specific zone (defaults to the current zone).

Provides detailed information about a specific Queued Resource, including its lifecycle state and expiry.

Estimates the cost of a TPU deployment. `topology` is a chip grid like '2x4'.

High-level dashboard covering both serving paths: GCE flex-start TPU VMs (all zones) and Queued Resources (current zone), plus vLLM health.

Returns the active vLLM service URL if available.

Queries the self-hosted model (the configured MODEL_NAME) on the active TPU deployment. With include_stats=True, streams the response and also reports TTFT, total generation time, and tokens/second.

Runs vLLM's internal benchmark tool in a separate container on the serving host (queued resource node by default, or a GCE flex-start TPU VM via instance_name). `model` defaults to the model actually being served (else MODEL_NAME).

Retrieves the last `tail` lines of the vLLM Docker container's logs from the serving host (queued resource node by default, or a GCE flex-start TPU VM via instance_name). Bounded — the full log of a long-serving container can run to megabytes.

Retrieves systemd logs for a specific service from the serving host (queued resource node by default, or a GCE flex-start TPU VM via instance_name).

Fetches logs from Google Cloud Logging.

Summarizes recent TPU errors from Cloud Logging using the self-hosted Gemma 4 model.

Retrieves detailed information about the running model, vLLM engine, and versions.

Provides help text and summarizes the configuration options and all available SRE/DevOps tools for this TPU VM MCP server.

Fetches Prometheus metrics from the running vLLM service's /metrics endpoint. Returns the key serving metrics by default; raw=True returns the full dump (large — mostly histogram buckets).

Finds available GPU resources (GCE VMs, Cloud Run services, and zones with available GPU quota) in the GCP project.