SkillSpector scans (1.0.0c Phase 2)
NVIDIA SkillSpector (Apache-2.0) is a command-line scanner for agent
skills: static rules for prompt injection, exfiltration, supply chain, MCP tool poisoning and least privilege, plus an
optional LLM semantic pass, summed into a 0–100 risk score — SAFE (≤ 20), CAUTION (≤ 50), DO_NOT_INSTALL.
Since 1.0.0c Phase 2 Ramen runs it on every deploy over the skills a group repo ships and over what the zone really
serves, keeps the report on a page per zone, and can gate the deploy on it. The same release teaches the bridge to
scan the skills on the developer's machine before Ramen is used. Ramen pins tag v2.13.4.
Contract: §30. Everything here is off until the engine is turned on in the chart.
Turning it on
skillspector:
engine: skillspector # none (default) | skillspector
image: ghcr.io/bkraad47/ramen-skillspector:2.13.4
SkillSpector has no PyPI package, no image and no server, so Ramen publishes one: deploy/docker/skillspector/Dockerfile
clones the pinned tag and runs the upstream multi-stage build (non-root, WORKDIR /scan, ENTRYPOINT ["skillspector"]),
released as ghcr.io/bkraad47/ramen-skillspector:2.13.4 and :latest next to the console and worker images
(scripts/check_versions.py keeps the tag in one place). engine: skillspector hands the console
RAMEN_SKILLSPECTOR_IMAGE (+ RAMEN_SKILLSPECTOR_TIMEOUT_SECS, 600). Nothing else runs; the Config page shows the
engine and the image under Evals & scans.
What is scanned, and when
A deploy of an environment, per zone, after the evals of that zone, when the engine is on:
- The console builds a scan directory with every immediate subdirectory of the repo's
mcp/skills/that has aSKILL.md(the same synced copy the image build uses) and one synthesized skill,ramen-served-<zone>/SKILL.md, written from the canary'stools/listandprompts/list— name, title, description and input schema as fenced JSON — so the instruction-text rules run against what the zone actually serves, not only against what the repo says. - A Job
ramen-scan-<group>-<zone>-<rand>inramen-system(ServiceAccountramen-evals, the evals Jobs' identity, never the builder's) runsskillspector scan /scan --recursive --format json -o /out/report.json(upstream caps a recursive scan at 32 skills), with--no-llmorSKILLSPECTOR_PROVIDER=<llm>from the group setting below. The directory travels as a ConfigMap (≤ 900 KiB; a largermcp/skillstree is an error of the run naming the size); the report is read back from the pod log. - The job records
job.result[<zone>].skillspector = {score, severity, recommendation, skills, findings, url}(findingsis a count) and keeps the full report (≤ 256 KiB, elsetruncated: true) on the job for the report page.
An optional mcp/skillspector-baseline.yaml in the repo is applied to single-skill scans only — upstream rejects it
with --recursive, so with a baseline Ramen scans skill by skill and merges the results.
The report page
GET /groups/<g>/scans/<job>/<zone> (group members) is a table per skill with each finding's rule, severity,
file:line and message. The tool has no UI of its own, so this page is where the group page's Prompts/Packages
section and the per-zone badge point.
Gate or report
Group admins set skillspector: {gate, required, llm} on the group page next to evals; the API is
PUT /api/v1/groups/{g}/skillspector.
| Setting | Means |
|---|---|
gate: false (default) |
Report only: the result on the job, the badge on the group page, the report page. |
gate: true |
DO_NOT_INSTALL or a run error fails the deploy exactly like a failed golden case (canary scaled down, stable workers untouched). CAUTION is a warning line. |
required |
The client policy switch, below. |
llm |
The name of a configured LLM provider for the semantic pass, or "" for the static rules only (--no-llm). |
LLM providers, part 2
The scan's semantic pass and the evals share Config → LLM providers. Phase 2 added four providers, which since
D82 are rows of that one card: Vertex AI (project, location; the Job's ServiceAccount through Workload Identity, no key), Bedrock
(region; the Job's IRSA identity, no key), Azure AI Foundry (endpoint, api_key, deployment) and any
OpenAI-compatible server (base_url, api_key, headers). Values show as set / not set; only providers that are set
are exported to the Jobs (GOOGLE_CLOUD_PROJECT / GOOGLE_CLOUD_LOCATION, AWS_REGION, AZURE_OPENAI_*,
SKILLSPECTOR_COMPAT_*).
Vertex and Bedrock are the operator's grant
The Jobs run as ramen-evals (§33.4a), which holds no AI role. For an evals provider on Vertex AI give that
identity roles/aiplatform.user; for Bedrock (evals or a scan's LLM pass), bedrock:InvokeModel. SkillSpector
has no Vertex provider: a scan with llm: vertex runs the static rules only (--no-llm). Documented, not automated. A group's
guardrails are different: they reach Vertex and Bedrock through the console's judge proxy as the console's
identity, which Terraform grants (console_vertex, console_bedrock).
Client policy: the bridge's local scan
When the chart runs the engine or the group sets required, the /oauth/token answer gains
skillspector: {version, required, llm, baseline} next to telemetry and evals; version is the first 12 hex
digits of a SHA-256 over required, llm and the baseline text, and /.well-known/ramen-policy answers
skillspector: {version, required}. With required the worker gate applies: a call must carry
_meta["io.ramen/skillspector"] = {"version"} (a 2025-06-18 body through the bridge: params.skillspector) —
missing → -32027 skillspector policy required (HTTP 428), another version → -32027 skillspector policy changed:
sign in again; it runs after the evals gate.
Bridge 1.0.0, the bridge for Ramen 1.0.0, keeps skillspector in its policy bundle, carries the version on every call and signs in again
once on -32027. When the group requires it, policy check — the Claude Code SessionStart hook, the Cursor rule,
policy show — also runs skillspector scan <dir> --recursive --no-llm --format json over the client's own skill
directories (~/.claude/skills, <cwd>/.claude/skills, ~/.codex/skills when present) with a local skillspector
(uv tool install 'git+https://github.com/NVIDIA/SkillSpector.git@v2.13.4' is the hint when it is missing). A
DO_NOT_INSTALL for any skill, or no binary, is exit 3 with the skills named and Ramen policy: fix or remove
these skills — Ramen cannot be used with them; CAUTION is printed. Results are cached per directory content hash in
~/.ramen/scans/<host>/ for 24 h so hooks stay fast. --no-client-scan turns the local scan off; the bundle and the
version contract stay. The same bundle carries the group's guardrails policy since 1.0.0c Phase 2.5 — the rails
the workers run, which the bridge runs locally too and the workers gate with -32029 (Guardrails).
The honest limit
A scan reads instruction text. Ramen blocks what SkillSpector's rules recognise in the repo's skills, in the tools and prompts a zone serves, and in the skill directories of a machine that goes through the bridge; it cannot see a skill a client loads from elsewhere, nor a malicious instruction phrased past the rules (the LLM pass narrows that gap, it does not close it). As with the evals policy, what runs through Ramen is covered; the rest is a convention the hook and the rule state.
What is verified
tests/conformance/test_console_skillspector.py— a real console with a fake scan runner and a group repo the test owns: the group setting, a SAFE scan with the synthesized skill and the report page for members only, a plantedSKILL.mdthat reads~/.sshandcurls it away reported with the gate off and blocked with it on, CAUTION through the gate, a run error, the baseline,skillspectorin the token answer and the policy endpoint.tests/cloud/test_release_features.py(RAMEN_SKILLSPECTOR=1) — on a cluster with the real image: a deploy of the demo repo reportsSAFEorCAUTION, the report page renders, the policy endpoint matches the token answer, the Config page names the engine and the providers.scripts/cloud_smoke.shprintsskillspector: <recommendation>, score <n>, <n> findingsafter the deploy.- The node's
-32027gate and the bridge's local scan (a fakeskillspectoron PATH: clean / caution / block / missing, the cache,--no-client-scan) are covered by their own unit tests.