> ## Documentation Index > Fetch the complete documentation index at: https://mcpjam-mintlify-docs-update-pr-4070-1786998819507.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Command Reference > Complete flag reference for every mcpjam CLI command Complete flag tables for every `mcpjam` command. For guides and recipes, see the individual command pages. Account-bound Cloud commands live under `mcpjam cloud`. Local MCP testing stays at the top level (`mcpjam server`, `mcpjam oauth login`, …). Credential flags `--api-key` / `--api-url` are declared on `mcpjam cloud` and work before or after descendants. Hosted `readiness` stays at the root and still takes leaf `--api-key`. See [Migrating to CLI 4.0](/cli/migration). ## Global flags | Flag | Default | Description | | ------------------- | --------------------------------- | --------------------------------------------------------- | | `--timeout ` | `30000` | Request timeout in milliseconds | | `--rpc` | off | Include raw JSON-RPC logs in JSON output under `_rpcLogs` | | `--quiet` | off | Suppress non-result progress output on stderr | | `--no-telemetry` | off | Disable anonymous telemetry for this invocation | | `--format ` | `human` on TTY, `json` when piped | Raw output format (`json` or `human`) | | `-v, --version` | | Print the CLI version | *** ## `server` commands All server commands accept the shared connection flags below, plus command-specific options. ### Shared connection flags | Flag | Description | | ------------------------------ | ---------------------------------------------------------------------------------------------------------------------- | | `--transport ` | Explicit transport type (`http` or `stdio`) | | `--url ` | HTTP MCP server URL | | `--access-token ` | Bearer access token | | `--oauth-access-token ` | OAuth bearer access token | | `--refresh-token ` | OAuth refresh token | | `--client-id ` | OAuth client ID (with `--refresh-token`) | | `--client-secret ` | OAuth client secret (with `--refresh-token`) | | `--credentials-file ` | Load OAuth credentials from a file created by `oauth login --credentials-out` or `oauth conformance --credentials-out` | | `--header
` | HTTP header `Key: Value` (repeatable) | | `--client-capabilities ` | Client capabilities as inline JSON, `@path`, or `-` for stdin | | `--command ` | Stdio server command | | `--args ` | Preferred stdio command arguments | | `--command-args ` | Legacy stdio command argument (repeatable) | | `-e, --env ` | Stdio environment `KEY=VALUE` values | | `--cwd ` | Working directory for the stdio child process | Transport selection is inferred from `--url` vs `--command` when `--transport` is omitted. Use `--transport http|stdio` when you want an explicit validation step. For stdio targets, child processes inherit the parent shell environment by default. `-e/--env` adds or overrides child env values, and structured debug artifacts only record the explicit env keys you passed on the command line. `--credentials-file` cannot be combined with individual token flags (`--access-token`, `--oauth-access-token`, `--refresh-token`, `--client-id`, `--client-secret`). The CLI rejects conflicting auth sources upfront. ### `server probe` No additional flags beyond shared connection flags. ### `server doctor` | Flag | Description | | -------------- | ---------------------------------------- | | `--out ` | Write the doctor JSON artifact to a file | ### `server info` No additional flags. ### `server validate` No additional flags. ### `server ping` No additional flags. ### `server capabilities` No additional flags. ### `server export` No additional flags. *** ## `tools` commands ### `tools list` Uses shared connection flags, plus: | Flag | Description | | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--host ` | Connect as this host (e.g. `claude`, `chatgpt`, `cursor`) — sends its `clientInfo`, `clientCapabilities`, and protocol version in `initialize`, and hides app-only tools its model can't see. Mutually exclusive with `--client-capabilities`. | | `--cursor ` | Pagination cursor | | `--model-id ` | Model ID used for token counting | When `--host` is set, the output includes a `host` field and a `toolsDroppedVisibility` count. Hosts that opt out of visibility filtering (e.g. `cursor`) keep all tools. ### `tools call` | Flag | Description | | ----------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--host ` | Connect as this host — sends its identity/capabilities/protocol in `initialize`, and rejects app-only tools the host's model can't call. Mutually exclusive with `--client-capabilities`. | | `--tool-name ` | Name of the tool to call | | `--name ` | Legacy alias for `--tool-name` | | `--interactive` | Drive the `input_required` multi-round-trip loop: render embedded elicitations to the terminal and collect responses from stdin. Cannot be combined with `--tool-args -`, `--tool-args-stdin`, or `--params -`. | | `--yes` | With `--interactive`, decline every embedded input request without prompting (non-interactive mode). | | `--tool-args ` | Tool arguments as inline JSON, `@path`, or `-` for stdin | | `--params ` | Legacy alias for `--tool-args` | | `--tool-args-stdin` | Read tool arguments JSON from stdin | | `--validate-response` | Validate the MCP tool-call envelope returned by the server | | `--expect-success` | Fail when the tool result reports `isError` | | `--reporter ` | `json-summary`, `junit-xml`, or `html` validation report output | | `--debug-out ` | Write debug artifact to file | | `--ui` | Attach to Inspector and render the completed tool result in App Builder; opens a browser by default in a TTY | | `--require-render` | Treat skipped Inspector renders as errors (requires `--ui`) | | `--open` | Open Inspector in the system browser before rendering (default with `--ui` in a TTY) | | `--no-open` | Start/use Inspector without opening a system browser | | `--attach-only` | Require an already-running Inspector browser client; do not start or open Inspector | | `--inspector-url ` | Local Inspector backend/API base URL | | `--frontend-url ` | Inspector frontend/browser base URL; skips frontend discovery | | `--server-name ` | Server name to use inside Inspector | | `--protocol ` | Render protocol: `mcp-apps` or `openai-sdk` | | `--device ` | Render device: `mobile`, `tablet`, `desktop`, or `custom` | | `--theme ` | Render theme: `light` or `dark` | | `--locale ` | Render locale | | `--time-zone ` | Render IANA timezone | Plus shared connection flags. Without `--ui`, `tools call` returns the raw tool result. When that result is a JSON object (the usual MCP `CallToolResult`), the default JSON output also includes `_durationMs` — wall-clock milliseconds spent in the tool call itself, excluding connection setup and the `--host` visibility probe, so it measures the same window as the `durationMs` returned by `POST /v1/projects/{projectId}/servers/{serverId}/tools/call`. Arrays and non-objects are left unchanged so a list-shaped payload is not rewritten. `--reporter` keeps its own `durationMs`, which is end-to-end (connection setup plus the call, stopped before validation, Inspector render, and debug I/O), and does not get `_durationMs`. With `--ui`, it opens Inspector by default in a TTY and returns a compact envelope with `result`, `inspectorBrowserUrl`, and `inspectorRender` status. Object-shaped `result` values carry `_durationMs`; the envelope root does not. `inspectorRender.status` is `rendered` when Inspector accepted the render, `skipped` when the tool succeeded but Inspector had no active browser client, an unsatisfied render precondition, or a render timeout, and `error` for non-recoverable render command failures. `inspectorRender.remediation` is always present and is one of `open_browser`, `retry`, `reconnect_server`, or `none`. Skipped renders are emitted as a stable root `warning` plus `inspectorRender.warning`, both with the shape `{ code, message, remediation, browserUrl?, hasActiveClient?, inspectorStarted? }`. Stable skipped-render codes are `no_active_client`, `timeout`, `disconnected_server`, and `unsupported_in_mode`. Skipped renders keep the tool-call exit code unless `--require-render` is set; tool failures, validation failures, non-skippable render command errors, and `--require-render` skipped renders all exit nonzero. `--attach-only` is an exception to the skipped-render rule for `no_active_client`: by default a missing browser client yields `inspectorRender.status = "skipped"` with `inspectorRender.remediation = "open_browser"`, but when `--attach-only` is set, `no_active_client` is treated as non-skippable, surfaces as a root `error` (not a downgraded `warning`), and exits nonzero like other non-skippable render failures. `--inspector-url` points to the Inspector backend/API; pass `--frontend-url` when you already know the browser/client URL and want to skip health-advertised frontend checks and local dev port discovery. Use `--no-open` when browser automation already opened `inspectorBrowserUrl`; use `--attach-only` when startup, browser opening, and discovery should all be disallowed. Default non-TTY `--ui` runs do not open a browser unless `--open` is passed. When `--open` is in effect (default in TTYs, opt-in elsewhere), the App Builder URL and the initial browser-client wait progress are emitted to stderr unless `--quiet` is set, regardless of whether stderr is a TTY; only the elapsed-seconds heartbeat is gated on stderr being a TTY. The Inspector path injects the completed tool result through `renderToolResult`; it does not call the tool a second time. Fresh tabs do not hydrate the injected render state; use the active Inspector client that received the render. Use `--debug-out` for the full render envelope including params and command responses. `--ui` cannot be combined with `--reporter`. ### Reading `tools call --ui` output as an agent Treat the tool result and the Inspector render as separate outcomes. An exit code of `0` means the tool call succeeded and no hard render error occurred; it does not, by itself, prove the UI rendered. Confirm UI delivery with `inspectorRender.status === "rendered"`. If `inspectorRender.status === "skipped"`, branch on `inspectorRender.remediation` or the stable root `warning.code`. If `--require-render` is set, the same skipped-render issue moves from root `warning` to root `error` and the command exits with code `1`. ```json theme={"theme":"css-variables"} { "success": true, "command": "tools call", "inspectorUi": true, "inspectorBrowserUrl": "http://127.0.0.1:6274/#app-builder", "result": { "content": [{ "type": "text", "text": "view created" }], "_durationMs": 42 }, "inspectorRender": { "status": "skipped", "remediation": "open_browser", "mode": "active-client", "urlHydratesRender": false, "browserUrl": "http://127.0.0.1:6274/#app-builder", "hasActiveClient": false, "inspectorStarted": false, "warning": { "code": "no_active_client", "message": "Inspector has no active browser client. Open the Inspector App Builder URL in your browser, then rerun `tools call --ui`; or pass `--open` to let the CLI open a system browser.", "remediation": "open_browser", "browserUrl": "http://127.0.0.1:6274/#app-builder", "hasActiveClient": false, "inspectorStarted": false } }, "warning": { "code": "no_active_client", "message": "Inspector has no active browser client. Open the Inspector App Builder URL in your browser, then rerun `tools call --ui`; or pass `--open` to let the CLI open a system browser.", "remediation": "open_browser", "browserUrl": "http://127.0.0.1:6274/#app-builder", "hasActiveClient": false, "inspectorStarted": false } } ``` *** ## `resources` commands ### `resources list` Uses shared connection flags, plus: | Flag | Description | | ------------------- | --------------------------------------------------------------------------------------------------------------------------------- | | `--host ` | Connect as this host — sends its identity/capabilities/protocol in `initialize`. Mutually exclusive with `--client-capabilities`. | | `--cursor ` | Pagination cursor | ### `resources read` | Flag | Description | | ---------------------- | --------------------------------------------------------------------------------------------------------------------------------- | | `--host ` | Connect as this host — sends its identity/capabilities/protocol in `initialize`. Mutually exclusive with `--client-capabilities`. | | `--resource-uri ` | URI of the resource to read | | `--uri ` | Legacy alias for `--resource-uri` | | `--interactive` | Drive the `input_required` multi-round-trip loop: render embedded elicitations to the terminal and collect responses from stdin. | | `--yes` | With `--interactive`, decline every embedded input request without prompting (non-interactive mode). | Plus shared connection flags. ### `resources templates` Uses shared connection flags, plus: | Flag | Description | | ------------------- | --------------------------------------------------------------------------------------------------------------------------------- | | `--host ` | Connect as this host — sends its identity/capabilities/protocol in `initialize`. Mutually exclusive with `--client-capabilities`. | | `--cursor ` | Pagination cursor | *** ## `prompts` commands ### `prompts list` Uses shared connection flags, plus: | Flag | Description | | ------------------- | --------------------------------------------------------------------------------------------------------------------------------- | | `--host ` | Connect as this host — sends its identity/capabilities/protocol in `initialize`. Mutually exclusive with `--client-capabilities`. | | `--cursor ` | Pagination cursor | ### `prompts get` | Flag | Description | | ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--host ` | Connect as this host — sends its identity/capabilities/protocol in `initialize`. Mutually exclusive with `--client-capabilities`. | | `--prompt-name ` | Name of the prompt | | `--name ` | Legacy alias for `--prompt-name` | | `--interactive` | Drive the `input_required` multi-round-trip loop: render embedded elicitations to the terminal and collect responses from stdin. Cannot be combined with `--prompt-args -`. | | `--yes` | With `--interactive`, decline every embedded input request without prompting (non-interactive mode). | | `--prompt-args ` | Prompt arguments as inline JSON, `@path`, or `-` for stdin | Plus shared connection flags. *** ## `xaa` commands ### `xaa run` Run the Cross-App Access (ID-JAG) debugger: self-issue an ID-JAG, redeem it at the target authorization server (RFC 7523), and call the MCP server with the resulting access token. See the [XAA Debugger guide](/cli/xaa) for the trust setup, registration strategies, and CI usage. | Flag | Required | Default | Description | | --------------------------------------- | -------- | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `--url ` | Yes | | Target MCP server URL (the protected resource) | | `--issuer-base-url ` | Yes | | Origin the local mock IdP issues from. It must publish the CLI's local signing key — typically a running [local inspector](/cli/xaa#before-you-run-make-the-issuer-reachable), not your real IdP | | `--sub ` | Yes | | Simulated end-user subject identifier | | `--client-id ` | No | | OAuth client ID. Required for `preregistered`; rejected for `dcr`/`cimd` | | `--registration ` | No | `preregistered` | `preregistered`, `dcr`, or `cimd` | | `--client-metadata-url ` | No | | CIMD only: the Client ID Metadata Document URL to present as the `client_id` | | `--client-auth ` | No | `none` | CIMD only: `none` (public) or `private-key-jwt` (confidential — generates a local EC P-256 key, publishes it via the hosted reflector, and signs a `client_assertion`) | | `--cimd-metadata-origin ` | No | `https://app.mcpjam.com` | Confidential CIMD only: bare origin hosting the metadata-document reflector. An `http://` loopback origin is a dev-only opt-in; rejected when `--https-only` is set | | `--authz-server-issuer ` | No | | Target AS issuer. When set, protected-resource metadata discovery is skipped | | `--token-endpoint ` | No | | AS token endpoint. When set, AS-metadata discovery is skipped. Not valid with `dcr` or `cimd` | | `--email ` | No | | Simulated end-user email claim | | `--client-secret ` | No | | OAuth client secret presented at redemption | | `--token-endpoint-auth-method ` | No | | `client_secret_basic`, `client_secret_post`, or `none`. Not valid with `dcr` or `cimd` | | `--assertion-format ` | No | `oidc` | Identity assertion format: `oidc` (ID token) or `saml` (SAML 2.0 assertion) | | `--scopes ` | No | | Space-separated scope string | | `--https-only` | No | | Reject non-HTTPS / private targets | #### Registration strategies | Strategy | Description | | --------------- | --------------------------------------------------------------------------------------- | | `preregistered` | Supply a pre-registered `--client-id` (and optional `--client-secret`) | | `dcr` | Dynamic Client Registration (RFC 7591) — the CLI registers a client at the RAS each run | | `cimd` | Client ID Metadata Document — the client publishes its identity at a URL | #### Confidential CIMD (`--client-auth private-key-jwt`) When `--registration cimd --client-auth private-key-jwt` is used, the CLI loads or generates a local EC P-256 key pair (stored in `~/.mcpjam`). The public key is encoded into a reflector URL on `app.mcpjam.com`, which becomes the `client_id`. The CLI signs a `client_assertion` with the private key at token-redemption time; the private key never leaves the machine. ```bash theme={"theme":"css-variables"} mcpjam inspector start # serves the issuer at http://localhost:6274/api/mcp/xaa mcpjam xaa run \ --url http://localhost:8080/mcp \ --issuer-base-url http://localhost:6274/api/mcp \ --sub user@example.com \ --registration cimd \ --client-auth private-key-jwt ``` For a cloud authorization server, expose the inspector origin through a tunnel and pass the public origin as `--issuer-base-url` — see [making the issuer reachable](/cli/xaa#before-you-run-make-the-issuer-reachable). **Key rotation:** the key is the identity. Deleting `~/.mcpjam/xaa-client-private.pem` or changing the `XAA_CLIENT_PRIVATE_KEY` environment variable generates a new `client_id`; any RAS-side allowlisting must be updated after rotation. *** ## `oauth` commands ### `oauth login` | Flag | Required | Default | Description | | ----------------------------- | -------- | -------------- | ------------------------------------------------------------------------ | | `--url ` | Yes | | MCP server URL | | `--protocol-version ` | Yes | | `2025-03-26`, `2025-06-18`, or `2025-11-25` | | `--registration ` | Yes | | `cimd`, `dcr`, or `preregistered` | | `--auth-mode ` | No | `interactive` | `headless`, `interactive`, or `client_credentials` | | `--client-id ` | No | | OAuth client ID | | `--client-secret ` | No | | OAuth client secret | | `--client-metadata-url ` | No | | CIMD metadata document URL | | `--redirect-url ` | No | Auto-generated | OAuth redirect URL | | `--scopes ` | No | | Space-separated scope string | | `--header
` | No | | HTTP header `Key: Value` (repeatable) | | `--step-timeout ` | No | `30000` | Per-step timeout | | `--verify-tools` | No | | After login, list tools | | `--verify-call-tool ` | No | | Also call the named tool | | `--credentials-out ` | No | | Write OAuth credentials to file (mode 0600); stdout has secrets redacted | | `--debug-out ` | No | | Write debug artifact to file | ### `oauth conformance` | Flag | Required | Default | Description | | ----------------------------- | -------- | -------------- | ---------------------------------------------------------------------------------------------------- | | `--url ` | Yes | | MCP server URL | | `--protocol-version ` | Yes | | `2025-03-26`, `2025-06-18`, or `2025-11-25` | | `--registration ` | Yes | | `cimd`, `dcr`, or `preregistered` | | `--auth-mode ` | No | `interactive` | `headless`, `interactive`, or `client_credentials` | | `--client-id ` | No | | OAuth client ID | | `--client-secret ` | No | | OAuth client secret | | `--client-metadata-url ` | No | | CIMD metadata document URL | | `--redirect-url ` | No | Auto-generated | OAuth redirect URL | | `--scopes ` | No | | Space-separated scope string | | `--header
` | No | | HTTP header `Key: Value` (repeatable) | | `--step-timeout ` | No | `30000` | Per-step timeout | | `--verify-tools` | No | | After OAuth, list tools | | `--verify-call-tool ` | No | | Also call the named tool | | `--conformance-checks` | No | | Run additional negative OAuth checks, including DCR redirect URI policy and redirect-mismatch probes | | `--credentials-out ` | No | | Write OAuth credentials to file (mode 0600); stdout has secrets redacted | | `--print-url` | No | | Print consent URL to stderr (interactive only) | | `--reporter ` | No | | `json-summary` or `junit-xml` CI report output | ### `oauth conformance-suite` | Flag | Required | Default | Description | | --------------------------- | -------- | ------- | ---------------------------------------------------------------------------------------- | | `--config ` | Yes | | Path to JSON config file | | `--verify-tools` | No | | Enable tool listing on all flows | | `--verify-call-tool ` | No | | Call the named tool after listing | | `--credentials-out ` | No | | Write OAuth credentials from the first flow that returns credentials to file (mode 0600) | | `--reporter ` | No | | `json-summary` or `junit-xml` CI report output | ### `oauth metadata` | Flag | Required | Description | | ------------- | -------- | --------------------------- | | `--url ` | Yes | OAuth metadata URL to fetch | ### `oauth proxy` / `oauth debug-proxy` | Flag | Required | Default | Description | | ------------------- | -------- | ------- | ----------------------------------------------------------- | | `--url ` | Yes | | OAuth request URL | | `--method ` | No | `GET` | HTTP method | | `--header
` | No | | HTTP header `Key: Value` (repeatable) | | `--body ` | No | | Request body as JSON, raw string, `@path`, or `-` for stdin | *** ## `protocol` commands ### `protocol conformance` | Flag | Required | Default | Description | | --------------------------- | -------- | ------- | ---------------------------------------------------------------------------------------------------------------------- | | `--url ` | Yes | | MCP server URL | | `--access-token ` | No | | Bearer access token | | `--credentials-file ` | No | | Load OAuth credentials from a file created by `oauth login --credentials-out` or `oauth conformance --credentials-out` | | `--header
` | No | | HTTP header `Key: Value` (repeatable) | | `--check-timeout ` | No | `15000` | Per-check timeout in milliseconds | | `--category ` | No | all | Restrict checks to one or more categories | | `--check-id ` | No | all | Restrict checks to one or more check IDs | | `--reporter ` | No | | `json-summary` or `junit-xml` CI report output | Use `--format json|human` for raw output and `--reporter json-summary|junit-xml` for CI reports. ### `protocol conformance-suite` | Flag | Required | Default | Description | | ----------------------- | -------- | ------- | ---------------------------------------------- | | `--config ` | Yes | | Path to JSON config file | | `--reporter ` | No | | `json-summary` or `junit-xml` CI report output | *** ## `apps` commands ### Shared connection flags | Flag | Description | | ------------------------------ | ---------------------------------------------------------------------------------------------------------------------- | | `--transport ` | Explicit transport type (`http` or `stdio`) | | `--url ` | HTTP MCP server URL | | `--access-token ` | Bearer access token | | `--oauth-access-token ` | OAuth bearer access token | | `--refresh-token ` | OAuth refresh token | | `--client-id ` | OAuth client ID (with `--refresh-token`) | | `--client-secret ` | OAuth client secret (with `--refresh-token`) | | `--credentials-file ` | Load OAuth credentials from a file created by `oauth login --credentials-out` or `oauth conformance --credentials-out` | | `--header
` | HTTP header `Key: Value` (repeatable) | | `--client-capabilities ` | Client capabilities as inline JSON, `@path`, or `-` for stdin | | `--command ` | Stdio server command | | `--args ` | Preferred stdio command arguments | | `--command-args ` | Legacy stdio command argument (repeatable) | | `-e, --env ` | Stdio environment `KEY=VALUE` values | | `--cwd ` | Working directory for the stdio child process | Apps commands share the same transport inference rules as the rest of the CLI: `--url` implies HTTP, `--command` implies stdio, and `--transport` is an optional explicit override. ### `apps conformance` MCP Apps server-side conformance checks. Uses shared connection flags plus: | Flag | Description | | ----------------------- | -------------------------------------------------------- | | `--category ` | Check category to run (`tools`, `resources`). Repeatable | | `--check-id ` | Specific check id to run. Repeatable | | `--reporter ` | `json-summary` or `junit-xml` CI report output | Use `--format json|human` for raw output and `--reporter json-summary|junit-xml` for CI reports. ### `apps conformance-suite` | Flag | Required | Default | Description | | ----------------------- | -------- | ------- | ---------------------------------------------- | | `--config ` | Yes | | Path to JSON config file | | `--reporter ` | No | | `json-summary` or `junit-xml` CI report output | *** ## `compat` command Check whether an MCP server's tools and widgets work on each AI host. Uses shared server connection flags (see [`server` commands](#server-commands)) plus: | Flag | Description | | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | | `--host ` | Only report this host ID. Repeat for several. Default: all. Valid IDs: `claude`, `chatgpt`, `cursor`, `copilot`, `codex`, `goose`, and others. | ```bash theme={"theme":"css-variables"} mcpjam compat --url https://your-server.com/mcp mcpjam compat --url https://your-server.com/mcp --host claude --host chatgpt ``` The result includes a per-host `verdict` (`works`, `degraded`, `blocked`, or `unknown`), a `findings` list explaining each verdict, and a top-level `summary` count. When the tool list is truncated (more than 50 pages), verdicts that would otherwise be `works` are downgraded to `unknown`. *** ## `cloud hosts` commands Manage the hosts saved in your hosted MCPJam projects. All `hosts` commands require an `sk_` API key (via `--api-key` or the `MCPJAM_API_KEY` environment variable) or a prior `mcpjam cloud login`. ### Shared platform flags | Flag | Description | | ----------------- | ----------------------------------------------------------------- | | `--api-key ` | MCPJam `sk_` API key (overrides `MCPJAM_API_KEY`) | | `--api-url ` | MCPJam API base URL (defaults to `https://app.mcpjam.com/api/v1`) | ### `cloud hosts templates` List the built-in host templates usable with `mcpjam cloud hosts create --template`. No additional flags. ### `cloud hosts list` | Flag | Description | | ------------------------ | ------------------------------------------------------------------ | | `--project ` | Project name or ID (defaults to the most recently updated project) | ### `cloud hosts get` | Flag | Required | Description | | ------------------------ | -------- | ------------------ | | `--host ` | Yes | Host name or ID | | `--project ` | No | Project name or ID | ### `cloud hosts create` | Flag | Required | Description | | ------------------------ | -------- | -------------------------------------------------------------------------------- | | `--name ` | Yes | Display name for the new host | | `--project ` | No | Project name or ID | | `--template ` | No | Built-in template ID (see `hosts templates`), e.g. `claude`, `chatgpt`, `cursor` | | `--theme ` | No | Theme for the seeded config: `light` or `dark` (template only) | | `--file ` | No | Host config v2 JSON file (or `-` for stdin) | | `--json ` | No | Inline host config v2 JSON (or `@file`, or `-`) | Provide either `--template` or `--file`/`--json`, not both. ### `cloud hosts update` | Flag | Required | Description | | ------------------------ | -------- | ----------------------------------------------------------- | | `--host ` | Yes | Host name or ID | | `--project ` | No | Project name or ID | | `--name ` | No | New display name | | `--file ` | No | Replacement host config v2 JSON (or `-` for stdin) | | `--json ` | No | Inline replacement host config v2 JSON (or `@file`, or `-`) | ### `cloud hosts delete` | Flag | Required | Description | | ------------------------ | -------- | ------------------ | | `--host ` | Yes | Host name or ID | | `--project ` | No | Project name or ID | *** ## `cloud environments` commands Manage **project environments** in your hosted MCPJam projects. A project environment is a named, live-editable execution bundle — one host, optionally a standalone server group, optionally a pinned skill selection and pinned plugin versions — that eval suites and journeys run against. A project environment is not a Computer sandbox image (those are `mcpjam cloud images`), and not the STDIO environment variables you pass with `--env`. All `environments` commands require an `sk_` API key (via `--api-key` or the `MCPJAM_API_KEY` environment variable) or a prior `mcpjam cloud login`. Reading requires project membership; **creating, updating, archiving, and restoring require project admin** — a key bound to a non-admin gets a `FORBIDDEN` error on those. ### Shared platform flags | Flag | Description | | ----------------- | ----------------------------------------------------------------- | | `--api-key ` | MCPJam `sk_` API key (overrides `MCPJAM_API_KEY`) | | `--api-url ` | MCPJam API base URL (defaults to `https://app.mcpjam.com/api/v1`) | ### The revision workflow Environments use optimistic concurrency. Every write takes `--expected-revision`, the `revision` you last read: ```bash theme={"theme":"css-variables"} mcpjam cloud environments get --environment Staging # note the "revision" field mcpjam cloud environments update --environment Staging --expected-revision 3 --name Prod ``` If someone else changed the environment in between, the write fails with a `CONFLICT` error (HTTP 409) instead of overwriting their edit — re-read it and retry. `CONFLICT` also covers a duplicate name and archive-state errors (archiving something already archived, editing something archived), so read the message. ### `cloud environments list` | Flag | Description | | ------------------------ | ------------------------------------------------------------------ | | `--project ` | Project name or ID (defaults to the most recently updated project) | | `--include-archived` | Include archived environments (needed to find one to restore) | ### `cloud environments get` | Flag | Required | Description | | ---------------------------- | -------- | ---------------------- | | `--environment ` | Yes | Environment name or ID | | `--project ` | No | Project name or ID | ### `cloud environments resolve` Preview what the environment resolves to right now: the host's current config, the closed server set, and the pinned plugin versions. Fails with `CONFLICT` when the environment can't currently produce a runnable configuration (for example a pinned plugin was disabled); `details.code` carries the specific reason. | Flag | Required | Description | | ---------------------------- | -------- | ---------------------- | | `--environment ` | Yes | Environment name or ID | | `--project ` | No | Project name or ID | ### `cloud environments create` | Flag | Required | Description | | ------------------------ | -------- | ------------------------------------------------------------------ | | `--name ` | Yes\* | Display name; must be unique among the project's live environments | | `--host-id ` | Yes\* | ID of the host this environment runs against | | `--project ` | No | Project name or ID | | `--description ` | No | Optional description | | `--file ` | No | Environment JSON file (or `-` for stdin) | | `--json ` | No | Inline environment JSON (or `@file`, or `-`) | \* `--name` and `--host-id` may instead be supplied inside `--file`/`--json`; explicit flags override the same key in the JSON body. Use the JSON body for the structured fields that have no flag — `serverAttachmentId`, `skillSelection`, and `pluginVersionIds`: ```bash theme={"theme":"css-variables"} mcpjam cloud environments create --project Acme --json '{ "name": "Staging", "hostId": "h_123", "skillSelection": { "mode": "explicit", "skillIds": ["sk_1"] } }' ``` ### `cloud environments update` | Flag | Required | Description | | ---------------------------- | -------- | ------------------------------------------------------------- | | `--environment ` | Yes | Environment name or ID | | `--expected-revision ` | Yes | The revision you last read (from `environments get`) | | `--project ` | No | Project name or ID | | `--name ` | No | New display name | | `--host-id ` | No | New host | | `--description ` | No | New description (empty string clears it) | | `--file ` | No | Environment JSON with the fields to change (or `-` for stdin) | | `--json ` | No | Inline environment JSON (or `@file`, or `-`) | Only the fields you pass change. To **clear** `serverAttachmentId`, `skillSelection`, or `pluginVersionIds`, send an explicit `null` in the JSON body — an empty array is rejected, it is not a way to clear: ```bash theme={"theme":"css-variables"} mcpjam cloud environments update --environment Staging --expected-revision 3 \ --json '{ "pluginVersionIds": null }' ``` ### `cloud environments archive` Archiving is reversible and frees the name for a new environment; the row is kept. | Flag | Required | Description | | ---------------------------- | -------- | -------------------------- | | `--environment ` | Yes | Environment name or ID | | `--expected-revision ` | Yes | The revision you last read | | `--project ` | No | Project name or ID | ### `cloud environments restore` Fails with `CONFLICT` if another live environment took the name while this one was archived. Plugin pins whose version no longer exists at all are dropped on restore — compare the returned `pluginVersionIds` against what you archived. | Flag | Required | Description | | ---------------------------- | -------- | ----------------------------------------------------------------------- | | `--environment ` | Yes | Environment name or ID | | `--expected-revision ` | Yes | The revision you last read (use `environments list --include-archived`) | | `--project ` | No | Project name or ID | ### `cloud environments ensure-adhoc` Get or create an **unnamed** environment for a composed stack. Deduplicated by content: the same stack always returns the same environment, with `created: false` on the second call. Ad-hoc environments do not appear in `environments list` — they exist so a one-off combination can be run and reproduced without adding a name to the project's list. Requires project membership, not admin (pinning plugin versions still requires admin). | Flag | Required | Description | | ------------------------- | -------- | ----------------------------------------------------------------------------- | | `--host ` | Yes | Host the stack runs as — the client whose configuration a run is stamped with | | `--server-group ` | No | Standalone server group to pin (omit to use the host's own servers) | | `--model ` | No | Model to run instead of the host's pinned one | | `--computer ` | No | Project-shared sandbox image to pin, so runs boot a fresh computer from it | | `--skill ` | No | Project-shared skill IDs to pin | | `--project ` | No | Project name or ID | ### `cloud environments name` Promote an ad-hoc environment to a named one **in place** — the same id every existing run already points at, so history stays attached. Fails with `CONFLICT` if the environment already has a name. | Flag | Required | Description | | ------------------------- | -------- | --------------------------------------------------------------------------------------------- | | `--environment ` | Yes | The ad-hoc environment to promote, by ID (an unnamed environment has no name to select it by) | | `--name ` | Yes | Display name for the promoted environment | | `--expected-revision ` | Yes | The revision you last read | | `--description ` | No | Optional description | | `--project ` | No | Project name or ID | ### What can be pinned Both pin fields are narrower than they look: * **`skillSelection`** accepts project-shared skills only. Personal skills, skills with supporting files or extra frontmatter, and plugin-component skills are all rejected. * **`pluginVersionIds`** accepts a plugin version only when its plugin is installed and enabled, the version is `ready`, at most one version per plugin is pinned, and none of the version's skills carry supporting files. *** ## `cloud images` commands Manage custom Computer sandbox images in your hosted MCPJam projects. An image is defined by a **blueprint** — YAML with a digest-pinned `base` image, `initialize` steps baked into the image at build time, and `maintenance` / `knowledge` text delivered to the agent at runtime (never executed automatically): ```yaml theme={"theme":"css-variables"} base: debian:bookworm-slim@sha256:<64-hex-digest> initialize: - name: Install tooling run: apt-get update && apt-get install -y git jq maintenance: - name: Refresh deps run: cd ~/app && npm install knowledge: - name: Test notes contents: Run `make test` before pushing. ``` All `images` commands require an `sk_` API key (via `--api-key` or the `MCPJAM_API_KEY` environment variable) or a prior `mcpjam cloud login`. ### Shared platform flags | Flag | Description | | ----------------- | ----------------------------------------------------------------- | | `--api-key ` | MCPJam `sk_` API key (overrides `MCPJAM_API_KEY`) | | `--api-url ` | MCPJam API base URL (defaults to `https://app.mcpjam.com/api/v1`) | ### `cloud images list` | Flag | Description | | ------------------------ | ------------------------------------------------------------------ | | `--project ` | Project name or ID (defaults to the most recently updated project) | ### `cloud images get` | Flag | Required | Description | | ------------------------ | -------- | ------------------------ | | `--image ` | Yes | Sandbox image name or ID | | `--project ` | No | Project name or ID | ### `cloud images validate` Lint a blueprint without saving it. Prints `ok` plus the resolved base digest, or structured errors with the YAML path of each violation. | Flag | Required | Description | | ------------------------ | -------- | ---------------------------------------------- | | `--file ` | Yes | Blueprint YAML path, or `-` to read from stdin | | `--project ` | No | Project name or ID | ### `cloud images create` | Flag | Required | Description | | ------------------------ | -------- | ---------------------------------------------- | | `--name ` | Yes | Display name for the new sandbox image | | `--file ` | Yes | Blueprint YAML path, or `-` to read from stdin | | `--project ` | No | Project name or ID | ### `cloud images edit` | Flag | Required | Description | | ------------------------ | -------- | -------------------------------------------------- | | `--image ` | Yes | Sandbox image name or ID | | `--project ` | No | Project name or ID | | `--name ` | No | New display name | | `--file ` | No | Replacement blueprint YAML path (or `-` for stdin) | ### `cloud images build` Trigger an image build for the sandbox image (async). Poll `images logs` to check build status. | Flag | Required | Description | | ------------------------ | -------- | ------------------------ | | `--image ` | Yes | Sandbox image name or ID | | `--project ` | No | Project name or ID | ### `cloud images logs` Show a sandbox image's builds (newest first) with their log preview. | Flag | Required | Description | | ------------------------ | -------- | ------------------------ | | `--image ` | Yes | Sandbox image name or ID | | `--project ` | No | Project name or ID | ### `cloud images use` Boot your computer from this sandbox image. This rebuilds the computer — installed files are wiped. | Flag | Required | Description | | ------------------------ | -------- | ------------------------ | | `--image ` | Yes | Sandbox image name or ID | | `--project ` | No | Project name or ID | ### `cloud images reset` Reset your computer to its current image, wiping mutable state. | Flag | Required | Description | | ------------------------ | -------- | ------------------ | | `--project ` | No | Project name or ID | ### `cloud images promote` Share a personal-draft sandbox image with the whole project. Requires project admin permissions. | Flag | Required | Description | | ------------------------ | -------- | ------------------------ | | `--image ` | Yes | Sandbox image name or ID | | `--project ` | No | Project name or ID | ### `cloud images delete` Permanently delete a sandbox image from a project. | Flag | Required | Description | | ------------------------ | -------- | ------------------------ | | `--image ` | Yes | Sandbox image name or ID | | `--project ` | No | Project name or ID | *** ## `inspector` commands ### `inspector open` Start or attach to the local Inspector and open the UI. | Flag | Required | Description | | ----------------------- | -------- | -------------------------------- | | `--inspector-url ` | No | Local Inspector base URL | | `--tab ` | No | Open Inspector on a specific tab | ### `inspector start` Start the local Inspector in the background without opening a browser. | Flag | Required | Description | | ----------------------- | -------- | ------------------------ | | `--inspector-url ` | No | Local Inspector base URL | ### `inspector stop` Stop the local Inspector if it is running. | Flag | Required | Description | | ----------------------- | -------- | ------------------------ | | `--inspector-url ` | No | Local Inspector base URL | *** ## `cloud tunnel` Expose a local MCP server through an MCPJam relay tunnel and register it as a server in your hosted project, so evals and scenarios can target it. Requires an `sk_` API key or a prior `mcpjam cloud login`. The tunnel stays up until Ctrl-C; the server record outlives the session (calls fail fast at the edge until you re-run, which revives the same URL slug with a fresh secret). ```bash theme={"theme":"css-variables"} # HTTP target mcpjam cloud tunnel http://localhost:9090/mcp --server my-server --project acme # stdio target (command goes after --) mcpjam cloud tunnel --server everything --project acme -- npx -y @modelcontextprotocol/server-everything ``` | Flag | Required | Description | | ------------------------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `--server ` | Yes | Server name to register in the project. An existing server with this name is pointed at the tunnel: its URL is overwritten, and stdio records are converted to HTTP. Hidden alias: `--id`. | | `--project ` | No | Project name or ID (defaults to the most recently updated project) | | `--api-key ` | No | MCPJam `sk_` API key (overrides `MCPJAM_API_KEY`) | | `--api-url ` | No | MCPJam API base URL (defaults to `https://app.mcpjam.com/api/v1`) | | `-e, --env ` | No | Stdio environment assignment in `KEY=VALUE` format (stdio targets only) | | `--cwd ` | No | Working directory for the stdio MCP server process (stdio targets only) | With `--format json`, a single machine-readable startup object (public URL, server ID, slug, project) is written to stdout; ongoing status goes to stderr in both formats. The public tunnel URL embeds a bearer secret and is stored on the project server record so the platform can call it. Every re-run rotates the secret and disconnects any previous tunnel session for the same server. *** ## `readiness` hosted commands Grade a saved server against a publisher's directory as the platform reaches it — through the saved server row and the authorize exchange. This is the hosted half of `readiness check`: it answers a different question (what the platform sees, not what your machine sees), can optionally spend credits for model observations, and leaves a persistent record. All `readiness` hosted commands require an `sk_` API key (via `--api-key` or the `MCPJAM_API_KEY` environment variable) or a prior `mcpjam cloud login`. ### Shared platform flags | Flag | Description | | ----------------- | ----------------------------------------------------------------- | | `--api-key ` | MCPJam `sk_` API key (overrides `MCPJAM_API_KEY`) | | `--api-url ` | MCPJam API base URL (defaults to `https://app.mcpjam.com/api/v1`) | ### `readiness start claude` Start a hosted readiness run that grades a saved server against Anthropic's connector-directory rules. Returns immediately with a run id; poll with `readiness status`. | Flag | Required | Description | | ------------------------- | -------- | ----------------------------------------------------------------------- | | `--server ` | Yes | Saved server to grade | | `--project ` | No | Project name or ID (defaults to the most recently updated project) | | `--ai-observations` | No | Add optional model observations. **Consumes MCPJam credits.** | | `--idempotency-key ` | No | Replay guard: a retry with the same key returns the run already started | ### `readiness start openai` Start a hosted readiness run that grades a saved server against OpenAI's app-directory rules. `--submission-mode` is required and never inferred — omitting it would silently report the package lane as not-applicable. | Flag | Required | Description | | -------------------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | | `--server ` | Yes | Saved server to grade | | `--submission-mode ` | Yes | Declared submission shape: `mcp-only` or `mcp-imported-skills`. The two package shapes need a local package — use `readiness check openai` for those. | | `--project ` | No | Project name or ID | | `--ai-observations` | No | Add optional model observations. **Consumes MCPJam credits.** | | `--idempotency-key ` | No | Replay guard | ### `readiness status` Read one hosted readiness run. The response carries three independent axes: `status` (whether the run finished), `overallStatus` (the grade — a `completed` run can be `not-ready`), and `llmObservations` (whether the optional model pass ran). | Flag | Required | Description | | ------------------------ | -------- | ------------------ | | `--run ` | Yes | Readiness run id | | `--project ` | No | Project name or ID | ### `readiness list` List hosted readiness runs, newest first. | Flag | Required | Description | | ------------------------ | -------- | ------------------------------ | | `--project ` | No | Project name or ID | | `--kind ` | No | Narrow to `claude` or `openai` | | `--server ` | No | Narrow to one saved server | | `--limit ` | No | Rows to return (1–100) | ### `readiness cancel` Stop a hosted readiness run that is still going. The run's real terminal state arrives on a later `readiness status` — this response reports the request, not the outcome. | Flag | Required | Description | | ------------------------ | -------- | ------------------ | | `--run ` | Yes | Readiness run id | | `--project ` | No | Project name or ID | ### `readiness report` Read a finished run's findings, ordered most-consequential-first and capped. `truncated` and `totalFindings` indicate when you are seeing a subset. | Flag | Required | Description | | ------------------------ | -------- | ------------------ | | `--run ` | Yes | Readiness run id | | `--project ` | No | Project name or ID | *** ## `cloud eval` commands All `eval` commands accept the shared platform flags below. ### Shared platform flags | Flag | Description | | ----------------- | ----------------------------------------------------------------- | | `--api-key ` | MCPJam `sk_` API key (overrides `MCPJAM_API_KEY`) | | `--api-url ` | MCPJam API base URL (defaults to `https://app.mcpjam.com/api/v1`) | ### `cloud eval create` Create a runnable eval suite from authored test cases (does not run it). | Flag | Required | Description | | -------------------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------- | | `--project ` | No | Project name or ID (defaults to the most recently updated project) | | `--file ` | No | Path to a **create-API JSON** body (or `-` for stdin). A versioned suite file (`schemaVersion: "1"`) belongs on `eval run --file` | | `--json ` | No | Inline suite definition JSON (or `@file`, or `-` for stdin) | | `--name ` | No | Suite name (overrides the file) | | `--model ` | No | Suite-level default model (overrides the file) | | `--provider ` | No | Suite-level default provider (overrides the file) | | `--server ` | No | Project HTTP server names or IDs (overrides the file) | ### `cloud eval list` List the eval suites saved in a project. | Flag | Required | Description | | ------------------------ | -------- | ------------------------------------------------------------------ | | `--project ` | No | Project name or ID (defaults to the most recently updated project) | ### `cloud eval run` Start an eval run of an existing suite, or upload a versioned suite file and run it. Provide **either** `--suite` **or** `--file`, not both. By default the command prints a launch receipt and returns immediately. `--wait` polls every launched run to a terminal state. **Exit codes (`--wait` only).** Without `--wait` this command's exit code is unchanged: `0` on any launch, `1` on a partial or wholly failed fan-out. `--wait` owns a separate six-code contract — a caller who asked this process to sit and watch a run to completion is asking it to say how the run went: | Code | Meaning | | ---- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `0` | Every waited run passed, unanimously, and (when a report was requested) every report was assembled | | `1` | A run **completed** with a **failed** verdict. The only condition that produces this code | | `2` | Usage error — bad flags, `--reporter`/`--out` without `--wait`, or an invalid suite file (after auth, before any server/model work) | | `3` | Auth failed — no credential, or the platform rejected it, at launch or mid-wait (a token that expired during a long poll). Zero credits spent if it happened before launch | | `4` | Connection/setup failed before evaluation — a launch failure this CLI itself observed (network, timeout, rate limit, a partial or failed fan-out, an unrecognized launch error), or a local `--out` write failure — unless a more severe outcome (`1` or `3`) already applies to the same run, per the merge order above | | `5` | No valid verdict — `inconclusive`, a null/unrecognized result, a run whose own status is failed/cancelled/timed-out, a wait that hit its deadline, a mid-wait failure that was not auth-shaped, or a report that could not be assembled | No infrastructure condition ever maps to `1` — see the same rule under `eval gate` below. A multi-target launch merges these worst-of across every waited run, in the order `1 > 3 > 4 > 5 > 0`: a real verdict failure is never masked by a sibling's infrastructure noise, and a credential failure (which may be *why* other targets show no verdict) outranks a plain connection failure. This six-code contract is new and deliberately **different** from `eval gate`'s four codes below — see that command's own doc for why the two are not unified. An invalid suite file exits **2** either way — `eval validate` still exits 1 for the same contract failure, because it is a verdict on the file and this command is not. | Flag | Required | Description | | -------------------------------------------- | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--suite ` | Yes, unless `--file` | Eval suite name or ID | | `--file ` | Yes, unless `--suite` | Versioned suite file to upload and run (`.yaml` or `.json`, or `-` for stdin) | | `--project ` | No | Project name or ID (defaults to the most recently updated project) | | `--server ` | No | Override the suite's saved server selection (HTTP servers only) | | `--environment ` | No | Attached project environment(s) to run. Several values start one **paid run each**. | | `--host ` | No | Attached host(s) to run, so the run is stamped with that host's config. Several values start one **paid run each**. | | `--all-targets` | No | Run **every** attached environment (or, if none, every attached host) — one **paid run per target** | | `--iterations ` | No | Run each case this many times (1-10), for this run only | | `--case ` | No | Run only these cases instead of the whole suite | | `--exclude-skills` | No | Run the "without skills" A/B arm: nothing is pinned, and the run is labelled as excluded | | `--refresh-snapshot` | No | **Persists a new host-config snapshot on the suite** (see below). Single-target runs only. | | `--notes ` | No | Free-text note stored on the run | | `--min-pass-rate ` | No | Pass threshold for this run (0-100), overriding the suite's own | | `--match-options ` | No | Tool-call match options for this run, e.g. `'{"toolCallOrder":"exact"}'` | | `--idempotency-key ` | No | Retry-safety key: repeating the call returns the run it already started | | `--wait` | No | Wait for every launched run to reach a terminal status | | `--wait-timeout ` | No | Maximum wait time (default 600000); requires `--wait` | | `--reporter ` | No | Write the completed structured report to stdout; requires `--wait`. `html` is a self-contained decision summary + failures page — traces/parity/history are not included. | | `--out ` | No | Atomically write the completed report to a file; defaults to `json-summary`, or uses `--reporter` when present; requires `--wait` | | `--compose-host ` | No | Compose a stack to run instead of naming a saved environment (see below). **Appends** the composed environment to the suite. | | `--compose-computer ` | No | Sandbox image to pin on the composed stack | | `--compose-model ` | No | Model to run on the composed stack, instead of the host's | | `--compose-server-group ` | No | Standalone server group to pin on the composed stack | | `--compose-skill ` | No | Project-shared skill IDs to pin on the composed stack | #### Composing a target instead of naming one `--compose-host ` builds an execution stack for this run instead of naming a saved environment, refined by `--compose-computer`, `--compose-model`, `--compose-server-group` and `--compose-skill`. The stack becomes an unnamed, content-addressed environment (the same one `environments ensure-adhoc` produces), so composing the same stack twice reuses one environment. **This edits the suite.** The composed environment is appended to the suite's environment list — deliberately, because an environment the suite does not list is one nobody can re-run from the app afterwards. The result reports both writes (`composed.environment.created`, `composed.attachment.attached`) even when the launch itself fails, so you always know what changed. Any `--compose-*` flag alongside `--environment`, `--host`, `--server` or `--all-targets` is a usage error: compose builds the stack the run uses, so naming a target alongside it describes two different runs. #### Which target runs **Fan-out is explicit and never inferred.** Choosing for you would be choosing how much to spend, so: * **nothing attached** → the suite's saved server selection runs (unchanged); * **exactly one** attached environment or host → that one runs automatically, and the run is stamped with its configuration; * **several attached** → the run fails with `TARGET_REQUIRED`, listing every choice. Name one with `--environment` / `--host`, several by repeating the flag, or run all of them with `--all-targets`. `--all-targets` expands **one axis**: attached environments if the suite has any, otherwise attached hosts — never a cross product, because an environment already resolves a host. It starts **one paid run per target**, launched as a single group that occupies one concurrency slot rather than N. `--environment` / `--host` and `--server` are mutually exclusive: an environment or host supplies a closed server set that an override cannot change. `--refresh-snapshot` **changes the suite**, not just this run: it re-derives and stores the suite's host-config snapshot, so every future run uses the new one. It is rejected on any multi-target launch — several runs racing to write one snapshot would leave the suite pinned to whichever finished last. #### Output and exit codes With `--format json`, stdout is **exactly one JSON document** — the launch receipt, with nothing appended — so CI can parse it directly. In human format the CLI prints a `View:` link per started run, a `Started N/M runs (group …)` summary when several launched, and a `Failed: — : ` line per failure. With `--wait`, the default output contains the launch receipt and completed runs. `--reporter` replaces that output with the canonical structured report. `--out` writes the same report atomically, including on failed-run and post-completion reporting-error paths, before the command sets its exit code. Without `--wait`, a partial or wholly failed fan-out **exits 1**; with `--wait` it exits **4** (see the exit-code table above). A per-target failure does not abort its siblings, so exiting 0 would let a pipeline read "1 of 3 runs never started" as a clean launch. #### `--file`: upload and run a suite file `eval run --file` reads the versioned suite file (`schemaVersion: "1"`), authenticates, then validates. A contract-invalid file exits **2** after the auth request — it does not start a run. `eval validate` on the same bytes still exits 1: that command is a verdict on the file; this one is not. The file's `suite.id` is the declared suite identity. The first run creates a file-owned suite stamped with that id; later runs of the same id in the same project update that suite rather than creating a second one. A UI-authored suite has no declared id, so no file can claim it. Resolve is never by name. File `passThreshold` is a fraction; the hosted suite grades on a percent. The conversion refuses rather than approximates. File `repetitions` above **10** are refused by name (`REPETITIONS_CAP`) and are not clamped. Cases are synced in batches of at most 100 (`create_eval_cases`). Every declared case is created or updated, including `disabled: true` rows. A case the file **no longer declares** is deleted from the hosted suite before launch. A case the file still declares but marks `disabled: true` is **kept, with its history** — it is simply left out of the launch, so parking a flaky test does not destroy its past results and re-enabling it later resumes the same case. The run is scoped to the enabled cases either way. A file with no enabled cases is refused (`NO_ENABLED_CASES`) rather than launching the whole persisted suite. `--case` may only name an enabled case. A later run updates existing cases by declared id and creates new ones. An enabled case that overrides `passThreshold` is refused: hosted grading is suite-wide. A disabled case with the same override is ignored, matching how repetitions are checked. Authored `defaults.toolPolicy` and non-empty `defaults.validity` gates are refused at hosted CLI launch (`TOOL_POLICY_UNSUPPORTED` / the corresponding validity refusal). Hosted platform-authored suites do not currently have a backend tool-policy field or enforcement path, so the CLI does not upload a suite that would claim those guarantees. When supported by an eval runner, `toolPolicy` is enforced at execution time with denied tools kept visible but blocked; server annotations are advisory and **UNTRUSTED**. File `defaults.repetitions` is inherited per case and is not uploaded as a suite `minIterations` floor. Authored `defaults.systemPrompt` and `defaults.temperature` are uploaded as execution defaults; omitted fields stay omitted so the platform applies its own defaults. `target.hosts` and `target.environment` are attached before launch. A file environment takes precedence over file hosts for that launch, while an explicit CLI target takes precedence over both. When an explicit CLI target is provided (for example `--host`), the file's `target.hosts` are **not** synchronized to the suite's persisted host attachments — the suite keeps whatever hosts it already has. Host attachment synchronization only runs when no explicit CLI target overrides the file. Environment attachment synchronization is unaffected and always runs. Without `--idempotency-key`, the CLI derives one from the file's SHA-256, declared suite id, project, target, and every run-affecting knob (`--iterations`, `--case`, `--min-pass-rate`, `--exclude-skills`, `--match-options`, `--environment` / `--host` / `--server` / `--all-targets`, `--refresh-snapshot`, and compose flags). Repeating the same file with the same knobs returns the run it already started; changing a knob starts a new one. `--notes` is not part of the key. `--file` pointing at create-API JSON (no `schemaVersion`) is a usage error that names `eval create --file`. The converse is also true: `eval create --file` on a versioned suite file names `eval run --file`. Export of a file-owned suite writes `declaredId` as `suite.id`. Export of a UI suite still writes the Convex id; running that file back is the ownership refusal. When the run targets an **attached project environment** (`eval environments set`), it executes against that environment's resolved host config, closed server set, and pinned plugin versions instead of the suite's saved selection. Every run records the environment and the exact revision it used, so `eval status` can answer "which configuration did this run actually execute against?" long after the environment has been edited. ### `cloud eval status` Get the status and summary of an eval run. | Flag | Required | Description | | ------------------------ | -------- | ------------------------------------------------------------------ | | `--run ` | Yes | Eval run ID (from `eval run`) | | `--project ` | No | Project name or ID (defaults to the most recently updated project) | The response carries a `judges` block with each advisory grader's state and per-case grades. `status: null` means that judge was never requested for the run — different from a judge that ran and graded nothing. In `--format human`, each judge that actually graded gets a one-line summary. ### `cloud eval judge` Grade a finished eval run with LLM as Judge. **Spends your organization's model budget.** Returns a pending receipt; read the grades from `eval status`. | Flag | Required | Description | | ------------------------- | -------- | ------------------------------------------------------------------ | | `--run ` | Yes | Eval run ID (from `eval run`) | | `--project ` | No | Project name or ID (defaults to the most recently updated project) | | `--force` | No | Re-grade a run that already has a judge result | | `--enable` | No | Grade this run even though the judge was off when it ran | | `--judge-model ` | No | Judge model for this run only | | `--judge-threshold <0-1>` | No | Pass threshold for this run only | A run's grading config is pinned when the run starts, so turning the judge on for the suite does not reach a run that was already recorded — `--enable` is what grades it, and it changes nothing beyond that run. Passing neither `--judge-model` nor `--judge-threshold` grades with the suite's own config, clearing any override a previous request left on the run. ### `cloud eval iterations` List per-iteration results for an eval run (pass/fail, tool calls, tokens, latency). Results are paginated. `--project` is optional. Selection follows `--project` → input selector → `MCPJAM_PROJECT` → nearest project link → automatic newest accessible project. | Flag | Required | Description | | ------------------------ | -------- | ------------------------------------------------------------------ | | `--run ` | Yes | Eval run ID (from `eval run`) | | `--project ` | No | Project name or ID (defaults to the most recently updated project) | | `--cursor ` | No | Pagination cursor from a previous response | | `--limit ` | No | Max iterations per page (1–200) | ### `cloud eval gate` Apply a pass/fail policy to an eval run. Add `--wait` to poll an in-progress run before evaluating it. | Flag | Required | Description | | -------------------------------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--run ` | Yes | Eval run ID (from `eval run`) | | `--project ` | No | Project name or ID (defaults to the most recently updated project) | | `--min-pass-rate-percent <0-100>` | No | Minimum share of iterations that must pass | | `--no-gating-score-errors` | No | Fail if any gating scorer errored | | `--min-scorer-pass-rate ` | No | Minimum pass rate for one scorer; repeatable | | `--min-mean-score ` | No | Minimum mean score for one scorer; repeatable | | `--baseline ` | No | Baseline run ID to gate a regression delta against, in addition to any threshold flags above. SHA baselines are not supported yet — pass a run id | | `--min-sample-size ` | No | Iterations required on **each** side before a pass-rate regression is decidable (default 5); requires `--baseline` | | `--min-effect-size-percent <0-100>` | No | Smallest pass-rate drop worth failing on, as a percentage (default 1); requires `--baseline` | | `--gate-deterministic-regressions` | No | Fail if a deterministic gating scorer flipped from passed to failed; requires `--baseline` | | `--max-p95-latency-increase-ms ` | No | Fail if p95 end-to-end latency rose by more than this many milliseconds vs the baseline; requires `--baseline` | | `--wait` | No | Poll until the run reaches a terminal status | | `--wait-timeout ` | No | Maximum wait time (default 600000) | | `--reporter ` | No | Write the structured eval and gate report to stdout. `html` is a self-contained decision summary + failures page — traces/parity/history are not included. | | `--out ` | No | Atomically write the report to a file; defaults to `json-summary`, or uses `--reporter` when present | Report files are flushed before gate exit codes are set, including failed and incomplete gate outcomes. **`--baseline`.** Passing a run ID fetches the same run comparison `cloud eval compare` uses and folds a regression verdict into the same report and exit code as the threshold flags above — `--baseline` alone enables the pass-rate regression gate with its default tuning; the four tuning flags need `--baseline` to mean anything and are a usage error without it. A case set that changed, was added or removed, or ran an unequal iteration count between the two runs makes the whole-run pass-rate and p95-latency gates non-gateable (exit `3`) rather than a silent pass or a misread regression; the deterministic per-case regression gate is exempt and can still fail (exit `1`) on a matching case. SHA baselines (`--baseline `) are not supported yet and are rejected as a usage error. **Exit codes.** `eval gate` is the command that fails a build, so it is the one that maps a verdict onto an exit code. It keeps four: | Code | Meaning | | ---- | ----------------------------------------------------------------------------------------------------------------------------------------- | | `0` | Every requested gate passed | | `1` | An eval **verdict** failed. Reserved for exactly that | | `2` | Usage error — an unknown scorer, or a threshold out of range | | `3` | Incomplete: the run was cancelled, the wait timed out, the network failed, or the run is non-gateable (its score evidence did not verify) | No infrastructure condition ever maps to `1`. A job that fails a release because a network call flaked, and reports it as a regression, teaches people to ignore the gate — so retry on `3`, and treat `1` as a real finding about the server. These four are deliberately **not** the same set `eval run --wait` uses (see that command above): `gate`'s exit `3` means "incomplete / non-gateable", and remapping it to the six-code scheme would invert pipelines that already retry on `3`. This is a deliberate compatibility exception for v1 — a six-code migration for `gate` is deferred behind a future opt-in flag, kept here so the exception stays documented rather than silently diverging. ### `cloud eval trace` Fetch the full trace for one eval iteration (complete message history and spans). Can be large. `--project` is optional. Selection follows `--project` → input selector → `MCPJAM_PROJECT` → nearest project link → automatic newest accessible project. | Flag | Required | Description | | ------------------------ | -------- | ------------------------------------------------------------------ | | `--run ` | Yes | Eval run ID (from `eval run`) | | `--iteration ` | Yes | Iteration ID (from `eval iterations`) | | `--project ` | No | Project name or ID (defaults to the most recently updated project) | ### `cloud eval get` Show an eval suite's full settings. | Flag | Required | Description | | ------------------------ | -------- | ------------------------------------------------------------------ | | `--suite ` | Yes | Eval suite name or ID | | `--project ` | No | Project name or ID (defaults to the most recently updated project) | ### `cloud eval validate` Validate a local eval **suite file** — the versioned declarative document that describes a suite, its defaults and its cases (`schemaVersion: "1"`, YAML canonical, JSON accepted, conventionally `.mcpjam/evals/*.yaml`). Offline: this command takes **no `--project`**, builds no API client and reads no API key. It never touches the network. | Flag | Required | Description | | --------------- | -------- | ------------------------------------------------------------- | | `--file ` | Yes | Suite file to validate, `.yaml` or `.json` (or `-` for stdin) | | Exit code | Meaning | | --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `0` | Valid | | `1` | Parsed, but invalid against the suite-file contract. Every finding is reported, not just the first. `eval run --file` on the same bytes exits **2** — 1 is reserved there for a real verdict | | `2` | Nothing was validated: unreadable path, input over the 1 MiB limit, or malformed YAML | Input is capped at **1,048,576 bytes (1 MiB) of UTF-8** and is never truncated — a file over the cap is rejected outright. Under `--format json`, stdout carries this envelope whether the file is valid or not: ```jsonc theme={"theme":"css-variables"} // exit 0 { "valid": true, "file": ".mcpjam/evals/s_billing.yaml", "suite": { "id": "s_billing", "name": "Billing smoke", "cases": 12, "enabledCases": 11 }, "findings": [] } // exit 1 or 2 { "valid": false, "file": ".mcpjam/evals/s_billing.yaml", "stage": "contract", // "contract" is exit 1; "input" and "parse" are exit 2 "findings": [ { "code": "SUITE_FILE_INVALID", // or SUITE_FILE_TOO_LARGE / _EMPTY / _YAML_INVALID / _MULTIPLE_DOCUMENTS "path": ["cases", 1, "id"], // field path as segments "pointer": "cases[1].id", // the same path, rendered "message": "duplicate case id \"c_refund\"" // `location` accompanies SUITE_FILE_YAML_INVALID findings only, where the // YAML parser reported a position: { "line": 24, "column": 9 } } ] } ``` Findings are deterministically ordered and carry no timestamps, so two runs over the same bytes produce byte-identical output. An unreadable or missing path is a usage error and goes to stderr in the CLI's standard error envelope rather than as findings — nothing was read, so there is nothing to report about. **What "valid" does not mean.** This command checks the file against the suite-file contract. It does **not** re-resolve tool names, server references or fixtures against a project's live discovery — that is project-aware validation, it needs a network round trip, and it is deliberately deferred. A file that validates here can still fail to run. ### `cloud eval export` Write a hosted eval suite to a local suite file. | Flag | Required | Description | | ------------------------ | -------- | ---------------------------------------------------------------------------- | | `--suite ` | Yes | Eval suite to export (name or ID) | | `--project ` | No | Project the suite belongs to (defaults to the most recently updated project) | | `--out ` | No | Where to write (default `.mcpjam/evals/.yaml`) | | `--force` | No | Replace an existing file at the output path | The default path uses the suite's **id**, not its display name: an id is stable and path-safe, so a rename does not leave a second file behind. **Fail-closed.** A suite file cannot express everything a hosted suite can, and this command refuses rather than writing a file that describes a different test than the dashboard runs. When any part of a suite is not representable it emits `UNSUPPORTED_SUITE_EXPORT` findings, exits `1`, and writes **no file at all** — not a partial one, not one with a warning comment. Every finding names the field it is about, so the command's output is the authoritative list; the constructs that refuse today include: * several attached project environments, or legacy servers combined with an attached environment (the suite file can name one environment); * a pinned sandbox image; * automatic LLM-as-judge grading, non-default match options, and an iterations floor that raises a case above its own count; * a compare-across-models case, a scenario-bound case, and cases that name **different model providers** (a suite file carries one suite-level provider); * `replace`/`extend` check overrides, a case that inherits the suite's checks while also carrying its own, and any suite check the predicate contract does not recognise; * a suite with no execution model, no minimum accuracy, no name, no runnable server or environment target, or no cases; a case with no steps; and a suite or case id that is not an opaque id; * a suite that serializes past the 1 MiB suite-file limit — the size is a property of the suite, and this never truncates. Case identity is never freshly minted: a case exports under its declared id, or under its platform row id, or it refuses. Exporting the same suite twice produces the same case ids. | Exit code | Meaning | | --------- | ----------------------------------------------------------------------------------------------------- | | `0` | Written | | `1` | The suite cannot be represented losslessly; nothing was written | | `2` | Usage: the output path exists and `--force` was not passed, or the suite's cases did not fit one page | The file is written through a sibling temp file and a rename, so an interrupted write leaves the previous file exactly as it was. ### `cloud eval update` Edit an eval suite's settings (only the flags you pass change). | Flag | Required | Description | | ------------------------------------------ | -------- | -------------------------------------------------------------------------------------------------------------------------------------------- | | `--suite ` | Yes | Eval suite name or ID | | `--project ` | No | Project name or ID (defaults to the most recently updated project) | | `--file ` | No | Suite-update JSON body (or `-` for stdin) | | `--json ` | No | Inline suite-update JSON (or `@file`, or `-`) | | `--name ` | No | Rename the suite | | `--description ` | No | Suite description | | `--server ` | No | Replace the suite's server selection (project server names) | | `--computer-image ` | No | Sandbox image eval runs boot a fresh computer from (list them with `mcpjam cloud images list`). `off` uses the provider's default base image | | `--host ` | No | Replace host attachments (by name/ID) | | `--model ` | No | Execution model ID | | `--system-prompt ` | No | Execution system prompt | | `--temperature ` | No | Execution temperature | | `--min-accuracy ` | No | Minimum accuracy, 0–100 | | `--min-iterations <1-10\|off>` | No | Floor on per-case iterations — every case runs at least this many times. `off` removes the floor | | `--tool-call-order ` | No | Tool call order | | `--arguments ` | No | Argument matching | | `--extra-tool-calls ` | No | Allowed extra tool calls | | `--judge ` | No | Turn LLM-as-judge grading on/off. `on` both enables the judge and sets it to grade every run as it completes; `off` stops both | | `--judge-model ` | No | Judge model ID | | `--judge-threshold <0-1>` | No | Judge pass threshold, 0–1 (a case passes when its score is at or above it) | ### `cloud eval checks list` List the repositories whose pull requests run an eval suite, plus the repositories the MCPJam GitHub App can reach. | Flag | Required | Description | | ------------------------ | -------- | ------------------------------------------------------------------------------------------------------- | | `--project ` | No | Project name or ID — only used to pick the organization (defaults to the most recently updated project) | `available: false` means GitHub Checks is not enabled for the organization at all, so connecting a repository will not help. `connectable: null` means the App could not be asked (GitHub unavailable, or no installation) — different from an empty list, which would mean it reaches nothing. ### `cloud eval checks connect` Run this suite on every pull request to a repository. **Affects everyone who opens a pull request on that repository**, and with `fail-closed` it can block their merges. | Flag | Required | Description | | ------------------------------------------ | -------- | ---------------------------------------------------------------------------- | | `--suite ` | Yes | Eval suite name or ID | | `--repo ` | Yes | Repository to connect. Must be one `eval checks list` reports as connectable | | `--outage-policy ` | Yes | What the check reports when MCPJam cannot conclude | | `--project ` | No | Project name or ID (defaults to the most recently updated project) | `--outage-policy` has no default on purpose: it decides what happens to other people's pull requests during an MCPJam outage, and it is not editable from this command afterwards. Retargeting a repository at a different suite, pausing it, and disconnecting all live in the app's Settings → Integrations, where every connected repository is visible at once. ### `cloud eval delete` Permanently delete an eval suite (and its cases and runs). | Flag | Required | Description | | ------------------------ | -------- | ------------------------------------------------------------------ | | `--suite ` | Yes | Eval suite name or ID | | `--project ` | No | Project name or ID (defaults to the most recently updated project) | ### `cloud eval schedule` Enable or disable scheduled runs for a suite. | Flag | Required | Description | | ---------------------------- | -------- | -------------------------------------------------------------------- | | `--suite ` | Yes | Eval suite name or ID | | `--project ` | No | Project name or ID (defaults to the most recently updated project) | | `--enable` | No | Enable scheduled runs | | `--disable` | No | Disable scheduled runs | | `--interval ` | No | Run interval in minutes (5–10080) | | `--environment ` | No | Project environment the scheduled runs launch (only with `--enable`) | Pass either `--enable` or `--disable` (not both). A scheduled run launches exactly one run, so a suite with several attached environments must pin one with `--environment`; a suite with exactly one defaults to it. `--environment` is rejected with `--disable`: disabling preserves the existing pin, so accepting one there would silently do nothing. ### `cloud eval environments` subcommands Attach or detach the project environments an eval suite runs against. Attaching is what makes `eval run --environment` available for the suite. #### `cloud eval environments set` Replace the suite's attached environments (this sets the whole list, in order). | Flag | Required | Description | | ------------------------------- | -------- | ------------------------------------------------------------------ | | `--suite ` | Yes | Eval suite name or ID | | `--environment ` | Yes | Project environments to attach, in order | | `--project ` | No | Project name or ID (defaults to the most recently updated project) | ```bash theme={"theme":"css-variables"} mcpjam cloud eval environments set --suite "Checkout smoke" --environment Staging Prod ``` Rejected if it would strand an enabled schedule pinned to an environment being removed — repoint the schedule (`eval schedule --enable --environment ...`) or disable it first. #### `cloud eval environments clear` Detach every environment, reverting the suite to its saved server selection. | Flag | Required | Description | | ------------------------ | -------- | ------------------------------------------------------------------ | | `--suite ` | Yes | Eval suite name or ID | | `--project ` | No | Project name or ID (defaults to the most recently updated project) | Subject to the same schedule guard as `set`: rejected if the suite has an enabled schedule pinned to one of the environments being removed. Repoint the schedule or disable it first. ### `cloud eval cases` subcommands #### `cloud eval cases list` List a suite's test cases. | Flag | Required | Description | | ------------------------ | -------- | ------------------------------------------------------------------ | | `--suite ` | Yes | Eval suite name or ID | | `--project ` | No | Project name or ID (defaults to the most recently updated project) | #### `cloud eval cases get` Show one test case. | Flag | Required | Description | | ------------------------ | -------- | ------------------------------------------------------------------ | | `--suite ` | Yes | Eval suite name or ID | | `--case ` | Yes | Eval case title or ID | | `--project ` | No | Project name or ID (defaults to the most recently updated project) | #### `cloud eval cases run` Run a single case as a persisted, fully-queryable run — inspect it with `eval iterations` / `eval steps` like any other run. | Flag | Required | Description | | --------------------------------- | -------- | ----------------------------------------------------------------------------------------------------------- | | `--suite ` | Yes | Eval suite name or ID | | `--case ` | Yes | Eval case title or ID | | `--project ` | No | Project name or ID (defaults to the most recently updated project) | | `--server ` | No | Override the suite's saved servers for this run | | `--environment ` | No | Project environment to run against (must be attached to the suite) | | `--host ` | No | Attached host to run against, so the run is stamped with that host's config | | `--compose-host ` | No | Compose a stack to run instead of naming a saved environment; appends the composed environment to the suite | | `--compose-computer ` | No | Sandbox image to pin on the composed stack | | `--compose-model ` | No | Model to run on the composed stack, instead of the host's | | `--compose-server-group ` | No | Standalone server group to pin on the composed stack | | `--compose-skill ` | No | Project-shared skill IDs to pin on the composed stack | | `--iterations ` | No | Run the case this many times (1-10) | | `--idempotency-key ` | No | Retry-safety key: repeating the call returns the run it already started | Every `--compose-*` flag behaves exactly as it does for `eval run` — see [Composing a target instead of naming one](#composing-a-target-instead-of-naming-one). Target selection follows the same rules ([Which target runs](#which-target-runs)), except that a single case run targets ONE thing — there is no `--all-targets` here. #### `cloud eval cases create` Add a test case to a suite (definition via `--file`/`--json`). | Flag | Required | Description | | ------------------------ | -------- | ------------------------------------------------------------------ | | `--suite ` | Yes | Eval suite name or ID | | `--project ` | No | Project name or ID (defaults to the most recently updated project) | | `--file ` | No | Case JSON body (or `-` for stdin) | | `--json ` | No | Inline case JSON (or `@file`, or `-`) | | `--title ` | No | Case title (overrides the body) | #### `cloud eval cases update` Edit a test case (definition via `--file`/`--json`). | Flag | Required | Description | | ------------------------ | -------- | ------------------------------------------------------------------ | | `--suite <id-or-name>` | Yes | Eval suite name or ID | | `--case <id-or-title>` | Yes | Eval case title or ID | | `--project <id-or-name>` | No | Project name or ID (defaults to the most recently updated project) | | `--file <path>` | No | Case JSON body (or `-` for stdin) | | `--json <json>` | No | Inline case JSON (or `@file`, or `-`) | | `--title <title>` | No | Rename the case | #### `cloud eval cases delete` Permanently delete a test case. | Flag | Required | Description | | ------------------------ | -------- | ------------------------------------------------------------------ | | `--suite <id-or-name>` | Yes | Eval suite name or ID | | `--case <id-or-title>` | Yes | Eval case title or ID | | `--project <id-or-name>` | No | Project name or ID (defaults to the most recently updated project) | #### `cloud eval cases generate` AI-generate test cases from the suite's tools (spends credits). | Flag | Required | Description | | ---------------------------- | -------- | ----------------------------------------------------------------------------------------------------------------- | | `--suite <id-or-name>` | Yes | Eval suite name or ID | | `--project <id-or-name>` | No | Project name or ID (defaults to the most recently updated project) | | `--mode <normal\|negative>` | No | Generation mode (default `normal`) | | `--server <id-or-name...>` | No | Servers to discover tools from (default: suite's) | | `--environment <id-or-name>` | No | Discover tools from this attached environment's server set | | `--case-model <id...>` | No | Execution model(s) for the generated cases | | `--simple <n>` | No | How many easy, single-tool cases | | `--multi-tool <n>` | No | How many medium, 2+ tool cases | | `--multi-turn <n>` | No | How many multi-turn follow-up cases | | `--complex <n>` | No | How many hard / cross-server cases | | `--negative <n>` | No | How many negative (no-tool) cases | | `--vary-user-styles` | No | Vary query phrasing across a realistic range of user styles | | `--idempotency-key <key>` | No | Retry-safety key: repeating the call replays the first attempt's drafts instead of generating (and billing) again | *** ## `mcp` command ### `mcp` Run MCPJam as an MCP server over stdio so MCP clients (Claude Desktop, Claude Code, Cursor, ...) can connect to, exercise, and debug other MCP servers. See [MCPJam as an MCP server](/cli/mcp-server) for the exposed tools and client setup. ```bash theme={"theme":"css-variables"} npx -y @mcpjam/cli@latest mcp ``` The command takes no flags of its own. The global `--timeout <ms>` flag sets the default per-request timeout against target servers, and `--quiet` suppresses the startup notice on stderr. Stdout carries only JSON-RPC; never pipe other output into it. *** ## `telemetry` commands Telemetry commands inspect and configure anonymous CLI telemetry. They never emit telemetry events themselves. ### `telemetry status` Shows the effective telemetry state, install ID state, state file path, debug mode, and disable reason when disabled. This command does not create an install ID. ### `telemetry disable` Persistently disables anonymous CLI telemetry by writing `enabled: false` to the telemetry state file. If no install ID exists yet, this command does not create one. ### `telemetry enable` Persistently enables anonymous CLI telemetry. If no install ID exists yet, this command creates a random install UUID. *** ## `cloud` workspace `mcpjam cloud` is the account-bound namespace. Log in, link a project, then run evals, tunnels, and the rest of the Cloud groups. ### Cloud account session See [`cloud login` / `logout` / `whoami`](#cloud-login--logout--whoami) below. ### `cloud link` Pin the current Git worktree (or `--here` cwd) to a Cloud project by writing `.mcpjam/project.json`. No secrets. Bare `link` ignores an existing file when choosing what to pin. | Flag | Description | | ----------- | ------------------------------------------------------------------------------------------------------------------- | | `[project]` | Optional project name or ID to pin. Omit to pick the most recently updated project. | | `--here` | Write the link in the current working directory instead of the Git worktree root. | | `--remove` | Remove the nearest project link (or the current-directory link with `--here`). Cannot be combined with `[project]`. | ### `cloud status` Zero-network. Prints credential source, deployment URL, and the project selector that Cloud commands would use (flag / env / link / automatic). API keys are redacted. The report includes `credential.valid` (`true` / `false` / `null`) and `deployment.valid`. `null` means no credential is configured and is a successful informational status. An invalid explicit `--api-key` or `--api-url` still emits the complete JSON report, sets `ok: false`, includes an `error` string, and exits `1`. Other Cloud commands reject those same values with exit `2` and no report. ### `cloud organizations list` List organizations and their ids. An `sk_` key sees only its own. Use the id with `mcpjam cloud projects list --org <id>`. ### `cloud projects list` | Flag | Description | | ------------ | ---------------------------------------------------------------------------- | | `--org <id>` | Restrict the listing to one organization (ID only; see `organizations list`) | ### `cloud sessions list` List Playground chat sessions in the selected project. Same project-selection rule as other Cloud commands. | Flag | Description | | ------------------------ | ------------------------------------------------------------------ | | `--project <id-or-name>` | Project name or ID (defaults to the most recently updated project) | | `--all-projects` | List across every accessible project. Conflicts with `--project`. | | `--status <status>` | Filter by session status | | `--limit <n>` | Maximum sessions to return (1–200) | ### `cloud sessions send` Send one message to a project's MCP servers and print the reply plus the raw tool calls, per-call latency and token usage — the telemetry a participant in the conversation cannot see. **This spends model credits on every call.** `--idempotency-key` is required and must be **stable for the intent**, not regenerated per attempt. It is what makes a retry safe: with the same key, a turn that already completed is replayed rather than re-run and re-billed. A fresh key per attempt deduplicates nothing. Omit `--session` to start a conversation; pass the `sessionId` the command returns to continue one. Configuration is fixed at the first turn, so `--model`, `--environment`, `--server`, `--system-prompt` and `--tool-mode` are refused alongside `--session` rather than silently repinning the session. | Flag | Description | | ------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--message <text>` | The message to send, as the user (required) | | `--idempotency-key <key>` | Stable key for this turn's intent (required). Reuse it when retrying. | | `--session <sessionId>` | Continue this session instead of starting one | | `--model <modelId>` | Provider-prefixed model id, e.g. `anthropic/claude-sonnet-5`. Required to start a session. A bare id is rejected rather than guessed — unprefixed ids are indistinguishable from local Ollama models. | | `--environment <environmentId>` | Target this environment's servers. Conflicts with `--server`. | | `--server <serverId...>` | Target these project servers. Conflicts with `--environment`. | | `--system-prompt <text>` | System prompt for the session | | `--tool-mode <mode>` | `read_only` (default) advertises only tools annotated `readOnlyHint: true`. `auto` advertises everything and **may cause real side effects**. | | `--max-steps <n>` | Maximum engine steps this turn (1–16) | | `--max-tool-calls <n>` | Cap the tool calls this turn may make. `0` answers without tools. | | `--temperature <n>` | Sampling temperature (0–2) | | `--project <id-or-name>` | Project name or ID (required to start a session) | `read_only` filters on a hint the SERVER asserts, so it is a policy this host applies rather than a guarantee it can verify. A server is free to annotate a mutating tool as read-only. ### `cloud sessions show` Print a session's metadata and a window of its raw messages. Message `index` values are **absolute transcript positions** — the same indices `cloud sessions trace` spans reference, so the two outputs join on that number. A transcript that could not be read reports `transcriptUnavailable: true` and a `null` `messageCount` — never `0`, which would claim the conversation is empty. | Flag | Description | | ------------------------- | ---------------------------------------------------------------------- | | `--session <sessionId>` | The session to read (required) | | `--after-message <index>` | Start the window at this absolute transcript index | | `--limit <n>` | Messages to return (1–200) | | `--project <id-or-name>` | Optional project scope; a session in another project answers not-found | ### `cloud sessions trace` Print a session's per-turn execution spans: per-tool-call latency, token usage, and indices into the transcript. Returns the **latest turn** by default rather than the whole session — a session's spans are the largest thing the API emits, and the turn you just took is the one you are debugging. A turn whose spans could not be read reports `spansUnavailable: true` rather than an empty `spans` array: "made no calls" and "could not fetch" are opposite conclusions. | Flag | Description | | ---------------------------- | -------------------------------------------------------- | | `--session <sessionId>` | The session to trace (required) | | `--turn <turnId>` | Return exactly this turn. Conflicts with `--after-turn`. | | `--after-turn <promptIndex>` | Page forward from this turn index | | `--limit <n>` | Turns to return (1–20). Defaults to 1. | | `--no-spans` | Per-turn summaries without span payloads | | `--project <id-or-name>` | Optional project scope | *** ## `cloud login` / `logout` / `whoami` These commands manage your MCPJam platform session. `login` opens a browser for OAuth and stores the session locally; `logout` removes it; `whoami` shows the account behind the current credentials. ### `cloud login` | Flag | Default | Description | | ----------------- | ------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--api-url <url>` | `https://app.mcpjam.com/api/v1` | MCPJam API base URL. The value is persisted with the session so subsequent cloud commands target the same deployment without needing `--api-url` again. Must be a valid `http(s)` URL — an invalid value exits with code `2` before any network call. | | `--no-browser` | | Print the login URL to stderr instead of opening a browser. | The `MCPJAM_API_URL` environment variable is equivalent to `--api-url`. An invalid value in either source is a hard error (exit 2). **URL resolution order for cloud commands after login:** 1. Explicit `--api-url` flag 2. `MCPJAM_API_URL` environment variable 3. The `apiUrl` stored with the OAuth session (only when the stored login is the credential; `sk_` API keys do not inherit the stored URL) 4. Production default (`https://app.mcpjam.com/api/v1`) **Login output fields:** After a successful login, the result includes: | Field | Description | | ----------- | ----------------------------------------------------------------------- | | `status` | `"logged_in"` | | `issuer` | OAuth issuer URL | | `apiUrl` | MCPJam API base URL persisted with the session | | `authFile` | Path to the stored session file | | `email` | Account email address (present when account lookup succeeded) | | `plan` | Account plan (present when account lookup succeeded) | | `expiresAt` | ISO 8601 access-token expiry (present when the token carries an expiry) | Account details (`email` and `plan`) are fetched from the platform after the OAuth exchange completes. If the lookup fails, login still succeeds and a warning is printed to stderr (suppressed with `--quiet`). ### `cloud logout` No additional flags. Removes the stored MCPJam session. When `MCPJAM_API_KEY` is set to an active `sk_` key, `logout` prints a warning to stderr (human format only) noting that the CLI remains authenticated via the environment variable even after the stored session is cleared. JSON output is unaffected. ### `cloud whoami` | Flag | Default | Description | | ----------------- | ------------------------------- | -------------------------------------------------- | | `--api-key <key>` | | MCPJam `sk_` API key (overrides `MCPJAM_API_KEY`). | | `--api-url <url>` | `https://app.mcpjam.com/api/v1` | MCPJam API base URL. | *** ## Exit codes | Code | Meaning | | ----- | ------------------------------------------------------------------------------------------------------------------------------------------ | | `0` | Success / all checks passed | | `1` | Command ran but reported a failure | | `2` | Invalid arguments or configuration | | `3` | `tasks conformance` only: the run was incomplete — its task-dependent checks never ran, so it is neither a pass nor a spec violation | | `6` | `tasks watch` only: the task needs input this run could not answer. Answer it with `tasks update`, or re-run with `--interactive` in a TTY | | `7` | `tasks watch` only: `--duration-ms` elapsed before the task reached a terminal status | | `130` | Interrupted by a signal (`tasks watch` aborted, or a second Ctrl-C during `tunnel` shutdown) | Codes above `2` are reserved for outcomes a script should act on *differently* — not merely for naming a failure. See [MCP Tasks](/cli/tasks#watch-exit-codes) for the full `tasks watch` mapping.