Diagnostics and recovery
Turn symptoms into checks, fixes, and durable evidence.
Good troubleshooting in MARS starts by preserving the repo,
reading the actionable error, checking the owning surface, and
recording evidence before retrying. This guide collects the
common user-facing failure paths and the safest recovery commands.
For ownership and cleanup questions, use the
Files And State Reference.
Triage Loop
-
Stop making the state worse
Pause the worker pool before broad inspection if autonomous work is still running.
Press p in the terminal or use authenticated dashboard Pause.
-
Capture the current state
Save the exact command, repo path, database path, log path, role, trace ID, and visible error.
git status --short --branch
mars doctor --repo /path/to/repo --json
-
Use the smallest relevant check
Prefer doctor, dry run, targeted tests, or dashboard status before rerunning the full autonomous loop.
-
Classify ownership
Decide whether the failure belongs to MARS foundation behavior, the deployed target repo, mixed configuration, or missing evidence.
-
Retry only after the fix is real
Do not repeat the same failed queue job unchanged unless the blocker was external and has changed.
Install And PATH
| Symptom | Check | Recovery |
|---|
mars command not found | command -v mars | Run mars path setup from a known installed binary, or rerun make install from the source checkout and open a new shell. See the Shell Integration Reference. |
| Old behavior after rebuild | mars version and which mars | Use make install; avoid go build ./cmd/mars; ./mars ... because a failed build can run an old source-root binary. |
| Shell profile not updated | mars path setup --dry-run --json | Run mars path setup --shell zsh or the detected shell option, then restart the terminal. Use Shell Integration for shell-specific profile and completion fixes. |
| Source checkout cannot update | git status --short --branch | Commit, stash, or deliberately handle local changes before make update-tool. |
Setup And Models
| Symptom | Likely cause | Recovery |
|---|
| Setup fails during model download | Network, disk, checksum, or ineligible hardware profile. | Run mars doctor --json, then retry mars setup --download --local-bundle auto. |
| No eligible local bundle | Hardware detection cannot prove enough RAM, VRAM, disk, OS, or backend support. | Use cloud/deferred routing or choose a smaller eligible bundle after mars models eligible --json. |
| Local generation is very slow | Large quantization, memory pressure, or CPU fallback. | Choose a smaller local bundle, use a balanced profile, or point to a real OpenAI-compatible endpoint with --model-endpoint. |
| Cloud provider auth fails | The configured api_key_env is not present in the process environment. | Export the named variable or write ignored local env with mars models credentials write-local-env. See the Authentication And Credentials Reference. |
| Ollama override fails | Ollama model is not installed or endpoint is not reachable. | Run mars models list --provider ollama and evaluate the candidate before using it. |
mars models eligible --json
mars models list --provider ollama
mars models evaluate --provider ollama --model qwen3.6:27b
mars models override --repo /path/to/repo --tier coding --provider ollama --model qwen3.6:27b
Harness And Config
| Symptom | Check | Recovery |
|---|
Missing .harness/manifest.yaml | ls .harness | Run mars init --repo /path/to/repo. For inspection only, use mars run engineer --repo /path/to/repo --dry-run --no-init. |
| Invalid manifest YAML | mars doctor --repo /path/to/repo | Fix YAML syntax, required name, required role prompt, and valid orchestration_mode. |
| Role registry drift | docs/roles/ROLES.md versus manifest. | Update registry rows for custom roles and rerun mars doctor --repo /path/to/repo. |
| Upgrade does not change an existing prompt | Expected preservation behavior. | Initialize a temporary repo with the new binary and manually copy wanted default prompt changes. |
| Need to remove MARS | Preview first. | mars eject --repo /path/to/repo, then mars eject --repo /path/to/repo --apply --confirm repo-name. |
Agent Runs
| Symptom | First check | Recovery |
|---|
| Role cannot run | Manifest role key, prompt file, tool list, trust, and model route. | mars run engineer --repo /path/to/repo --dry-run, then fix the manifest or prompt path. |
| Role has no tools | Strict tool allowlist is empty or missing. | Add only the required tools in .harness/manifest.yaml. |
| Role loops or hits max turns | Trace summary, terminal phase, and ticket state. | Inspect trace evidence and add a skill, clearer stop condition, smaller ticket, or better context route before retrying. |
| Role completes without useful work | No-op outcome or stale ticket state. | Route to Janitor or update the ticket/plan so the next role has a concrete action. |
| Tool is blocked | Trust level, guardrail message, and allowlist. | Prefer changing the implementation or policy narrowly over bypassing the guardrail. |
| Worktree dirty after a run | git status --short | Review changes, validate, and commit or deliberately revert target-owned throwaway output. |
mars run engineer --repo /path/to/repo --dry-run
mars run engineer --repo /path/to/repo --debug
mars trust --repo /path/to/repo
mars tools list --json
Dashboard And Ports
| Symptom | Check | Recovery |
|---|
| Dashboard does not open | Printed dashboard URL and port binding. | Pass --dashboard-addr 127.0.0.1:9090 or another loopback port. |
/healthz works on one port but dashboard does not | Control/webhook address versus dashboard address. | Use the dashboard URL printed by start or serve. |
Control returns 503 | MARS_DASHBOARD_CONTROL_SECRET is missing. | Set a new random value of at least 32 bytes and restart MARS; do not place it in flags or YAML. |
Control returns 401/403 | Session, exact Origin, or CSRF is missing or expired. | Use the exact printed dashboard URL and log in again. |
| Pause button appears ineffective | Running jobs continue after authenticated pause. | Pause stops new claims only. Wait for current jobs or use graceful stop if needed. |
| SSE stream disconnects | Browser/network interruption or process restart. | Refresh the dashboard or reconnect curl -N http://localhost:9090/api/events. |
| Dashboard stop times out | Older or interrupted shutdown path. | Use terminal q or Ctrl+C, then inspect the command log. |
mars start --repo /path/to/repo --dashboard-addr 127.0.0.1:9090 --addr 127.0.0.1:9091
curl http://localhost:9091/healthz
curl http://localhost:9090/api/status
Git And Worktree
| Symptom | Meaning | Recovery |
|---|
| Dirty worktree blocks success | Roles cannot hide uncommitted plans, tickets, code, docs, or evidence. | Review, validate, commit, or deliberately clean the target-owned changes. |
| Push rejected | Remote moved or auth failed. | Fetch, rebase or resolve intentionally, rerun relevant checks, then push. |
| Generated files tracked accidentally | Workspace hygiene issue. | Update .gitignore, remove generated output from the index, and rerun checks. |
| Need to inspect without mutation | Observer-safe dry run. | Use mars run engineer --repo /path/to/repo --dry-run --no-init. |
Release And Auth
| Symptom | Check | Recovery |
|---|
| Private release auth fails | mars auth github check | Authenticate with GitHub CLI, set GH_TOKEN or GITHUB_TOKEN, or run mars auth github setup. See the Authentication And Credentials Reference. |
| Update check cannot reach latest release | Network or remote release lookup. | Use --skip-remote for local drift checks, then retry remote lookup later. |
| Release notes missing narrative sections | mars release backfill-notes --repo . --check | Run mars release backfill-notes --repo . and commit compliant changelog updates. |
| Release assets missing | Repository-approved local artifact-contract gate. | Rerun the repository-approved producer and verifier. For MARS source, follow the exact publication-disabled F-018 snapshot workflow; the dist/ directory alone is not proof. |
GitHub mirror missing or mismatched assets / mirror_incomplete | Repository-approved remote artifact-contract gate. | Keep immutable refs fixed and publication blocked until the approved gate proves exact artifact identity and inventory convergence. MARS source specifically requires F-018 signed identity; missing or unverifiable state is never clean. |
Integrations
| Symptom | Likely cause | Recovery |
|---|
| JIRA webhook returns not found | No board-driven config or route is disabled. | Copy .harness/integrations.example.yaml to local .harness/integrations.yaml and set flow_profile: board-driven only when opting in. |
| JIRA issue is ignored | Project repo map, workspace, board, label, or JQL scope rejected it. | Read sanitized logs and narrow config deliberately. Do not remove containment broadly. |
| Atlassian MCP exposes no JIRA tools | Credential scopes or provider capabilities are insufficient. | Probe with the configured provider, then fix token/admin scopes or use the documented stdio proxy path. |
| Integration secret appears in config | Raw value was committed instead of env var name. | Rotate the secret, remove it from history if needed, and store only *_env names in config. |
DocSync
| Symptom | Meaning | Recovery |
|---|
docsync missing metadata | An audited source file has no top-of-file MarsDocSync block. | Add the block and list the owning durable docs. |
| Listed doc is missing | Metadata points at a nonexistent artifact. | Fix the path or create the owning doc. |
| Audit passes but docs are semantically stale | The audit checks metadata shape and paths, not prose truth. | Read the listed docs and update behavior text manually. |
| CLI docs stale after command change | CLI tool/skill sync rule was missed. | Update CLI reference, mirrored tool guidance, generated target docs, and affected skills. |
mars docsync audit --repo .
go test ./internal/docsconsistency ./internal/docsync
Evidence Bundle For Help Or Handoff
When handing a problem to another operator or agent, include enough
evidence to reproduce the state without exposing secrets.
| Include | How to collect | Do not include |
|---|
| Exact command and repo path shape | Copy the command with sensitive paths redacted if needed. | Tokens, private user names, or credential values. |
| Version and commit | mars version, git log --oneline -3. | Unrelated private branch history. |
| Worktree state | git status --short --branch. | Raw secret diffs. |
| Doctor output | mars doctor --repo /path/to/repo --json. | Unredacted environment variables. |
| Log and trace identifiers | Command log path, trace ID, job ID, role, and database path. | Full raw prompts or source content unless the recipient is allowed to see them. |
| Quality and ticket evidence | docs/QUALITY_SCORE.md, relevant ticket, validation report, or changelog entry. | Chat-only summaries as the only evidence. |