Troubleshooting

Common issues and how to resolve them.

Authentication

"Unauthorized" error

Your token may be expired or invalid.

1. Sign in again from the Connecting page
2. Verify your MCTL account and workspace membership with the operator
3. Check the Authorization: Bearer header format

For OAuth tokens (Claude.ai connector): disconnect and reconnect the MCP server.

"Forbidden" on tenant operations

Your authenticated identity lacks tenant access or the required operation role.

• Run mctl_whoami and check the accessible namespaces
• Ask your operator to verify the tenant membership record and role
• Deploy/configuration needs developer; retirement and tenant deletion need owner
• A 503 during authorization is a failed role lookup, not evidence that you lack membership

"Forbidden" on workflows.mctl.ai

If you deploy or check logs via mctl_deploy_service / MCP, requests go through mctl-api's service account and the tenant authorization checks described above. Browsing or submitting workflows directly at workflows.mctl.ai is a separate path: it uses Argo Workflows' own SSO, which maps your session to a per-tenant ServiceAccount by group membership rather than by the mctl-api RBAC model.

A forbidden ... cannot get resource "workflows" ... in namespace "<tenant>" error here usually means your SSO session resolved to a different tenant's ServiceAccount than the one you're trying to act in. This happens when you belong to more than one tenant and their SSO group-resolution rules tie — most commonly because you were once part of a duplicate or since-renamed tenant that was never fully deleted. Ask a platform admin to check for stale tenants tied to your account and remove them; recreating your session (sign out and back in) after cleanup clears the stale group claim.

Token rejected

Clients sign in through MCTL and hold a token issued by mctl-api; you never paste one. If your client keeps getting 401, remove the server from the client and add it again with the URL https://api.mctl.ai/mcp only, so it signs in from scratch. Remove any Authorization header you added by hand.

MCP Connection

Tools not appearing in Claude.ai

1. Disconnect the MCP server from Claude.ai settings
2. Clear your browser cache
3. Reconnect from the Connecting page
4. Start a new conversation (existing conversations don't pick up tool changes)

Timeout errors on tool calls

Operations like deploy-service and scale-service are async. The tool returns a workflow name immediately — the actual work happens in the background.

If the MCP endpoint itself times out:

• Check if api.mctl.ai is reachable
• Client and server timeouts vary; inspect the error and check workflow status before retrying a write
• Try again — transient network issues resolve quickly

"Connection refused" in Claude Code / Cursor

Verify the MCP server URL is exactly https://api.mctl.ai/mcp (not /api/v1/mcp or other paths).

MCTL uses the Streamable HTTP transport. Configure it as type: "http" in Claude Code; the CLI option is --transport http:

{
  "mcpServers": {
    "mctl": {
      "type": "http",
      "url": "https://api.mctl.ai/mcp"
    }
  }
}

Deployment

Deployment stuck in "Pending" or "Progressing"

1. Check the operation status:

   "What's the status of workflow deploy-service-xyz?"

2. Common causes:

   • Image pull error: the container image doesn't exist or requires authentication
   • Insufficient resources: the tenant's resource quota is exhausted
   • Probe/configuration error: the app is not listening on its configured container port or health path

3. Check resource usage:

   "What's the resource usage for my-team?"

Service shows "OutOfSync" in ArgoCD

ArgoCD detected a difference between the GitOps repo and the cluster state.

• Expected after a deploy: ArgoCD normally reconciles automatically; timing depends on sync policy and resource health
• Persistent OutOfSync: someone may have manually modified the cluster. ArgoCD will reconcile on next sync
• Check the ArgoCD UI at ops.mctl.ai for diff details

Service shows "Degraded" health

ArgoCD reports an unhealthy resource; the application may not be running. Common causes:

1. Crash loop: the container is repeatedly crashing

   "Show me logs for my-app in staging"

2. Failed readiness probe: the /healthz or /readyz endpoint is not responding
   • Check that your app listens on the configured port
   • Verify environment variables are set correctly
3. OOMKilled: the container exceeds its memory limit

   "What's the resource usage for staging?"

   Consider scaling up memory limits or optimizing your application.

"Operation not found" error

The operation name must match a registered operation exactly. Available operations:

Operation	Description
deploy-service	Deploy or update a service
create-tenant	Create a new tenant
provision-database	Provision PostgreSQL
retire-service	Remove a service
delete-tenant	Delete a tenant
rollback-service	Rollback to a previous tag
preview-deploy	Create preview environment
preview-delete	Delete preview environment

Use mctl_list_operations to see the full list with parameters.

Image pull errors

If you see ImagePullBackOff or ErrImagePull:

1. Check the actual build first: your source repo's own Actions job only triggers the build — it doesn't run it. MCTL builds and pushes the image centrally via an Argo Workflow, so a green check on your repo's Actions tab just means the trigger request was sent, not that the build itself succeeded. Use mctl_get_workflow_status (or mctl_list_workflows) to check the actual build/deploy workflow before assuming the image exists — this saves time over guessing at the steps below.
2. Verify the image exists:

   docker pull ghcr.io/your-org/your-image:tag

3. For private images, grant repository access:

   "Grant access to repo myorg/my-app for my-team"

4. Check that the image tag is correct — tags are case-sensitive
5. If the image builds and pushes successfully but still won't pull, your tenant namespace may be missing the ghcr-credentials secret used to authenticate to ghcr.io. This is provisioned automatically for new tenants — contact the platform team if you're on an older tenant that predates it. Repository access (step 3) only covers the build; the registry pull is a separate credential.

Databases

Database provisioning fails

The provision-database operation creates a PostgreSQL database on the shared CNPG cluster.

Common issues:

• Database name conflict: a database with that name already exists in the tenant
• Workflow failure: inspect the provisioning workflow and database cluster connectivity
• Vault permissions: the tenant's Vault policy may not include the teams/{team}/* path

Cannot connect to database

Database credentials are stored in Vault and synced to Kubernetes secrets via ExternalSecrets.

1. Ask your operator to verify ExternalSecret synchronization and the generated database Secret. Service status alone does not expose secret contents.
2. Check that your application reads the correct environment variables:
   • the generated DB_HOST, DB_PORT, DB_USER, DB_PASSWORD, DB_NAME; configure DATABASE_URL yourself if your application needs it
3. Use the host and port from the provisioned DB_HOST and DB_PORT; do not substitute an assumed pooler address

Domains

Custom domain shows "pending"

After registration, create the returned TXT challenge and routing record, then call domain verification; see Custom Domains.

1. Add the CNAME record at your DNS provider:

   api.example.com  CNAME  staging-my-app.mctl.ai

2. Wait for DNS propagation (usually 1-5 minutes, up to 48 hours)
3. Verify the domain:

   "Verify domain api.example.com for my-app in staging"

TLS certificate not issued

TLS certificates are provisioned by cert-manager with Let's Encrypt after DNS verification succeeds.

• Check that the CNAME record is correctly configured
• Let's Encrypt has rate limits — if you've recently issued many certificates, you may need to wait
• Verify at ops.mctl.ai that the Certificate resource exists and check its status

Preview Environments

Preview not accessible

Previews get a temporary domain based on the preview name. Check:

1. That the preview was created successfully:

   "List all previews in staging"

2. The preview's health status — it follows the same deployment pipeline as regular services
3. Preview TTL — previews auto-delete after the configured TTL (default: 24 hours)

Self-Healing Agent

Agent not creating PRs for incidents

The mctl-agent processes alerts from AlertManager. If PRs aren't appearing:

1. Check that the alert is firing in AlertManager
2. The agent may have classified the alert as informational (no action needed)
3. Some alerts are suppressed to reduce noise (e.g. monitoring meta-alerts)
4. Check agent logs at agent.mctl.ai

Agent PR was not merged

Agent PRs must satisfy the configured review and approval gates for their current head. Check:

• The PR in the mctl-gitops repository on GitHub
• Telegram notifications for operator approval requests
• The incident status via MCP: "Show me details of incident INC-xxx"

Getting Help

If your issue isn't covered here:

1. Check the FAQ
2. Search the MCP Tools Reference for the relevant tool
3. Open an issue on GitHub
4. Contact support at [email protected]