Logs, sessions, traces and evals

#Where data lives

~/.config is $XDG_CONFIG_HOME and ~/.local/share is $XDG_DATA_HOME when those are set. The kimiflare directory names are kept for compatibility with earlier versions.

#Logs

Autopilot writes structured JSON logs of agent-side activity: tool calls, permission decisions, MCP and LSP lifecycle, session events and errors. There is one file per day, and files older than 7 days are pruned at startup.

The logs exclude prompts and completions.

autopilot logs path     # today's file
autopilot logs dir      # log directory
autopilot logs prune    # delete files older than 7 days

# Tail this session's activity, formatted
tail -f $(autopilot logs path) | jq

# Find the slowest tool calls in the last day
jq -r 'select(.event == "tool:end") | "\(.data.duration_ms)\t\(.data.tool)"' \
  $(autopilot logs path) | sort -rn | head

#Log settings

On OpenRouter, each generation is also visible in your OpenRouter activity log.

#Ship logs to OpenTelemetry

Set KIMIFLARE_OTEL_ENDPOINT to also send each log entry to an OTLP/HTTP endpoint, such as Datadog, Honeycomb, Grafana Loki or your own collector:

# Full path
export KIMIFLARE_OTEL_ENDPOINT="https://otel.example.com/v1/logs"
# Or the base URL (/v1/logs is appended)
export KIMIFLARE_OTEL_ENDPOINT="https://otel.example.com"

# Optional headers, comma-separated key=value pairs
export KIMIFLARE_OTEL_HEADERS="Authorization=Bearer xyz,X-Tenant=acme"

• Entries are batched every 5 seconds or every 100 entries, whichever comes first. Export is best-effort and never blocks the agent loop.
• Each entry becomes one OTel LogRecord.
• Correlation ids (session_id, turn_id, request_id) become record attributes, and data.* fields are flattened to attributes.
• The resource carries service.name=kimiflare and service.version.

#Sessions

Each conversation is saved as a session. Resume one with autopilot resume [id], autopilot -c, -S <id>, or /resume in the UI.

Autopilot also appends a Claude Code-format JSONL transcript per session. Hook payloads point transcript_path at it. Set KIMIFLARE_TRANSCRIPT=0 to turn it off.

#Local history

Autopilot never deletes your saved sessions or traces on its own. Once they pass 1 GB, a new session mentions how much space they take.

autopilot history                         # counts, sizes and paths
autopilot history prune --older-than 30   # delete sessions and traces untouched for 30 days

#Cost

The status bar shows the cost of each turn. On OpenRouter this is the billed cost; a local estimate marked ≈$ shows until the billed number arrives. On other providers, cost is estimated from the provider's published prices.

/cost totals cost by session, day, month and all time. Daily usage is kept in an append-only history.jsonl, so totals survive across sessions and version updates.

From the command line, autopilot cost shows cost attribution by task type:

#Session traces

Tracing is off by default. Turn it on for one run, or always:

KIMIFLARE_TRACE=1 autopilot

{ "trace": { "enabled": true } }

Each session then writes a trace.jsonl under ~/.local/share/kimiflare/traces/, one event per line. Change the location with KIMIFLARE_TRACE_DIR or trace.root. A trace records:

• every model request and response, with tokens, cost and latency;
• tool calls and their results, and each permission decision;
• compactions, completion checks, subagents and turn ends;
• outcome signals: the git changes a session made, test, lint, typecheck and build commands and whether they passed, your /good and /bad ratings, and whether your next message corrected the agent or told it to carry on.

Large bodies are stored once, by hash, next to the trace. Likely secrets are redacted before anything is written. Nothing is uploaded. A single session stops recording at 200 MB.

#Evals

autopilot eval runs Autopilot headlessly on a set of tasks and compares two configurations: a model, a setting or a harness change.

autopilot eval run --tasks evals/tasks/autopilot-bench --config evals/configs/baseline.json --model <id> --out /tmp/eval-a
autopilot eval run --tasks evals/tasks/autopilot-bench --config evals/configs/my-change.json --model <id> --out /tmp/eval-b
autopilot eval compare /tmp/eval-a/results.jsonl /tmp/eval-b/results.jsonl

#eval run options

Each run gets its own git worktree at the task's base commit, with tracing forced on and permissions auto-approved. The task passes if its test command exits 0. Results are appended to <out>/results.jsonl.

eval compare <results-a> <results-b> prints success rates with a paired significance test (McNemar), cost, turns and the tasks that flipped. Add --json for machine-readable output.

#Task format

Each task is a JSON file in the task set directory:

A harness config is { "name", "model"?, "temperature"?, "config": <partial config> }. config is merged over your own config for every run.

The repository ships autopilot-bench, twenty tasks mined from its own merged pull requests. See evals/README.md for the recipe to build a set from your own repository.