diff --git a/CHANGELOG.md b/CHANGELOG.md index f03bc81..734e940 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,25 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [6.9.7] - 2026-10-06 + +### Added + +- **Effort selector for GLM 5.3, GLM 5.3 Flash, Gemini 3.8 Flash and DeepSeek V4.1 Flash.** `/effort` opens a Faster ↔ Smarter slider (low · medium · high · max) for the current model: ←/→ to adjust, Enter to save, `s` to apply it to this session only. `/effort ` sets it directly, ←/→ in `/model` adjusts the highlighted model's effort, and the status bar shows the level in effect. The default is Medium. Picks are saved per model in `~/.orbcode/config.json` and read on every request, so every chat on the machine (new, resumed or already running) uses them until changed. OrbCode sends the level as `X-MATTERAI-REASONING-EFFORT`; the backend maps it to what the serving gateway accepts (e.g. GLM has no medium tier, so it runs at high; Fireworks tops out at high). Which models get the selector comes from the catalog's `reasoning_efforts`, with a built-in fallback for older backends. +- **Confirm before resuming a session with a cold prompt cache.** The gateway keeps a conversation's prompt cache for 5 minutes; after that, the first resumed turn re-reads the whole context at the uncached input price. `/resume` and `orbcode --resume ` now show how long the session has been idle, how large it is, and what resuming will cost as a share of your weekly usage limit (monthly on Lite, dollars for your own provider keys). The share comes from the backend's new `GET /axoncode/usage/estimate`, which prices the request exactly like a real one; if it can't be reached, the prompt shows the context size without a cost figure. You can then resume, or start a new conversation and pull the old one in later with `/task`. The prompt appears for any cold session with 100k+ tokens of context (the share fills in once the backend answers), and for smaller ones when the cost is at least 1% of the window; estimates are prefetched while the `/resume` picker is open so picking a session doesn't wait on the network. It's skipped when a prompt is passed alongside `--resume `. + +### Fixed + +- **Compaction no longer leaves a conversation stuck.** + - A "context too long" error from the provider now compacts and retries the step instead of failing the turn; before, every following turn hit the same error. + - A history that has already outgrown the window (e.g. a session resumed on a smaller-window model) still compacts: the summary request is trimmed to fit (older tool results stubbed, oldest turns dropped), with smaller budgets tried if the provider's real window is tighter than the catalog's. + - The 80% check now counts tool output and new input added since the last usage report, not just the last reported size. + - A failed auto-compaction is retried on the next turn instead of staying off for the rest of the session. + - A user message that triggers compaction as a turn starts is kept verbatim after the summary. +- **`read_file` caps characters as well as lines** (100k per file, 200k per call), so a minified bundle or a file of very long lines can't flood the context in one read. +- **Your model and its effort now survive a restart.** Models that only exist in the backend catalog (e.g. DeepSeek V4.1 Flash) weren't known until `/v1/models` answered, so on startup the saved model fell back to GLM 5.3 Flash and its effort to medium. The last fetched catalog is now cached in `~/.orbcode/models-cache.json` and loaded before settings, the saved model is restored once the live catalog arrives, and the plan-default logic reads the saved model from disk instead of the fallback. +- **A replaced conversation no longer leaks into the next one.** `/new`, `/resume`, sign-in/out and MCP reloads now abort the in-flight turn and drop any events, approval requests or follow-up questions it emits while unwinding, so a turn that was still running can't stream into the new conversation's transcript. + ## [6.9.6] - 2026-10-06 ### Added diff --git a/README.md b/README.md index b3ce3be..18f83d1 100644 --- a/README.md +++ b/README.md @@ -84,7 +84,13 @@ Then, from any project directory: orbcode ``` -To update later: `npm update -g @matterailab/orbcode` (or re-run the install command). +Updates install in the background: a new release (once it has been public for +24h) is downloaded into `~/.orbcode/versions/`, verified, and used from your +next launch — the banner just says to restart. If a new version fails to start, +OrbCode falls back to the previous one. To opt out, set +`ORBCODE_DISABLE_AUTOUPDATE=1` or `"autoUpdates": false` in +`~/.orbcode/settings.json`; you can always update manually with +`orbcode update` or `npm update -g @matterailab/orbcode`. ### From source (development) diff --git a/RELEASE.md b/RELEASE.md index ab89cf5..55750d9 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -20,6 +20,19 @@ Source (`src/`), tests, and workflow files are not published. or broken `dist/`. The CLI reads its version from `package.json` at runtime — bumping the version is all that's needed; no source change required. +## Background updates — compatibility rules + +Installed CLIs stage new releases under `~/.orbcode/versions/` and roll back to +the previous version if a release fails to start (see `src/utils/autoUpdate.ts` +and `bin/select-version.js`). Releases auto-install only after 24h on npm, so +**ship a fix within a day to keep a bad release from spreading**, and keep these +invariants so old and new versions can run against the same `~/.orbcode`: + +- `config.json`, `settings.json` and session files: additive changes only — + never rename, remove or change the type of an existing field. +- `bin/orbcode.js` must keep accepting `--version` (prints the bare version) + and `--self-test` (prints `ok `); updates are verified with them. + ## One-time setup 1. **Create an npm Automation token** (npmjs.com → Access Tokens → diff --git a/bin/orbcode.js b/bin/orbcode.js index 2df2f90..1acf7a1 100755 --- a/bin/orbcode.js +++ b/bin/orbcode.js @@ -1,5 +1,6 @@ #!/usr/bin/env node import { spawn } from "node:child_process" +import { readFileSync } from "node:fs" import { createRequire } from "node:module" import { fileURLToPath } from "node:url" @@ -7,11 +8,10 @@ process.title = "orbcode" const args = process.argv.slice(2) const first = args[0] +const versionOrHelpOnly = + args.includes("--help") || args.includes("-h") || args.includes("--version") || args.includes("-v") const runsWithoutTui = - args.includes("--help") || - args.includes("-h") || - args.includes("--version") || - args.includes("-v") || + versionOrHelpOnly || args.includes("--print") || args.includes("-p") || first === "update" || @@ -19,16 +19,9 @@ const runsWithoutTui = first === "plugin" || first === "plugins" -if (!runsWithoutTui && !process.versions.bun) { - const entrypoint = fileURLToPath(new URL("../dist/index.js", import.meta.url)) - let bunExecutable - try { - bunExecutable = createRequire(import.meta.url).resolve("bun/bin/bun.exe") - } catch { - console.error("OrbCode's bundled UI runtime is missing. Reinstall @matterailab/orbcode and try again.") - process.exit(1) - } - const child = spawn(bunExecutable, [entrypoint, ...args], { stdio: "inherit" }) +/** Run a child with inherited stdio, forwarding termination signals to it. */ +async function runChild(command, childArgs, env) { + const child = spawn(command, childArgs, { stdio: "inherit", env }) const forwardedSignals = ["SIGINT", "SIGTERM", "SIGHUP"] const forwarders = new Map() for (const signal of forwardedSignals) { @@ -41,12 +34,54 @@ if (!runsWithoutTui && !process.versions.bun) { child.once("exit", (code, signal) => resolve({ code, signal })) }) for (const [signal, forward] of forwarders) process.off(signal, forward) + return result +} + +function exitLike(result) { + if (result.signal) process.kill(process.pid, result.signal) + process.exit(result.code ?? 1) +} + +// Hand off to a newer version staged by the background updater, if any. Any +// problem selecting or starting it falls through to running this copy. +if (!process.versions.bun) { + let staged = null + try { + const { selectStagedVersion } = await import("./select-version.js") + const selfVersion = JSON.parse(readFileSync(new URL("../package.json", import.meta.url), "utf8")).version + staged = selectStagedVersion({ selfVersion, countAttempt: !versionOrHelpOnly }) + } catch { + staged = null + } + if (staged) { + const result = await runChild(process.execPath, [staged.bin, ...args], { + ...process.env, + ORBCODE_DELEGATED: "1", + }) + if (!result.error) exitLike(result) + } +} + +if (!runsWithoutTui && !process.versions.bun) { + const entrypoint = fileURLToPath(new URL("../dist/index.js", import.meta.url)) + let bunExecutable + try { + bunExecutable = createRequire(import.meta.url).resolve("bun/bin/bun.exe") + } catch { + console.error("OrbCode's bundled UI runtime is missing. Reinstall @matterailab/orbcode and try again.") + process.exit(1) + } + // The background updater verifies new versions through this same Node + // launcher, so it needs to know which Node binary started us. + const result = await runChild(bunExecutable, [entrypoint, ...args], { + ...process.env, + ORBCODE_NODE_PATH: process.env.ORBCODE_NODE_PATH || process.execPath, + }) if (result.error) { console.error(`Unable to launch OrbCode's UI runtime: ${result.error.message}`) process.exit(1) } - if (result.signal) process.kill(process.pid, result.signal) - process.exit(result.code ?? 1) + exitLike(result) } await import("../dist/index.js") diff --git a/bin/select-version.js b/bin/select-version.js new file mode 100644 index 0000000..6c1d9de --- /dev/null +++ b/bin/select-version.js @@ -0,0 +1,97 @@ +// Picks which OrbCode install should handle this launch: the copy this file +// ships in, or a newer version the background updater staged under +// ~/.orbcode/versions//. Every failure path returns null, which means +// "run the current copy" — exactly the behaviour before background updates. +// +// Plain JS with no dependencies: it runs before anything else, under Node. +import * as fs from "node:fs" +import * as os from "node:os" +import * as path from "node:path" + +const PACKAGE_PATH = ["node_modules", "@matterailab", "orbcode"] +const VERSION_RE = /^\d+\.\d+\.\d+(?:-[0-9A-Za-z.-]+)?$/ + +export const HEALTHY_MARKER = ".healthy" +export const ATTEMPTS_MARKER = ".attempts" +/** Launches that may fail to reach a healthy state before a version is skipped. */ +export const MAX_UNHEALTHY_LAUNCHES = 1 + +export function getVersionsDir(env = process.env) { + const configDir = env.MATTERAI_CONFIG_DIR || path.join(os.homedir(), ".orbcode") + return path.join(configDir, "versions") +} + +export function getVersionBin(versionDir) { + return path.join(versionDir, ...PACKAGE_PATH, "bin", "orbcode.js") +} + +export function isVersionString(value) { + return typeof value === "string" && VERSION_RE.test(value) +} + +/** -1 / 0 / 1, matching src/utils/updateCheck.ts#compareVersions. */ +export function compareVersions(a, b) { + const split = (v) => { + const [main, pre = ""] = v.split("-", 2) + const parts = main.split(".").map((p) => { + const n = Number.parseInt(p, 10) + return Number.isFinite(n) ? n : 0 + }) + while (parts.length < 3) parts.push(0) + return [parts, pre] + } + const [aMain, aPre] = split(a) + const [bMain, bPre] = split(b) + for (let i = 0; i < 3; i++) { + if (aMain[i] !== bMain[i]) return aMain[i] < bMain[i] ? -1 : 1 + } + if (aPre === bPre) return 0 + if (aPre === "") return 1 + if (bPre === "") return -1 + return aPre < bPre ? -1 : 1 +} + +export function readAttempts(versionDir) { + try { + const n = Number.parseInt(fs.readFileSync(path.join(versionDir, ATTEMPTS_MARKER), "utf8"), 10) + return Number.isFinite(n) && n > 0 ? n : 0 + } catch { + return 0 + } +} + +export function isHealthy(versionDir) { + return fs.existsSync(path.join(versionDir, HEALTHY_MARKER)) +} + +/** A staged version that failed to start and never proved itself healthy. */ +export function isBad(versionDir) { + return !isHealthy(versionDir) && readAttempts(versionDir) >= MAX_UNHEALTHY_LAUNCHES +} + +/** + * Returns `{ version, bin }` for a staged version to delegate to, or null to + * run the current copy. `countAttempt` is false for launches that never reach + * a health check (--version / --help) so they can't mark a version bad. + */ +export function selectStagedVersion({ selfVersion, env = process.env, countAttempt = true }) { + try { + if (env.ORBCODE_DELEGATED) return null + if (!isVersionString(selfVersion)) return null + const versionsDir = getVersionsDir(env) + const current = fs.readFileSync(path.join(versionsDir, "current"), "utf8").trim() + if (!isVersionString(current)) return null + if (compareVersions(current, selfVersion) <= 0) return null + const versionDir = path.join(versionsDir, current) + const bin = getVersionBin(versionDir) + if (!fs.existsSync(bin)) return null + if (!isHealthy(versionDir)) { + const attempts = readAttempts(versionDir) + if (attempts >= MAX_UNHEALTHY_LAUNCHES) return null + if (countAttempt) fs.writeFileSync(path.join(versionDir, ATTEMPTS_MARKER), String(attempts + 1)) + } + return { version: current, bin } + } catch { + return null + } +} diff --git a/package-lock.json b/package-lock.json index d3c37ea..b332798 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,17 +1,17 @@ { "name": "@matterailab/orbcode", - "version": "6.9.6", + "version": "6.9.7", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@matterailab/orbcode", - "version": "6.9.6", + "version": "6.9.7", "license": "MIT", "dependencies": { "@ai-sdk/anthropic": "^3.0.85", "@ai-sdk/openai-compatible": "^2.0.51", - "@modelcontextprotocol/sdk": "^1.29.0", + "@modelcontextprotocol/sdk": "^1.32.1", "@opentui/core": "0.4.3", "@opentui/react": "0.4.3", "ai": "^6.0.207", @@ -738,12 +738,12 @@ } }, "node_modules/@modelcontextprotocol/sdk": { - "version": "1.29.0", - "resolved": "https://registry.npmjs.org/@modelcontextprotocol/sdk/-/sdk-1.29.0.tgz", - "integrity": "sha512-zo37mZA9hJWpULgkRpowewez1y6ML5GsXJPY8FI0tBBCd77HEvza4jDqRKOXgHNn867PVGCyTdzqpz0izu5ZjQ==", + "version": "1.32.1", + "resolved": "https://registry.npmjs.org/@modelcontextprotocol/sdk/-/sdk-1.32.1.tgz", + "integrity": "sha512-2DdE+SJDtzLEEWzY1ZjY7Q+VcPhcV1KisD3zI4u0XZyktsjHum1mwbMI+JaulUBi2OZk+KJAi2uPXzxichPkdw==", "license": "MIT", "dependencies": { - "@hono/node-server": "^1.19.9", + "@hono/node-server": "^1.19.9 || ^2.0.5", "ajv": "^8.17.1", "ajv-formats": "^3.0.1", "content-type": "^1.0.5", diff --git a/package.json b/package.json index b840535..8b6f57a 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@matterailab/orbcode", - "version": "6.9.6", + "version": "6.9.7", "description": "OrbCode CLI — agentic coding in your terminal, by MatterAI", "type": "module", "bin": { @@ -42,13 +42,18 @@ "test:search": "node --import tsx --test test/search-files.test.ts", "test:rewind": "node --import tsx --test test/rewind.test.ts", "test:headless": "node --import tsx --test test/headless.test.ts", - "test:ui": "bun test test/ui-viewport.test.tsx test/session-picker.test.tsx test/rewind-picker.test.tsx", + "test:autoupdate": "node --import tsx --test test/auto-update.test.ts", + "test:warmup": "node --import tsx --test test/cache-warmup.test.ts", + "test:resume": "node --import tsx --test test/resume-cost.test.ts", + "test:compaction": "node --import tsx --test test/compaction.test.ts test/agent-context.test.ts test/files.test.ts", + "test:effort": "node --import tsx --test test/model-effort.test.ts test/model-persistence.test.ts", + "test:ui": "bun test test/ui-viewport.test.tsx test/session-picker.test.tsx test/rewind-picker.test.tsx test/resume-confirm.test.tsx test/model-picker.test.tsx test/effort-picker.test.tsx", "prepublishOnly": "npm run typecheck && npm run build" }, "dependencies": { "@ai-sdk/anthropic": "^3.0.85", "@ai-sdk/openai-compatible": "^2.0.51", - "@modelcontextprotocol/sdk": "^1.29.0", + "@modelcontextprotocol/sdk": "^1.32.1", "@opentui/core": "0.4.3", "@opentui/react": "0.4.3", "ai": "^6.0.207", diff --git a/src/api/client.ts b/src/api/client.ts index c821651..83e0410 100644 --- a/src/api/client.ts +++ b/src/api/client.ts @@ -5,11 +5,14 @@ import { DEFAULT_HEADERS, getClientMetadataHeaders, X_AXONCODE_TASKID, + X_AXONCODE_WARMUP, X_AXON_REPO, X_ORGANIZATIONID, + X_REASONING_EFFORT, } from "./headers.js"; import { stripReasoningDetails, type LLMClient } from "./llmClient.js"; import { getGatewayModelId, getModel } from "./models.js"; +import { getModelEffort, loadModelEfforts } from "../config/settings.js"; import type { ApiStreamChunk } from "./stream.js"; interface CompletionUsage { @@ -62,17 +65,21 @@ export class AxonClient implements LLMClient { if (this.options.organizationId) headers[X_ORGANIZATIONID] = this.options.organizationId; if (this.options.repo) headers[X_AXON_REPO] = this.options.repo; + // Read per request (not per session): an effort picked in any chat on + // this machine applies to every other chat's next turn. + const effort = getModelEffort(loadModelEfforts(), model); + if (effort) headers[X_REASONING_EFFORT] = effort; return headers; } - async *createMessage( + /** Shared by createMessage and warmup: the prompt cache only hits when the + * model, system prompt and tools are identical between the two. */ + private buildRequest( + model: ReturnType, systemPrompt: string, messages: OpenAI.Chat.ChatCompletionMessageParam[], tools: OpenAI.Chat.ChatCompletionTool[], - abortSignal?: AbortSignal, - ): AsyncGenerator { - const model = getModel(this.options.modelId); - + ): OpenAI.Chat.Completions.ChatCompletionCreateParamsStreaming { const requestOptions: OpenAI.Chat.Completions.ChatCompletionCreateParamsStreaming = { model: getGatewayModelId(model), @@ -90,6 +97,45 @@ export class AxonClient implements LLMClient { requestOptions.tool_choice = "auto"; requestOptions.parallel_tool_calls = true; } + return requestOptions; + } + + async warmup( + systemPrompt: string, + tools: OpenAI.Chat.ChatCompletionTool[], + abortSignal?: AbortSignal, + ): Promise { + const model = getModel(this.options.modelId); + const requestOptions = this.buildRequest( + model, + systemPrompt, + [{ role: "user", content: "Hi" }], + tools, + ); + requestOptions.max_tokens = 1; + const stream = await this.client.chat.completions.create(requestOptions, { + headers: { ...this.requestHeaders(model), [X_AXONCODE_WARMUP]: "1" }, + signal: abortSignal, + }); + // Drain so the gateway sees a completed request (and bills/logs it). + for await (const _chunk of stream) { + // discard + } + } + + async *createMessage( + systemPrompt: string, + messages: OpenAI.Chat.ChatCompletionMessageParam[], + tools: OpenAI.Chat.ChatCompletionTool[], + abortSignal?: AbortSignal, + ): AsyncGenerator { + const model = getModel(this.options.modelId); + const requestOptions = this.buildRequest( + model, + systemPrompt, + messages, + tools, + ); const stream = await this.client.chat.completions.create(requestOptions, { headers: this.requestHeaders(model), diff --git a/src/api/headers.ts b/src/api/headers.ts index a27d564..0cab18c 100644 --- a/src/api/headers.ts +++ b/src/api/headers.ts @@ -6,11 +6,15 @@ import { getShell } from "../utils/shell.js" // Header names ported from the Orbital extension (shared/kilocode/headers.ts). export const X_AXONCODE_VERSION = "X-AxonCode-Version" export const X_AXONCODE_TASKID = "X-AxonCode-TaskId" +/** Marks a prompt-cache priming request; the backend skips title generation for it. */ +export const X_AXONCODE_WARMUP = "X-AxonCode-Warmup" export const X_ORGANIZATIONID = "X-KiloCode-OrganizationId" export const X_AXON_REPO = "X-AXON-REPO" export const X_MODEL_CONTEXT_WINDOW = "X-Model-Context-Window" export const X_DEVICE_OS = "X-Device-OS" export const X_CLIENT_USER_AGENT = "X-Client-User-Agent" +/** low | medium | high | max — mapped by the backend to the serving provider's levels. */ +export const X_REASONING_EFFORT = "X-MATTERAI-REASONING-EFFORT" export const DEFAULT_HEADERS = { "HTTP-Referer": "https://matterai.so", diff --git a/src/api/llmClient.ts b/src/api/llmClient.ts index 66f1d4a..0c6d322 100644 --- a/src/api/llmClient.ts +++ b/src/api/llmClient.ts @@ -17,6 +17,17 @@ export interface LLMClient { tools: OpenAI.Chat.ChatCompletionTool[], abortSignal?: AbortSignal, ): AsyncGenerator + + /** + * Prime the provider's prompt cache with `systemPrompt` + `tools` (one + * output token) so the task's first real request starts warm. Optional: + * clients without a shared prompt cache simply don't implement it. + */ + warmup?( + systemPrompt: string, + tools: OpenAI.Chat.ChatCompletionTool[], + abortSignal?: AbortSignal, + ): Promise } /** diff --git a/src/api/models.ts b/src/api/models.ts index c93cd4c..5ffbc2c 100644 --- a/src/api/models.ts +++ b/src/api/models.ts @@ -6,6 +6,16 @@ import * as path from "node:path" import { DEFAULT_HEADERS } from "./headers.js" +/** settings' getConfigDir, inlined: settings imports this module. */ +function configDir(): string { + return process.env.MATTERAI_CONFIG_DIR || path.join(os.homedir(), ".orbcode") +} + +/** Last successful /v1/models response, so catalog-only models resolve offline and at startup. */ +function catalogCachePath(): string { + return path.join(configDir(), "models-cache.json") +} + /** * Read the currently selected model id from disk without triggering the full * settings-load side effects (env application, custom-model registration). @@ -13,8 +23,7 @@ import { DEFAULT_HEADERS } from "./headers.js" */ function loadSettingsModel(): string { try { - const dir = process.env.MATTERAI_CONFIG_DIR || path.join(os.homedir(), ".orbcode") - const raw = fs.readFileSync(path.join(dir, "config.json"), "utf8") + const raw = fs.readFileSync(path.join(configDir(), "config.json"), "utf8") const parsed = JSON.parse(raw) return typeof parsed.model === "string" ? parsed.model : "" } catch { @@ -28,6 +37,26 @@ function loadSettingsModel(): string { */ export type ModelEffort = "low" | "medium" | "high" | "xhigh" | "max"; +/** + * Effort levels the MatterAI gateway accepts for models with an effort + * selector. The backend maps each level to what the serving provider supports. + */ +export const GATEWAY_EFFORT_LEVELS = ["low", "medium", "high", "max"] as const; +export type GatewayEffort = (typeof GATEWAY_EFFORT_LEVELS)[number]; +export const DEFAULT_GATEWAY_EFFORT: GatewayEffort = "medium"; + +/** Used until the catalog (which advertises `reasoning_efforts`) has loaded. */ +const FALLBACK_EFFORT_MODELS = new Set([ + "zai/glm-5.3", + "zai/glm-5.3-flash", + "gemini-3.8-flash", + "deepseek/deepseek-v4.1-flash", +]); + +export function isGatewayEffort(value: unknown): value is GatewayEffort { + return (GATEWAY_EFFORT_LEVELS as readonly unknown[]).includes(value); +} + export interface AxonModel { id: string; /** Model ID sent to the MatterAI gateway when it differs from the local selection ID. */ @@ -50,6 +79,8 @@ export interface AxonModel { costMultiplier?: number; /** True when the backend catalog marks this model as the free-plan model. */ freePlan?: boolean; + /** Effort levels the gateway accepts for this model (catalog `reasoning_efforts`). */ + reasoningEfforts?: GatewayEffort[]; /** * Which transport serves this model. Absent (or "matterai"/"axon") routes * through the MatterAI gateway (OpenAI `/chat/completions`). Any other value @@ -406,6 +437,14 @@ export function getModel(modelId: string): AxonModel { return AXON_MODELS[modelId] ?? AXON_MODELS[DEFAULT_MODEL_ID]; } +/** Effort levels for the gateway effort selector; empty when the model has none. */ +export function getModelEffortLevels(model: AxonModel): GatewayEffort[] { + // Own-provider models (AI SDK) carry their own `effort` setting instead. + if (model.provider && model.provider !== "matterai" && model.provider !== "axon") return []; + if (model.reasoningEfforts) return model.reasoningEfforts; + return FALLBACK_EFFORT_MODELS.has(model.id) ? [...GATEWAY_EFFORT_LEVELS] : []; +} + /** Whether an AxonCode plan string is the free tier (a missing plan counts as free). */ export function isFreePlan(plan?: string): boolean { const normalized = plan?.trim().toLowerCase() ?? ""; @@ -439,6 +478,76 @@ export function getGatewayModelId(model: AxonModel): string { * into BUILTIN_AXON_MODELS and AXON_MODELS so the model picker and agent loops can * dynamically use newly added models without hardcoding. */ +function parseCatalogItems(items: unknown[]): AxonModel[] { + const models: AxonModel[] = []; + for (const raw of items) { + const item = raw as any; + if (!item?.id || typeof item.id !== "string" || item.id.startsWith("axon-")) { + continue; + } + models.push({ + id: item.id, + name: item.name || item.id, + description: item.description || `${item.name || item.id} open model`, + contextWindow: item.context_length || 232000, + maxOutputTokens: item.max_output_length || 64000, + supportsImages: Array.isArray(item.input_modalities) + ? item.input_modalities.includes("image") + : true, + inputPrice: + typeof item.pricing?.prompt === "string" + ? parseFloat(item.pricing.prompt) || 0 + : typeof item.pricing?.prompt === "number" + ? item.pricing.prompt + : 0, + outputPrice: + typeof item.pricing?.completion === "string" + ? parseFloat(item.pricing.completion) || 0 + : typeof item.pricing?.completion === "number" + ? item.pricing.completion + : 0, + free: false, + freePlan: item.freePlan === true, + iconUrl: typeof item.iconUrl === "string" ? item.iconUrl : undefined, + costMultiplier: + typeof item.costMultiplier === "number" ? item.costMultiplier : undefined, + // Absent on older backends: fall back to the built-in selector list. + reasoningEfforts: Array.isArray(item.reasoning_efforts) + ? item.reasoning_efforts.filter(isGatewayEffort) + : undefined, + }); + } + return models; +} + +function registerCatalogModels(models: AxonModel[]): void { + for (const model of models) { + managedModelIds.add(model.id); + BUILTIN_AXON_MODELS[model.id] = model; + AXON_MODELS[model.id] = model; + } +} + +/** + * Register the catalog from the last successful fetch (models-cache.json). + * Call at startup before loadSettings(): a saved selection of a model that + * only exists in the backend catalog (e.g. DeepSeek V4.1 Flash) is otherwise + * "unknown" until /v1/models answers, and loadSettings() would fall back to + * the default model — dropping both the model and its saved effort. Nothing + * is pruned here; the next fetch reconciles. + */ +export function loadCachedModelCatalog(): void { + try { + const cached = JSON.parse(fs.readFileSync(catalogCachePath(), "utf8")); + const models = parseCatalogItems(Array.isArray(cached?.data) ? cached.data : []); + if (models.length === 0) return; + catalogOrder = models.map((model) => model.id); + registerCatalogModels(models); + } catch { + // no cache yet (first run) or unreadable: the fetch fills it in + } +} + export async function fetchDynamicModels( token?: string, organizationId?: string, @@ -480,41 +589,8 @@ export async function fetchDynamicModels( } const json = (await res.json()) as any; - const items = Array.isArray(json?.data) ? json.data : []; - - const fetched: AxonModel[] = []; - for (const item of items) { - if (!item?.id || typeof item.id !== "string" || item.id.startsWith("axon-")) { - continue; - } - fetched.push({ - id: item.id, - name: item.name || item.id, - description: item.description || `${item.name || item.id} open model`, - contextWindow: item.context_length || 232000, - maxOutputTokens: item.max_output_length || 64000, - supportsImages: Array.isArray(item.input_modalities) - ? item.input_modalities.includes("image") - : true, - inputPrice: - typeof item.pricing?.prompt === "string" - ? parseFloat(item.pricing.prompt) || 0 - : typeof item.pricing?.prompt === "number" - ? item.pricing.prompt - : 0, - outputPrice: - typeof item.pricing?.completion === "string" - ? parseFloat(item.pricing.completion) || 0 - : typeof item.pricing?.completion === "number" - ? item.pricing.completion - : 0, - free: false, - freePlan: item.freePlan === true, - iconUrl: typeof item.iconUrl === "string" ? item.iconUrl : undefined, - costMultiplier: - typeof item.costMultiplier === "number" ? item.costMultiplier : undefined, - }); - } + const items: unknown[] = Array.isArray(json?.data) ? json.data : []; + const fetched = parseCatalogItems(items); // Reconcile only when the backend returned a usable catalog — an empty or // failed response must never wipe the offline fallback. Retired models @@ -531,10 +607,12 @@ export async function fetchDynamicModels( delete AXON_MODELS[id]; } managedModelIds.clear(); - for (const model of fetched) { - managedModelIds.add(model.id); - BUILTIN_AXON_MODELS[model.id] = model; - AXON_MODELS[model.id] = model; + registerCatalogModels(fetched); + try { + fs.mkdirSync(configDir(), { recursive: true }); + fs.writeFileSync(catalogCachePath(), JSON.stringify({ data: items })); + } catch { + // best-effort: without the cache, startup just waits for the fetch } } return BUILTIN_AXON_MODELS; diff --git a/src/auth/auth.ts b/src/auth/auth.ts index 303ae47..c7403d9 100644 --- a/src/auth/auth.ts +++ b/src/auth/auth.ts @@ -1,6 +1,8 @@ // Auth ported from the Orbital extension: a MatterAI JWT acts as the API key. // Backend URL resolution mirrors getKiloBaseUriFromToken / getKiloUrlFromToken. +import { DEFAULT_HEADERS } from "../api/headers.js"; + export const DEFAULT_BACKEND_URL = "https://api.matterai.so"; export const API_GATEWAY_PATH = "https://api2.matterai.so/v1/web/"; @@ -183,6 +185,55 @@ export async function fetchProfile(token: string): Promise { return (await response.json()) as ProfileData; } +/** Share of the plan windows a prospective request would use (no credit amounts). */ +export interface UsageEstimate { + weeklyPercentage: number; + monthlyPercentage: number; +} + +/** + * Price `inputTokens` uncached input tokens on `model` against the user's plan + * windows (GET /axoncode/usage/estimate). The backend applies the same rate, + * per-org overrides and OSS cost multiplier as a real request. + */ +export async function fetchUsageEstimate( + token: string, + model: string, + inputTokens: number, +): Promise { + const params = new URLSearchParams({ + model, + inputTokens: String(Math.max(0, Math.round(inputTokens))), + }); + const url = getUrlFromToken( + `https://api.matterai.so/axoncode/usage/estimate?${params}`, + token, + ); + const response = await fetch(url, { + headers: { + Authorization: `Bearer ${token}`, + "Content-Type": "application/json", + // Priced by user agent: first-party clients pay list price. + "User-Agent": DEFAULT_HEADERS["User-Agent"], + }, + signal: AbortSignal.timeout(3000), + }); + if (!response.ok) { + throw new Error(`Usage estimate failed (${response.status}).`); + } + const data = (await response.json()) as Partial; + if ( + typeof data.weeklyPercentage !== "number" || + typeof data.monthlyPercentage !== "number" + ) { + throw new Error("Usage estimate returned no percentages."); + } + return { + weeklyPercentage: data.weeklyPercentage, + monthlyPercentage: data.monthlyPercentage, + }; +} + export async function resetWeeklyUsage( token: string, ): Promise { diff --git a/src/config/settings.ts b/src/config/settings.ts index a60e852..530ee69 100644 --- a/src/config/settings.ts +++ b/src/config/settings.ts @@ -3,7 +3,17 @@ import * as fs from "node:fs" import * as os from "node:os" import * as path from "node:path" -import { DEFAULT_MODEL_ID, isValidAxonModel, registerCustomModels, type CustomModelConfig } from "../api/models.js" +import { + DEFAULT_GATEWAY_EFFORT, + DEFAULT_MODEL_ID, + getModelEffortLevels, + isGatewayEffort, + isValidAxonModel, + registerCustomModels, + type AxonModel, + type CustomModelConfig, + type GatewayEffort, +} from "../api/models.js" import { isHookEvent, type HookMatcher, type HooksConfig } from "../core/hooks.js" import type { McpServerConfig } from "../mcp/types.js" @@ -25,6 +35,12 @@ export interface OrbCodeSettings { autoApproveSafeCommands: boolean /** OrbCode-owned TUI palette; terminal theme detection never overrides it. */ theme: OrbCodeThemeMode + /** + * Effort picked per model in the effort selector. Lives in config.json and + * is read on every request, so every chat on this machine — new, resumed or + * already running — uses it until it's changed again. + */ + modelEfforts?: Record // The fields below come from settings.json files (and env vars) and are // never persisted back to config.json. @@ -45,6 +61,8 @@ export interface OrbCodeSettings { enabledMcpServers?: string[] /** MCP server names the user has explicitly disabled. */ disabledMcpServers?: string[] + /** false turns off background updates (same as ORBCODE_DISABLE_AUTOUPDATE=1). */ + autoUpdates?: boolean } const DEFAULTS: OrbCodeSettings = { @@ -65,6 +83,7 @@ const SETTINGS_KEYS = [ "env", "enabledMcpServers", "disabledMcpServers", + "autoUpdates", ] as const export function getConfigDir(): string { @@ -251,9 +270,62 @@ export function loadSettings(): OrbCodeSettings { if (settings.theme !== "dark" && settings.theme !== "light") { settings.theme = DEFAULTS.theme } + if (settings.modelEfforts && typeof settings.modelEfforts === "object") { + settings.modelEfforts = Object.fromEntries( + Object.entries(settings.modelEfforts).filter(([, effort]) => isGatewayEffort(effort)), + ) + } else { + settings.modelEfforts = undefined + } return settings } +// Efforts picked "for this session only" (`s` in the /effort picker): held in +// this process, layered over the saved ones, never written to config.json. +const sessionModelEfforts: Record = {} + +/** Set (or with no effort, clear) this process's effort override for a model. */ +export function setSessionModelEffort(modelId: string, effort?: GatewayEffort): void { + if (effort) sessionModelEfforts[modelId] = effort + else delete sessionModelEfforts[modelId] +} + +/** Saved efforts with this session's overrides on top. */ +export function withSessionModelEfforts( + saved: Record | undefined, +): Record { + return { ...saved, ...sessionModelEfforts } +} + +/** Per-model efforts in effect for this process — cheap enough to read per request. */ +export function loadModelEfforts(): Pick { + const stored = readJson(getConfigPath())?.modelEfforts + const saved = + stored && typeof stored === "object" + ? Object.fromEntries( + Object.entries(stored as Record).filter( + (entry): entry is [string, GatewayEffort] => isGatewayEffort(entry[1]), + ), + ) + : undefined + return { modelEfforts: withSessionModelEfforts(saved) } +} + +/** + * Effort to send for `model`: the user's pick for that model, else medium. + * Undefined when the model has no effort selector. + */ +export function getModelEffort( + settings: Pick, + model: AxonModel, +): GatewayEffort | undefined { + const levels = getModelEffortLevels(model) + if (levels.length === 0) return undefined + const picked = settings.modelEfforts?.[model.id] + if (picked && levels.includes(picked)) return picked + return levels.includes(DEFAULT_GATEWAY_EFFORT) ? DEFAULT_GATEWAY_EFFORT : levels[0] +} + /** * Effective credential: MATTERAI_TOKEN env > apiKey (settings.json / env) > * stored login token. @@ -274,6 +346,8 @@ export function saveSettings(settings: OrbCodeSettings): void { autoApproveEdits: settings.autoApproveEdits, autoApproveSafeCommands: settings.autoApproveSafeCommands, theme: settings.theme, + modelEfforts: + settings.modelEfforts && Object.keys(settings.modelEfforts).length > 0 ? settings.modelEfforts : undefined, } fs.writeFileSync(getConfigPath(), JSON.stringify(toPersist, null, "\t") + "\n", { mode: 0o600 }) } diff --git a/src/core/agent.ts b/src/core/agent.ts index 3c52368..8e4508d 100644 --- a/src/core/agent.ts +++ b/src/core/agent.ts @@ -62,6 +62,13 @@ import { reportUsageEvent, } from "../api/metrics.js" +/** + * Prompt-cache warmup on launch and /new (see Agent.warmCache). Off until it + * has had more testing; while off, agents are created lazily on the first + * message exactly as before. + */ +export const CACHE_WARMUP_ENABLED = false + const MAX_STEPS_PER_TURN = 50 const RESULT_PREVIEW_LINES = 6 /** Maximum number of independent read-only tools started at once. */ @@ -100,6 +107,13 @@ const PRUNE_MIN_CHARS = 1500 const PRUNABLE_TOOLS = new Set(["read_file", "search_files", "list_files", "Bash", "execute_command", "web_fetch", "web_search"]) /** Summarize the history before a step once context passes this fraction of the window. */ const AUTO_COMPACT_FRACTION = 0.8 +/** Rough chars-per-token for content the gateway hasn't measured yet. */ +const CHARS_PER_TOKEN = 4 +/** Tokens counted per image part: its base64 length says nothing about its token cost. */ +const IMAGE_TOKEN_ESTIMATE = 1500 +/** Share of the window the summary request may use, tried in order. The + * smaller budgets cover upstreams whose real window is under the catalog's. */ +const SUMMARY_BUDGET_FRACTIONS = [0.6, 0.3, 0.15] /** Warn the model when the same call returns the same output this many times in a row. */ const REPEAT_WARN_AT = 3 /** A successful edit changes the files, so earlier identical calls may now differ. */ @@ -109,6 +123,8 @@ const EDIT_TOOLS = new Set(["file_edit", "multi_file_edit", "file_write"]) * usable HTTP status — socket reset, DNS, timeout, TLS drop) plus 5xx/408/429 * server responses. Real 4xx client errors (auth, bad request) are not retried. */ function isRetryableStreamError(error: unknown): boolean { + // Resending the same oversized request can't succeed; the turn compacts instead. + if (isContextOverflowError(error)) return false const err = error as { status?: number; code?: number | string } const status = Number(err?.status ?? err?.code) if (Number.isFinite(status) && status !== 0) { @@ -117,6 +133,75 @@ function isRetryableStreamError(error: unknown): boolean { return true } +/** The provider rejected the request because the conversation doesn't fit its context window. */ +function isContextOverflowError(error: unknown): boolean { + const err = error as { + status?: number + statusCode?: number + code?: unknown + message?: unknown + error?: unknown + } + const status = Number(err?.status ?? err?.statusCode) + if (Number.isFinite(status) && status !== 0 && ![400, 413, 422].includes(status)) return false + let body = "" + try { + body = JSON.stringify(err?.error ?? "") + } catch { + // unserializable error body — the message alone decides + } + const text = `${String(err?.message ?? "")} ${String(err?.code ?? "")} ${body}` + return /context[ _-]?(length|window|limit)|maximum context|too many (input )?tokens|prompt is too long|input is too long|request too large|exceeds? (the )?(maximum|max|context|token)|reduce the length/i.test( + text, + ) +} + +/** Token estimate for a message the gateway hasn't measured yet. */ +function messageTokenEstimate(message: OpenAI.Chat.ChatCompletionMessageParam): number { + let chars = 0 + let images = 0 + const content = (message as { content?: unknown }).content + if (typeof content === "string") { + chars += content.length + } else if (Array.isArray(content)) { + for (const part of content as Array<{ type?: string; text?: string }>) { + if (part?.type === "text") chars += part.text?.length ?? 0 + else if (part?.type === "image_url") images++ + } + } + if (message.role === "assistant") { + for (const call of message.tool_calls ?? []) { + if (call.type === "function") chars += call.function.name.length + call.function.arguments.length + } + } + return Math.ceil(chars / CHARS_PER_TOKEN) + images * IMAGE_TOKEN_ESTIMATE +} + +/** Cut a message's text down to `maxChars` (structure and tool-call links intact). */ +function truncateMessage( + message: OpenAI.Chat.ChatCompletionMessageParam, + maxChars: number, +): OpenAI.Chat.ChatCompletionMessageParam { + const marker = "\n[… truncated to fit the context window …]" + const content = (message as { content?: unknown }).content + if (typeof content === "string") { + return content.length > maxChars + ? ({ ...message, content: content.slice(0, maxChars) + marker } as OpenAI.Chat.ChatCompletionMessageParam) + : message + } + if (Array.isArray(content)) { + return { + ...message, + content: (content as Array<{ type?: string; text?: string }>).map((part) => + part?.type === "text" && (part.text?.length ?? 0) > maxChars + ? { ...part, text: part.text!.slice(0, maxChars) + marker } + : part, + ), + } as OpenAI.Chat.ChatCompletionMessageParam + } + return message +} + function retryBackoffMs(attempt: number): number { return Math.min(500 * 2 ** attempt, 8000) } @@ -406,6 +491,8 @@ export class Agent { private sessionApproveEdits: boolean private sessionApproveCommands = false private abortController?: AbortController + /** in-flight prompt-cache warmup (see warmCache) */ + private warmupController?: AbortController private totalCost = 0 /** * Latest context window usage (input + output tokens from the most recent @@ -433,8 +520,12 @@ export class Agent { private stopHookActive = false /** tool results at message indexes below this are sent as short stubs (see pruneStaleToolResults) */ private prunedBefore = 0 - /** set after a failed auto-compaction so it isn't retried on every step */ + /** set after a failed auto-compaction so it isn't retried on every step of + * the same turn; cleared at the start of each turn and on any success */ private autoCompactFailed = false + /** how many messages the last usage report (`contextTokens`) covered; later + * ones (tool results, new user input) are estimated until the next report */ + private measuredMessages = 0 /** consecutive identical call+output tracking for loop warnings */ private readonly repeatTracker = new Map() /** in-flight fire-and-forget hook promises (Notification), awaited on exit */ @@ -480,6 +571,8 @@ export class Agent { this.todos = normalizeTodoList(options.resume.todos) this.totalCost = options.resume.totalCost this.contextTokens = options.resume.contextTokens + // Sessions without a recorded size are estimated from their messages. + this.measuredMessages = this.contextTokens > 0 ? this.messages.length : 0 this.title = options.resume.title this.createdAt = options.resume.createdAt this.firstMessageSent = this.messages.length > 0 @@ -525,6 +618,28 @@ export class Agent { }) } + /** + * Prime the gateway's prompt cache for this task before the user's first + * message: same taskId (provider session affinity), system prompt and tools + * as the first real turn, one output token. Only for a fresh conversation; + * best-effort and silent, since a miss just means a normal cold start. + */ + async warmCache(): Promise { + if (this.messages.length > 0 || !this.client.warmup) return + this.warmupController?.abort() + const controller = new AbortController() + this.warmupController = controller + try { + await this.mcp?.whenStarted() + if (controller.signal.aborted || this.messages.length > 0) return + await this.client.warmup(this.systemPrompt, getActiveTools(this.mcp), controller.signal) + } catch { + // ignore — warmup is an optimization only + } finally { + if (this.warmupController === controller) this.warmupController = undefined + } + } + get modelId(): string { return this.options.modelId } @@ -615,6 +730,7 @@ export class Agent { this.todos = normalizeTodoList(target.todos) this.firstMessageSent = this.messages.length > 0 if (this.messages.length === 0) this.contextTokens = 0 + this.measuredMessages = Math.min(this.measuredMessages, this.messages.length) this.stopHookActive = false this.autoCompactFailed = false this.prunedBefore = Math.min(this.prunedBefore, this.messages.length) @@ -730,6 +846,7 @@ export class Agent { this.stopHookActive = false this.prunedBefore = 0 this.autoCompactFailed = false + this.measuredMessages = 0 this.repeatTracker.clear() this.lastGitHead = getGitHead(this.options.cwd) persistGitHead(this.options.cwd, this.lastGitHead) @@ -790,12 +907,19 @@ export class Agent { abort(): void { this.abortController?.abort() + this.warmupController?.abort() } get isIdle(): boolean { return this.abortController === undefined } + /** False for an agent created ahead of its first message (to warm the + * prompt cache) that hasn't run a turn yet — there is no session to end. */ + get hasSession(): boolean { + return this.sessionStarted || this.messages.length > 0 + } + private buildEnvironmentDetails(): string { const files = walkFiles(this.options.cwd, true, 200) const git = getGitSummary(this.options.cwd) @@ -1048,9 +1172,26 @@ User time zone: ${timeZone}, UTC${timeZoneOffsetStr}` try { this.stopHookActive = false + // A failed auto-compaction only pauses retries for the rest of its turn. + this.autoCompactFailed = false + let recoveredOverflow = false for (let step = 0; step < MAX_STEPS_PER_TURN; step++) { await this.autoCompactIfNeeded() - const done = await this.runStep() + let done: boolean + try { + done = await this.runStep() + } catch (error) { + // The provider says the history no longer fits (e.g. a resumed + // session on a smaller-window model, or tool output the last + // usage report didn't cover): compact and retry the step once. + if (recoveredOverflow || this.abortController.signal.aborted || !isContextOverflowError(error)) throw error + recoveredOverflow = true + const compacted = await this.autoCompact( + "The conversation no longer fits the model's context window — compacting and retrying…", + ) + if (!compacted) throw error + done = await this.runStep() + } if (!done) continue // The model is ready to stop; Stop hooks may force it to continue. if (await this.shouldContinueAfterStop()) continue @@ -1197,7 +1338,7 @@ User time zone: ${timeZone}, UTC${timeZoneOffsetStr}` this.abortController = new AbortController() const signal = this.abortController.signal try { - const summary = await this.summarizeHistory(signal, startContext, true) + const summary = await this.summarizeWithinWindow(signal, startContext, true) if (summary) { onEvent({ type: "text-done" }) this.replaceHistoryWithSummary(summary, false) @@ -1218,11 +1359,76 @@ User time zone: ${timeZone}, UTC${timeZoneOffsetStr}` } } + /** + * Summarize, shrinking the request if the provider says it doesn't fit. + * Throws when every budget fails (or on any non-overflow error). + */ + private async summarizeWithinWindow(signal: AbortSignal, startContext: string, showText: boolean): Promise { + const window = getModel(this.options.modelId).contextWindow + let lastError: unknown + for (const fraction of SUMMARY_BUDGET_FRACTIONS) { + try { + return await this.summarizeHistory(signal, startContext, showText, Math.floor(window * fraction)) + } catch (error) { + if (signal.aborted || !isContextOverflowError(error)) throw error + lastError = error + } + } + throw lastError + } + + /** + * History for the summary request. It is the whole conversation, so once + * the history outgrows the window it would fail exactly when compaction is + * needed; past `budgetTokens` it is trimmed: older tool results are + * stubbed, oversized messages cut, and the oldest whole turns dropped + * (keeping the first message — the original request or the last summary). + * Under budget it is the normal outgoing history, which keeps the + * gateway's prompt cache warm. + */ + private summarySource(budgetTokens: number): OpenAI.Chat.ChatCompletionMessageParam[] { + let messages = this.outgoingMessages() + const size = (list: OpenAI.Chat.ChatCompletionMessageParam[]) => + list.reduce((total, message) => total + messageTokenEstimate(message), 0) + if (size(messages) <= budgetTokens) return messages + + const toolIndexes = messages.flatMap((message, index) => (message.role === "tool" ? [index] : [])) + const keepToolsFrom = toolIndexes[toolIndexes.length - KEEP_RECENT_TOOL_RESULTS] ?? Number.POSITIVE_INFINITY + messages = messages.map((message, index) => { + if (message.role !== "tool" || index >= keepToolsFrom) return message + const lines = contentToText(message.content).split("\n").length + return { ...message, content: `[Tool result (${lines} lines) omitted to fit the summary request.]` } + }) + const maxMessageChars = Math.floor((budgetTokens * CHARS_PER_TOKEN) / 4) + messages = messages.map((message) => truncateMessage(message, maxMessageChars)) + if (size(messages) <= budgetTokens || messages.length < 3) return messages + + // Drop whole turns from the front so no tool result loses its call. + const [first, ...rest] = messages + let start = 0 + while (size([first, ...rest.slice(start)]) > budgetTokens) { + const next = rest.findIndex((message, index) => index > start && message.role === "user") + if (next === -1) break + start = next + } + if (start === 0) return messages + return [ + first, + { role: "user", content: `[${start} earlier messages omitted to fit the context window.]` }, + ...rest.slice(start), + ] + } + /** Ask the model to summarize the conversation so far. `showText` streams the summary to the UI. */ - private async summarizeHistory(signal: AbortSignal, startContext: string, showText: boolean): Promise { + private async summarizeHistory( + signal: AbortSignal, + startContext: string, + showText: boolean, + budgetTokens: number, + ): Promise { const { onEvent } = this.options.callbacks const request: OpenAI.Chat.ChatCompletionMessageParam[] = [ - ...this.outgoingMessages(), + ...this.summarySource(budgetTokens), { role: "user", content: @@ -1260,8 +1466,14 @@ User time zone: ${timeZone}, UTC${timeZoneOffsetStr}` return summary } - /** Swap the whole history for a single summary message. */ + /** + * Swap the whole history for a single summary message. A user message the + * model hasn't answered yet (compaction right as a turn starts) is kept + * verbatim after it, so the new request isn't lost in the summary. + */ private replaceHistoryWithSummary(summary: string, continueWork: boolean): void { + const last = this.messages[this.messages.length - 1] + const pending = continueWork && last?.role === "user" ? last : undefined this.messages = [ { role: "user", @@ -1271,9 +1483,12 @@ User time zone: ${timeZone}, UTC${timeZoneOffsetStr}` ? "\n\nContinue the work described above from where it left off. Do not restart or repeat steps that are already done." : ""), }, + ...(pending ? [pending] : []), ] // The summary is small; the next usage report replaces this estimate. - this.contextTokens = Math.ceil(summary.length / 4) + this.contextTokens = this.messages.reduce((total, message) => total + messageTokenEstimate(message), 0) + this.measuredMessages = this.messages.length + this.autoCompactFailed = false // Checkpoints index into the history that was just replaced. deleteBackups(this.taskId, this.checkpoints) this.checkpoints = [] @@ -1289,19 +1504,35 @@ User time zone: ${timeZone}, UTC${timeZoneOffsetStr}` private async autoCompactIfNeeded(): Promise { if (this.autoCompactFailed || this.messages.length < 4) return const window = getModel(this.options.modelId).contextWindow - if (this.contextTokens < window * AUTO_COMPACT_FRACTION) return + const estimated = this.estimatedContextTokens() + if (estimated < window * AUTO_COMPACT_FRACTION) return + const percent = Math.round((estimated / window) * 100) + await this.autoCompact(`Context is ${percent}% full — compacting the conversation…`) + } + + /** Context the next request will carry: the last measured size plus an + * estimate for everything added since (tool results, new input). */ + private estimatedContextTokens(): number { + let unmeasured = 0 + for (const message of this.messages.slice(this.measuredMessages)) unmeasured += messageTokenEstimate(message) + return this.contextTokens + unmeasured + } + + /** Compact mid-turn and continue. Returns false (and pauses auto-compaction + * for the rest of the turn) when it fails. */ + private async autoCompact(message: string): Promise { const { onEvent } = this.options.callbacks const signal = this.abortController!.signal - const percent = Math.round((this.contextTokens / window) * 100) - onEvent({ type: "system", message: `Context is ${percent}% full — compacting the conversation…`, isError: false }) + onEvent({ type: "system", message, isError: false }) if (this.hooks.hasHooks("PreCompact")) { await this.hooks.run("PreCompact", { trigger: "auto", custom_instructions: "" }) } try { - const summary = await this.summarizeHistory(signal, "", false) + const summary = await this.summarizeWithinWindow(signal, "", false) if (!summary) throw new Error("no summary produced") this.replaceHistoryWithSummary(summary, true) onEvent({ type: "system", message: "Conversation compacted; continuing.", isError: false }) + return true } catch (error) { if ((error as Error).name === "AbortError" || signal.aborted) throw error this.autoCompactFailed = true @@ -1310,6 +1541,7 @@ User time zone: ${timeZone}, UTC${timeZoneOffsetStr}` message: `Auto-compaction failed (${sanitizeErrorMessage(error)}); continuing with the full history.`, isError: true, }) + return false } } @@ -1381,6 +1613,7 @@ User time zone: ${timeZone}, UTC${timeZoneOffsetStr}` } const toolCallsByIndex = new Map() let nextSyntheticIndex = 10000 + let usageReported = false // Roll back this step's partial output so streamWithRetry can restart a // dropped stream mid-flight. Tools only run after the stream completes, so @@ -1443,6 +1676,7 @@ User time zone: ${timeZone}, UTC${timeZoneOffsetStr}` case "usage": this.totalCost += chunk.totalCost ?? 0 this.contextTokens = (chunk.inputTokens ?? 0) + (chunk.outputTokens ?? 0) + usageReported = true onEvent({ type: "usage", inputTokens: chunk.inputTokens, @@ -1479,6 +1713,8 @@ User time zone: ${timeZone}, UTC${timeZoneOffsetStr}` ;(assistantMessage as unknown as Record)[REASONING_DETAILS_FIELD] = reasoningDetails } this.messages.push(assistantMessage) + // The report's output tokens already cover this assistant message. + if (usageReported) this.measuredMessages = this.messages.length if (toolCalls.length === 0) { return true diff --git a/src/core/resumeCost.ts b/src/core/resumeCost.ts new file mode 100644 index 0000000..506c14d --- /dev/null +++ b/src/core/resumeCost.ts @@ -0,0 +1,111 @@ +import type { AxonModel } from "../api/models.js" +import type { UsageEstimate } from "../auth/auth.js" +import type { SessionData } from "./sessions.js" + +/** + * How long the gateway keeps a conversation's prompt prefix cached. OrbCode + * doesn't send `x-axon-extended-cache`, so the backend's 5-minute default + * applies; Anthropic's prompt cache (direct `provider: "anthropic"`) also + * defaults to 5 minutes. Past this, the next turn re-reads the whole context + * at the uncached input price. + */ +export const PROMPT_CACHE_TTL_MS = 5 * 60_000 + +/** Ask before a cold resume that costs at least this share of the plan window… */ +export const RESUME_CONFIRM_MIN_PERCENT = 1 + +/** …and before any cold resume this large, whatever it costs (slow and wasteful to re-read). */ +export const RESUME_CONFIRM_MIN_TOKENS = 100_000 + +export interface ResumeCostEstimate { + idleMs: number + contextTokens: number + /** Uncached input cost of the first resumed turn in USD, at the model's catalog price. */ + cost: number + /** Whether that turn draws from the plan (vs. the user's own provider key). */ + billsPlan: boolean + /** Plan-window share of that turn, from the backend's estimate. */ + percent?: number + window?: "weekly" | "monthly" +} + +/** Models served through the MatterAI gateway draw from the plan; BYO-key providers don't. */ +export function billsPlan(model: AxonModel): boolean { + return !model.provider || model.provider === "matterai" || model.provider === "axon" +} + +/** + * Estimate what the first turn of a resumed session will cost once its prompt + * cache has expired. `share` is the backend's plan-window estimate for that + * many uncached tokens (the profile API exposes percentages, not credit + * limits, so the share can't be derived locally). Returns null when the cache + * is still warm or the session holds no context. + */ +export function estimateResumeCost( + session: Pick, + model: AxonModel, + share?: UsageEstimate, + plan?: string, + now = Date.now(), +): ResumeCostEstimate | null { + const idleMs = now - Date.parse(session.updatedAt) + const contextTokens = session.contextTokens ?? 0 + if (!Number.isFinite(idleMs) || idleMs < PROMPT_CACHE_TTL_MS || contextTokens <= 0) return null + const onPlan = billsPlan(model) + const cost = contextTokens * model.inputPrice + const estimate: ResumeCostEstimate = { idleMs, contextTokens, cost, billsPlan: onPlan } + if (onPlan && share) { + // Lite plans have no separate weekly allowance (the backend mirrors the + // monthly limit into the weekly window), so call it monthly there. + estimate.percent = share.weeklyPercentage + estimate.window = plan === "lite" ? "monthly" : "weekly" + } + return estimate +} + +/** + * Whether to confirm a cold resume: always for a large context, otherwise + * only when the plan share (known once the backend answers) is significant. + */ +export function shouldConfirmResume(estimate: ResumeCostEstimate | null): estimate is ResumeCostEstimate { + if (!estimate) return false + return ( + estimate.contextTokens >= RESUME_CONFIRM_MIN_TOKENS || + (estimate.percent ?? 0) >= RESUME_CONFIRM_MIN_PERCENT + ) +} + +/** `1d 1h 17m`, `3h 5m`, `42m`. */ +export function formatIdle(ms: number): string { + const totalMinutes = Math.max(0, Math.floor(ms / 60_000)) + const days = Math.floor(totalMinutes / 1440) + const hours = Math.floor((totalMinutes % 1440) / 60) + const minutes = totalMinutes % 60 + const parts: string[] = [] + if (days) parts.push(`${days}d`) + if (days || hours) parts.push(`${hours}h`) + parts.push(`${minutes}m`) + return parts.join(" ") +} + +/** `785k`, `1.2M`, `950`. */ +export function formatTokens(tokens: number): string { + if (tokens >= 1_000_000) return `${+(tokens / 1_000_000).toFixed(1)}M` + if (tokens >= 1_000) return `${Math.round(tokens / 1_000)}k` + return String(tokens) +} + +/** The sentence after "Resuming it will…", e.g. "use about 22% of your weekly usage limit". */ +export function describeResumeCost(estimate: ResumeCostEstimate, estimating = false): string { + if (estimate.percent !== undefined) { + const pct = estimate.percent < 1 ? "<1" : String(Math.round(estimate.percent)) + return `use about ${pct}% of your ${estimate.window} usage limit` + } + // Your own provider key: there's no plan window, so show what it'll bill. + if (!estimate.billsPlan && estimate.cost > 0) { + return `cost about $${estimate.cost < 0.01 ? "0.01" : estimate.cost.toFixed(2)}` + } + return estimating + ? "re-read the full context without the prompt cache (checking your usage…)" + : "re-read the full context without the prompt cache" +} diff --git a/src/index.tsx b/src/index.tsx index 87deb23..b4eab87 100644 --- a/src/index.tsx +++ b/src/index.tsx @@ -5,6 +5,7 @@ import { runPluginCommand } from "./commands/plugin.js" import { runUsageCommand } from "./commands/usage.js" import { loadSessionById, type SessionData } from "./core/sessions.js" import { loadSettings } from "./config/settings.js" +import { loadCachedModelCatalog } from "./api/models.js" import { clearUpdateCache, compareVersions, @@ -14,6 +15,7 @@ import { isGlobalInstall, runNpmUpdate, } from "./utils/updateCheck.js" +import { markRunningVersionHealthy, resolveUpdateNotice } from "./utils/autoUpdate.js" const PACKAGE_NAME = "@matterailab/orbcode" @@ -89,6 +91,26 @@ async function runUpdate(force: boolean): Promise { return code } +/** + * Hidden `--self-test`: load the full interactive module graph under the + * bundled Bun runtime without opening a terminal UI. The background updater + * runs this against a freshly staged version before switching to it. + */ +async function runSelfTest(): Promise { + if (!process.versions.bun) { + console.error("--self-test must run under the bundled Bun runtime") + return 1 + } + await Promise.all([ + import("@opentui/core"), + import("@opentui/react"), + import("./ui/App.js"), + import("./ui/theme.js"), + ]) + console.log(`ok ${VERSION}`) + return 0 +} + /** Pop `flag ` (accepting -flag and --flag) out of args; returns the value. */ function takeFlagValue(args: string[], name: string): string | undefined { const index = args.findIndex((a) => a === `--${name}` || a === `-${name}`) @@ -134,6 +156,13 @@ async function main(): Promise { // name", VSCode terminal status, etc.) don't append " (node)" next to our // own title. The bundled bin/orbcode.js also does this for the npm case. process.title = "orbcode" + if (process.argv.includes("--self-test")) { + process.exit(await runSelfTest()) + } + // Before anything reads settings: a saved catalog-only model (and its + // effort) must resolve before /v1/models answers, or it falls back to the + // default model for the whole session. + loadCachedModelCatalog() const args = process.argv.slice(2) if (args.includes("--version") || args.includes("-v")) { @@ -144,6 +173,15 @@ async function main(): Promise { printHelp() return } + + // A staged version proves itself healthy once its module graph has loaded + // (management/headless commands) or its renderer is up (the TUI, below). + const launchesTui = + !["update", "mcp", "plugin", "plugins", "usage"].includes(args[0] ?? "") && + !args.includes("-p") && + !args.includes("--print") + if (!launchesTui) markRunningVersionHealthy() + if (args[0] === "update") { const force = args.includes("--force") || args.includes("-f") const code = await runUpdate(force) @@ -236,8 +274,12 @@ async function main(): Promise { const initialPrompt = initialView ? undefined : args.find((a) => !a.startsWith("-")) // Fire-and-forget: a stale or no-network state just means the header shows - // the "current version" line instead of an upgrade prompt. - const updateCheckPromise = getUpdateInfo(PACKAGE_NAME, VERSION) + // the "current version" line instead of an upgrade prompt. When background + // updates are on, a newer release is staged here and the banner only asks + // for a restart. + const updateCheckPromise = getUpdateInfo(PACKAGE_NAME, VERSION).then((info) => + resolveUpdateNotice(PACKAGE_NAME, info, loadSettings()), + ) const bunVersion = (process.versions as Record).bun const [bunMajor = 0, bunMinor = 0] = (bunVersion ?? "").split(".").map(Number) @@ -280,6 +322,7 @@ async function main(): Promise { onDestroy: markDestroyed, }) renderer.setTerminalTitle("orbcode") + markRunningVersionHealthy() const root = createRoot(renderer) root.render( createElement( diff --git a/src/mcp/manager.ts b/src/mcp/manager.ts index 2a0d26b..847745d 100644 --- a/src/mcp/manager.ts +++ b/src/mcp/manager.ts @@ -26,6 +26,8 @@ export class McpManager { /** project-scope server names that require user approval before connecting. */ private pendingApproval: Set = new Set() private started = false + /** the initial connection pass from start(), for whenStarted() */ + private startup: Promise = Promise.resolve() constructor(cwd: string, disabled: string[], enabled: string[]) { this.cwd = cwd @@ -37,6 +39,18 @@ export class McpManager { async start(): Promise { if (this.started) return this.snapshot() this.started = true + const startup = this.connectAll() + this.startup = startup + return startup + } + + /** Settles once the connection pass begun by start() has finished, so the + * tool list is final (immediately when start() was never called). */ + async whenStarted(): Promise { + await this.startup.catch(() => {}) + } + + private async connectAll(): Promise { const { servers } = loadMcpConfig(this.cwd) this.configs.clear() this.states.clear() diff --git a/src/tools/executors/files.ts b/src/tools/executors/files.ts index a73f24f..77c7081 100644 --- a/src/tools/executors/files.ts +++ b/src/tools/executors/files.ts @@ -6,6 +6,10 @@ import { unifiedDiff } from "../../utils/diff.js" const MAX_READ_LINES = 1000 const MIN_READ_LINES = 200 +/** Line limits don't bound minified or generated files (one line can be megabytes). */ +const MAX_READ_CHARS_PER_FILE = 100_000 +const MAX_READ_CHARS_PER_CALL = 200_000 +const MIN_READ_BUDGET_CHARS = 1_000 /** Format file content as right-aligned `LINE_NUMBER|LINE_CONTENT` (6-char padding). */ function withLineNumbers(lines: string[], startLine: number): string { @@ -62,6 +66,7 @@ export async function readFile(args: Record, context: ToolConte } const results: { text: string; isError?: boolean }[] = [] + let charsReturned = 0 for (const entry of entries) { const relPath = entry.file_path @@ -92,19 +97,44 @@ export async function readFile(args: Record, context: ToolConte continue } - const formattedContent = withLineNumbers(slice, offset) - const lastLineShown = offset - 1 + slice.length + const budget = Math.min(MAX_READ_CHARS_PER_FILE, MAX_READ_CHARS_PER_CALL - charsReturned) + // A sliver of leftover budget would return a useless fragment. + if (budget < MIN_READ_BUDGET_CHARS) { + results.push({ + text: `Skipped ${relPath}: this call reached its ${MAX_READ_CHARS_PER_CALL}-character limit. Read it in a separate call.`, + isError: true, + }) + continue + } + let formattedContent = withLineNumbers(slice, offset) + let lastLineShown = offset - 1 + slice.length + let capNote = "" + if (formattedContent.length > budget) { + // Whole lines when possible; a single line longer than the budget is cut mid-line. + const lineEnd = formattedContent.lastIndexOf("\n", budget) + formattedContent = lineEnd > 0 ? formattedContent.slice(0, lineEnd) : formattedContent.slice(0, budget) + lastLineShown = offset - 1 + formattedContent.split("\n").length + capNote = + lineEnd > 0 + ? `Output capped at ${budget} characters; showing through line ${lastLineShown}. Use offset/limit to read more.` + : `Line ${lastLineShown} alone exceeds ${budget} characters and was cut off. Use Bash (e.g. head -c, cut -c) to read part of it.` + } + charsReturned += formattedContent.length if (entries.length === 1) { let output = formattedContent - if (lastLineShown < totalLines) { + if (capNote) { + output += `\n\n(${capNote})` + } else if (lastLineShown < totalLines) { output += `\n\n(Showing lines ${offset}-${lastLineShown} of ${totalLines}. Use offset/limit to read more.)` } results.push({ text: output }) } else { const rangeLabel = totalLines > 0 ? ` (lines ${offset}-${lastLineShown} of ${totalLines})` : "" let output = `--- ${relPath}${rangeLabel} ---\n${formattedContent}` - if (lastLineShown < totalLines) { + if (capNote) { + output += `\n(${capNote})` + } else if (lastLineShown < totalLines) { output += `\n(Showing lines ${offset}-${lastLineShown} of ${totalLines}. Use offset/limit to read more.)` } results.push({ text: output }) diff --git a/src/ui/App.tsx b/src/ui/App.tsx index d79b195..9dac682 100644 --- a/src/ui/App.tsx +++ b/src/ui/App.tsx @@ -37,7 +37,12 @@ import { fetchDynamicModels, get232kAxonFallback, getDefaultModelId, + getGatewayModelId, getModel, + getModelEffortLevels, + isGatewayEffort, + type GatewayEffort, + type AxonModel, is400kAxonModel, isEidoBaseAxonModel, isEidoProAxonModel, @@ -49,11 +54,17 @@ import { APP_URL, fetchProfile, fetchTaskTitle, + fetchUsageEstimate, + type UsageEstimate, resetWeeklyUsage, type ProfileData, } from "../auth/auth.js"; import { getAuthToken, + getModelEffort, + loadModelEfforts, + setSessionModelEffort, + withSessionModelEfforts, getPendingProjectHooks, loadSettings, saveMcpApproval, @@ -61,9 +72,9 @@ import { trustProjectHooks, type OrbCodeSettings, } from "../config/settings.js"; -import { Agent } from "../core/agent.js"; +import { Agent, CACHE_WARMUP_ENABLED } from "../core/agent.js"; import { McpManager } from "../mcp/manager.js"; -import type { UpdateInfo } from "../utils/updateCheck.js"; +import type { UpdateNotice } from "../utils/autoUpdate.js"; import { normalizeTodoList } from "../utils/todos.js"; import type { AgentEvent, @@ -91,10 +102,18 @@ import { import { StatusBar, type ApprovalMode } from "./components/StatusBar.js"; import { ModelPicker } from "./components/ModelPicker.js"; import { ThemePicker } from "./components/ThemePicker.js"; +import { EffortPicker } from "./components/EffortPicker.js"; import { SessionPicker } from "./components/SessionPicker.js"; +import { ResumeConfirm, type ResumeDecision } from "./components/ResumeConfirm.js"; import { RewindPicker } from "./components/RewindPicker.js"; import type { RewindMode, RewindPoint } from "../core/checkpoints.js"; import { listSessions, type SessionData } from "../core/sessions.js"; +import { + billsPlan, + estimateResumeCost, + shouldConfirmResume, + type ResumeCostEstimate, +} from "../core/resumeCost.js"; import { diffViewHeight, formatToolName, @@ -124,6 +143,10 @@ const SLASH_COMMANDS: SlashCommand[] = [ { name: "/help", description: "show available commands" }, { name: "/attach", description: "choose files to attach" }, { name: "/model", description: "select the Axon model to use" }, + { + name: "/effort", + description: "set the model's effort: low, medium, high or max", + }, { name: "/theme", description: "choose the OrbCode dark or light theme" }, { name: "/clear", @@ -377,6 +400,20 @@ interface PendingFollowup { } let rowCounter = 0; +/** Header model line, e.g. "Opus 5.5 with high effort". */ +function headerModelLabel(modelId: string): string { + const model = getModel(modelId); + const effort = getModelEffort(loadModelEfforts(), model); + return effort ? `${model.name} with ${effort} effort` : model.name; +} + +/** Rows with the intro header's model line refreshed for `modelId`. */ +function withHeaderModel(rows: Row[], modelId: string): Row[] { + return rows.map((row) => + row.kind === "header" ? { ...row, modelName: headerModelLabel(modelId) } : row, + ); +} + function rowId(): string { return `row-${rowCounter++}`; } @@ -400,7 +437,7 @@ export function App({ /** Optional override replacing the default system prompt (from `-s`). */ systemPromptOverride?: string; /** Promise resolving to the latest-npm-version comparison; resolved after first paint. */ - updateCheck?: Promise; + updateCheck?: Promise; }) { const { exit } = useApp(); const renderer = useRenderer(); @@ -435,14 +472,14 @@ export function App({ const [view, setView] = useState<"login" | "chat">( initialView ?? (getAuthToken(settings) ? "chat" : "login"), ); - const [updateInfo, setUpdateInfo] = useState(null); + const [updateInfo, setUpdateInfo] = useState(null); const [rows, setRows] = useState(() => [ { kind: "header", id: "header", cwd: process.cwd(), - modelName: getModel(loadSettings().model).name, + modelName: headerModelLabel(loadSettings().model), }, ]); const [busy, setBusy] = useState(false); @@ -471,6 +508,7 @@ export function App({ const [queuedMessages, setQueuedMessages] = useState([]); const [modelPickerOpen, setModelPickerOpen] = useState(false); const [themePickerOpen, setThemePickerOpen] = useState(false); + const [effortPickerOpen, setEffortPickerOpen] = useState(false); // Set while the /rewind picker is open: the user turns that can be rewound to. const [rewindPoints, setRewindPoints] = useState(null); // Puts a rewound message back in the prompt so it can be edited and resent. @@ -482,6 +520,13 @@ export function App({ here: SessionData[]; everywhere: SessionData[]; } | null>(null); + // A cold-cache resume awaiting confirmation (see ResumeConfirm). + const [pendingResume, setPendingResume] = useState<{ + session: SessionData; + estimate: ResumeCostEstimate; + /** Plan share still loading; the prompt fills it in when it arrives. */ + estimating?: boolean; + } | null>(null); const [taskPickerSessions, setTaskPickerSessions] = useState< SessionData[] | null >(null); @@ -673,7 +718,7 @@ export function App({ kind: "header", id: rowId(), cwd: process.cwd(), - modelName: getModel(loadSettings().model).name, + modelName: headerModelLabel(loadSettings().model), }, ]); }, []); @@ -819,7 +864,12 @@ export function App({ current.enabledMcpServers ?? [], ); } - return new Agent({ + // Set once constructed. Events from an agent that has since been + // replaced (/new, /resume, …) are dropped so a turn still unwinding + // can't stream into the next conversation's transcript. + let agent: Agent | null = null; + const isStale = () => agent !== null && agentRef.current !== agent; + agent = new Agent({ cwd: process.cwd(), token: getAuthToken(current)!, modelId: current.model, @@ -834,29 +884,68 @@ export function App({ // mid-session still applies it to the new agent. systemPromptOverride: systemPromptOverrideRef.current, callbacks: { - onEvent: handleEvent, + onEvent: (event) => { + if (!isStale()) handleEvent(event); + }, requestApproval: (request) => - new Promise((resolve) => - setPendingApproval({ request, resolve }), - ), + isStale() + ? Promise.resolve("no") + : new Promise((resolve) => + setPendingApproval({ request, resolve }), + ), requestFollowup: (question, suggestions) => - new Promise((resolve) => - setPendingFollowup({ question, suggestions, resolve }), - ), + isStale() + ? Promise.resolve("") + : new Promise((resolve) => + setPendingFollowup({ question, suggestions, resolve }), + ), }, }); + return agent; }, [handleEvent], ); const getAgent = useCallback((): Agent => { - if (!agentRef.current) { - agentRef.current = createAgent(); - process.env.ORBCODE_LAST_SESSION_ID = agentRef.current.taskId; - } + if (!agentRef.current) agentRef.current = createAgent(); + // Set on use rather than creation: an agent from prepareFreshAgent has + // no saved session until its first message. + process.env.ORBCODE_LAST_SESSION_ID = agentRef.current.taskId; return agentRef.current; }, [createAgent]); + // Create the next conversation's agent now instead of on its first message, + // so its taskId already exists and the gateway prompt cache can be warmed + // under it. The first message then reuses this agent (and taskId) and its + // request hits the cached system prompt + tools. + const prepareFreshAgent = useCallback(() => { + // Never replace an agent a message has already started on. + if ( + !CACHE_WARMUP_ENABLED || + agentRef.current || + !getAuthToken(loadSettings()) + ) + return; + const agent = createAgent(); + agentRef.current = agent; + void agent.warmCache(); + }, [createAgent]); + + // Retire the active agent so the next message starts a fresh one. Any + // in-flight turn is aborted; because the agent is detached first, the + // events it emits while unwinding ("Interrupted.", turn-end) are dropped + // by `createAgent`'s stale check, so the streaming UI is reset here. + const discardAgent = useCallback(() => { + const agent = agentRef.current; + agentRef.current = null; + agent?.abort(); + textBufferRef.current = ""; + reasoningBufferRef.current = ""; + setStreamingText(""); + setStreamingReasoning(""); + setBusy(false); + }, []); + // Force-send: jump a queued message to the front of the queue, then skip // the wait for the in-flight turn. Aborting makes the agent's `finally` // emit `turn-end`, whose handler drains the queue and starts the next @@ -901,9 +990,10 @@ export function App({ [busy, drainQueue, getAgent, pushRow], ); - const handleResume = useCallback( + const resumeSession = useCallback( (session: SessionData) => { setResumableSessions(null); + setPendingResume(null); // A session from another directory continues in that directory, so its // tools, AGENTS.md, project settings, hooks and MCP servers match the // conversation. @@ -938,6 +1028,8 @@ export function App({ const pendingHooks = getPendingProjectHooks(process.cwd()); if (pendingHooks) setPendingHookTrust(pendingHooks); } + clearQueue(); + discardAgent(); const resumedAgent = createAgent(session); agentRef.current = resumedAgent; process.env.ORBCODE_LAST_SESSION_ID = resumedAgent.taskId; @@ -972,7 +1064,97 @@ export function App({ text: `Resumed session: ${session.title || session.id}${switchedDir ? ` · working directory is now ${session.cwd}` : ""}`, }); }, - [createAgent, pushRow, resetTranscript], + [createAgent, clearQueue, discardAgent, pushRow, resetTranscript], + ); + + // Resume, but first confirm when the session's prompt cache has expired and + // re-reading its context would take a noticeable bite out of the plan. + // Plan-share estimates by model + context size. The backend knows the plan + // limits (the profile API only exposes percentages), so each answer costs a + // round trip; /resume prefetches them while the picker is open. + const shareCacheRef = useRef( + new Map>(), + ); + const fetchResumeShare = useCallback( + (model: AxonModel, tokens: number): Promise => { + const token = getAuthToken(loadSettings()); + if (!token || !billsPlan(model)) return Promise.resolve(undefined); + const key = `${model.id}:${tokens}`; + let pending = shareCacheRef.current.get(key); + if (!pending) { + pending = fetchUsageEstimate(token, getGatewayModelId(model), tokens).catch( + () => { + // Let a later resume retry instead of caching the failure. + shareCacheRef.current.delete(key); + return undefined; + }, + ); + shareCacheRef.current.set(key, pending); + } + return pending; + }, + [], + ); + + // Start estimates for the cold sessions a /resume picker lists. + const prefetchResumeShares = useCallback( + (sessions: SessionData[]) => { + const model = getModel(loadSettings().model); + for (const session of sessions.slice(0, 10)) { + const cold = estimateResumeCost(session, model); + if (cold) void fetchResumeShare(model, cold.contextTokens); + } + }, + [fetchResumeShare], + ); + + const handleResume = useCallback( + async (session: SessionData) => { + const model = getModel(loadSettings().model); + const cold = estimateResumeCost(session, model); + if (!cold) { + resumeSession(session); + return; + } + setResumableSessions(null); + const plan = usage?.plan ?? usage?.tieredUsage?.plan; + const sharePromise = fetchResumeShare(model, cold.contextTokens); + if (shouldConfirmResume(cold)) { + // Large enough to ask regardless of cost: show the prompt now and + // fill in the plan share when it arrives. + setPendingResume({ + session, + estimate: cold, + estimating: !!getAuthToken(loadSettings()) && billsPlan(model), + }); + const share = await sharePromise; + setPendingResume((pending) => + pending?.session.id === session.id + ? { + session, + estimate: + estimateResumeCost(session, model, share, plan) ?? + pending.estimate, + } + : pending, + ); + return; + } + // Smaller contexts only ask when the share is significant. The answer + // is usually already in from the picker's prefetch. + const estimate = estimateResumeCost( + session, + model, + await sharePromise, + plan, + ); + if (shouldConfirmResume(estimate)) { + setPendingResume({ session, estimate }); + return; + } + resumeSession(session); + }, + [fetchResumeShare, resumeSession, usage], ); // `quiet`: opened by double-Esc, which also interrupts a running turn — stay @@ -1070,8 +1252,15 @@ export function App({ ); const switchModel = useCallback( - /** `auto`: an automatic switch (plan default / plan fallback), not a user pick. */ - (modelId: string, options?: { silent?: boolean; auto?: boolean }) => { + /** + * `auto`: an automatic switch (plan default / plan fallback), not a user pick. + * `effort`: saved for this model in config.json, so every chat on the + * machine uses it until it's changed again. + */ + ( + modelId: string, + options?: { silent?: boolean; auto?: boolean; effort?: GatewayEffort }, + ) => { if (isLumenAxonModel(modelId) && !hasLumenAccess) { pushRow({ kind: "error", @@ -1100,25 +1289,35 @@ export function App({ }); return; } - const updated = { ...loadSettings(), model: modelId, modelExplicit: !options?.auto }; + const previous = loadSettings(); + const updated = { + ...previous, + model: modelId, + modelExplicit: !options?.auto, + modelEfforts: options?.effort + ? { ...previous.modelEfforts, [modelId]: options.effort } + : previous.modelEfforts, + }; + // A saved pick replaces any "this session only" override for the model. + if (options?.effort) setSessionModelEffort(modelId, undefined); setSettings(updated); saveSettings(updated); + const sameModel = previous.model === modelId; agentRef.current?.setModel(modelId); - setRows((prev) => { - const headerIndex = prev.findIndex((row) => row.kind === "header"); - if (headerIndex === -1) return prev; - const updatedHeader = { - ...prev[headerIndex]!, - modelName: getModel(modelId).name, - }; - const next = [...prev]; - next[headerIndex] = updatedHeader; - return next; - }); + // Re-prime for the new model if the conversation hasn't started yet + // (warmCache is a no-op once it has). + if (CACHE_WARMUP_ENABLED && !sameModel) void agentRef.current?.warmCache(); + setRows((prev) => withHeaderModel(prev, modelId)); if (!options?.silent) { + const model = getModel(modelId); + const effort = getModelEffort(updated, model); + const effortText = effort ? `${effort[0].toUpperCase()}${effort.slice(1)} effort` : ""; pushRow({ kind: "info", - text: `Model switched to ${getModel(modelId).name}`, + text: + sameModel && effort + ? `${model.name} · ${effortText}` + : `Model switched to ${model.name}${effort ? ` · ${effortText}` : ""}`, }); } }, @@ -1131,10 +1330,26 @@ export function App({ // an explicit user pick is never overwritten. useEffect(() => { if (!catalogReady || !planLoaded) return; - if (settings.modelExplicit || settings.model !== DEFAULT_MODEL_ID) return; + // Decide from disk, not state: before the catalog loaded, a saved + // catalog-only model reads as unknown and the state falls back to the + // default, which must not be mistaken for "never chose a model". + const saved = loadSettings(); + if (saved.modelExplicit || saved.model !== DEFAULT_MODEL_ID) return; const preferred = getDefaultModelId(activePlan); - if (preferred !== settings.model) switchModel(preferred, { silent: true, auto: true }); - }, [activePlan, catalogReady, planLoaded, settings.model, settings.modelExplicit, switchModel]); + if (preferred !== saved.model) switchModel(preferred, { silent: true, auto: true }); + }, [activePlan, catalogReady, planLoaded, switchModel]); + + // Once the live catalog is in, restore the saved model if startup had to + // fall back (no models-cache.json yet, e.g. the first run of this version). + useEffect(() => { + if (!catalogReady) return; + const saved = loadSettings(); + if (saved.model !== settings.model) { + switchModel(saved.model, { silent: true, auto: !saved.modelExplicit }); + } + // Only on catalog load; later model changes go through switchModel. + // eslint-disable-next-line react-hooks/exhaustive-deps + }, [catalogReady]); useEffect(() => { if (!activePlan) { @@ -1183,8 +1398,10 @@ export function App({ const endAndExit = useCallback( (reason: string) => { const agent = agentRef.current; - if (!agent) { - // No agent yet, but the MCP manager may have started; tear it down. + if (!agent || !agent.hasSession) { + // No conversation yet (at most a prepared agent warming the cache), + // but the MCP manager may have started; tear it down. + agent?.abort(); void mcpManagerRef.current?.stop().catch(() => {}); exit(); return; @@ -1223,6 +1440,31 @@ export function App({ ).join("\n"), }); break; + case "/effort": { + const model = getModel(settings.model); + const levels = getModelEffortLevels(model); + if (levels.length === 0) { + pushRow({ + kind: "info", + text: `${model.name} doesn't have an effort setting.`, + }); + break; + } + const requested = arg.trim().toLowerCase(); + if (!requested) { + setEffortPickerOpen(true); + break; + } + if (!isGatewayEffort(requested) || !levels.includes(requested)) { + pushRow({ + kind: "error", + text: `Unknown effort "${arg.trim()}". Use one of: ${levels.join(", ")}.`, + }); + break; + } + switchModel(model.id, { effort: requested }); + break; + } case "/model": { // The interactive picker is restricted to Axon's own models for now. // Third-party providers (Anthropic, OpenAI-compatible) are still @@ -1283,15 +1525,17 @@ export function App({ resetTranscript(); break; case "/new": - // Drop the agent entirely so the next message starts a fresh session. + // Drop the agent entirely; the next message goes to a fresh session + // whose agent is created (and its cache warmed) right away. clearQueue(); - agentRef.current = null; + discardAgent(); titleTaskRef.current = null; setSessionTitle(""); setTerminalTitle("orbcode"); setTasks(""); setContextTokens(0); resetTranscript(); + prepareFreshAgent(); break; case "/analytics": { const url = `${APP_URL}/orbital`; @@ -1308,6 +1552,7 @@ export function App({ break; } setResumableSessions({ here, everywhere }); + prefetchResumeShares(everywhere); break; } case "/rewind": @@ -1517,7 +1762,7 @@ export function App({ const updated = { ...settings, token: undefined }; setSettings(updated); saveSettings(updated); - agentRef.current = null; + discardAgent(); setView("login"); break; } @@ -1545,7 +1790,10 @@ export function App({ switchTheme, resetTranscript, clearQueue, + discardAgent, + prepareFreshAgent, openRewind, + prefetchResumeShares, ], ); @@ -1661,7 +1909,7 @@ export function App({ if (oldManager) { await oldManager.stop().catch(() => {}); } - agentRef.current = null; + discardAgent(); const refreshed = loadSettings(); setSettings(refreshed); if (getAuthToken(refreshed)) { @@ -1676,7 +1924,7 @@ export function App({ if (pendingMcp.length > 0) setPendingMcpApproval(pendingMcp); }); } - }, []); + }, [discardAgent]); // Apply a user-confirmed migration selection: write the chosen entries to // ~/.orbcode/settings.json, then rebuild the MCP manager so the new @@ -1744,12 +1992,38 @@ export function App({ [pushRow, rebuildMcpManager], ); + const handleResumeDecision = useCallback( + (decision: ResumeDecision) => { + const session = pendingResume?.session; + setPendingResume(null); + if (!session) return; + if (decision === "resume") { + resumeSession(session); + return; + } + handleCommand("/new"); + pushRow({ + kind: "info", + text: + session.cwd === process.cwd() + ? `Started a new conversation. Use /task to bring in "${session.title || session.id}" if you need it.` + : `Started a new conversation. "${session.title || session.id}" is still available from /resume.`, + }); + }, + [pendingResume, resumeSession, handleCommand, pushRow], + ); + // Apply --resume and an initial prompt (`orbcode "do something"`) on startup. const bootedRef = useRef(false); useEffect(() => { if (bootedRef.current) return; bootedRef.current = true; - if (initialSession) handleResume(initialSession); + // A prompt given alongside --resume is meant for that session, so + // don't stop to confirm a cold resume first. + if (initialSession) { + if (initialPrompt) resumeSession(initialSession); + else void handleResume(initialSession); + } if (initialAction === "resume") handleCommand("/resume"); // If this project ships untrusted hooks, ask before running anything; the // startup prompt waits until the user decides. @@ -1776,14 +2050,22 @@ export function App({ if (pendingMcp.length > 0) setPendingMcpApproval(pendingMcp); }); } + // A plain launch waits for the user's first message: warm the prompt + // cache meanwhile. (An initial prompt is sent right away, and a resumed + // session's prefix is its whole history, so neither is warmed.) + if (!initialSession && !initialPrompt && initialAction !== "resume") { + prepareFreshAgent(); + } refreshUsage(); }, [ initialSession, initialAction, initialPrompt, + resumeSession, handleResume, handleCommand, handleSubmit, + prepareFreshAgent, refreshUsage, ]); @@ -1906,9 +2188,12 @@ export function App({ const updated = { ...loadSettings(), token }; setSettings(updated); saveSettings(updated); - agentRef.current = null; + discardAgent(); setView("chat"); - fetchDynamicModels(token).catch(() => {}); + // Warm once the catalog is in, so the warmup resolves the right model. + fetchDynamicModels(token) + .catch(() => {}) + .finally(prepareFreshAgent); setUsage({ plan: profile.plan, usagePercentage: profile.usagePercentage, @@ -1920,7 +2205,7 @@ export function App({ text: `Signed in${who ? ` as ${who}` : ""}. Ready when you are.`, }); }, - [pushRow], + [discardAgent, prepareFreshAgent, pushRow], ); const taskLines = useMemo( @@ -1940,9 +2225,11 @@ export function App({ !pendingMcpApproval && !modelPickerOpen && !themePickerOpen && + !effortPickerOpen && !mcpPickerOpen && !mcpMigrationEntries && !resumableSessions && + !pendingResume && !rewindPoints && !taskPickerSessions && !linkManagerOpen && @@ -1952,9 +2239,11 @@ export function App({ !!pendingFollowup || modelPickerOpen || themePickerOpen || + effortPickerOpen || mcpPickerOpen || !!mcpMigrationEntries || !!resumableSessions || + !!pendingResume || !!rewindPoints || !!taskPickerSessions || linkManagerOpen || @@ -2003,7 +2292,11 @@ export function App({ const rowsHeight = rowLayout.height; let dynamicHeight = 0; - if (updateInfo?.updateAvailable && updateInfo.latest) dynamicHeight += 6; + if ( + updateInfo?.stagedVersion || + (updateInfo?.updateAvailable && updateInfo.latest) + ) + dynamicHeight += 6; const reasoningWrapWidth = Math.max(20, wrapWidth - 2); const streamingReasoningDisplay = streamingReasoning ? tailForHeight(streamingReasoning, 3, reasoningWrapWidth) @@ -2203,7 +2496,26 @@ export function App({ {rowBottomSpacerHeight > 0 && ( )} - {updateInfo?.updateAvailable && updateInfo.latest && ( + {updateInfo?.stagedVersion && ( + + + ✓ Update installed: v{updateInfo.stagedVersion} + + + Restart orbcode to apply. + + + )} + {!updateInfo?.stagedVersion && + updateInfo?.updateAvailable && + updateInfo.latest && ( { + modelEfforts={withSessionModelEfforts(settings.modelEfforts)} + onSelect={(modelId, effort) => { setModelPickerOpen(false); - switchModel(modelId); + switchModel(modelId, { effort }); }} onCancel={() => setModelPickerOpen(false)} /> )} + {effortPickerOpen && ( + { + setEffortPickerOpen(false); + if (scope === "saved") { + switchModel(settings.model, { effort }); + return; + } + setSessionModelEffort(settings.model, effort); + // Re-render so the status bar shows the override. + setSettings((current) => ({ ...current })); + setRows((prev) => withHeaderModel(prev, settings.model)); + pushRow({ + kind: "info", + text: `${getModel(settings.model).name} · ${effort[0].toUpperCase()}${effort.slice(1)} effort (this session only)`, + }); + }} + onCancel={() => setEffortPickerOpen(false)} + /> + )} {themePickerOpen && ( setResumableSessions(null)} /> )} + {pendingResume && ( + setPendingResume(null)} + /> + )} {rewindPoints && ( void + onCancel: () => void +} + +const COLUMN_WIDTH = 13 +const MIN_TRACK_WIDTH = 36 + +/** + * Slider geometry: the track spans `width` columns, the first label starts at + * its left end, the last ends at its right end, and the others are centred on + * evenly spaced stops. The ▲ marker sits over the middle of the selected label. + */ +export function effortSliderLayout(levels: readonly string[], selectedIndex: number) { + const width = Math.max(MIN_TRACK_WIDTH, (levels.length - 1) * COLUMN_WIDTH) + const step = levels.length > 1 ? width / (levels.length - 1) : 0 + const starts = levels.map((label, i) => { + if (i === 0) return 0 + if (i === levels.length - 1) return width - label.length + return Math.round(i * step - label.length / 2) + }) + const selectedLabel = levels[selectedIndex] ?? "" + const markerX = (starts[selectedIndex] ?? 0) + Math.floor(selectedLabel.length / 2) + const line = "─".repeat(markerX) + "▲" + "─".repeat(Math.max(0, width - markerX - 1)) + return { width, starts, markerX, line } +} + +export function EffortPicker({ modelName, levels, current, onSelect, onCancel }: EffortPickerProps) { + const [selected, setSelected] = useState(() => Math.max(0, levels.indexOf(current))) + + useInput((input, key) => { + if (key.leftArrow) { + setSelected((index) => Math.max(0, index - 1)) + return + } + if (key.rightArrow) { + setSelected((index) => Math.min(levels.length - 1, index + 1)) + return + } + if (key.return) { + onSelect(levels[selected], "saved") + return + } + if (input === "s") { + onSelect(levels[selected], "session") + return + } + if (key.escape) onCancel() + }) + + const { width, starts, line } = effortSliderLayout(levels, selected) + const ends = ["Faster", "Smarter"] as const + return ( + + + Effort · {modelName} + + + + + {ends[0]} + {" ".repeat(Math.max(1, width - ends[0].length - ends[1].length))} + {ends[1]} + + {line} + + {levels.map((level, i) => { + const previousEnd = i === 0 ? 0 : starts[i - 1] + levels[i - 1].length + const gap = " ".repeat(Math.max(i === 0 ? 0 : 1, starts[i] - previousEnd)) + return ( + + {gap} + {i === selected ? ( + + {level} + + ) : ( + {level} + )} + + ) + })} + + + + + + ←/→ to adjust · Enter to confirm · s for this session only · Esc to cancel + + + + ) +} diff --git a/src/ui/components/ModelPicker.tsx b/src/ui/components/ModelPicker.tsx index c7d8c68..66aafce 100644 --- a/src/ui/components/ModelPicker.tsx +++ b/src/ui/components/ModelPicker.tsx @@ -2,7 +2,16 @@ import React, { useState } from "react" import { Box, Text, useInput } from "../primitives.js" import { COLORS } from "../../branding.js" -import { BUILTIN_AXON_MODELS, isEidoBaseAxonModel, isEidoProAxonModel, isLumenAxonModel, type AxonModel } from "../../api/models.js" +import { + BUILTIN_AXON_MODELS, + getModelEffortLevels, + isEidoBaseAxonModel, + isEidoProAxonModel, + isLumenAxonModel, + type AxonModel, + type GatewayEffort, +} from "../../api/models.js" +import { getModelEffort } from "../../config/settings.js" import { PopoverBox } from "./PopoverBox.js" const VISIBLE_ROWS = 12 @@ -45,10 +54,20 @@ interface ModelPickerProps { canUseEidoBase: boolean canUseEidoPro: boolean canUseLumen: boolean - onSelect: (modelId: string) => void + /** Saved effort per model (config.json `modelEfforts`). */ + modelEfforts?: Record + /** `effort` is set for models with an effort selector. */ + onSelect: (modelId: string, effort?: GatewayEffort) => void onCancel: () => void } +const EFFORT_LABELS: Record = { + low: "Low", + medium: "Medium", + high: "High", + max: "Max", +} + function formatPrice(model: AxonModel): string { if (model.pricingLabel) return model.pricingLabel if (model.free) return "free" @@ -64,7 +83,20 @@ function displayName(model: AxonModel): string { return model.name.replace(/\s*\(\d+K context\)$/, "") } -export function ModelPicker({ currentId, canUse400k, canUseEidoBase, canUseEidoPro, canUseLumen, onSelect, onCancel }: ModelPickerProps) { +export function ModelPicker({ + currentId, + canUse400k, + canUseEidoBase, + canUseEidoPro, + canUseLumen, + modelEfforts, + onSelect, + onCancel, +}: ModelPickerProps) { + // Efforts adjusted in this picker (←/→), layered over the saved ones. + const [efforts, setEfforts] = useState>(() => ({ ...modelEfforts })) + const effortFor = (model: AxonModel) => getModelEffort({ modelEfforts: efforts }, model) + const select = (model: AxonModel) => onSelect(model.id, effortFor(model)) const models = Object.values(BUILTIN_AXON_MODELS).sort((a, b) => { const aIndex = CONTEXT_WINDOW_ORDER.indexOf(a.contextWindow) const bIndex = CONTEXT_WINDOW_ORDER.indexOf(b.contextWindow) @@ -100,8 +132,18 @@ export function ModelPicker({ currentId, canUse400k, canUseEidoBase, canUseEidoP setSelected((s) => nextSelectableIndex(s, 1)) return } + if (key.leftArrow || key.rightArrow) { + const model = models[selected] + const levels = getModelEffortLevels(model) + const current = effortFor(model) + if (!current) return + const index = levels.indexOf(current) + (key.rightArrow ? 1 : -1) + if (index < 0 || index >= levels.length) return + setEfforts((prev) => ({ ...prev, [model.id]: levels[index] })) + return + } if (key.return) { - if (!isLocked(models[selected])) onSelect(models[selected].id) + if (!isLocked(models[selected])) select(models[selected]) return } if (key.escape) { @@ -110,13 +152,14 @@ export function ModelPicker({ currentId, canUse400k, canUseEidoBase, canUseEidoP } if (/^[1-9]$/.test(input)) { const index = Number(input) - 1 - if (index < models.length && !isLocked(models[index])) onSelect(models[index].id) + if (index < models.length && !isLocked(models[index])) select(models[index]) } }) // Keep the selection inside the visible window when the list is long. const windowStart = Math.max(0, Math.min(selected - VISIBLE_ROWS + 1, models.length - VISIBLE_ROWS)) const visible = models.slice(windowStart, windowStart + VISIBLE_ROWS) + const effortHint = models[selected] && effortFor(models[selected]) ? " · ←/→ effort" : "" return ( @@ -129,6 +172,7 @@ export function ModelPicker({ currentId, canUse400k, canUseEidoBase, canUseEidoP const isSelected = index === selected const isCurrent = model.id === currentId const locked = isLocked(model) + const effort = locked ? undefined : effortFor(model) const providerTag = providerLabel(model.id) // The 400k group header already carries the plan note, so only badge // rows whose lock isn't explained by the context header. @@ -160,6 +204,7 @@ export function ModelPicker({ currentId, canUse400k, canUseEidoBase, canUseEidoP {displayName(model)} {isCurrent && ✓ current} · {formatPrice(model)} + {effort && !isSelected && · {EFFORT_LABELS[effort]} effort} {showPlanBadge && · {planBadgeText}} {isSelected && ( @@ -169,6 +214,25 @@ export function ModelPicker({ currentId, canUse400k, canUseEidoBase, canUseEidoP )} + {isSelected && effort && ( + + + Effort + {getModelEffortLevels(model).map((level, levelIndex) => ( + + {levelIndex > 0 && · } + {level === effort ? ( + + [{EFFORT_LABELS[level]}] + + ) : ( + {EFFORT_LABELS[level]} + )} + + ))} + + + )} ) @@ -176,7 +240,7 @@ export function ModelPicker({ currentId, canUse400k, canUseEidoBase, canUseEidoP {windowStart + VISIBLE_ROWS < models.length && ( ↓ {models.length - windowStart - VISIBLE_ROWS} more )} - ↑/↓ select · enter confirm · esc cancel + ↑/↓ select{effortHint} · enter confirm · esc cancel ) } diff --git a/src/ui/components/ResumeConfirm.tsx b/src/ui/components/ResumeConfirm.tsx new file mode 100644 index 0000000..04dda8a --- /dev/null +++ b/src/ui/components/ResumeConfirm.tsx @@ -0,0 +1,68 @@ +import React, { useState } from "react" +import { Text, useInput } from "../primitives.js" + +import { COLORS } from "../../branding.js" +import { describeResumeCost, formatIdle, formatTokens, type ResumeCostEstimate } from "../../core/resumeCost.js" +import { PopoverBox } from "./PopoverBox.js" + +export type ResumeDecision = "resume" | "new" + +interface ResumeConfirmProps { + estimate: ResumeCostEstimate + /** The plan share is still being fetched; it fills in when it arrives. */ + estimating?: boolean + onDecision: (decision: ResumeDecision) => void + onCancel: () => void +} + +const OPTIONS: Array<{ decision: ResumeDecision; label: string; hint?: string }> = [ + { decision: "resume", label: "Resume" }, + { + decision: "new", + label: "Start a new conversation", + hint: "OrbCode can look this one up if you refer to it later", + }, +] + +/** + * Shown before resuming a session whose prompt cache has expired: the first + * turn re-reads the whole context at the uncached price, so a long-idle, + * large session gets a chance to start fresh instead. + */ +export function ResumeConfirm({ estimate, estimating = false, onDecision, onCancel }: ResumeConfirmProps) { + const [selected, setSelected] = useState(0) + + useInput((input, key) => { + if (key.upArrow) setSelected((s) => (s - 1 + OPTIONS.length) % OPTIONS.length) + else if (key.downArrow || key.tab) setSelected((s) => (s + 1) % OPTIONS.length) + else if (key.return) onDecision(OPTIONS[selected].decision) + else if (key.escape) onCancel() + else if (/^[1-9]$/.test(input) && Number(input) <= OPTIONS.length) onDecision(OPTIONS[Number(input) - 1].decision) + }) + + return ( + + + Resume this conversation? + + + + This conversation has been inactive for {formatIdle(estimate.idleMs)} and is{" "} + {formatTokens(estimate.contextTokens)} tokens long. Resuming it will{" "} + {describeResumeCost(estimate, estimating)}. + + + {OPTIONS.map((option, i) => { + const isSelected = i === selected + return ( + + {isSelected ? "❯ " : " "} + {i + 1}. {option.label} + {option.hint && · {option.hint}} + + ) + })} + ↑/↓ select · enter confirm · esc cancel + + ) +} diff --git a/src/ui/components/StatusBar.tsx b/src/ui/components/StatusBar.tsx index c9673af..b2395bd 100644 --- a/src/ui/components/StatusBar.tsx +++ b/src/ui/components/StatusBar.tsx @@ -42,6 +42,8 @@ function formatRelativeTime(isoStr?: string): string { interface StatusBarProps { modelId: string; + /** Effort sent with requests, for models with an effort selector. */ + effort?: string; contextTokens: number; totalCost: number; state: string; @@ -104,6 +106,7 @@ function usageSummary(tu: AxonCodeTieredUsage | undefined): string | null { export function StatusBar({ modelId, + effort, contextTokens, totalCost: _totalCost, state, @@ -153,7 +156,8 @@ export function StatusBar({ > {title ? `${truncate(title, 32)} · ` : ""} - {model.name} · ctx {contextTokens.toLocaleString()} ({contextPct}%) + {model.name} + {effort ? ` · ${effort}` : ""} · ctx {contextTokens.toLocaleString()} ({contextPct}%) diff --git a/src/utils/autoUpdate.ts b/src/utils/autoUpdate.ts new file mode 100644 index 0000000..6f59f56 --- /dev/null +++ b/src/utils/autoUpdate.ts @@ -0,0 +1,501 @@ +import { execFile, spawn } from "node:child_process"; +import * as fs from "node:fs"; +import * as path from "node:path"; +import { fileURLToPath } from "node:url"; +import { promisify } from "node:util"; + +import { VERSION } from "../branding.js"; +import { getConfigDir, type OrbCodeSettings } from "../config/settings.js"; +import { compareVersions, isGlobalInstall, type UpdateInfo } from "./updateCheck.js"; + +// Background updates stage each new version in its own directory, +// ~/.orbcode/versions//, and flip the one-line `current` pointer only +// after the install has been verified. bin/select-version.js reads that +// pointer on the next launch. The running install is never modified, so a +// failed or interrupted update leaves everything exactly as it was. +// +// The directory layout and marker names are shared with bin/select-version.js. + +const execFileAsync = promisify(execFile); + +/** Only auto-install releases that have been public this long, so a bad + * release can be caught (and superseded) before it reaches everyone. */ +export const MIN_RELEASE_AGE_MS = 24 * 60 * 60 * 1_000; +/** After a failed install, stay quiet (and show the manual banner) this long. */ +const FAILURE_BACKOFF_MS = 24 * 60 * 60 * 1_000; +/** Old version dirs are removed once they have been superseded this long. */ +const PRUNE_AFTER_MS = 7 * 24 * 60 * 60 * 1_000; +const LOCK_STALE_MS = 30 * 60 * 1_000; +const REGISTRY_TIMEOUT_MS = 10_000; +const INSTALL_TIMEOUT_MS = 10 * 60 * 1_000; +const VERIFY_TIMEOUT_MS = 60_000; + +const PACKAGE_PATH = ["node_modules", "@matterailab", "orbcode"]; +const VERSION_RE = /^\d+\.\d+\.\d+(?:-[0-9A-Za-z.-]+)?$/; +const HEALTHY_MARKER = ".healthy"; +const ATTEMPTS_MARKER = ".attempts"; +const SUPERSEDED_MARKER = ".superseded"; +const MAX_UNHEALTHY_LAUNCHES = 1; + +export function getVersionsDir(): string { + return path.join(getConfigDir(), "versions"); +} + +function versionDir(version: string): string { + return path.join(getVersionsDir(), version); +} + +function versionBin(dir: string): string { + return path.join(dir, ...PACKAGE_PATH, "bin", "orbcode.js"); +} + +export function isVersionString(value: unknown): value is string { + return typeof value === "string" && VERSION_RE.test(value); +} + +export function readCurrentPointer(): string | null { + try { + const value = fs.readFileSync(path.join(getVersionsDir(), "current"), "utf8").trim(); + return isVersionString(value) ? value : null; + } catch { + return null; + } +} + +function readAttempts(dir: string): number { + try { + const n = Number.parseInt(fs.readFileSync(path.join(dir, ATTEMPTS_MARKER), "utf8"), 10); + return Number.isFinite(n) && n > 0 ? n : 0; + } catch { + return 0; + } +} + +/** A staged version that failed to start and never proved itself healthy. */ +export function isBadVersion(version: string): boolean { + const dir = versionDir(version); + return !fs.existsSync(path.join(dir, HEALTHY_MARKER)) && readAttempts(dir) >= MAX_UNHEALTHY_LAUNCHES; +} + +/** The staged-version directory this process is running from, if any. */ +function getRunningVersionDir(): string | null { + try { + // dist/utils/autoUpdate.js -> package root + const packageRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..", ".."); + const dir = path.resolve(packageRoot, ...PACKAGE_PATH.map(() => "..")); + if (path.dirname(dir) !== path.resolve(getVersionsDir())) return null; + return isVersionString(path.basename(dir)) ? dir : null; + } catch { + return null; + } +} + +/** + * Record that this staged version started successfully, so the launcher keeps + * using it. No-op for installs outside ~/.orbcode/versions. + */ +export function markRunningVersionHealthy(): void { + const dir = getRunningVersionDir(); + if (!dir) return; + try { + fs.writeFileSync(path.join(dir, HEALTHY_MARKER), new Date().toISOString()); + } catch { + // best-effort + } +} + +/** Why background updates are off for this process, or null when they're on. */ +export function autoUpdateDisabledReason(settings: Pick): string | null { + const optOut = process.env.ORBCODE_DISABLE_AUTOUPDATE; + if (optOut && optOut !== "0" && optOut !== "false") return "ORBCODE_DISABLE_AUTOUPDATE is set"; + if (process.env.CI) return "running in CI"; + if (settings.autoUpdates === false) return "autoUpdates is false in settings"; + if (!isGlobalInstall()) return "not a global install"; + if (!getNodePath()) return "no Node.js binary to verify updates with"; + return null; +} + +/** The Node binary that runs bin/orbcode.js — what a real launch uses. */ +function getNodePath(): string | null { + if (process.env.ORBCODE_NODE_PATH) return process.env.ORBCODE_NODE_PATH; + return process.versions.bun ? null : process.execPath; +} + +/** Node version reported by the launcher's Node binary (not Bun's shim). */ +async function getNodeVersion(nodePath: string): Promise { + if (!process.versions.bun && nodePath === process.execPath) return process.versions.node; + try { + const { stdout } = await execFileAsync(nodePath, ["-p", "process.versions.node"], { timeout: 10_000 }); + const v = stdout.trim(); + return isVersionString(v) ? v : null; + } catch { + return null; + } +} + +/** + * Only the `>=X[.Y[.Z]]` form OrbCode publishes is understood; anything else + * counts as unsatisfied so we never stage a version we can't vouch for. + */ +export function satisfiesNodeEngine(range: string | undefined, nodeVersion: string): boolean { + if (!range || !range.trim()) return true; + const match = /^\s*>=\s*v?(\d+(?:\.\d+){0,2})\s*$/.exec(range); + if (!match) return false; + return compareVersions(nodeVersion, match[1]) >= 0; +} + +interface ReleaseInfo { + publishedAt: number | null; + nodeEngine?: string; + deprecated: boolean; +} + +async function fetchReleaseInfo(pkg: string, version: string): Promise { + const controller = new AbortController(); + const timer = setTimeout(() => controller.abort(), REGISTRY_TIMEOUT_MS); + try { + const res = await fetch(`https://registry.npmjs.org/${encodeURIComponent(pkg)}`, { + signal: controller.signal, + headers: { accept: "application/json" }, + }); + if (!res.ok) return null; + const doc = (await res.json()) as { + time?: Record; + versions?: Record; + }; + const manifest = doc.versions?.[version]; + if (!manifest) return null; + const published = Date.parse(doc.time?.[version] ?? ""); + return { + publishedAt: Number.isFinite(published) ? published : null, + nodeEngine: typeof manifest.engines?.node === "string" ? manifest.engines.node : undefined, + deprecated: manifest.deprecated !== undefined && manifest.deprecated !== false, + }; + } catch { + return null; + } finally { + clearTimeout(timer); + } +} + +// ── lock ──────────────────────────────────────────────────────────────────── + +function isPidAlive(pid: number): boolean { + try { + process.kill(pid, 0); + return true; + } catch (error) { + return (error as NodeJS.ErrnoException).code === "EPERM"; + } +} + +/** Exclusive cross-process lock so concurrent sessions don't install at once. */ +function acquireLock(): (() => void) | null { + const lockPath = path.join(getVersionsDir(), ".lock"); + const token = `${process.pid}:${Date.now()}`; + for (let attempt = 0; attempt < 2; attempt++) { + try { + fs.writeFileSync(lockPath, token, { flag: "wx" }); + return () => { + try { + if (fs.readFileSync(lockPath, "utf8") === token) fs.unlinkSync(lockPath); + } catch { + // ignore + } + }; + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== "EEXIST") return null; + try { + const raw = fs.readFileSync(lockPath, "utf8"); + const [pidText, atText] = raw.split(":"); + const pid = Number(pidText); + const at = Number(atText); + const stale = !isPidAlive(pid) || !Number.isFinite(at) || Date.now() - at > LOCK_STALE_MS; + if (!stale) return null; + // Another process may have taken over the stale lock since we read it; + // only remove the exact lock we judged stale. + if (fs.readFileSync(lockPath, "utf8") !== raw) return null; + fs.unlinkSync(lockPath); + } catch { + return null; + } + } + } + return null; +} + +// ── install / verify ──────────────────────────────────────────────────────── + +function npmInstall(pkg: string, version: string, cwd: string): Promise { + return new Promise((resolve) => { + const child = spawn( + "npm", + [ + "install", + "--prefix", + ".", + "--no-audit", + "--no-fund", + "--no-save", + "--no-package-lock", + // The bundled Bun runtime is placed by its postinstall script. + "--ignore-scripts=false", + "--loglevel=error", + `${pkg}@${version}`, + ], + { cwd, stdio: "ignore", shell: process.platform === "win32" }, + ); + const killChild = () => child.kill(); + const timer = setTimeout(killChild, INSTALL_TIMEOUT_MS); + // An interrupted install only leaves a *.tmp dir, which the launcher + // ignores and the next update removes. + process.once("exit", killChild); + const done = (reason: string | null) => { + clearTimeout(timer); + process.off("exit", killChild); + resolve(reason); + }; + child.once("error", (error) => done(`npm could not start: ${error.message}`)); + child.once("close", (code) => done(code === 0 ? null : `npm install exited with ${code}`)); + }); +} + +/** + * Run the staged copy exactly the way a real launch would (Node launcher → + * bundled Bun → dist) and require it to report the expected version, then + * load its whole UI module graph via the hidden --self-test flag. + */ +async function verifyInstall(dir: string, version: string, nodePath: string): Promise { + const bin = versionBin(dir); + try { + const pkg = JSON.parse(fs.readFileSync(path.join(dir, ...PACKAGE_PATH, "package.json"), "utf8")) as { + version?: unknown; + }; + if (pkg.version !== version) return `installed package reports ${String(pkg.version)}`; + } catch { + return "installed package.json is missing"; + } + const options = { + cwd: dir, + timeout: VERIFY_TIMEOUT_MS, + env: { ...process.env, ORBCODE_DELEGATED: "1", ORBCODE_NODE_PATH: nodePath }, + }; + try { + const { stdout } = await execFileAsync(nodePath, [bin, "--version"], options); + if (stdout.trim() !== version) return `--version printed "${stdout.trim()}"`; + } catch (error) { + return `--version failed: ${(error as Error).message}`; + } + try { + const { stdout } = await execFileAsync(nodePath, [bin, "--self-test"], options); + const lastLine = stdout.trim().split(/\r?\n/).pop(); + if (lastLine !== `ok ${version}`) return `--self-test printed "${lastLine ?? ""}"`; + } catch (error) { + return `--self-test failed: ${(error as Error).message}`; + } + return null; +} + +/** rename() with retries — on Windows, scanners briefly lock fresh files. */ +async function renameWithRetry(from: string, to: string): Promise { + for (let attempt = 0; ; attempt++) { + try { + fs.renameSync(from, to); + return; + } catch (error) { + const code = (error as NodeJS.ErrnoException).code; + if (attempt >= 5 || (code !== "EPERM" && code !== "EBUSY" && code !== "EACCES")) throw error; + await new Promise((r) => setTimeout(r, 200 * (attempt + 1))); + } + } +} + +async function writeCurrentPointer(version: string): Promise { + const versionsDir = getVersionsDir(); + const tmp = path.join(versionsDir, `current.${process.pid}.tmp`); + fs.writeFileSync(tmp, `${version}\n`); + await renameWithRetry(tmp, path.join(versionsDir, "current")); +} + +function removeDir(dir: string): void { + try { + fs.rmSync(dir, { recursive: true, force: true, maxRetries: 3 }); + } catch { + // best-effort; tried again on the next update + } +} + +/** Drop leftover *.tmp dirs and versions superseded more than a week ago. */ +function pruneVersions(keep: Set): void { + const versionsDir = getVersionsDir(); + let entries: fs.Dirent[]; + try { + entries = fs.readdirSync(versionsDir, { withFileTypes: true }); + } catch { + return; + } + for (const entry of entries) { + if (!entry.isDirectory()) continue; + const dir = path.join(versionsDir, entry.name); + if (entry.name.endsWith(".tmp")) { + removeDir(dir); + continue; + } + if (!isVersionString(entry.name) || keep.has(entry.name)) continue; + let since: number; + try { + since = fs.statSync(path.join(dir, SUPERSEDED_MARKER)).mtimeMs; + } catch { + try { + since = fs.statSync(dir).mtimeMs; + } catch { + continue; + } + } + if (Date.now() - since > PRUNE_AFTER_MS) removeDir(dir); + } +} + +/** + * Install `version` into its own directory, verify it, and point `current` at + * it. Returns null on success or a reason string. Never touches the running + * install; on any failure the pointer is left unchanged. + */ +export async function stageVersion( + pkg: string, + version: string, + nodePath: string, +): Promise { + const versionsDir = getVersionsDir(); + fs.mkdirSync(versionsDir, { recursive: true }); + const release = acquireLock(); + if (!release) return "locked"; + try { + const finalDir = versionDir(version); + const previous = readCurrentPointer(); + if (fs.existsSync(finalDir)) { + if (isBadVersion(version)) return `v${version} failed to start previously`; + if (!fs.existsSync(versionBin(finalDir))) removeDir(finalDir); + } + if (!fs.existsSync(finalDir)) { + const tmpDir = `${finalDir}.tmp`; + removeDir(tmpDir); + fs.mkdirSync(tmpDir, { recursive: true }); + const installError = await npmInstall(pkg, version, tmpDir); + if (installError) { + removeDir(tmpDir); + return installError; + } + const verifyError = await verifyInstall(tmpDir, version, nodePath); + if (verifyError) { + removeDir(tmpDir); + return verifyError; + } + await renameWithRetry(tmpDir, finalDir); + } + await writeCurrentPointer(version); + if (previous && previous !== version) { + try { + fs.writeFileSync(path.join(versionDir(previous), SUPERSEDED_MARKER), new Date().toISOString()); + } catch { + // previous dir may already be gone + } + } + pruneVersions(new Set([version, VERSION, ...(previous ? [previous] : [])])); + return null; + } catch (error) { + return (error as Error).message; + } finally { + release(); + } +} + +// ── orchestration ─────────────────────────────────────────────────────────── + +interface FailureRecord { + at: number; + version: string; + reason: string; +} + +function failurePath(): string { + return path.join(getVersionsDir(), ".last-failure"); +} + +function readRecentFailure(): FailureRecord | null { + try { + const record = JSON.parse(fs.readFileSync(failurePath(), "utf8")) as FailureRecord; + return Date.now() - record.at < FAILURE_BACKOFF_MS ? record : null; + } catch { + return null; + } +} + +function writeFailure(version: string, reason: string): void { + try { + fs.mkdirSync(getVersionsDir(), { recursive: true }); + fs.writeFileSync(failurePath(), JSON.stringify({ at: Date.now(), version, reason })); + } catch { + // best-effort + } +} + +function clearFailure(): void { + try { + fs.unlinkSync(failurePath()); + } catch { + // ignore + } +} + +export interface UpdateNotice extends UpdateInfo { + /** A newer version is staged and will run on the next launch. */ + stagedVersion?: string; +} + +/** + * Turn an npm version check into what the UI should show, staging the update + * in the background when allowed. The manual "run orbcode update" banner only + * appears when background updates are off or have failed. + */ +export async function resolveUpdateNotice( + pkg: string, + info: UpdateInfo, + settings: Pick, +): Promise { + try { + if (autoUpdateDisabledReason(settings)) return info; + const quiet: UpdateNotice = { ...info, updateAvailable: false }; + const staged = readCurrentPointer(); + const stagedNotice = + staged && compareVersions(staged, info.current) > 0 && !isBadVersion(staged) + ? { ...quiet, stagedVersion: staged } + : null; + const latest = info.latest; + if (!info.updateAvailable || !latest) return stagedNotice ?? info; + // Registry data becomes a path and an npm argument below. + if (!isVersionString(latest)) return stagedNotice ?? quiet; + if (staged && compareVersions(staged, latest) >= 0) return stagedNotice ?? info; + if (readRecentFailure()) return info; + + const release = await fetchReleaseInfo(pkg, latest); + // Registry hiccup or a pulled release: stay quiet and retry next launch. + if (!release || release.deprecated || release.publishedAt === null) return stagedNotice ?? quiet; + if (Date.now() - release.publishedAt < MIN_RELEASE_AGE_MS) return stagedNotice ?? quiet; + + const nodePath = getNodePath()!; + const nodeVersion = await getNodeVersion(nodePath); + if (!nodeVersion || !satisfiesNodeEngine(release.nodeEngine, nodeVersion)) return info; + + const error = await stageVersion(pkg, latest, nodePath); + if (error === "locked") return stagedNotice ?? quiet; + if (error) { + writeFailure(latest, error); + return info; + } + clearFailure(); + return { ...quiet, stagedVersion: latest }; + } catch { + return info; + } +} diff --git a/test/auto-update.test.ts b/test/auto-update.test.ts new file mode 100644 index 0000000..099e2d1 --- /dev/null +++ b/test/auto-update.test.ts @@ -0,0 +1,184 @@ +import assert from "node:assert/strict" +import * as fs from "node:fs" +import * as os from "node:os" +import * as path from "node:path" +import test from "node:test" + +// @ts-ignore -- plain-JS launcher helper, no type declarations +import { selectStagedVersion } from "../bin/select-version.js" +import { + isBadVersion, + readCurrentPointer, + resolveUpdateNotice, + satisfiesNodeEngine, + stageVersion, +} from "../src/utils/autoUpdate.js" + +const PKG = "@matterailab/orbcode" + +function freshConfigDir(): string { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), "orbcode-autoupdate-")) + process.env.MATTERAI_CONFIG_DIR = dir + return dir +} + +function stageFake(configDir: string, version: string): string { + const binDir = path.join(configDir, "versions", version, "node_modules", "@matterailab", "orbcode", "bin") + fs.mkdirSync(binDir, { recursive: true }) + fs.writeFileSync(path.join(binDir, "orbcode.js"), "") + return path.join(configDir, "versions", version) +} + +function setPointer(configDir: string, version: string): void { + fs.mkdirSync(path.join(configDir, "versions"), { recursive: true }) + fs.writeFileSync(path.join(configDir, "versions", "current"), `${version}\n`) +} + +const launcherEnv = (configDir: string) => ({ MATTERAI_CONFIG_DIR: configDir }) + +test("launcher runs itself when nothing is staged or the pointer is unusable", () => { + const configDir = freshConfigDir() + assert.equal(selectStagedVersion({ selfVersion: "1.0.0", env: launcherEnv(configDir) }), null) + setPointer(configDir, "not-a-version") + assert.equal(selectStagedVersion({ selfVersion: "1.0.0", env: launcherEnv(configDir) }), null) + // Pointer to a version whose files are missing. + setPointer(configDir, "2.0.0") + assert.equal(selectStagedVersion({ selfVersion: "1.0.0", env: launcherEnv(configDir) }), null) +}) + +test("launcher prefers its own copy when it is the same or newer", () => { + const configDir = freshConfigDir() + stageFake(configDir, "1.5.0") + setPointer(configDir, "1.5.0") + assert.equal(selectStagedVersion({ selfVersion: "1.5.0", env: launcherEnv(configDir) }), null) + assert.equal(selectStagedVersion({ selfVersion: "2.0.0", env: launcherEnv(configDir) }), null) +}) + +test("launcher delegates to a newer staged version and stops after a failed start", () => { + const configDir = freshConfigDir() + const dir = stageFake(configDir, "2.0.0") + setPointer(configDir, "2.0.0") + const env = launcherEnv(configDir) + + // --version / --help don't count toward the failure budget. + assert.equal(selectStagedVersion({ selfVersion: "1.0.0", env, countAttempt: false })?.version, "2.0.0") + assert.equal(fs.existsSync(path.join(dir, ".attempts")), false) + + const picked = selectStagedVersion({ selfVersion: "1.0.0", env }) + assert.equal(picked?.version, "2.0.0") + assert.match(picked?.bin ?? "", /2\.0\.0.*orbcode\.js$/) + + // It never wrote .healthy, so the next launch falls back to the old copy. + assert.equal(selectStagedVersion({ selfVersion: "1.0.0", env }), null) + assert.equal(isBadVersion("2.0.0"), true) +}) + +test("a healthy staged version keeps being used", () => { + const configDir = freshConfigDir() + const dir = stageFake(configDir, "2.0.0") + setPointer(configDir, "2.0.0") + fs.writeFileSync(path.join(dir, ".healthy"), "") + const env = launcherEnv(configDir) + for (let i = 0; i < 3; i++) { + assert.equal(selectStagedVersion({ selfVersion: "1.0.0", env })?.version, "2.0.0") + } + assert.equal(selectStagedVersion({ selfVersion: "1.0.0", env: { ...env, ORBCODE_DELEGATED: "1" } }), null) +}) + +test("satisfiesNodeEngine only accepts ranges it understands", () => { + assert.equal(satisfiesNodeEngine(undefined, "20.1.0"), true) + assert.equal(satisfiesNodeEngine(">=20.0.0", "20.1.0"), true) + assert.equal(satisfiesNodeEngine(">=22", "20.1.0"), false) + assert.equal(satisfiesNodeEngine("^20 || ^22", "20.1.0"), false) +}) + +test("resolveUpdateNotice leaves the manual banner alone when disabled", async () => { + freshConfigDir() + process.env.ORBCODE_DISABLE_AUTOUPDATE = "1" + try { + const info = { current: "1.0.0", latest: "2.0.0", updateAvailable: true, unknown: false } + assert.deepEqual(await resolveUpdateNotice(PKG, info, {}), info) + } finally { + delete process.env.ORBCODE_DISABLE_AUTOUPDATE + } +}) + +// ── stageVersion against a fake npm ──────────────────────────────────────── + +const FAKE_NPM = `#!/bin/sh +[ -n "$FAKE_NPM_FAIL" ] && exit 1 +for arg in "$@"; do spec="$arg"; done +version="\${spec##*@}" +root="node_modules/@matterailab/orbcode" +mkdir -p "$root/bin" +printf '{"name":"@matterailab/orbcode","version":"%s"}' "$version" > "$root/package.json" +cat > "$root/bin/orbcode.js" < void { + const binDir = fs.mkdtempSync(path.join(os.tmpdir(), "orbcode-fake-npm-")) + fs.writeFileSync(path.join(binDir, "npm"), FAKE_NPM, { mode: 0o755 }) + const originalPath = process.env.PATH + process.env.PATH = `${binDir}${path.delimiter}${originalPath}` + return () => { + process.env.PATH = originalPath + } +} + +const posixOnly = { skip: process.platform === "win32" ? "fake npm is a shell script" : false } + +test("stageVersion installs, verifies and flips the pointer", posixOnly, async () => { + const configDir = freshConfigDir() + const restore = withFakeNpm() + try { + // An old version superseded long ago gets pruned. + const oldDir = stageFake(configDir, "0.1.0") + const marker = path.join(oldDir, ".superseded") + fs.writeFileSync(marker, "") + const longAgo = new Date(Date.now() - 30 * 24 * 60 * 60 * 1000) + fs.utimesSync(marker, longAgo, longAgo) + fs.mkdirSync(path.join(configDir, "versions", "1.9.0.tmp")) + + assert.equal(await stageVersion(PKG, "2.0.0", process.execPath), null) + assert.equal(readCurrentPointer(), "2.0.0") + const entries = fs.readdirSync(path.join(configDir, "versions")).sort() + assert.deepEqual(entries, ["2.0.0", "current"]) + } finally { + restore() + } +}) + +test("a failed install or self-test leaves the pointer untouched", posixOnly, async () => { + const configDir = freshConfigDir() + const restore = withFakeNpm() + try { + setPointer(configDir, "1.0.0") + + process.env.FAKE_NPM_FAIL = "1" + assert.match((await stageVersion(PKG, "2.0.0", process.execPath)) ?? "", /npm install exited/) + delete process.env.FAKE_NPM_FAIL + + process.env.FAKE_SELFTEST_FAIL = "1" + assert.match((await stageVersion(PKG, "2.0.0", process.execPath)) ?? "", /--self-test failed/) + delete process.env.FAKE_SELFTEST_FAIL + + assert.equal(readCurrentPointer(), "1.0.0") + const entries = fs.readdirSync(path.join(configDir, "versions")) + assert.deepEqual(entries, ["current"]) + } finally { + restore() + } +}) + +test("a concurrent install is skipped rather than raced", posixOnly, async () => { + const configDir = freshConfigDir() + fs.mkdirSync(path.join(configDir, "versions"), { recursive: true }) + fs.writeFileSync(path.join(configDir, "versions", ".lock"), `${process.pid}:${Date.now()}`) + assert.equal(await stageVersion(PKG, "2.0.0", process.execPath), "locked") + assert.equal(readCurrentPointer(), null) +}) diff --git a/test/cache-warmup.test.ts b/test/cache-warmup.test.ts new file mode 100644 index 0000000..4a05ee2 --- /dev/null +++ b/test/cache-warmup.test.ts @@ -0,0 +1,124 @@ +import assert from "node:assert/strict" +import test from "node:test" +import * as fs from "node:fs" +import * as http from "node:http" +import type { AddressInfo } from "node:net" +import * as os from "node:os" +import * as path from "node:path" +import type OpenAI from "openai" + +import { AxonClient } from "../src/api/client.js" +import type { LLMClient } from "../src/api/llmClient.js" +import type { ApiStreamChunk } from "../src/api/stream.js" +import { Agent } from "../src/core/agent.js" + +type Tools = OpenAI.Chat.ChatCompletionTool[] + +/** Records the prefix (system prompt + tools) of every warmup and real request. */ +class RecordingClient implements LLMClient { + warmups: { system: string; tools: Tools }[] = [] + requests: { system: string; tools: Tools }[] = [] + + async warmup(system: string, tools: Tools): Promise { + this.warmups.push({ system, tools }) + } + + async *createMessage( + system: string, + _messages: OpenAI.Chat.ChatCompletionMessageParam[], + tools: Tools, + ): AsyncGenerator { + this.requests.push({ system, tools }) + yield { type: "text", text: "done" } + yield { type: "usage", inputTokens: 100, outputTokens: 1, totalCost: 0 } + } +} + +function makeAgent(client: LLMClient) { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), "orbcode-warmup-")) + process.env.MATTERAI_CONFIG_DIR = path.join(dir, ".config") + return new Agent({ + cwd: dir, + token: "", + modelId: "zai/glm-5.3-flash", + autoApproveEdits: true, + autoApproveSafeCommands: true, + client, + callbacks: { + onEvent: () => {}, + requestApproval: async () => "yes", + requestFollowup: async () => "ok", + }, + }) +} + +test("warmup primes the exact system prompt and tools the first turn sends", async () => { + const client = new RecordingClient() + const agent = makeAgent(client) + assert.equal(agent.hasSession, false) + + await agent.warmCache() + await agent.runTurn("hello") + + assert.equal(client.warmups.length, 1) + assert.equal(client.requests.length, 1) + assert.equal(client.warmups[0].system, client.requests[0].system) + assert.deepEqual(client.warmups[0].tools, client.requests[0].tools) + assert.equal(agent.hasSession, true) +}) + +test("warmup is skipped once the conversation has started", async () => { + const client = new RecordingClient() + const agent = makeAgent(client) + await agent.runTurn("hello") + await agent.warmCache() + assert.equal(client.warmups.length, 0) +}) + +test("warmup failures stay silent", async () => { + const agent = makeAgent({ + warmup: async () => { + throw new Error("gateway down") + }, + createMessage: async function* () {}, + }) + await agent.warmCache() +}) + +test("AxonClient.warmup marks the request and caps output at one token", async () => { + let received: { headers: http.IncomingHttpHeaders; body: any } | undefined + const server = http.createServer((req, res) => { + let raw = "" + req.on("data", (chunk) => (raw += chunk)) + req.on("end", () => { + received = { headers: req.headers, body: JSON.parse(raw) } + res.writeHead(200, { "Content-Type": "text/event-stream" }) + res.write(`data: ${JSON.stringify({ id: "1", object: "chat.completion.chunk", choices: [{ index: 0, delta: { content: "H" } }] })}\n\n`) + res.end("data: [DONE]\n\n") + }) + }) + await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve)) + const { port } = server.address() as AddressInfo + try { + const client = new AxonClient({ + token: "test-token", + modelId: "zai/glm-5.3-flash", + taskId: "task-123", + baseUrl: `http://127.0.0.1:${port}/v1/`, + }) + const tools: Tools = [ + { type: "function", function: { name: "read_file", parameters: { type: "object", properties: {} } } }, + ] + await client.warmup("SYSTEM", tools) + + assert.ok(received) + assert.equal(received.headers["x-axoncode-warmup"], "1") + assert.equal(received.headers["x-axoncode-taskid"], "task-123") + assert.equal(received.body.max_tokens, 1) + assert.equal(received.body.stream, true) + assert.deepEqual(received.body.messages[0], { role: "system", content: "SYSTEM" }) + assert.deepEqual(received.body.tools, tools) + } finally { + server.close() + } +}) diff --git a/test/compaction.test.ts b/test/compaction.test.ts new file mode 100644 index 0000000..858fa60 --- /dev/null +++ b/test/compaction.test.ts @@ -0,0 +1,210 @@ +import assert from "node:assert/strict" +import test from "node:test" +import * as fs from "node:fs" +import * as os from "node:os" +import * as path from "node:path" +import type OpenAI from "openai" + +import type { LLMClient } from "../src/api/llmClient.js" +import type { ApiStreamChunk } from "../src/api/stream.js" +import { Agent } from "../src/core/agent.js" +import type { SessionData } from "../src/core/sessions.js" + +// glm-5.3-flash: 232k-token window, auto-compaction at 80% (185.6k). +type Messages = OpenAI.Chat.ChatCompletionMessageParam[] + +const tooLong = () => + Object.assign(new Error("This model's maximum context length is 232000 tokens. Please reduce the length of the messages."), { + status: 400, + }) + +/** Rough request size in tokens, as a provider would count it (4 chars/token). */ +const requestTokens = (messages: Messages) => Math.ceil(JSON.stringify(messages).length / 4) + +class Client implements LLMClient { + requests: { messages: Messages; tools: number }[] = [] + constructor(private readonly answer: (messages: Messages, tools: number, n: number) => Array) {} + async *createMessage(_s: string, messages: Messages, tools: OpenAI.Chat.ChatCompletionTool[]): AsyncGenerator { + this.requests.push({ messages: structuredClone(messages), tools: tools.length }) + for (const chunk of this.answer(messages, tools.length, this.requests.length)) { + if (chunk instanceof Error) throw chunk + yield chunk + } + } + get summaries() { + return this.requests.filter((r) => r.tools === 0) + } +} + +const summary = (): ApiStreamChunk[] => [ + { type: "text", text: "SUMMARY" }, + { type: "usage", inputTokens: 1000, outputTokens: 50, totalCost: 0 }, +] +const final = (inputTokens = 5000): ApiStreamChunk[] => [ + { type: "text", text: "done" }, + { type: "usage", inputTokens, outputTokens: 10, totalCost: 0 }, +] +const readCall = (inputTokens: number, file: string): ApiStreamChunk[] => [ + { + type: "native_tool_calls", + toolCalls: [ + { + index: 0, + id: `call_${file}_${inputTokens}`, + type: "function", + function: { name: "read_file", arguments: JSON.stringify({ files: [{ file_path: file, offset: null, limit: null }] }) }, + }, + ], + }, + { type: "usage", inputTokens, outputTokens: 10, totalCost: 0 }, +] + +function makeAgent(client: LLMClient, resume?: SessionData) { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), "orbcode-compact-")) + process.env.MATTERAI_CONFIG_DIR = path.join(dir, ".config") + fs.writeFileSync(path.join(dir, "small.txt"), "x".repeat(2000)) + // 1000 lines x 200 chars: read_file returns its 100k-char cap (~25k tokens). + fs.writeFileSync(path.join(dir, "wide.txt"), Array.from({ length: 1000 }, (_, i) => `${i} ${"w".repeat(200)}`).join("\n")) + const events: string[] = [] + const agent = new Agent({ + cwd: dir, + token: "", + modelId: "zai/glm-5.3-flash", + autoApproveEdits: true, + autoApproveSafeCommands: true, + client, + resume, + callbacks: { + onEvent: (event) => { + if (event.type === "system" || event.type === "error") events.push(event.message) + }, + requestApproval: async () => "yes", + requestFollowup: async () => "ok", + }, + }) + return { agent, events } +} + +function session(contextTokens: number, opts: { turns?: number; charsPerReply?: number } = {}): SessionData { + const messages: Messages = [] + for (let i = 0; i < (opts.turns ?? 4); i++) { + messages.push({ role: "user", content: `question ${i}` }) + messages.push({ role: "assistant", content: `answer ${i} ${"z".repeat(opts.charsPerReply ?? 10)}` }) + } + const now = new Date().toISOString() + return { + id: `s-${Math.random()}`, + cwd: "/tmp", + model: "zai/glm-5.3-flash", + title: "t", + createdAt: now, + updatedAt: now, + totalCost: 0, + contextTokens, + todos: "", + messages, + } +} + +test("a resumed session past 80% compacts before its first request and keeps the new message verbatim", async () => { + const client = new Client((_m, tools) => (tools === 0 ? summary() : final())) + const { agent } = makeAgent(client, session(200_000)) + await agent.runTurn("now add tests") + assert.equal(client.requests[0].tools, 0, "first request is the summary") + const next = client.requests[1].messages + assert.equal(next.length, 2) + assert.match(String(next[0].content), /SUMMARY/) + assert.match(String(next[1].content), /now add tests/) +}) + +test("tool output added since the last usage report counts toward the threshold", async () => { + // 170k reported (73%), then read_file adds ~25k tokens: ~84% before the next step. + let step = 0 + const client = new Client((_m, tools) => { + if (tools === 0) return summary() + step++ + return step === 1 ? readCall(170_000, "wide.txt") : final() + }) + const { agent, events } = makeAgent(client, session(10_000)) + await agent.runTurn("read it") + assert.equal(client.summaries.length, 1) + assert.ok(events.some((e) => /Context is 8\d% full/.test(e)), events.join(" | ")) +}) + +test("a failed auto-compaction is retried on the next turn", async () => { + let attempts = 0 + const client = new Client((_m, tools) => { + if (tools === 0) { + attempts++ + return attempts === 1 ? [Object.assign(new Error("upstream hiccup"), { status: 400 })] : summary() + } + return final(210_000) + }) + const { agent, events } = makeAgent(client, session(210_000)) + await agent.runTurn("one") + assert.equal(attempts, 1) + assert.ok(events.some((e) => /Auto-compaction failed/.test(e))) + await agent.runTurn("two") + assert.equal(attempts, 2) + assert.ok(events.some((e) => /Conversation compacted/.test(e))) +}) + +test("a 'context too long' error compacts and retries the step instead of failing the turn", async () => { + let overflowed = false + const client = new Client((_m, tools) => { + if (tools === 0) return summary() + if (!overflowed) { + overflowed = true + return [tooLong()] + } + return final() + }) + const { agent, events } = makeAgent(client, session(120_000)) + await agent.runTurn("go") + assert.ok(events.some((e) => /no longer fits the model's context window/.test(e)), events.join(" | ")) + assert.equal(client.summaries.length, 1) + // No transient-error retries of the oversized request, and no error surfaced. + assert.ok(!events.some((e) => /Retrying/.test(e))) + assert.ok(!events.some((e) => /maximum context length/.test(e))) + assert.match(String(client.requests.at(-1)!.messages[0].content), /SUMMARY/) +}) + +test("a history already past the window still compacts: the summary request is trimmed to fit", async () => { + // ~800k tokens of history on a 232k model (e.g. resumed from a bigger-window model). + const WINDOW = 232_000 + const client = new Client((messages, tools) => { + if (requestTokens(messages) > WINDOW) return [tooLong()] + return tools === 0 ? summary() : final() + }) + const { agent, events } = makeAgent(client, session(800_000, { turns: 8, charsPerReply: 400_000 })) + await agent.runTurn("continue") + const summaryRequest = client.summaries.find((r) => requestTokens(r.messages) <= WINDOW) + assert.ok(summaryRequest, "a summary request fit the window") + assert.ok(events.some((e) => /Conversation compacted/.test(e)), events.join(" | ")) + assert.ok(!events.some((e) => /maximum context length/.test(e))) + // The turn's real request went out compacted. + assert.ok(requestTokens(client.requests.at(-1)!.messages) < 10_000) +}) + +test("a provider window smaller than the catalog's: the summary budget shrinks until it fits", async () => { + // Real upstream limit ~60k tokens while the catalog says 232k. + const REAL_WINDOW = 60_000 + const client = new Client((messages, tools) => { + if (requestTokens(messages) > REAL_WINDOW) return [tooLong()] + return tools === 0 ? summary() : final() + }) + const { agent, events } = makeAgent(client, session(300_000, { turns: 8, charsPerReply: 150_000 })) + await agent.runTurn("continue") + assert.ok(client.summaries.length >= 2, "retried with a smaller budget") + assert.ok(events.some((e) => /Conversation compacted/.test(e)), events.join(" | ")) +}) + +test("compaction still gives up cleanly when nothing fits", async () => { + const client = new Client(() => [tooLong()]) + const { agent, events } = makeAgent(client, session(200_000)) + await agent.runTurn("go") + assert.ok(events.some((e) => /Auto-compaction failed/.test(e))) + assert.ok(events.some((e) => /maximum context length/.test(e)), "the original error is reported") + // One proactive attempt (3 budgets) + one overflow-recovery attempt (3 budgets), then stop. + assert.equal(client.summaries.length, 6) +}) diff --git a/test/effort-picker.test.tsx b/test/effort-picker.test.tsx new file mode 100644 index 0000000..7a83b0f --- /dev/null +++ b/test/effort-picker.test.tsx @@ -0,0 +1,91 @@ +import assert from "node:assert/strict"; +import { test } from "node:test"; +import React, { act } from "react"; +import { testRender } from "@opentui/react/test-utils"; + +import { EffortPicker, effortSliderLayout, type EffortScope } from "../src/ui/components/EffortPicker.js"; +import type { GatewayEffort } from "../src/api/models.js"; + +const LEVELS: GatewayEffort[] = ["low", "medium", "high", "max"]; + +test("slider layout pins the ends and puts the marker over the selected label", () => { + const { width, starts, line } = effortSliderLayout(LEVELS, 2); + assert.equal(starts[0], 0); + assert.equal(starts[3] + "max".length, width); + assert.equal(line.length, width); + // ▲ sits inside the "high" label's span. + const marker = line.indexOf("▲"); + assert.ok(marker >= starts[2] && marker < starts[2] + "high".length); +}); + +async function renderPicker(current: GatewayEffort) { + const picks: Array<[GatewayEffort, EffortScope]> = []; + let cancelled = false; + const screen = await testRender( + picks.push([effort, scope])} + onCancel={() => { + cancelled = true; + }} + />, + { width: 100, height: 12 }, + ); + const press = async (action: () => void) => { + await act(async () => { + action(); + await screen.flush(); + }); + await screen.renderOnce(); + }; + return { screen, picks, press, wasCancelled: () => cancelled }; +} + +test("renders Faster/Smarter, the track and the levels; ←/→ then Enter saves", async () => { + const { screen, picks, press } = await renderPicker("medium"); + try { + await screen.renderOnce(); + const frame = screen.captureCharFrame(); + assert.match(frame, /Effort · GLM 5\.3/); + assert.match(frame, /Faster\s+Smarter/); + assert.match(frame, /─+▲─+/); + assert.match(frame, /low\s+medium\s+high\s+max/); + assert.match(frame, /←\/→ to adjust · Enter to confirm · s for this session only · Esc to cancel/); + + await press(() => screen.mockInput.pressArrow("right")); + await press(() => screen.mockInput.pressArrow("right")); + await press(() => screen.mockInput.pressArrow("right")); // clamped at max + await press(() => screen.mockInput.pressEnter()); + assert.deepEqual(picks, [["max", "saved"]]); + } finally { + act(() => screen.renderer.destroy()); + } +}); + +test("s applies to this session only; Esc cancels", async () => { + const first = await renderPicker("high"); + try { + await first.screen.renderOnce(); + await first.press(() => first.screen.mockInput.pressArrow("left")); + await first.press(() => first.screen.mockInput.pressKey("s")); + assert.deepEqual(first.picks, [["medium", "session"]]); + } finally { + act(() => first.screen.renderer.destroy()); + } + + const second = await renderPicker("low"); + try { + await second.screen.renderOnce(); + await second.press(() => second.screen.mockInput.pressEscape()); + // A lone Esc is held back briefly in case it starts an escape sequence. + await second.press(() => {}); + await new Promise((resolve) => setTimeout(resolve, 100)); + await second.press(() => {}); + assert.equal(second.wasCancelled(), true); + assert.deepEqual(second.picks, []); + } finally { + act(() => second.screen.renderer.destroy()); + } +}); diff --git a/test/files.test.ts b/test/files.test.ts index d59e760..0ea8636 100644 --- a/test/files.test.ts +++ b/test/files.test.ts @@ -77,3 +77,30 @@ test("readFile executes single file and batched file region reads", async () => fs.rmSync(tmpDir, { recursive: true, force: true }) }) + +test("readFile caps characters, not just lines (minified files, many files per call)", async () => { + const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), "orbcode-test-")) + // One 2 MB line: the 1000-line limit alone would return all of it. + fs.writeFileSync(path.join(tmpDir, "bundle.min.js"), "x".repeat(2_000_000)) + // 1000 lines of 300 chars: well under the line limit, over the char cap. + fs.writeFileSync(path.join(tmpDir, "wide.txt"), Array.from({ length: 1000 }, (_, i) => `${i} ${"w".repeat(300)}`).join("\n")) + const context = { cwd: tmpDir, setTodos: () => {} } + + const minified = await readFile({ files: [{ file_path: "bundle.min.js" }] }, context) + assert.ok(minified.text.length < 110_000, `got ${minified.text.length} chars`) + assert.match(minified.text, /Line 1 alone exceeds 100000 characters and was cut off/) + + const wide = await readFile({ files: [{ file_path: "wide.txt" }] }, context) + assert.ok(wide.text.length < 110_000) + assert.match(wide.text, /Output capped at 100000 characters; showing through line \d+\. Use offset\/limit/) + // Whole lines only: the last shown line is complete. + const shown = wide.text.split("\n\n(")[0].split("\n") + assert.match(shown.at(-1)!, /w{300}$/) + + const batch = await readFile( + { files: [{ file_path: "wide.txt" }, { file_path: "bundle.min.js" }, { file_path: "wide.txt", offset: 1, limit: 10 }] }, + context, + ) + assert.ok(batch.text.length < 215_000, `got ${batch.text.length} chars`) + assert.match(batch.text, /Skipped wide\.txt: this call reached its 200000-character limit/) +}) diff --git a/test/model-effort.test.ts b/test/model-effort.test.ts new file mode 100644 index 0000000..f25fd89 --- /dev/null +++ b/test/model-effort.test.ts @@ -0,0 +1,108 @@ +import assert from "node:assert/strict" +import * as fs from "node:fs" +import * as os from "node:os" +import * as path from "node:path" +import { test } from "node:test" + +import { AxonClient } from "../src/api/client.js" +import { X_REASONING_EFFORT } from "../src/api/headers.js" +import { getModel, getModelEffortLevels, type AxonModel } from "../src/api/models.js" +import { + getModelEffort, + loadModelEfforts, + loadSettings, + saveSettings, + setSessionModelEffort, +} from "../src/config/settings.js" + +// Settings read the config dir lazily, so this isolates every call below. +const configDir = fs.mkdtempSync(path.join(os.tmpdir(), "orbcode-effort-")) +process.env.MATTERAI_CONFIG_DIR = configDir + +function model(id: string, overrides: Partial = {}): AxonModel { + return { + id, + name: id, + description: "", + contextWindow: 232_000, + maxOutputTokens: 64_000, + supportsImages: true, + inputPrice: 0, + outputPrice: 0, + free: false, + ...overrides, + } +} + +test("the four gateway models have the selector before the catalog loads; others don't", () => { + for (const id of ["zai/glm-5.3", "zai/glm-5.3-flash", "gemini-3.8-flash", "deepseek/deepseek-v4.1-flash"]) { + assert.deepEqual(getModelEffortLevels(model(id)), ["low", "medium", "high", "max"]) + } + assert.deepEqual(getModelEffortLevels(model("gpt-5.6-sol")), []) + // Own-provider models use their own `effort` setting instead. + assert.deepEqual(getModelEffortLevels(model("zai/glm-5.3", { provider: "anthropic" })), []) +}) + +test("the catalog's reasoning_efforts list wins over the built-in one", () => { + assert.deepEqual(getModelEffortLevels(model("gpt-5.6-sol", { reasoningEfforts: ["low", "high"] })), ["low", "high"]) + assert.deepEqual(getModelEffortLevels(model("zai/glm-5.3", { reasoningEfforts: [] })), []) +}) + +test("effort defaults to medium and follows the saved pick", () => { + const glm = model("zai/glm-5.3") + assert.equal(getModelEffort({}, glm), "medium") + assert.equal(getModelEffort({ modelEfforts: { "zai/glm-5.3": "max" } }, glm), "max") + // A pick the model no longer supports falls back to the default. + assert.equal( + getModelEffort({ modelEfforts: { "zai/glm-5.3": "max" } }, model("zai/glm-5.3", { reasoningEfforts: ["low", "medium"] })), + "medium", + ) + assert.equal(getModelEffort({ modelEfforts: { "gpt-5.6-sol": "max" } }, model("gpt-5.6-sol")), undefined) +}) + +test("efforts persist in config.json, per model, and are readable without a full settings load", () => { + const settings = loadSettings() + saveSettings({ ...settings, modelEfforts: { "zai/glm-5.3": "high", "gemini-3.8-flash": "low" } }) + assert.deepEqual(loadModelEfforts(), { modelEfforts: { "zai/glm-5.3": "high", "gemini-3.8-flash": "low" } }) + assert.deepEqual(loadSettings().modelEfforts, { "zai/glm-5.3": "high", "gemini-3.8-flash": "low" }) + + // Hand-edited junk is ignored. + const configPath = path.join(configDir, "config.json") + const raw = JSON.parse(fs.readFileSync(configPath, "utf8")) + raw.modelEfforts["zai/glm-5.3-flash"] = "ultra" + fs.writeFileSync(configPath, JSON.stringify(raw)) + assert.equal(loadModelEfforts().modelEfforts?.["zai/glm-5.3-flash"], undefined) + assert.equal(loadSettings().modelEfforts?.["zai/glm-5.3-flash"], undefined) +}) + +test("every request sends the effort currently saved on this machine", () => { + const headersFor = (modelId: string) => { + const client = new AxonClient({ token: "t", modelId, taskId: "task" }) + return (client as unknown as { requestHeaders(m: AxonModel): Record }).requestHeaders( + getModel(modelId), + ) + } + saveSettings({ ...loadSettings(), modelEfforts: {} }) + assert.equal(headersFor("zai/glm-5.3-flash")[X_REASONING_EFFORT], "medium") + + // Another chat (process) saves a new pick: the next request picks it up. + saveSettings({ ...loadSettings(), modelEfforts: { "zai/glm-5.3-flash": "max" } }) + assert.equal(headersFor("zai/glm-5.3-flash")[X_REASONING_EFFORT], "max") + + // Models without the selector send no effort header. + assert.equal(headersFor("gpt-5.6-sol")[X_REASONING_EFFORT], undefined) +}) + +test("a session-only pick overrides the saved one in this process without being saved", () => { + saveSettings({ ...loadSettings(), modelEfforts: { "zai/glm-5.3": "high" } }) + setSessionModelEffort("zai/glm-5.3", "low") + try { + assert.equal(loadModelEfforts().modelEfforts?.["zai/glm-5.3"], "low") + assert.equal(loadSettings().modelEfforts?.["zai/glm-5.3"], "high") + const configPath = path.join(configDir, "config.json") + assert.equal(JSON.parse(fs.readFileSync(configPath, "utf8")).modelEfforts["zai/glm-5.3"], "high") + } finally { + setSessionModelEffort("zai/glm-5.3", undefined) + } + assert.equal(loadModelEfforts().modelEfforts?.["zai/glm-5.3"], "high") +}) diff --git a/test/model-persistence.test.ts b/test/model-persistence.test.ts new file mode 100644 index 0000000..d0dab41 --- /dev/null +++ b/test/model-persistence.test.ts @@ -0,0 +1,52 @@ +import assert from "node:assert/strict" +import * as fs from "node:fs" +import * as os from "node:os" +import * as path from "node:path" +import { test } from "node:test" + +import { DEFAULT_MODEL_ID, getModel, loadCachedModelCatalog } from "../src/api/models.js" +import { getModelEffort, loadSettings } from "../src/config/settings.js" + +// Settings and the catalog cache read the config dir lazily. +const configDir = fs.mkdtempSync(path.join(os.tmpdir(), "orbcode-persist-")) +process.env.MATTERAI_CONFIG_DIR = configDir + +const CATALOG_ONLY = "deepseek/deepseek-v4.1-flash" + +fs.writeFileSync( + path.join(configDir, "config.json"), + JSON.stringify({ model: CATALOG_ONLY, theme: "dark", modelEfforts: { [CATALOG_ONLY]: "high" } }), +) +fs.writeFileSync( + path.join(configDir, "models-cache.json"), + JSON.stringify({ + data: [ + { + id: CATALOG_ONLY, + name: "DeepSeek V4.1 Flash", + context_length: 232000, + pricing: { prompt: "0.0000003", completion: "0.0000012" }, + costMultiplier: 1, + }, + ], + }), +) + +test("a saved catalog-only model and its effort survive a restart", () => { + // Without the cached catalog (the old startup), the saved model reads as + // unknown and falls back to the default — losing its effort too. + assert.equal(loadSettings().model, DEFAULT_MODEL_ID) + + // Startup now registers the cached catalog before reading settings. + loadCachedModelCatalog() + const settings = loadSettings() + assert.equal(settings.model, CATALOG_ONLY) + assert.equal(getModelEffort(settings, getModel(settings.model)), "high") +}) + +test("a missing or corrupt cache is ignored", () => { + fs.writeFileSync(path.join(configDir, "models-cache.json"), "{not json") + assert.doesNotThrow(() => loadCachedModelCatalog()) + fs.rmSync(path.join(configDir, "models-cache.json")) + assert.doesNotThrow(() => loadCachedModelCatalog()) +}) diff --git a/test/model-picker.test.tsx b/test/model-picker.test.tsx new file mode 100644 index 0000000..486be84 --- /dev/null +++ b/test/model-picker.test.tsx @@ -0,0 +1,70 @@ +import assert from "node:assert/strict"; +import { test } from "node:test"; +import React, { act } from "react"; +import { testRender } from "@opentui/react/test-utils"; + +import { ModelPicker } from "../src/ui/components/ModelPicker.js"; +import type { GatewayEffort } from "../src/api/models.js"; + +async function renderPicker(currentId: string, modelEfforts?: Record) { + const picks: Array<[string, GatewayEffort | undefined]> = []; + const screen = await testRender( + picks.push([id, effort])} + onCancel={() => {}} + />, + { width: 140, height: 40 }, + ); + return { screen, picks }; +} + +const press = async (screen: Awaited>, action: () => void) => { + await act(async () => { + action(); + await screen.flush(); + }); + await screen.renderOnce(); +}; + +test("←/→ moves the effort for the highlighted model; enter saves both", async () => { + const { screen, picks } = await renderPicker("zai/glm-5.3-flash"); + try { + await screen.renderOnce(); + let frame = screen.captureCharFrame(); + assert.match(frame, /Effort Low · \[Medium\] · High · Max/); + assert.match(frame, /←\/→ effort/); + + await press(screen, () => screen.mockInput.pressArrow("right")); + await press(screen, () => screen.mockInput.pressArrow("right")); + // Clamped at the top. + await press(screen, () => screen.mockInput.pressArrow("right")); + frame = screen.captureCharFrame(); + assert.match(frame, /Effort Low · Medium · High · \[Max\]/); + + await press(screen, () => screen.mockInput.pressArrow("left")); + await press(screen, () => screen.mockInput.pressEnter()); + assert.deepEqual(picks, [["zai/glm-5.3-flash", "high"]]); + } finally { + act(() => screen.renderer.destroy()); + } +}); + +test("saved efforts are shown and models without the selector pick no effort", async () => { + const { screen, picks } = await renderPicker("gpt-5.6-sol", { "zai/glm-5.3": "low" }); + try { + await screen.renderOnce(); + const frame = screen.captureCharFrame(); + assert.match(frame, /GLM 5\.3 .*Low effort/); + assert.doesNotMatch(frame, /←\/→ effort/); + await press(screen, () => screen.mockInput.pressEnter()); + assert.deepEqual(picks, [["gpt-5.6-sol", undefined]]); + } finally { + act(() => screen.renderer.destroy()); + } +}); diff --git a/test/resume-confirm.test.tsx b/test/resume-confirm.test.tsx new file mode 100644 index 0000000..6327ca8 --- /dev/null +++ b/test/resume-confirm.test.tsx @@ -0,0 +1,46 @@ +import assert from "node:assert/strict"; +import { test } from "node:test"; +import React, { act } from "react"; +import { testRender } from "@opentui/react/test-utils"; + +import { ResumeConfirm, type ResumeDecision } from "../src/ui/components/ResumeConfirm.js"; + +test("resume confirm explains the cold-cache cost and defaults to Resume", async () => { + const decisions: ResumeDecision[] = []; + const screen = await testRender( + decisions.push(d)} + onCancel={() => {}} + />, + { width: 140, height: 12 }, + ); + try { + await screen.renderOnce(); + const frame = screen.captureCharFrame(); + assert.match(frame, /Resume this conversation\?/); + assert.match(frame, /inactive for 1d 1h 17m and is 785k tokens long/); + assert.match(frame, /22% of your weekly usage limit/); + assert.match(frame, /❯ 1\. Resume/); + assert.match(frame, /2\. Start a new conversation/); + + await act(async () => { + screen.mockInput.pressArrow("down"); + await screen.flush(); + }); + await act(async () => { + screen.mockInput.pressEnter(); + await screen.flush(); + }); + assert.deepEqual(decisions, ["new"]); + } finally { + act(() => screen.renderer.destroy()); + } +}); diff --git a/test/resume-cost.test.ts b/test/resume-cost.test.ts new file mode 100644 index 0000000..1382c46 --- /dev/null +++ b/test/resume-cost.test.ts @@ -0,0 +1,102 @@ +import assert from "node:assert/strict" +import { test } from "node:test" + +import type { AxonModel } from "../src/api/models.js" +import { + describeResumeCost, + estimateResumeCost, + formatIdle, + formatTokens, + PROMPT_CACHE_TTL_MS, + shouldConfirmResume, +} from "../src/core/resumeCost.js" + +const NOW = Date.parse("2026-10-06T12:00:00Z") +const MINUTE = 60_000 + +function model(overrides: Partial = {}): AxonModel { + return { + id: "m", + name: "M", + description: "", + contextWindow: 1_000_000, + maxOutputTokens: 64_000, + supportsImages: true, + inputPrice: 0.000005, + outputPrice: 0.000025, + free: false, + ...overrides, + } +} + +function session(minutesIdle: number, contextTokens: number) { + return { updatedAt: new Date(NOW - minutesIdle * MINUTE).toISOString(), contextTokens } +} + +const share = (weeklyPercentage: number) => ({ weeklyPercentage, monthlyPercentage: weeklyPercentage / 2 }) + +test("a warm cache needs no confirmation", () => { + const estimate = estimateResumeCost(session(4, 800_000), model(), share(30), "pro", NOW) + assert.equal(estimate, null) + assert.equal(shouldConfirmResume(estimate), false) + assert.notEqual(estimateResumeCost(session(PROMPT_CACHE_TTL_MS / MINUTE, 800_000), model(), share(30), "pro", NOW), null) +}) + +test("a cold resume reports the backend's weekly share", () => { + const estimate = estimateResumeCost(session(24 * 60 + 77, 785_000), model(), share(21.6), "pro", NOW) + assert.ok(estimate) + assert.equal(estimate.percent, 21.6) + assert.equal(estimate.window, "weekly") + assert.equal(shouldConfirmResume(estimate), true) + assert.equal(describeResumeCost(estimate), "use about 22% of your weekly usage limit") +}) + +test("lite plans call the (mirrored) window monthly", () => { + const estimate = estimateResumeCost(session(30, 500_000), model(), share(12.5), "lite", NOW) + assert.equal(estimate!.window, "monthly") + assert.equal(describeResumeCost(estimate!), "use about 13% of your monthly usage limit") +}) + +test("plan-billed models never fall back to a dollar figure", () => { + // Backend unreachable: no share. Big context still asks, without a $ amount. + const estimate = estimateResumeCost(session(60, 150_000), model(), undefined, "pro", NOW) + assert.equal(shouldConfirmResume(estimate), true) + assert.equal(describeResumeCost(estimate!), "re-read the full context without the prompt cache") + assert.equal(shouldConfirmResume(estimateResumeCost(session(60, 50_000), model(), undefined, "pro", NOW)), false) +}) + +test("own-provider-key models report dollars and ignore plan shares", () => { + const estimate = estimateResumeCost(session(30, 100_000), model({ provider: "anthropic" }), share(40), "pro", NOW) + assert.equal(estimate!.percent, undefined) + assert.equal(describeResumeCost(estimate!), "cost about $0.50") +}) + +test("large cold contexts always ask, showing even a small share", () => { + // 115k tokens on a cheap model: 0.2% of the week, still worth a prompt. + const estimate = estimateResumeCost(session(60, 115_227), model(), share(0.2), "pro_plus", NOW) + assert.equal(shouldConfirmResume(estimate), true) + assert.equal(describeResumeCost(estimate!), "use about <1% of your weekly usage limit") +}) + +test("smaller cold contexts ask only when the share is significant", () => { + assert.equal(shouldConfirmResume(estimateResumeCost(session(60, 50_000), model(), share(0.4), "pro", NOW)), false) + assert.equal(shouldConfirmResume(estimateResumeCost(session(60, 50_000), model(), share(2), "pro", NOW)), true) +}) + +test("while the share loads, the prompt says so", () => { + const estimate = estimateResumeCost(session(60, 150_000), model(), undefined, "pro", NOW)! + assert.equal( + describeResumeCost(estimate, true), + "re-read the full context without the prompt cache (checking your usage…)", + ) +}) + +test("formatting", () => { + assert.equal(formatIdle((24 * 60 + 77) * MINUTE), "1d 1h 17m") + assert.equal(formatIdle(185 * MINUTE), "3h 5m") + assert.equal(formatIdle(42 * MINUTE), "42m") + assert.equal(formatIdle(24 * 60 * MINUTE), "1d 0h 0m") + assert.equal(formatTokens(785_123), "785k") + assert.equal(formatTokens(1_234_567), "1.2M") + assert.equal(formatTokens(950), "950") +})