Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,8 +90,10 @@ the target journey, contract, evidence, governance, and DX-measurement system.
cross-profile host-id and alias conflicts. `profile-builders.mjs` owns typed
construction helpers for evidence, operations, surfaces, and steps rather
than duplicating their invariants, and `profiles.mjs` only composes the
registry. Every provider-specific filesystem root read outside the selected
workspace must have an explicit `inventoryHomeRoutes` entry, including state
registry. `qoder-home.mjs` owns the shared Qoder asset/session home precedence
used by analysis and inventory consumers. Every provider-specific filesystem
root read outside the selected workspace must have an explicit
`inventoryHomeRoutes` entry, including state
files, shared caches, and compatibility roots such as a user-level
`.agents/` directory. Each primary and secondary route declares its provider
option, an isolated path relative to `--host-home`, and a redacted safe
Expand Down
102 changes: 102 additions & 0 deletions docs/specs/2026-10-24-harness-analysis-defects.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
# Harness analysis host, history, instruction, and lead-binding defects

## Traceability

- Spec ID: 2026-10-24-harness-analysis-defects
- Story: QoderAI/better-harness#212, QoderAI/better-harness#195, QoderAI/better-harness#164
- Status: Implemented

## Intent

Repair the confirmed analysis defects found during Windows-hosted validation. The
defects fall into five groups: Qoder home discovery and CLI documentation,
bounded history windows that collapse to the same sample, instruction and gate
discovery gaps, an incomplete Claude Rules projection, and a missing regression
for the already-correct lead/session binding path. The change also brings the
file-explorer fixture in line with Windows environment and stream-lifecycle
behavior.

This work prefers honest coverage metadata over confident-looking but incomplete
data. A truncated or low-confidence history window must be visible as such, and
analysis must continue to fail closed when real binding conflicts exist.

## Acceptance scenarios

- AC-1: `--qoder-home` remains explicit and wins over environment/defaults, and
help for `harness analyze`, `harness evidence-bundle`, and task-loop source
advertises the option. `QODER_HOME` is honored by the Qoder session analyzer
only when it names an asset home (including `.qoder-cn`); a shared-client-cache
value is ignored. The default prefers `~/.qoder-cn` only when `~/.qoder` is
absent or has no session data.
- AC-2: history windows report independent ranges, so 30/90/180-day windows can
report different commit counts and hotspots. A window reuses the bounded main
scan when that scan reaches older than the window, and queries Git for the
window otherwise. Every window exposes its commit limit, truncation state, and
coverage status, with truncation exact at the limit.
- AC-3: `history-window-coverage` is `review` rather than `ready` when history
confidence is low or any window is truncated, while retaining the existing
ready behavior for complete high/medium-confidence history. The matrix note
explains that raising `--max-commits` completes the 30/90/180-day windows in a
very active repository.
- AC-4: instruction analysis recognizes root and nested `AGENTS.md` and
`CLAUDE.md` carriers, counts each directory scope once when both carriers
exist, excludes scratch directories from suggested scopes and the suggested
instruction minimum, and reports carrier kinds without changing the existing
density thresholds.
- AC-5: evidence packs expose project-owned package and Just gates, and the
`validate-impact` action includes those exact commands when available. Gates
that start long-running watchers, dev servers, or UI/open-headed sessions are
not presented as validation commands.
- AC-6: Claude project Rules inventory includes a root `AGENTS.md` compatibility
entry without duplicating `CLAUDE.md`. This relies on Claude Code reading
`AGENTS.md` as project instructions; see
<https://code.claude.com/docs/en/memory#agents-md>.
- AC-7: an integration test proves an injected session population keeps the lead
lane available and carries `sessionBinding`, while the genuine conflict test
still downgrades the lead to `SESSION_POPULATION_BINDING_MISMATCH`.
- AC-8: the file-explorer test sets both `HOME` and `USERPROFILE` on Windows and
shuts down without asserting while child-process pipes are still closing.

## Non-goals

- Adding new host adapters or changing host support levels.
- Replacing the bounded `maxCommits` safety limit or adding unbounded history
scans.
- Redesigning report/Canvas UI or changing the binding reconciliation contract.
- Treating scratch or generated files as instruction carriers.

## Plan and tasks

1. Align Qoder home resolution across asset and session analysis, and document
the accepted host-home option.
2. Fetch and filter each history window independently, reuse the main scan when
it covers the window, add coverage metadata, and make the review matrix
consume it.
3. Extend instruction carriers, scope filtering, and project gate extraction.
4. Add root `AGENTS.md` to the Claude Rules inventory.
5. Add a lead/session binding integration regression and keep the conflict test.
6. Make the file-explorer fixture portable and close child streams explicitly.
7. Run focused tests, then the broader affected suites.

## Test and review evidence

- `test/agents/coding-agent-practices-inventory.test.mjs` covers Qoder asset
homes, `QODER_HOME` separation, and an implicit `.qoder-cn` home.
- `test/agents/agent-customize.test.mjs` covers Qoder provider `QODER_HOME`
resolution and the Claude Rules root `AGENTS.md` compatibility entry.
- `test/sessions/session-analysis.test.mjs` exercises Qoder session roots from
`QODER_HOME`, rejects a non-asset `QODER_HOME` (shared client cache), the
`~/.qoder-cn` default, and the both-homes preference.
- `test/governance/core-change-watch.test.mjs` covers independent history
windows, exact truncation at the commit limit, instruction scope deduplication,
scratch-scope exclusion, non-terminating gate exclusion, and project gates in
the `validate-impact` action.
- `test/reporting/better-harness-evidence-bundle.test.mjs` exercises the real
Claude lead/session binding path and keeps the genuine conflict regression.
- `packages/harness-studio-apps/test/test_file_explorer.mjs` passes on Windows
with `HOME`/`USERPROFILE` isolation and orderly child shutdown.
- `test/support/path-links.mjs` keeps Windows fixtures portable with directory
junctions and skips only the two file-symlink cases when the host denies them.
- The DSH explicit-root fixture isolates `HOME`/`USERPROFILE`, so external roots
are not accidentally classified as user-home paths.
- Full `npm test` on 2026-10-25: 1767 passed, 30 skipped, 0 failed.
15 changes: 13 additions & 2 deletions packages/harness-studio-apps/test/test_file_explorer.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ function startBackend() {
env: {
...process.env,
HOME: HOME_DIR,
USERPROFILE: HOME_DIR,
KIROCREW_HOME: HOME_DIR,
KIROCREW_PROXY_SECRET: SECRET,
PORT: String(PORT),
Expand Down Expand Up @@ -72,6 +73,16 @@ async function call(path) {
return fetch(`http://127.0.0.1:${PORT}${path}`, { headers: { 'X-KiroCrew-Proxy': sig } })
}

async function stopBackend(backend) {
if (backend.exitCode === null && backend.signalCode === null) {
const exited = new Promise((resolve) => backend.once('exit', resolve))
backend.kill('SIGTERM')
await Promise.race([exited, new Promise((resolve) => setTimeout(resolve, 2000))])
}
backend.stdout?.destroy()
backend.stderr?.destroy()
}

const child = startBackend()

try {
Expand Down Expand Up @@ -112,8 +123,8 @@ try {
r = await call(`${BASE}/nope`)
check('unknown route -> 404', r.status === 404, `got ${r.status}`)
} finally {
child.kill('SIGTERM')
await stopBackend(child)
}

console.log(failures === 0 ? '\nALL PASS' : `\n${failures} FAILURES`)
process.exit(failures === 0 ? 0 : 1)
process.exitCode = failures === 0 ? 0 : 1
9 changes: 7 additions & 2 deletions scripts/agent-customize/providers/claude.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -167,12 +167,17 @@ async function collectProjectRules(workspace, sourceLabel, claudeRoot = path.joi
"claude-project",
],
[path.join(workspace, "CLAUDE.local.md"), "CLAUDE.local.md", "claude-local", "claude-local"],
// Claude Code reads a root AGENTS.md when no CLAUDE.md claims the project.
// See https://code.claude.com/docs/en/memory#agents-md.
[path.join(workspace, "AGENTS.md"), "AGENTS.md", "agents-md-compat", "agents-md-compat"],
];
const files = [];
const visited = new Set();
for (const [filePath, name, sourceKind, precedence] of fileDefinitions) {
if (visited.has(filePath)) continue;
visited.add(filePath);
const resolvedPath = path.resolve(filePath);
const key = await realpath(resolvedPath).catch(() => resolvedPath);
if (visited.has(key)) continue;
visited.add(key);
files.push(...await collectMarkdownFileRuleItem(filePath, "project", sourceLabel, workspace, {
name,
sourceKind,
Expand Down
51 changes: 5 additions & 46 deletions scripts/agent-customize/providers/qoder.mjs
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
import os from "node:os";
import path from "node:path";

import {
resolveQoderAssetHome,
resolveQoderSharedClientCacheRoot,
} from "../../host-support/qoder-home.mjs";
import { expandHome, normalizeWorkspace, pathExists } from "../../session-analysis/index.mjs";
import { MANAGE_TABS } from "../constants.mjs";
import {
Expand Down Expand Up @@ -32,50 +35,6 @@ import {

const QODER_PLUGIN_MANIFEST = [".qoder-plugin", "plugin.json"];

function defaultQoderHome() {
return path.join(os.homedir(), ".qoder");
}

function defaultQoderSharedClientCacheRoot() {
if (process.platform === "win32") {
const appData = process.env.APPDATA ?? path.join(os.homedir(), "AppData", "Roaming");
return path.join(appData, "Qoder", "SharedClientCache");
}
if (process.platform === "darwin") {
return path.join(os.homedir(), "Library", "Application Support", "Qoder", "SharedClientCache");
}
const xdgConfig = process.env.XDG_CONFIG_HOME ?? path.join(os.homedir(), ".config");
return path.join(xdgConfig, "Qoder", "SharedClientCache");
}

function looksLikeQoderAssetHome(candidate) {
const base = path.basename(String(candidate ?? "")).toLowerCase();
return base === ".qoder" || base === "qoder";
}

function resolveQoderAssetHome(options = {}) {
const explicit = options.qoderHome ?? options["qoder-home"];
if (explicit) return path.resolve(expandHome(explicit));
const environmentHome = process.env.QODER_HOME;
if (environmentHome && looksLikeQoderAssetHome(environmentHome)) {
return path.resolve(expandHome(environmentHome));
}
return defaultQoderHome();
}

function resolveQoderSharedClientCacheRoot(options = {}) {
const explicit =
options.qoderSharedClientCacheRoot ??
options["qoder-shared-client-cache-root"] ??
options.sharedClientCacheRoot;
if (explicit) return path.resolve(expandHome(explicit));
const environmentHome = process.env.QODER_HOME;
if (environmentHome && !looksLikeQoderAssetHome(environmentHome)) {
return path.resolve(expandHome(environmentHome));
}
return defaultQoderSharedClientCacheRoot();
}

export function qoderWorkspaceSlugs(workspace) {
const expanded = expandHome(workspace ?? process.cwd());
const normalized = path.win32.isAbsolute(expanded)
Expand Down Expand Up @@ -149,7 +108,7 @@ function mergeQoderInstalledRecord(existing, next) {
}

async function readQoderInstalledPluginState(options = {}) {
const qoderHome = path.resolve(expandHome(options.qoderHome ?? defaultQoderHome()));
const qoderHome = resolveQoderAssetHome(options);
const workspace = normalizeWorkspace(options.workspace ?? process.cwd());
if (Array.isArray(options.qoderInstalledPluginRecords) || Array.isArray(options.installedPluginRecords)) {
const records = (options.qoderInstalledPluginRecords ?? options.installedPluginRecords)
Expand Down
3 changes: 2 additions & 1 deletion scripts/agent-lint/host-instructions.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import { readdir, readFile } from "node:fs/promises";
import os from "node:os";
import path from "node:path";

import { resolveQoderAssetHome } from "../host-support/qoder-home.mjs";
import { expandHome, normalizeWorkspace, pathExists } from "../session-analysis/index.mjs";

const MARKDOWN_EXTENSIONS = new Set([".md", ".mdc", ".markdown"]);
Expand Down Expand Up @@ -61,7 +62,7 @@ function ancestors(workspace) {

async function qoderSources(options) {
const workspace = normalizeWorkspace(options.workspace);
const home = path.resolve(expandHome(options.qoderHome ?? options["qoder-home"] ?? process.env.QODER_HOME ?? "~/.qoder"));
const home = resolveQoderAssetHome(options);
const candidates = [];
if (options.includeUserHome ?? options["include-user-home"]) {
for (const filePath of await markdownFiles(path.join(home, "rules"), 4)) {
Expand Down
16 changes: 13 additions & 3 deletions scripts/coding-agent-practices/checkup/sources.mjs
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
import os from "node:os";
import path from "node:path";

import {
looksLikeQoderAssetHome,
resolveQoderAssetHome,
} from "../../host-support/qoder-home.mjs";
import { expandHome, pathExists } from "../../session-analysis/index.mjs";

function resolved(value) {
Expand Down Expand Up @@ -30,8 +33,15 @@ export async function collectQoderSourceResolution(options = {}, inventory = {})

const explicitHome = options.qoderHome;
const environmentHome = process.env.QODER_HOME;
const environmentIsAssetHome = looksLikeQoderAssetHome(environmentHome);
const inventoryHome = inventory.qoderHome;
const activeHome = resolved(explicitHome ?? environmentHome ?? inventoryHome ?? path.join(os.homedir(), ".qoder"));
const activeHome = explicitHome
? resolved(explicitHome)
: environmentIsAssetHome
? resolved(environmentHome)
: inventoryHome
? resolved(inventoryHome)
: resolveQoderAssetHome();
const workspace = path.resolve(options.workspace ?? process.cwd());
const sharedRoot = inventory.sharedClientCacheRoot
? path.resolve(inventory.sharedClientCacheRoot)
Expand Down Expand Up @@ -81,7 +91,7 @@ export async function collectQoderSourceResolution(options = {}, inventory = {})
state: "selected",
activeHomeEvidence: explicitHome
? "explicit-option"
: environmentHome
: environmentIsAssetHome
? "environment"
: inventoryHome
? "inventory-resolver"
Expand Down
11 changes: 6 additions & 5 deletions scripts/coding-agent-practices/inventory.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@ import {
walkFiles,
} from "../session-analysis/index.mjs";
import { collectAgentCustomizeInventory } from "../agent-customize/index.mjs";
import {
looksLikeQoderAssetHome,
resolveQoderAssetHome,
} from "../host-support/qoder-home.mjs";
import {
getHostDescriptor,
HOST_CAPABILITIES,
Expand Down Expand Up @@ -439,11 +443,8 @@ function publicScope(options = {}, dependencies = {}) {
}, dependencies);
const { workspace, cwd } = configuredScope;
const environmentHome = process.env.QODER_HOME;
const environmentBase = path.basename(String(environmentHome ?? "")).toLowerCase();
const environmentIsAssetHome = environmentBase === ".qoder" || environmentBase === "qoder";
const qoderHome = path.resolve(expandHome(
options.qoderHome ?? options["qoder-home"] ?? (environmentIsAssetHome ? environmentHome : "~/.qoder"),
));
const environmentIsAssetHome = looksLikeQoderAssetHome(environmentHome);
const qoderHome = resolveQoderAssetHome(options);
const explicitSharedCache = options.sharedCache ?? options["shared-cache"];
const sharedCache = explicitSharedCache
? path.resolve(expandHome(explicitSharedCache))
Expand Down
Loading
Loading