Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,8 +90,10 @@ the target journey, contract, evidence, governance, and DX-measurement system.
cross-profile host-id and alias conflicts. `profile-builders.mjs` owns typed
construction helpers for evidence, operations, surfaces, and steps rather
than duplicating their invariants, and `profiles.mjs` only composes the
registry. Every provider-specific filesystem root read outside the selected
workspace must have an explicit `inventoryHomeRoutes` entry, including state
registry. `qoder-home.mjs` owns the shared Qoder asset/session home precedence
used by analysis and inventory consumers. Every provider-specific filesystem
root read outside the selected workspace must have an explicit
`inventoryHomeRoutes` entry, including state
files, shared caches, and compatibility roots such as a user-level
`.agents/` directory. Each primary and secondary route declares its provider
option, an isolated path relative to `--host-home`, and a redacted safe
Expand Down
89 changes: 89 additions & 0 deletions docs/specs/2026-10-24-harness-analysis-defects.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# Harness analysis host, history, instruction, and lead-binding defects

## Traceability

- Spec ID: 2026-10-24-harness-analysis-defects
- Story: QoderAI/better-harness#212, QoderAI/better-harness#195, QoderAI/better-harness#164
- Status: Implemented

## Intent

Repair the confirmed analysis defects found during Windows-hosted validation. The
defects fall into five groups: Qoder home discovery and CLI documentation,
bounded history windows that collapse to the same sample, instruction and gate
discovery gaps, an incomplete Claude Rules projection, and a missing regression
for the already-correct lead/session binding path. The change also brings the
file-explorer fixture in line with Windows environment and stream-lifecycle
behavior.

This work prefers honest coverage metadata over confident-looking but incomplete
data. A truncated or low-confidence history window must be visible as such, and
analysis must continue to fail closed when real binding conflicts exist.

## Acceptance scenarios

- AC-1: `--qoder-home` remains explicit and wins over environment/defaults, and
help for `harness analyze`, `harness evidence-bundle`, and task-loop source
advertises the option. `QODER_HOME` is honored by the Qoder session analyzer,
including a `.qoder-cn` asset home, and the default selects an existing
`~/.qoder-cn` before `~/.qoder`.
- AC-2: history windows use independent git ranges instead of slicing one global
commit list, so 30/90/180-day windows can report different commit counts and
hotspots. Every window exposes its commit limit, truncation state, and coverage
status.
- AC-3: `history-window-coverage` is `review` rather than `ready` when history
confidence is low or any window is truncated, while retaining the existing
ready behavior for complete high/medium-confidence history.
- AC-4: instruction analysis recognizes root and nested `AGENTS.md` and
`CLAUDE.md` carriers, excludes scratch directories from suggested scopes, and
reports carrier kinds without changing the existing density thresholds.
- AC-5: evidence packs expose project-owned package and Just gates, and the
`validate-impact` action includes those exact commands when available.
- AC-6: Claude project Rules inventory includes a root `AGENTS.md` compatibility
entry without duplicating `CLAUDE.md`.
- AC-7: an integration test proves an injected session population keeps the lead
lane available and carries `sessionBinding`, while the genuine conflict test
still downgrades the lead to `SESSION_POPULATION_BINDING_MISMATCH`.
- AC-8: the file-explorer test sets both `HOME` and `USERPROFILE` on Windows and
shuts down without asserting while child-process pipes are still closing.

## Non-goals

- Adding new host adapters or changing host support levels.
- Replacing the bounded `maxCommits` safety limit or adding unbounded history
scans.
- Redesigning report/Canvas UI or changing the binding reconciliation contract.
- Treating scratch or generated files as instruction carriers.

## Plan and tasks

1. Align Qoder home resolution across asset and session analysis, and document
the accepted host-home option.
2. Fetch and filter each history window independently, add coverage metadata,
and make the review matrix consume it.
3. Extend instruction carriers, scope filtering, and project gate extraction.
4. Add root `AGENTS.md` to the Claude Rules inventory.
5. Add a lead/session binding integration regression and keep the conflict test.
6. Make the file-explorer fixture portable and close child streams explicitly.
7. Run focused tests, then the broader affected suites.

## Test and review evidence

- `test/agents/coding-agent-practices-inventory.test.mjs` covers Qoder asset
homes, `QODER_HOME` separation, and an implicit `.qoder-cn` home.
- `test/agents/agent-customize.test.mjs` covers Qoder provider `QODER_HOME`
resolution and the Claude Rules root `AGENTS.md` compatibility entry.
- `test/sessions/session-analysis.test.mjs` exercises Qoder session roots from
`QODER_HOME` and the `~/.qoder-cn` default.
- `test/governance/core-change-watch.test.mjs` covers independent history
windows, truncation downgrade, instruction carriers, scratch-scope exclusion,
and project gates in the `validate-impact` action.
- `test/reporting/better-harness-evidence-bundle.test.mjs` exercises the real
Claude lead/session binding path and keeps the genuine conflict regression.
- `packages/harness-studio-apps/test/test_file_explorer.mjs` passes on Windows
with `HOME`/`USERPROFILE` isolation and orderly child shutdown.
- `test/support/path-links.mjs` keeps Windows fixtures portable with directory
junctions and skips only the two file-symlink cases when the host denies them.
- The DSH explicit-root fixture isolates `HOME`/`USERPROFILE`, so external roots
are not accidentally classified as user-home paths.
- Full `npm test` on 2026-10-25: 1765 passed, 30 skipped, 0 failed.
15 changes: 13 additions & 2 deletions packages/harness-studio-apps/test/test_file_explorer.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ function startBackend() {
env: {
...process.env,
HOME: HOME_DIR,
USERPROFILE: HOME_DIR,
KIROCREW_HOME: HOME_DIR,
KIROCREW_PROXY_SECRET: SECRET,
PORT: String(PORT),
Expand Down Expand Up @@ -72,6 +73,16 @@ async function call(path) {
return fetch(`http://127.0.0.1:${PORT}${path}`, { headers: { 'X-KiroCrew-Proxy': sig } })
}

async function stopBackend(backend) {
if (backend.exitCode === null && backend.signalCode === null) {
const exited = new Promise((resolve) => backend.once('exit', resolve))
backend.kill('SIGTERM')
await Promise.race([exited, new Promise((resolve) => setTimeout(resolve, 2000))])
}
backend.stdout?.destroy()
backend.stderr?.destroy()
}

const child = startBackend()

try {
Expand Down Expand Up @@ -112,8 +123,8 @@ try {
r = await call(`${BASE}/nope`)
check('unknown route -> 404', r.status === 404, `got ${r.status}`)
} finally {
child.kill('SIGTERM')
await stopBackend(child)
}

console.log(failures === 0 ? '\nALL PASS' : `\n${failures} FAILURES`)
process.exit(failures === 0 ? 0 : 1)
process.exitCode = failures === 0 ? 0 : 1
1 change: 1 addition & 0 deletions scripts/agent-customize/providers/claude.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,7 @@ async function collectProjectRules(workspace, sourceLabel, claudeRoot = path.joi
"claude-project",
],
[path.join(workspace, "CLAUDE.local.md"), "CLAUDE.local.md", "claude-local", "claude-local"],
[path.join(workspace, "AGENTS.md"), "AGENTS.md", "agents-md-compat", "agents-md-compat"],
];
const files = [];
const visited = new Set();
Expand Down
51 changes: 5 additions & 46 deletions scripts/agent-customize/providers/qoder.mjs
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
import os from "node:os";
import path from "node:path";

import {
resolveQoderAssetHome,
resolveQoderSharedClientCacheRoot,
} from "../../host-support/qoder-home.mjs";
import { expandHome, normalizeWorkspace, pathExists } from "../../session-analysis/index.mjs";
import { MANAGE_TABS } from "../constants.mjs";
import {
Expand Down Expand Up @@ -32,50 +35,6 @@ import {

const QODER_PLUGIN_MANIFEST = [".qoder-plugin", "plugin.json"];

function defaultQoderHome() {
return path.join(os.homedir(), ".qoder");
}

function defaultQoderSharedClientCacheRoot() {
if (process.platform === "win32") {
const appData = process.env.APPDATA ?? path.join(os.homedir(), "AppData", "Roaming");
return path.join(appData, "Qoder", "SharedClientCache");
}
if (process.platform === "darwin") {
return path.join(os.homedir(), "Library", "Application Support", "Qoder", "SharedClientCache");
}
const xdgConfig = process.env.XDG_CONFIG_HOME ?? path.join(os.homedir(), ".config");
return path.join(xdgConfig, "Qoder", "SharedClientCache");
}

function looksLikeQoderAssetHome(candidate) {
const base = path.basename(String(candidate ?? "")).toLowerCase();
return base === ".qoder" || base === "qoder";
}

function resolveQoderAssetHome(options = {}) {
const explicit = options.qoderHome ?? options["qoder-home"];
if (explicit) return path.resolve(expandHome(explicit));
const environmentHome = process.env.QODER_HOME;
if (environmentHome && looksLikeQoderAssetHome(environmentHome)) {
return path.resolve(expandHome(environmentHome));
}
return defaultQoderHome();
}

function resolveQoderSharedClientCacheRoot(options = {}) {
const explicit =
options.qoderSharedClientCacheRoot ??
options["qoder-shared-client-cache-root"] ??
options.sharedClientCacheRoot;
if (explicit) return path.resolve(expandHome(explicit));
const environmentHome = process.env.QODER_HOME;
if (environmentHome && !looksLikeQoderAssetHome(environmentHome)) {
return path.resolve(expandHome(environmentHome));
}
return defaultQoderSharedClientCacheRoot();
}

export function qoderWorkspaceSlugs(workspace) {
const expanded = expandHome(workspace ?? process.cwd());
const normalized = path.win32.isAbsolute(expanded)
Expand Down Expand Up @@ -149,7 +108,7 @@ function mergeQoderInstalledRecord(existing, next) {
}

async function readQoderInstalledPluginState(options = {}) {
const qoderHome = path.resolve(expandHome(options.qoderHome ?? defaultQoderHome()));
const qoderHome = resolveQoderAssetHome(options);
const workspace = normalizeWorkspace(options.workspace ?? process.cwd());
if (Array.isArray(options.qoderInstalledPluginRecords) || Array.isArray(options.installedPluginRecords)) {
const records = (options.qoderInstalledPluginRecords ?? options.installedPluginRecords)
Expand Down
3 changes: 2 additions & 1 deletion scripts/agent-lint/host-instructions.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import { readdir, readFile } from "node:fs/promises";
import os from "node:os";
import path from "node:path";

import { resolveQoderAssetHome } from "../host-support/qoder-home.mjs";
import { expandHome, normalizeWorkspace, pathExists } from "../session-analysis/index.mjs";

const MARKDOWN_EXTENSIONS = new Set([".md", ".mdc", ".markdown"]);
Expand Down Expand Up @@ -61,7 +62,7 @@ function ancestors(workspace) {

async function qoderSources(options) {
const workspace = normalizeWorkspace(options.workspace);
const home = path.resolve(expandHome(options.qoderHome ?? options["qoder-home"] ?? process.env.QODER_HOME ?? "~/.qoder"));
const home = resolveQoderAssetHome(options);
const candidates = [];
if (options.includeUserHome ?? options["include-user-home"]) {
for (const filePath of await markdownFiles(path.join(home, "rules"), 4)) {
Expand Down
11 changes: 6 additions & 5 deletions scripts/coding-agent-practices/inventory.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@ import {
walkFiles,
} from "../session-analysis/index.mjs";
import { collectAgentCustomizeInventory } from "../agent-customize/index.mjs";
import {
looksLikeQoderAssetHome,
resolveQoderAssetHome,
} from "../host-support/qoder-home.mjs";
import {
getHostDescriptor,
HOST_CAPABILITIES,
Expand Down Expand Up @@ -439,11 +443,8 @@ function publicScope(options = {}, dependencies = {}) {
}, dependencies);
const { workspace, cwd } = configuredScope;
const environmentHome = process.env.QODER_HOME;
const environmentBase = path.basename(String(environmentHome ?? "")).toLowerCase();
const environmentIsAssetHome = environmentBase === ".qoder" || environmentBase === "qoder";
const qoderHome = path.resolve(expandHome(
options.qoderHome ?? options["qoder-home"] ?? (environmentIsAssetHome ? environmentHome : "~/.qoder"),
));
const environmentIsAssetHome = looksLikeQoderAssetHome(environmentHome);
const qoderHome = resolveQoderAssetHome(options);
const explicitSharedCache = options.sharedCache ?? options["shared-cache"];
const sharedCache = explicitSharedCache
? path.resolve(expandHome(explicitSharedCache))
Expand Down
32 changes: 26 additions & 6 deletions scripts/core-change-watch/evidence-pack.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ function supportingFilesByRole(history, diff, role, currentPaths) {
return actionFiles([...fromDiff, ...fromHistory], 8, currentPaths);
}

function buildFollowUpActions(core, diff, history, changeDrift, maxActions, currentPaths) {
function buildFollowUpActions(core, diff, history, changeDrift, projectProfile, maxActions, currentPaths) {
const actions = [];
const topCoreFiles = actionFiles(core.candidates.slice(0, 4).flatMap((candidate) => [
...candidate.evidence.hotFiles,
Expand Down Expand Up @@ -201,7 +201,8 @@ function buildFollowUpActions(core, diff, history, changeDrift, maxActions, curr
});
}

if (testFiles.length > 0 || diff.reviewRecommended || topCoreFiles.length > 0) {
const gates = projectProfile.projectGates ?? [];
if (testFiles.length > 0 || gates.length > 0 || diff.reviewRecommended || topCoreFiles.length > 0) {
actions.push({
id: "validate-impact",
type: "validation",
Expand All @@ -211,8 +212,11 @@ function buildFollowUpActions(core, diff, history, changeDrift, maxActions, curr
? "Diff impact requires attention because the change touches configured or detected core code."
: testFiles.length > 0
? "Detected tests or fixtures can anchor validation."
: "Core candidates need focused validation after implementation changes.",
: gates.length > 0
? "Project-owned validation gates were discovered from tracked manifests and Justfiles."
: "Core candidates need focused validation after implementation changes.",
files: testFiles,
...(gates.length > 0 ? { commands: gates } : {}),
passCheck: "Relevant focused tests, build, lint, smoke, or caller/API checks pass or failures are reported with exact output.",
});
}
Expand Down Expand Up @@ -352,6 +356,7 @@ function summarizeAgentInstructions(projectProfile) {
count: instructions.count,
rootCount: instructions.rootCount,
nestedCount: instructions.nestedCount,
carriers: instructions.carriers,
suggestedMinimum: instructions.suggestedMinimum,
suggestedAdditional: instructions.suggestedAdditional,
sourceFilesPerInstruction: instructions.sourceFilesPerInstruction,
Expand Down Expand Up @@ -396,9 +401,19 @@ function buildReviewMatrix(projectProfile, historyProfile, coreAnalysis, diffImp
matrixItem(
"history-window-coverage",
"History uses 30/90/180 style windows with confidence notes",
historyProfile.historyWindows.length >= 3 && historyProfile.confidence ? "ready" : "review",
{ windows: historyProfile.historyWindows.map((item) => item.days), confidence: historyProfile.confidence?.confidence ?? "unknown" },
"Low history confidence should downgrade hotspot certainty, not block source inspection.",
historyProfile.historyWindows.length >= 3 &&
historyProfile.confidence &&
historyProfile.confidence.confidence !== "low" &&
historyProfile.historyWindows.every((item) => item.coverage === "complete")
? "ready"
: "review",
{
windows: historyProfile.historyWindows.map((item) => item.days),
confidence: historyProfile.confidence?.confidence ?? "unknown",
truncatedWindows: historyProfile.historyWindows.filter((item) => item.truncated).map((item) => item.days),
coverage: historyProfile.historyWindows.map((item) => ({ days: item.days, status: item.coverage })),
},
"Low history confidence or a bounded-window truncation should downgrade hotspot certainty, not block source inspection.",
),
matrixItem(
"large-repo-safety",
Expand Down Expand Up @@ -492,6 +507,9 @@ export async function buildEvidencePack(options = {}) {
days,
since: null,
analyzedCommits: 0,
commitLimit: 0,
truncated: false,
coverage: "skipped",
trendLabel: days <= 30 ? "recent-hotspots" : days <= 90 ? "sustained-hotspots" : "legacy-or-long-range-hotspots",
interpretation: "History scan skipped; this window is present for schema stability only.",
hotFiles: [],
Expand Down Expand Up @@ -542,6 +560,7 @@ export async function buildEvidencePack(options = {}) {
diffImpact,
historyProfile,
changeDrift,
projectProfile,
maxFollowUpActions,
currentPaths,
);
Expand All @@ -562,6 +581,7 @@ export async function buildEvidencePack(options = {}) {
topLanguages: projectProfile.languages.slice(0, 5).map((item) => item.language),
projectInfo: summarizeProjectInfo(projectProfile),
agentInstructions: summarizeAgentInstructions(projectProfile),
projectGates: projectProfile.projectGates,
coreCandidates: coreAnalysis.summary,
changedFiles: diffImpact.metrics.changedFiles,
changedLines: diffImpact.metrics.changedLines,
Expand Down
Loading
Loading