From 9dc749e13ca9aed916d5f5e8a956024a8582bf7d Mon Sep 17 00:00:00 2001 From: Norm Brandinger Date: Tue, 3 Nov 2026 16:32:39 -0500 Subject: [PATCH 1/6] Hold a TFPS action whose answer was lost in doubt POST /v1/tfps/ban against a TFPS whose tfps_ctl wrote `expires` as an RFC 3339 string: TFPS applied the ban, sipnab answered 502 "TFPS could not be asked: ... cannot read", and journaled the outcome as `failed`. Every later unban answered 409 "sipnab holds no ban on this address", admin token included. sipnab had lost track of a ban it caused, and no surface could lift it. Root cause: once tfps_ctl has run, TFPS may have acted whatever becomes of its answer, but action_reply() turned every TfpsError into the same Err, and finish() journaled every Err as `failed`. The ledger reads `failed` as "did not happen": it drops the intent from in-doubt and takes no effect. So a timeout, a non-zero exit with no result, too much output, or an answer outside the JSON contract all recorded an unknown effect as no effect. The parse error only exposed it. Only a tfps_ctl that never started (TfpsError::Spawn) is known to have done nothing. Fix: - TfpsError::ran(): every error but Spawn. action_reply() maps an error from a peer that ran to TfpsReply::Unanswered. - outcome_of(), one rule for action and revert records: Unanswered is journaled as `unknown`; `failed` is kept for a peer that never ran. - The ledger keeps an action whose outcome is `unknown` in doubt. It carries its address, caller and time, and `sipnab --journal-show` lists it. - unban(): an action in doubt on the address being unbanned no longer refuses the unban (one on any other address still does). The caller of a ban in doubt may lift it, as may an admin token; anyone else gets 403 naming the owner. TFPS's answer settles the doubt with `reconciled` records: an applied unban means the address was banned; not-blocked means it was not; any other answer leaves the doubt in place. - ActionError::Unanswered: still 502 (REST) and refusal `tfps` (MCP), now worded "TFPS was asked, and sipnab could not read what it did", and saying the action is held in doubt and how to settle it. InDoubt's wording covers both causes. - An unparseable answer names the contract sipnab reads: sippulse/tfps e7ee57a (2026-09-16) and later, where every time is epoch seconds; the draft of that contract (before 54dc12d on json-output-contract-history) wrote `expires` as RFC 3339. sipnab does not parse RFC 3339. Other mutating actions: TFPS ban and unban are the only action target (ActionTarget::Tfps); revert sends an unban and now records a lost answer as `unknown` too. The kill-scanner packet expects no answer. Tests (written first, seen failing for the right reason): - tests/actions_tfps_ctl_test.rs: a fake tfps_ctl that bans and then answers garbage, a pre-contract RFC 3339 answer, a non-zero exit, or hangs past the timeout gives Unanswered; one that cannot start is still an error. - tests/actions_service_test.rs: the lost ban is in doubt with address, caller and time; the caller and an admin can lift it and TFPS settles it; another caller gets NotYours and TFPS is not asked; not-blocked settles it as never applied; a lost unban answer keeps the doubt; other actions wait; a restart resolves it against TFPS's list; a revert whose answer is lost is held in doubt too. - sipnab --journal-show and the InDoubt refusal now name both causes of doubt and the unban that settles one. - tests/actions_journal_rest_test.rs: over REST with the real binary, for garbage, a non-zero exit and a timeout, each lifted by its caller and by an admin: 502 saying in doubt, outcome `unknown`, the unban reaches TFPS, and a new ban runs afterwards. - tests/support/server.rs gains post_json_bearer_waiting: the hang case answers only after sipnab's own 10 s tfps_ctl timeout, which the shared client's 10 s read timeout could not outlast (it failed with EAGAIN under load). The ban step waits 30 s; under the ledger mutation the hang cases now fail on the 409 the report describes. - tfps_failing_is_a_bad_gateway_and_the_outcome_is_journaled_as_failed asserted the defect itself (a tfps_ctl that ran and exited 1 with no result journaled as `failed`); it now asserts `unknown` and is renamed. Found against a real TFPS during the 2026-11-03 documentation review. Ratchet (tests/rust_style_known.tsv), per file: - AssertWithoutMessage tests/actions_journal_rest_test.rs 15 -> 14: the corrected `unknown` assertion carries a message; the `failed` one it replaces did not. (cherry picked from commit d032017a3dc84bd2f0b2b0a31f783c854dc935ca) --- CHANGELOG.md | 1 + docs/mcp-tools-tfps.md | 21 +- docs/rest-api.md | 19 +- docs/tfps-sipnab.md | 13 ++ src/app/journal_cli.rs | 6 +- src/journal/ledger.rs | 7 +- src/mcp/tools/tfps.rs | 2 +- src/output/api.rs | 2 +- src/security/actions/service.rs | 131 +++++++++--- src/security/tfps.rs | 19 +- tests/actions_journal_rest_test.rs | 115 ++++++++++- tests/actions_service_test.rs | 274 ++++++++++++++++++++++++- tests/actions_tfps_ctl_test.rs | 95 +++++++++ tests/rust_style_known.tsv | 2 +- tests/support/fake_tfps.rs | 16 ++ tests/support/fake_tfps_ctl.rs | 25 ++- tests/support/server.rs | 34 ++- website/content/docs/api.md | 19 +- website/content/docs/mcp-tools-tfps.md | 21 +- website/content/docs/tfps-sipnab.md | 13 ++ website/static/llms-full.txt | 53 ++++- 21 files changed, 827 insertions(+), 61 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index aaed8831d..328d126d9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,7 @@ entry that carries them. ### Fixed - **A `[[vcon.rule]]` writes its containers on a run that also serves `--api` or `--mcp`.** The persistence gate both doors share took its ceiling from the export flags alone (`--export-vcon`, `--export-vcon-when`), so on a run whose only writer was a configured rule the gate was never authorized. The exporter consults that gate, so the rule wrote nothing and printed nothing, and `GET /v1/persistence` answered `{"authorized":false,"enabled":false}`; the same config and capture without a door wrote the call. `Cli::persists_content` now reads the config file as well as the flags, and it is the one answer the gate and the `--redact` refusal both use. A rule authorizes the gate as a flag does, so `POST /v1/persistence` can close and reopen it. Found during the 2026-11-03 documentation review. +- **A ban TFPS applied but whose answer sipnab could not read is held in doubt, and can be lifted.** Once `tfps_ctl` has run, TFPS may have acted, whatever became of its answer: a timeout, a non-zero exit with no result, or an answer that is not the JSON contract. sipnab journaled each of those as `failed`, which its ledger reads as "did not happen". So the ban stood in TFPS, the journal held nothing to act on, and every unban answered `409` "sipnab holds no ban on this address", with an `admin` token too. The outcome is now journaled as `unknown`, and the ledger keeps the action in doubt. `sipnab --journal-show` lists it with its address, caller and time. The caller who asked, or a token with the `admin` scope, can unban the address over REST or MCP. The unban goes to TFPS, and TFPS's answer settles the doubt: it lifted a ban, or it held none. Only a `tfps_ctl` that never started is still `failed`. The `502` still says what went wrong, now as "TFPS was asked, and sipnab could not read what it did" rather than "TFPS could not be asked", and an answer sipnab cannot read names the JSON contract it reads: sippulse/tfps `e7ee57a` and later, where every time is epoch seconds. A revert whose answer is lost is held in doubt the same way. Found against a real TFPS whose `tfps_ctl` gave `expires` as an [RFC 3339](https://www.rfc-editor.org/rfc/rfc3339.html) string. - **A release build stays reproducible on a host that configures a compiler wrapper.** With sccache set as `build.rustc-wrapper`, the `cc` crate routed mimalloc's C compile through the sccache server, whose clock, not `SOURCE_DATE_EPOCH`, set `__DATE__` and `__TIME__`, so two builds of one commit differed (run 41514782899). `scripts/reproducible-build.sh build` now runs cargo with `RUSTC_WRAPPER`, `RUSTC_WORKSPACE_WRAPPER` and their `CARGO_BUILD_` forms set to the empty string, which overrides a configured wrapper. ## [0.5.232] - 2026-11-03 diff --git a/docs/mcp-tools-tfps.md b/docs/mcp-tools-tfps.md index 0bc82581f..ea71f7121 100644 --- a/docs/mcp-tools-tfps.md +++ b/docs/mcp-tools-tfps.md @@ -286,9 +286,20 @@ it: } ``` -`refusal` is `rate` (with `retry_after_secs`), `in_doubt` (actions a crash -left unfinished are waiting for TFPS to answer for them), `journal` (sipnab -cannot use the journal), or `tfps` (sipnab could not ask TFPS). +`refusal` is `rate` (with `retry_after_secs`), `in_doubt`, `journal` (sipnab +cannot use the journal), or `tfps`. + +- `in_doubt`: actions wait for TFPS to answer for them, or for an unban of + their address. Either sipnab stopped before TFPS answered, or TFPS was + asked and sipnab could not read its answer. +- `tfps`: either sipnab could not ask TFPS, or it asked and could not read + the answer. + - In the second case TFPS may have acted. That covers a timeout, a + non-zero exit with no result, and an answer that is not the JSON TFPS + has given since [`e7ee57a`](https://github.com/sippulse/tfps/commit/e7ee57a). + - The journal then holds the ban in doubt, and the message says so. + - The caller who asked, or a token with the `admin` scope, can lift it + with `tfps_unban`, and TFPS's answer settles it. The tool is `readOnlyHint: false` and `destructiveHint: true`, so a host that confirms destructive calls asks before this one. `idempotentHint: true`, @@ -299,8 +310,8 @@ because banning a banned source changes nothing. Ask TFPS to release a source **sipnab banned**. Off by default, behind the same enabling, rules and limits as `tfps_ban`. -sipnab lifts only a ban it placed and still holds, as its journal records. A -ban TFPS placed itself, or anyone else did, is an error result with `refusal` +sipnab lifts only a ban it placed and still holds, as its journal records, +or one it asked for and holds in doubt. A ban TFPS placed itself, or anyone else did, is an error result with `refusal` `not_owned`, and sipnab does not ask TFPS. A ban belongs to the caller that placed it: another caller gets `refusal` `not_yours`, and the message names the owner, unless its token holds the `admin` scope. Because an address rests for a diff --git a/docs/rest-api.md b/docs/rest-api.md index a89cd3c92..2638f72d8 100644 --- a/docs/rest-api.md +++ b/docs/rest-api.md @@ -2270,10 +2270,24 @@ That is the answer TFPS gave, not an error: `tfps_ctl` missing or failing is `502`, with its reason in `detail`. +- If `tfps_ctl` could not start, TFPS did nothing, and the journal records + the ban as failed. +- If it ran and sipnab could not read its answer, TFPS may have banned the + address. That covers a timeout, a non-zero exit with no result, and an + answer that is not the JSON TFPS has given since + [`e7ee57a`](https://github.com/sippulse/tfps/commit/e7ee57a). In that case: + - the journal holds the ban **in doubt**, and `detail` says so; + - `sipnab --journal-show` lists it with its address, its caller and its + time; + - the caller who asked, or a token with scope `admin`, can lift it with + `POST /v1/tfps/unban`, and TFPS's answer settles it. + `503` means no action may run just now. Either: - sipnab cannot use the journal, or -- actions a crash left unfinished are waiting for TFPS to answer for them. +- actions are in doubt: sipnab stopped before TFPS answered for them, or + sipnab asked TFPS and could not read its answer. They wait for TFPS + to answer for them, or for an unban of their address. --- @@ -2282,7 +2296,8 @@ That is the answer TFPS gave, not an error: Ask TFPS to release a source **sipnab banned**. Off by default, behind the same two locks, rules and limits as the ban. -sipnab lifts only a ban it placed and still holds, as its journal records. +sipnab lifts only a ban it placed and still holds, as its journal records, +or one it asked for and holds in doubt. - A ban TFPS placed itself, or anyone else did, answers `409`. TFPS is not asked. diff --git a/docs/tfps-sipnab.md b/docs/tfps-sipnab.md index 35534fd61..cee0c78b3 100644 --- a/docs/tfps-sipnab.md +++ b/docs/tfps-sipnab.md @@ -446,6 +446,19 @@ The `curl` commands from the same-machine section work unchanged. HTTP token has the `full` scope. Mint one with `--token-scope actions`. - **`/v1/tfps/dropped` answers `502`.** Expected: no TFPS build has `dropped` yet. +- **A ban answers `502` with `answered something this sipnab cannot read`.** + - sipnab reads the JSON of + [`e7ee57a`](https://github.com/sippulse/tfps/commit/e7ee57a) and later, + where every time is epoch seconds. A `tfps_ctl` built from the draft of + that JSON gives `expires` as an [RFC 3339](https://www.rfc-editor.org/rfc/rfc3339.html) date string instead, and sipnab + cannot read it. + - TFPS may still have applied the ban, so sipnab holds it in doubt. + `sipnab --journal-show` lists it, and no other action runs until it is + settled. + - Unban the address with the same token, or with a token with scope + `admin`. TFPS's answer settles it. + - Then build TFPS from master, as in the "Check that TFPS answers in JSON" + step. - **`installed: false`, and the reason says `tfps_ctl` was not found on `PATH`.** - The service runs without `--tfps-ctl`, and its `PATH` has no `tfps_ctl`. diff --git a/src/app/journal_cli.rs b/src/app/journal_cli.rs index e4b94fe3a..b20f4d57c 100644 --- a/src/app/journal_cli.rs +++ b/src/app/journal_cli.rs @@ -122,8 +122,10 @@ pub fn show( } else { let _ = writeln!( out, - "In doubt: actions that started and never finished ({}); new actions wait until \ - TFPS answers for them", + "In doubt: actions TFPS has not answered for ({}): sipnab stopped before the \ + answer, or could not read it, so TFPS may have acted. New actions wait until TFPS \ + answers for them, or until their caller, or a token with the admin scope, unbans \ + their address", doubt.len() ); for i in &doubt { diff --git a/src/journal/ledger.rs b/src/journal/ledger.rs index 351fa8d74..57cc925d2 100644 --- a/src/journal/ledger.rs +++ b/src/journal/ledger.rs @@ -8,7 +8,9 @@ //! action path writes (see the approved spec "sipnab Operations Journal"): //! //! * `action_intent` — admitted and about to run; in doubt until resolved; -//! * `action_outcome` — `applied`, `refused` or `failed`; +//! * `action_outcome` — `applied`, `refused`, `failed` (`tfps_ctl` never +//! ran) or `unknown` (it ran and its answer was lost, so the action stays +//! in doubt); //! * `revert_intent` / `revert_outcome` — the same, for an unban that backs //! out an earlier ban; //! * `reconciled` — an in-doubt action resolved against TFPS at startup; @@ -207,6 +209,9 @@ impl Ledger { self.in_doubt.insert(intent.id.clone(), intent); } } + // `unknown`: TFPS was asked and may have acted, so the action + // stays in doubt until TFPS answers for it. + "action_outcome" | "revert_outcome" if body["result"].as_str() == Some("unknown") => {} "action_outcome" | "revert_outcome" => { if let Some(intent) = id().and_then(|id| self.in_doubt.remove(&id)) && body["result"].as_str() == Some("applied") diff --git a/src/mcp/tools/tfps.rs b/src/mcp/tools/tfps.rs index 544e7a9ac..44dc45422 100644 --- a/src/mcp/tools/tfps.rs +++ b/src/mcp/tools/tfps.rs @@ -177,7 +177,7 @@ fn action_refusal(e: ActionError) -> Result { ActionError::NotYours { .. } => ("not_yours", None), ActionError::InDoubt(_) => ("in_doubt", None), ActionError::JournalUnusable(_) => ("journal", None), - ActionError::Tfps(_) => ("tfps", None), + ActionError::Tfps(_) | ActionError::Unanswered(_) => ("tfps", None), }; let mut body = serde_json::json!({ "error": message, "refusal": refusal }); if let Some(secs) = retry_after { diff --git a/src/output/api.rs b/src/output/api.rs index 647242b84..ed76d9056 100644 --- a/src/output/api.rs +++ b/src/output/api.rs @@ -1727,7 +1727,7 @@ fn action_problem(e: crate::security::actions::ActionError) -> Problem { E::InDoubt(_) | E::JournalUnusable(_) => { Problem::detailed(StatusCode::SERVICE_UNAVAILABLE, detail) } - E::Tfps(_) => Problem::detailed(StatusCode::BAD_GATEWAY, detail), + E::Tfps(_) | E::Unanswered(_) => Problem::detailed(StatusCode::BAD_GATEWAY, detail), } } diff --git a/src/security/actions/service.rs b/src/security/actions/service.rs index 4211741f5..430b2b8bd 100644 --- a/src/security/actions/service.rs +++ b/src/security/actions/service.rs @@ -23,7 +23,7 @@ use std::path::{Path, PathBuf}; use std::sync::{Arc, Mutex}; use std::time::{Duration, Instant}; -use crate::journal::ledger::{Intent, Ledger, Owned, TfpsView}; +use crate::journal::ledger::{Intent, Ledger, Owned, Resolution, TfpsView, Verb}; use crate::journal::{Journal, JournalError, JournalLimits, Record}; use super::{ @@ -41,6 +41,10 @@ pub enum TfpsReply { Applied, /// TFPS refused, in its own words. Refused(String), + /// `tfps_ctl` ran, so TFPS may have acted, and sipnab could not read what + /// it did: an answer that is not the JSON contract, a non-zero exit with no + /// result, a timeout, or too much output. Carries what went wrong. + Unanswered(String), } /// The three things the service asks TFPS. A trait so the service can be @@ -97,6 +101,8 @@ fn action_reply( value.refused.unwrap_or_else(|| "unnamed".to_string()), )), Ok(Reply::NotInstalled { reason }) => Err(reason), + // It ran, so TFPS may have acted: the effect is unknown, not absent. + Err(e) if e.ran() => Ok(TfpsReply::Unanswered(e.to_string())), Err(e) => Err(e.to_string()), } } @@ -150,13 +156,18 @@ pub enum ActionError { /// Who placed the ban. owner: String, }, - /// Actions from before a crash are still unresolved; nothing new runs - /// until TFPS can be asked about them. + /// Actions are unresolved -- sipnab stopped before TFPS answered for + /// them, or TFPS was asked and its answer could not be read -- and + /// nothing new runs until TFPS answers for them or their address is + /// unbanned. InDoubt(usize), /// The journal cannot be used, so no action may run. JournalUnusable(String), /// TFPS could not be asked. Tfps(String), + /// TFPS was asked, so it may have acted, and its answer could not be + /// read. The action is held in doubt. Carries what went wrong. + Unanswered(String), } impl std::fmt::Display for ActionError { @@ -178,11 +189,20 @@ impl std::fmt::Display for ActionError { ), Self::InDoubt(n) => write!( f, - "{n} action(s) from before sipnab last stopped are unresolved, because TFPS could \ - not be asked about them; no new action runs until it can" + "{n} action(s) are unresolved: sipnab stopped before TFPS answered for them, or \ + TFPS was asked and sipnab could not read what it did. No new action runs until \ + TFPS answers for them, or until their caller, or a token with the admin scope, \ + unbans their address; `sipnab --journal-show` lists them" ), Self::JournalUnusable(m) => write!(f, "actions are off: {m}"), Self::Tfps(m) => write!(f, "TFPS could not be asked: {m}"), + Self::Unanswered(m) => write!( + f, + "TFPS was asked, and sipnab could not read what it did: {m}. TFPS may have \ + acted, so sipnab holds the action in doubt: `sipnab --journal-show` lists it, \ + no other action runs until it is settled, and the caller who asked, or a \ + token with the admin scope, can unban the address to settle it" + ), } } } @@ -593,19 +613,42 @@ impl ActionService { if let Err(r) = self.policy.permit(ActionTarget::Tfps, surface) { return Err(refuse(&mut inner, "policy", ActionError::NotEnabled(r))); } - let doubt = inner.ledger.in_doubt().len(); - if doubt > 0 { - return Err(refuse(&mut inner, "in_doubt", ActionError::InDoubt(doubt))); + // An action in doubt on THIS address is what an unban here settles, + // so it does not refuse the unban; one on any other address still + // does, as it refuses every action. + let address = ip.to_string(); + let (here, elsewhere): (Vec, Vec) = inner + .ledger + .in_doubt() + .into_iter() + .partition(|i| i.address == address); + if !elsewhere.is_empty() { + return Err(refuse( + &mut inner, + "in_doubt", + ActionError::InDoubt(elsewhere.len()), + )); } inner.ledger.set_now(now_unix); - let Some((owned_id, owner)) = inner + let owned_id = inner.ledger.owned(&address).map(|o| o.id.clone()); + // Who may lift it: the caller of the ban sipnab holds, and of every + // ban left in doubt here. A ban in doubt is the caller's as an owned + // one is: TFPS may hold it, and nobody else may lift it for them. + let owners: Vec = inner .ledger - .owned(&ip.to_string()) - .map(|o| (o.id.clone(), o.caller.clone())) - else { + .owned(&address) + .map(|o| o.caller.clone()) + .into_iter() + .chain( + here.iter() + .filter(|i| i.verb == Verb::Ban) + .map(|i| i.caller.clone()), + ) + .collect(); + if owners.is_empty() { return Err(refuse(&mut inner, "ownership", ActionError::NotOwned)); - }; - if !may_lift(caller, &owner) { + } + if let Some(owner) = owners.into_iter().find(|o| !may_lift(caller, o)) { return Err(refuse( &mut inner, "ownership", @@ -622,18 +665,39 @@ impl ActionService { inner.write( "action_intent", serde_json::json!({ - "id": id, "target": "tfps", "verb": "unban", "address": ip.to_string(), + "id": id, "target": "tfps", "verb": "unban", "address": address, "at": now_unix, "surface": surface.name(), "caller": caller.name, }), )?; let reply = self.tfps.unban(v4); let gone = matches!(&reply, Ok(TfpsReply::Refused(why)) if why == "not-blocked"); + // TFPS's answer settles what was in doubt here: it lifted a ban, so + // the address was banned; or it held none, so it was not. Any other + // answer settles nothing, and the doubt stays. + let was_banned = match &reply { + Ok(TfpsReply::Applied) => Some(true), + _ if gone => Some(false), + _ => None, + }; + if let Some(was_banned) = was_banned { + for intent in &here { + let resolution = if (intent.verb == Verb::Ban) == was_banned { + Resolution::Applied + } else { + Resolution::NotApplied + }; + inner.write( + "reconciled", + serde_json::json!({ "id": intent.id, "resolution": resolution }), + )?; + } + } let done = inner.finish(&id, reply)?; - if gone { + if gone && let Some(owned_id) = owned_id { // TFPS no longer held it: the ban sipnab placed has ended. inner.write( "lapsed_by_peer", - serde_json::json!({ "id": owned_id, "address": ip.to_string(), "at": now_unix }), + serde_json::json!({ "id": owned_id, "address": address, "at": now_unix }), )?; } Ok(done) @@ -820,11 +884,7 @@ impl ActionService { }), )?; let reply = self.tfps.unban(v4); - let (result, detail) = match &reply { - Ok(TfpsReply::Applied) => ("applied", None), - Ok(TfpsReply::Refused(why)) => ("refused", Some(why.clone())), - Err(e) => ("failed", Some(e.clone())), - }; + let (result, detail) = outcome_of(&reply); inner.write( "revert_outcome", serde_json::json!({ "id": id, "result": result, "detail": detail }), @@ -844,10 +904,13 @@ impl ActionService { address: owned.address.clone(), why, }), + Ok(TfpsReply::Unanswered(e)) if matches!(target, RevertTarget::One(_)) => { + return Err(ActionError::Unanswered(e)); + } Err(e) if matches!(target, RevertTarget::One(_)) => { return Err(ActionError::Tfps(e)); } - Err(e) => report.failed.push(RevertFailure { + Ok(TfpsReply::Unanswered(e)) | Err(e) => report.failed.push(RevertFailure { id: owned.id.clone(), address: owned.address.clone(), why: e, @@ -1059,11 +1122,7 @@ impl Inner { id: &str, reply: Result, ) -> Result { - let (result, detail) = match &reply { - Ok(TfpsReply::Applied) => ("applied", None), - Ok(TfpsReply::Refused(why)) => ("refused", Some(why.clone())), - Err(e) => ("failed", Some(e.clone())), - }; + let (result, detail) = outcome_of(&reply); self.write( "action_outcome", serde_json::json!({ "id": id, "result": result, "detail": detail }), @@ -1079,11 +1138,27 @@ impl Inner { applied: false, refused: Some(why), }), + Ok(TfpsReply::Unanswered(e)) => Err(ActionError::Unanswered(e)), Err(e) => Err(ActionError::Tfps(e)), } } } +/// The `result` and `detail` an outcome record carries for `reply`. +/// +/// `unknown` for an answer that was lost after `tfps_ctl` ran: the ledger +/// keeps that action in doubt, because TFPS may have acted. `failed` only for +/// a `tfps_ctl` that never ran, which the ledger reads as "did not happen". +/// One rule for the action and the revert records alike. +fn outcome_of(reply: &Result) -> (&'static str, Option) { + match reply { + Ok(TfpsReply::Applied) => ("applied", None), + Ok(TfpsReply::Refused(why)) => ("refused", Some(why.clone())), + Ok(TfpsReply::Unanswered(e)) => ("unknown", Some(e.clone())), + Err(e) => ("failed", Some(e.clone())), + } +} + /// What TFPS's list says about `address`. fn view_of(list: &[(Ipv4Addr, Option)], address: &str) -> TfpsView { match address.parse::() { diff --git a/src/security/tfps.rs b/src/security/tfps.rs index 2f9936e7b..522c6497b 100644 --- a/src/security/tfps.rs +++ b/src/security/tfps.rs @@ -607,6 +607,20 @@ pub fn peer_capability_hint(subcommand: &str, stderr: &str) -> Option<&'static s } } +impl TfpsError { + /// Whether `tfps_ctl` ran: every error but [`Self::Spawn`]. + /// + /// A peer that ran may have acted before whatever went wrong with its + /// answer -- a timeout, a non-zero exit with no result, output sipnab + /// cannot read or too much of it -- so for a command that changes TFPS the + /// effect is unknown, not absent. Only a peer that never started is known + /// to have done nothing. + #[must_use] + pub fn ran(&self) -> bool { + !matches!(self, Self::Spawn { .. }) + } +} + impl fmt::Display for TfpsError { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { match self { @@ -636,7 +650,10 @@ impl fmt::Display for TfpsError { what, } => write!( f, - "{} {subcommand} answered something this sipnab cannot read: {what}", + "{} {subcommand} answered something this sipnab cannot read: {what}. sipnab reads \ + the JSON contract of sippulse/tfps e7ee57a (2026-09-16) and later, where every \ + time is epoch seconds; a tfps_ctl built from that contract's draft wrote times, \ + `expires` among them, as RFC 3339 strings", ctl.display() ), Self::TimedOut { ctl, after, stderr } => write!( diff --git a/tests/actions_journal_rest_test.rs b/tests/actions_journal_rest_test.rs index 863f933ac..3ba5bb302 100644 --- a/tests/actions_journal_rest_test.rs +++ b/tests/actions_journal_rest_test.rs @@ -436,8 +436,13 @@ fn an_unban_straight_after_a_ban_waits_out_the_address_cooldown() -> Result<(), Ok(()) } +/// A `tfps_ctl` that ran and exited non-zero with no result is a `502`, and +/// its outcome is journaled as `unknown`, not `failed`: it ran, so sipnab +/// cannot know that TFPS did not act. `failed` is the ledger's "did not +/// happen", and recording it here lost track of bans TFPS did apply. #[test] -fn tfps_failing_is_a_bad_gateway_and_the_outcome_is_journaled_as_failed() -> Result<(), TestError> { +fn tfps_failing_after_it_ran_is_a_bad_gateway_and_the_outcome_is_unknown() -> Result<(), TestError> +{ let fake = Fake::new()?; fake.fail_bans()?; let journal = tempfile::tempdir()?; @@ -453,7 +458,10 @@ fn tfps_failing_is_a_bad_gateway_and_the_outcome_is_journaled_as_failed() -> Res let all = records(journal.path())?; let outcome = of_kind(&all, "action_outcome"); assert_eq!(outcome.len(), 1, "{all:?}"); - assert_eq!(outcome[0]["result"], "failed"); + assert_eq!( + outcome[0]["result"], "unknown", + "tfps_ctl ran, so whether TFPS acted is unknown: {all:?}" + ); Ok(()) } @@ -719,3 +727,106 @@ fn forged_expired_tampered_revoked_and_foreign_tokens_cannot_act() -> Result<(), ); Ok(()) } + +// ── a ban whose answer was lost ────────────────────────────────────────── +// +// Found against a real TFPS whose tfps_ctl predated the JSON contract: the ban +// took effect, the answer could not be read, sipnab answered 502 and +// journaled nothing it could act on, and every later unban answered 409. + +/// Ban an address through a TFPS that applies the ban and loses the answer +/// `how`, then lift it as `lifter`: the ban is held in doubt, the unban +/// reaches TFPS, and TFPS's answer settles it. +fn a_lost_answer_is_held_in_doubt_and_can_be_lifted( + how: &str, + lifter: &str, +) -> Result<(), TestError> { + let fake = Fake::new()?; + fake.lose_ban_answers(how)?; + let journal = tempfile::tempdir()?; + let (_cfg, config) = quick_cooldown()?; + let srv = server(&fake, journal.path(), &["--config", &config])?; + // `hang` answers only when sipnab's own 10 s `tfps_ctl` timeout stops + // the peer, so the client waits longer than that. + let resp = srv.post_json_bearer_waiting( + "/v1/tfps/ban", + &ban_body("198.51.100.40"), + &token_for("ops-console"), + 30, + )?; + assert_eq!(resp.status, 502, "{}", resp.body); + assert!( + resp.body.contains("in doubt"), + "the 502 says the ban is held in doubt: {}", + resp.body + ); + assert_eq!(fake.held().trim(), "198.51.100.40", "TFPS applied it"); + let all = records(journal.path())?; + let outcome = of_kind(&all, "action_outcome"); + assert_eq!(outcome.len(), 1, "{all:?}"); + assert_eq!(outcome[0]["result"], "unknown", "{all:?}"); + + // Past the one-second address cooldown. + std::thread::sleep(Duration::from_millis(1100)); + let token = if lifter == "admin" { + sipnab::auth::mint( + SIGNING_KEY.as_bytes(), + "lead", + chrono::Utc::now().timestamp() + 3600, + sipnab::auth::AUDIENCE_API, + sipnab::auth::SCOPE_ADMIN, + ) + } else { + token_for("ops-console") + }; + let resp = srv.post_json_bearer("/v1/tfps/unban", &ban_body("198.51.100.40"), &token)?; + assert_eq!(resp.status, 200, "{lifter} lifts it: {}", resp.body); + assert_eq!(resp.json()?["applied"], true, "{}", resp.body); + assert_eq!( + fake.count("unban"), + 1, + "the unban reached TFPS: {}", + fake.calls() + ); + assert_eq!(fake.held().trim(), "", "TFPS no longer holds it"); + + // Settled: a new action runs again. + fake.answer_bans()?; + let resp = srv.post_json_bearer( + "/v1/tfps/ban", + &ban_body("198.51.100.41"), + &token_for("ops-console"), + )?; + assert_eq!(resp.status, 200, "nothing is left in doubt: {}", resp.body); + Ok(()) +} + +#[test] +fn a_ban_answered_with_garbage_can_be_lifted_by_its_caller() -> Result<(), TestError> { + a_lost_answer_is_held_in_doubt_and_can_be_lifted("garbage", "caller") +} + +#[test] +fn a_ban_answered_with_garbage_can_be_lifted_by_an_admin() -> Result<(), TestError> { + a_lost_answer_is_held_in_doubt_and_can_be_lifted("garbage", "admin") +} + +#[test] +fn a_ban_that_exits_non_zero_after_acting_can_be_lifted_by_its_caller() -> Result<(), TestError> { + a_lost_answer_is_held_in_doubt_and_can_be_lifted("nonzero", "caller") +} + +#[test] +fn a_ban_that_exits_non_zero_after_acting_can_be_lifted_by_an_admin() -> Result<(), TestError> { + a_lost_answer_is_held_in_doubt_and_can_be_lifted("nonzero", "admin") +} + +#[test] +fn a_ban_that_times_out_can_be_lifted_by_its_caller() -> Result<(), TestError> { + a_lost_answer_is_held_in_doubt_and_can_be_lifted("hang", "caller") +} + +#[test] +fn a_ban_that_times_out_can_be_lifted_by_an_admin() -> Result<(), TestError> { + a_lost_answer_is_held_in_doubt_and_can_be_lifted("hang", "admin") +} diff --git a/tests/actions_service_test.rs b/tests/actions_service_test.rs index 5aa41309b..a6f810234 100644 --- a/tests/actions_service_test.rs +++ b/tests/actions_service_test.rs @@ -26,7 +26,7 @@ use std::time::{Duration, Instant}; use sipnab::security::actions::{ ActionCaller, ActionError, ActionLimits, ActionPolicy, ActionService, ActionSurface, BanRule, - RevertTarget, Reverter, + RevertTarget, Reverter, TfpsActions, }; #[path = "support/fake_tfps.rs"] @@ -542,3 +542,275 @@ fn a_second_sipnab_on_the_same_journal_is_refused_at_start() -> Result<(), TestE assert!(matches!(second, Err(ActionError::JournalUnusable(_)))); Ok(()) } + +// ── a ban whose answer was lost ────────────────────────────────────────── +// +// Once `tfps_ctl` has run, TFPS may have acted whatever became of its answer: +// a timeout, a non-zero exit with no result, or an answer in a shape sipnab +// cannot read. sipnab recorded that as "failed", which the ledger reads as +// "did not happen": the ban stood in TFPS, sipnab held no record of it, and +// every surface refused to lift it ("sipnab holds no ban on this address"). +// Found against a real TFPS whose tfps_ctl predated the JSON contract. + +/// A rig whose TFPS applies every ban and loses its answer, and, with +/// `unban_lost`, every unban's too. +fn lossy(unban_not_blocked: bool, unban_lost: bool) -> Result { + let tmp = tempfile::tempdir()?; + let dir = tmp.path().join("journal"); + let tfps = Arc::new(FakeTfps { + journal_dir: dir.clone(), + ban_answer_lost: true, + unban_not_blocked, + unban_answer_lost: unban_lost, + ..FakeTfps::default() + }); + Ok(Rig { + _tmp: tmp, + dir, + tfps, + t0: Instant::now(), + }) +} + +/// Ban `addr(20)` as `token:ops` through a TFPS that loses the answer, and +/// check that it is held in doubt rather than forgotten. +fn ban_and_lose_the_answer(r: &Rig, svc: &ActionService) -> Result<(), TestError> { + let err = svc + .ban(ActionSurface::Rest, "token:ops", addr(20), None, T0, r.t0) + .err() + .ok_or("a ban whose answer was lost is not a success")?; + assert!( + matches!(err, ActionError::Unanswered(_)), + "TFPS was asked, so this is not `could not be asked`: {err:?}" + ); + let doubt = svc.in_doubt(); + assert_eq!(doubt.len(), 1, "the ban is held in doubt: {doubt:?}"); + assert_eq!(doubt[0].address, "198.51.100.20", "with the address banned"); + assert_eq!(doubt[0].caller, "token:ops", "with who asked"); + assert_eq!(doubt[0].at, T0, "and when"); + assert!(svc.owned(addr(20)).is_none(), "not owned: it is not proven"); + Ok(()) +} + +#[test] +fn a_ban_whose_answer_is_lost_is_held_in_doubt() -> Result<(), TestError> { + let r = lossy(false, false)?; + let svc = start(&r, enabled("tfps:rest")?)?; + ban_and_lose_the_answer(&r, &svc)?; + assert!( + r.tfps.journal_text().contains("\"unknown\""), + "the outcome is journaled as unknown: {}", + r.tfps.journal_text() + ); + Ok(()) +} + +#[test] +fn the_caller_can_lift_a_ban_left_in_doubt_and_tfps_settles_it() -> Result<(), TestError> { + let r = lossy(false, false)?; + let svc = start(&r, enabled("tfps:rest")?)?; + ban_and_lose_the_answer(&r, &svc)?; + let done = svc.unban( + ActionSurface::Rest, + caller("token:ops", false), + addr(20), + T0 + 61, + r.t0 + Duration::from_secs(61), + )?; + assert!(done.applied, "TFPS lifted it: {done:?}"); + assert_eq!( + r.tfps.calls()?, + ["ban 198.51.100.20 3600", "unban 198.51.100.20"], + "the unban reached TFPS" + ); + assert!(svc.in_doubt().is_empty(), "TFPS's answer settled it"); + assert!(svc.owned(addr(20)).is_none(), "and nothing is left owned"); + assert!(r.tfps.banned()?.is_empty(), "and TFPS holds nothing"); + Ok(()) +} + +#[test] +fn an_admin_can_lift_another_callers_ban_left_in_doubt() -> Result<(), TestError> { + let r = lossy(false, false)?; + let svc = start(&r, enabled("tfps:rest")?)?; + ban_and_lose_the_answer(&r, &svc)?; + let done = svc.unban( + ActionSurface::Rest, + caller("token:lead", true), + addr(20), + T0 + 61, + r.t0 + Duration::from_secs(61), + )?; + assert!(done.applied, "{done:?}"); + assert_eq!( + r.tfps.calls()?.last().map(String::as_str), + Some("unban 198.51.100.20"), + "the admin's unban reached TFPS" + ); + assert!( + svc.in_doubt().is_empty(), + "TFPS lifting it settled the doubt" + ); + Ok(()) +} + +#[test] +fn another_caller_cannot_lift_a_ban_left_in_doubt_without_admin() -> Result<(), TestError> { + let r = lossy(false, false)?; + let svc = start(&r, enabled("tfps:rest")?)?; + ban_and_lose_the_answer(&r, &svc)?; + let err = svc + .unban( + ActionSurface::Rest, + caller("token:intern", false), + addr(20), + T0 + 61, + r.t0 + Duration::from_secs(61), + ) + .err() + .ok_or("another caller's ban is refused")?; + assert_eq!( + err, + ActionError::NotYours { + owner: "token:ops".to_string() + }, + "the refusal names who placed the ban" + ); + assert_eq!( + r.tfps.calls()?, + ["ban 198.51.100.20 3600"], + "TFPS not asked" + ); + assert_eq!(svc.in_doubt().len(), 1, "still in doubt"); + Ok(()) +} + +#[test] +fn an_unban_tfps_answers_not_blocked_settles_the_doubt_as_never_applied() -> Result<(), TestError> { + let r = lossy(true, false)?; + let svc = start(&r, enabled("tfps:rest")?)?; + ban_and_lose_the_answer(&r, &svc)?; + let done = svc.unban( + ActionSurface::Rest, + caller("token:ops", false), + addr(20), + T0 + 61, + r.t0 + Duration::from_secs(61), + )?; + assert_eq!(done.refused.as_deref(), Some("not-blocked"), "{done:?}"); + assert!(svc.in_doubt().is_empty(), "TFPS said it holds no ban"); + assert!( + svc.owned(addr(20)).is_none(), + "a ban TFPS never held is not owned" + ); + Ok(()) +} + +#[test] +fn an_unban_whose_answer_is_lost_too_leaves_the_address_in_doubt() -> Result<(), TestError> { + let r = lossy(false, true)?; + let svc = start(&r, enabled("tfps:rest")?)?; + ban_and_lose_the_answer(&r, &svc)?; + let err = svc + .unban( + ActionSurface::Rest, + caller("token:ops", false), + addr(20), + T0 + 61, + r.t0 + Duration::from_secs(61), + ) + .err() + .ok_or("an unban whose answer was lost is not a success")?; + assert!(matches!(err, ActionError::Unanswered(_)), "{err:?}"); + assert!( + !svc.in_doubt().is_empty(), + "nothing TFPS said settles the address" + ); + Ok(()) +} + +#[test] +fn while_a_ban_is_in_doubt_other_actions_wait() -> Result<(), TestError> { + let r = lossy(false, false)?; + let svc = start(&r, enabled("tfps:rest")?)?; + ban_and_lose_the_answer(&r, &svc)?; + let err = svc + .ban( + ActionSurface::Rest, + "token:ops", + addr(21), + None, + T0 + 1, + r.t0, + ) + .err() + .ok_or("a new ban waits for the doubt to be settled")?; + assert_eq!( + err, + ActionError::InDoubt(1), + "one action elsewhere is in doubt" + ); + assert_eq!( + r.tfps.calls()?, + ["ban 198.51.100.20 3600"], + "the second ban never reached TFPS" + ); + Ok(()) +} + +#[test] +fn a_restart_resolves_a_lost_answer_against_tfps() -> Result<(), TestError> { + let r = lossy(false, false)?; + { + let svc = start(&r, enabled("tfps:rest")?)?; + ban_and_lose_the_answer(&r, &svc)?; + } + // TFPS lists the ban with the expiry sipnab asked for, so the restart + // proves sipnab placed it, and owns it. + let svc = start(&r, enabled("tfps:rest")?)?; + assert!( + svc.in_doubt().is_empty(), + "the restart settled it against TFPS" + ); + assert!( + svc.owned(addr(20)).is_some(), + "the journal kept it in doubt" + ); + Ok(()) +} + +#[test] +fn a_revert_whose_answer_is_lost_is_held_in_doubt() -> Result<(), TestError> { + let r = lossy(false, true)?; + // This TFPS answers bans; only unbans lose their answer. + let tfps = Arc::new(FakeTfps { + journal_dir: r.dir.clone(), + unban_answer_lost: true, + ..FakeTfps::default() + }); + let r = Rig { tfps, ..r }; + let svc = start(&r, enabled("tfps:rest")?)?; + let id = svc + .ban(ActionSurface::Rest, "token:ops", addr(20), None, T0, r.t0)? + .id; + let err = svc + .revert( + Reverter::Surface { + surface: ActionSurface::Rest, + caller: caller("token:ops", false), + }, + RevertTarget::One(id), + T0 + 61, + r.t0 + Duration::from_secs(61), + ) + .err() + .ok_or("a revert whose answer was lost is not a success")?; + assert!(matches!(err, ActionError::Unanswered(_)), "{err:?}"); + let doubt = svc.in_doubt(); + assert_eq!(doubt.len(), 1, "the revert is held in doubt: {doubt:?}"); + assert_eq!( + doubt[0].address, "198.51.100.20", + "on the address it reverts" + ); + Ok(()) +} diff --git a/tests/actions_tfps_ctl_test.rs b/tests/actions_tfps_ctl_test.rs index e683a97fd..41d629b86 100644 --- a/tests/actions_tfps_ctl_test.rs +++ b/tests/actions_tfps_ctl_test.rs @@ -124,3 +124,98 @@ fn an_unban_asks_tfps_to_unban_and_nothing_else() -> Result<(), TestError> { assert_eq!(verbs, ["unban"], "{argv}"); Ok(()) } + +// ── A ban TFPS may have applied, with an answer sipnab cannot read ──────── + +/// A `tfps_ctl` that bans (it notes the address in `held`, as TFPS writes +/// its block map) and then answers with `answer`: shell lines run after the +/// ban took effect. +fn acts_then(answer: &str) -> Result<(tempfile::TempDir, TfpsCtl), TestError> { + let dir = tempfile::tempdir()?; + let here = dir.path().display(); + let script = format!( + "#!/bin/sh\ncase \"$1\" in\n\ + ban) echo \"$3\" >> \"{here}/held\"; {answer};;\n\ + esac\n" + ); + let path = dir.path().join("tfps_ctl"); + executable::write_executable(&path, &script)?; + let ctl = TfpsCtl::new(TfpsLocator::new(Some(path), None)); + Ok((dir, ctl)) +} + +/// Ban 198.51.100.20 through `ctl`, expecting TFPS to have acted and +/// sipnab to report the answer as unreadable rather than as an error. +fn unanswered(dir: &tempfile::TempDir, ctl: &TfpsCtl) -> Result { + let reply = ctl.ban(Ipv4Addr::new(198, 51, 100, 20), 600, 0); + let held = std::fs::read_to_string(dir.path().join("held")).unwrap_or_default(); + assert_eq!(held.trim(), "198.51.100.20", "the fake applied the ban"); + match reply { + Ok(TfpsReply::Unanswered(why)) => Ok(why), + other => Err(format!( + "tfps_ctl ran and may have banned, so the answer is Unanswered, not {other:?}" + ) + .into()), + } +} + +/// An answer that is not JSON at all. +#[test] +fn a_ban_answered_with_garbage_is_unanswered_not_failed() -> Result<(), TestError> { + let (dir, ctl) = acts_then("echo 'segmentation fault (core dumped)'")?; + let why = unanswered(&dir, &ctl)?; + assert!(why.contains("tfps_ctl"), "names what ran: {why}"); + Ok(()) +} + +/// The answer a `tfps_ctl` built before TFPS's JSON contract gives: the +/// same object, with `expires` an RFC 3339 string instead of epoch seconds. +/// sipnab does not read that shape, and says why it may have met it. +#[test] +fn a_ban_answered_by_a_pre_contract_tfps_ctl_is_unanswered_and_says_why() -> Result<(), TestError> { + let (dir, ctl) = acts_then( + "echo '{\"ip\":\"198.51.100.20\",\"action\":\"ban\",\"applied\":true,\ + \"refused\":null,\"expires\":\"2026-11-03T17:00:00Z\",\"source\":\"operator\"}'", + )?; + let why = unanswered(&dir, &ctl)?; + assert!( + why.contains("e7ee57a"), + "names the TFPS commit whose contract this answer does not meet: {why}" + ); + Ok(()) +} + +/// A non-zero exit with no result, after the ban took effect. +#[test] +fn a_ban_that_exits_non_zero_with_no_result_is_unanswered() -> Result<(), TestError> { + let (dir, ctl) = acts_then("echo 'tfps: lost the map' >&2; exit 3")?; + let why = unanswered(&dir, &ctl)?; + assert!(why.contains("lost the map"), "carries its stderr: {why}"); + Ok(()) +} + +/// A `tfps_ctl` that never answers, after the ban took effect. +#[test] +fn a_ban_that_times_out_is_unanswered() -> Result<(), TestError> { + let (dir, ctl) = acts_then("sleep 30")?; + let why = unanswered(&dir, &ctl)?; + assert!(why.contains("did not finish"), "says it timed out: {why}"); + Ok(()) +} + +/// A `tfps_ctl` that cannot be started never ran, so nothing was banned: +/// that is still an error, not an unreadable answer. +#[test] +fn a_tfps_ctl_that_cannot_start_is_an_error_not_unanswered() -> Result<(), TestError> { + let dir = tempfile::tempdir()?; + let path = dir.path().join("tfps_ctl"); + std::fs::write(&path, "not executable")?; + let ctl = TfpsCtl::new(TfpsLocator::new(Some(path), None)); + match ctl.ban(Ipv4Addr::new(198, 51, 100, 20), 600, 0) { + Err(e) => { + assert!(e.contains("tfps_ctl"), "{e}"); + Ok(()) + } + Ok(reply) => Err(format!("a peer that never ran answered {reply:?}").into()), + } +} diff --git a/tests/rust_style_known.tsv b/tests/rust_style_known.tsv index a6e130741..44ee3d930 100644 --- a/tests/rust_style_known.tsv +++ b/tests/rust_style_known.tsv @@ -317,7 +317,7 @@ AssertWithoutMessage tests/actions_cli_test.rs 2 AssertWithoutMessage tests/actions_config_test.rs 15 AssertWithoutMessage tests/actions_governor_test.rs 4 AssertWithoutMessage tests/actions_journal_mcp_test.rs 4 -AssertWithoutMessage tests/actions_journal_rest_test.rs 15 +AssertWithoutMessage tests/actions_journal_rest_test.rs 14 AssertWithoutMessage tests/actions_ledger_test.rs 26 AssertWithoutMessage tests/actions_optin_rest_test.rs 1 AssertWithoutMessage tests/actions_recovery_test.rs 36 diff --git a/tests/support/fake_tfps.rs b/tests/support/fake_tfps.rs index e03cee876..71648aa96 100644 --- a/tests/support/fake_tfps.rs +++ b/tests/support/fake_tfps.rs @@ -40,6 +40,12 @@ pub struct FakeTfps { pub ban_takes: Option, /// Set as a ban starts, before it waits `ban_takes`. pub ban_started: std::sync::atomic::AtomicBool, + /// Apply every ban, then lose its answer, as a `tfps_ctl` does that + /// writes the block map and then times out, exits with no result, or + /// answers in a shape sipnab cannot read. + pub ban_answer_lost: bool, + /// The same, for every unban. + pub unban_answer_lost: bool, } impl FakeTfps { @@ -89,6 +95,11 @@ impl TfpsActions for FakeTfps { .lock() .map_err(|e| e.to_string())? .push((ip, Some(now_unix + ttl_secs))); + if self.ban_answer_lost { + return Ok(TfpsReply::Unanswered( + "tfps_ctl did not finish within 10.0s and was stopped".into(), + )); + } Ok(TfpsReply::Applied) } @@ -103,6 +114,11 @@ impl TfpsActions for FakeTfps { let mut b = self.banned.lock().map_err(|e| e.to_string())?; let before = b.len(); b.retain(|(a, _)| *a != ip); + if self.unban_answer_lost { + return Ok(TfpsReply::Unanswered( + "tfps_ctl exited 1 with no result".into(), + )); + } Ok(if b.len() < before { TfpsReply::Applied } else { diff --git a/tests/support/fake_tfps_ctl.rs b/tests/support/fake_tfps_ctl.rs index 86fb20260..8313d15f4 100644 --- a/tests/support/fake_tfps_ctl.rs +++ b/tests/support/fake_tfps_ctl.rs @@ -32,7 +32,13 @@ impl Fake { printf '%s\\n' \"$@\" >> \"{here}/argv\"\n\ case \"$1\" in\n\ ban) if [ -e \"{here}/fail\" ]; then echo 'tfps: map write failed' >&2; exit 1; fi; \ - echo \"$3\" >> \"{here}/held\"; echo '{BAN}';;\n\ + echo \"$3\" >> \"{here}/held\"; \ + if [ -e \"{here}/lose\" ]; then case $(cat \"{here}/lose\") in \ + garbage) echo 'segmentation fault (core dumped)'; exit 0;; \ + nonzero) echo 'tfps: lost the map' >&2; exit 3;; \ + hang) sleep 30;; \ + esac; fi; \ + echo '{BAN}';;\n\ unban) : > \"{here}/held.new\"; while read -r ip; do [ \"$ip\" = \"$3\" ] || echo \"$ip\" >> \"{here}/held.new\"; done < \"{here}/held\"; \ mv \"{here}/held.new\" \"{here}/held\"; \ echo '{UNBAN}';;\n\ @@ -56,6 +62,23 @@ impl Fake { std::fs::write(self.dir.path().join("fail"), "") } + /// Apply every ban, then lose its answer: `garbage` answers something + /// that is not JSON, `nonzero` exits 3 with no result, `hang` never + /// answers. TFPS has acted each time; what sipnab reads is the loss. + pub fn lose_ban_answers(&self, how: &str) -> std::io::Result<()> { + std::fs::write(self.dir.path().join("lose"), how) + } + + /// Answer bans again. + pub fn answer_bans(&self) -> std::io::Result<()> { + std::fs::remove_file(self.dir.path().join("lose")) + } + + /// The addresses this TFPS holds banned now, one a line. + pub fn held(&self) -> String { + std::fs::read_to_string(self.dir.path().join("held")).unwrap_or_default() + } + pub fn calls(&self) -> String { std::fs::read_to_string(self.dir.path().join("argv")).unwrap_or_default() } diff --git a/tests/support/server.rs b/tests/support/server.rs index 73eb3d45c..7742f216d 100644 --- a/tests/support/server.rs +++ b/tests/support/server.rs @@ -317,6 +317,26 @@ impl ApiServer { http_post(&self.addr, path, body, Some(&format!("Bearer {token}"))) } + /// [`Self::post_json_bearer`], reading the answer for up to `wait_secs` + /// rather than 10 s: for a route that answers only after a wait of its + /// own, such as a ban whose `tfps_ctl` runs into sipnab's 10 s timeout. + pub fn post_json_bearer_waiting( + &self, + path: &str, + body: &str, + token: &str, + wait_secs: u64, + ) -> Result { + http_post_waiting( + &self.addr, + &self.addr, + path, + body, + Some(&format!("Bearer {token}")), + wait_secs, + ) + } + /// `GET path` with no auth header and `host` as the `Host` header. pub fn get_as_host(&self, path: &str, host: &str) -> Result { http_get_as(&self.addr, host, path, None) @@ -419,9 +439,21 @@ fn http_post_as( path: &str, body: &str, auth: Option<&str>, +) -> Result { + http_post_waiting(addr, host, path, body, auth, 10) +} + +/// [`http_post_as`], reading the answer for up to `wait_secs`. +fn http_post_waiting( + addr: &str, + host: &str, + path: &str, + body: &str, + auth: Option<&str>, + wait_secs: u64, ) -> Result { let mut stream = TcpStream::connect(addr).map_err(|e| format!("connect {addr}: {e}"))?; - stream.set_read_timeout(Some(test_timeout(10))).ok(); + stream.set_read_timeout(Some(test_timeout(wait_secs))).ok(); let mut req = format!( "POST {path} HTTP/1.1\r\nHost: {host}\r\nConnection: close\r\n\ diff --git a/website/content/docs/api.md b/website/content/docs/api.md index ade550ead..4bb3ccf1d 100644 --- a/website/content/docs/api.md +++ b/website/content/docs/api.md @@ -2278,10 +2278,24 @@ That is the answer TFPS gave, not an error: `tfps_ctl` missing or failing is `502`, with its reason in `detail`. +- If `tfps_ctl` could not start, TFPS did nothing, and the journal records + the ban as failed. +- If it ran and sipnab could not read its answer, TFPS may have banned the + address. That covers a timeout, a non-zero exit with no result, and an + answer that is not the JSON TFPS has given since + [`e7ee57a`](https://github.com/sippulse/tfps/commit/e7ee57a). In that case: + - the journal holds the ban **in doubt**, and `detail` says so; + - `sipnab --journal-show` lists it with its address, its caller and its + time; + - the caller who asked, or a token with scope `admin`, can lift it with + `POST /v1/tfps/unban`, and TFPS's answer settles it. + `503` means no action may run just now. Either: - sipnab cannot use the journal, or -- actions a crash left unfinished are waiting for TFPS to answer for them. +- actions are in doubt: sipnab stopped before TFPS answered for them, or + sipnab asked TFPS and could not read its answer. They wait for TFPS + to answer for them, or for an unban of their address. --- @@ -2290,7 +2304,8 @@ That is the answer TFPS gave, not an error: Ask TFPS to release a source **sipnab banned**. Off by default, behind the same two locks, rules and limits as the ban. -sipnab lifts only a ban it placed and still holds, as its journal records. +sipnab lifts only a ban it placed and still holds, as its journal records, +or one it asked for and holds in doubt. - A ban TFPS placed itself, or anyone else did, answers `409`. TFPS is not asked. diff --git a/website/content/docs/mcp-tools-tfps.md b/website/content/docs/mcp-tools-tfps.md index 3a07251c3..2b38ff93e 100644 --- a/website/content/docs/mcp-tools-tfps.md +++ b/website/content/docs/mcp-tools-tfps.md @@ -295,9 +295,20 @@ it: } ``` -`refusal` is `rate` (with `retry_after_secs`), `in_doubt` (actions a crash -left unfinished are waiting for TFPS to answer for them), `journal` (sipnab -cannot use the journal), or `tfps` (sipnab could not ask TFPS). +`refusal` is `rate` (with `retry_after_secs`), `in_doubt`, `journal` (sipnab +cannot use the journal), or `tfps`. + +- `in_doubt`: actions wait for TFPS to answer for them, or for an unban of + their address. Either sipnab stopped before TFPS answered, or TFPS was + asked and sipnab could not read its answer. +- `tfps`: either sipnab could not ask TFPS, or it asked and could not read + the answer. + - In the second case TFPS may have acted. That covers a timeout, a + non-zero exit with no result, and an answer that is not the JSON TFPS + has given since [`e7ee57a`](https://github.com/sippulse/tfps/commit/e7ee57a). + - The journal then holds the ban in doubt, and the message says so. + - The caller who asked, or a token with the `admin` scope, can lift it + with `tfps_unban`, and TFPS's answer settles it. The tool is `readOnlyHint: false` and `destructiveHint: true`, so a host that confirms destructive calls asks before this one. `idempotentHint: true`, @@ -308,8 +319,8 @@ because banning a banned source changes nothing. Ask TFPS to release a source **sipnab banned**. Off by default, behind the same enabling, rules and limits as `tfps_ban`. -sipnab lifts only a ban it placed and still holds, as its journal records. A -ban TFPS placed itself, or anyone else did, is an error result with `refusal` +sipnab lifts only a ban it placed and still holds, as its journal records, +or one it asked for and holds in doubt. A ban TFPS placed itself, or anyone else did, is an error result with `refusal` `not_owned`, and sipnab does not ask TFPS. A ban belongs to the caller that placed it: another caller gets `refusal` `not_yours`, and the message names the owner, unless its token holds the `admin` scope. Because an address rests for a diff --git a/website/content/docs/tfps-sipnab.md b/website/content/docs/tfps-sipnab.md index 9b3d67b1b..fa7bb9fd6 100644 --- a/website/content/docs/tfps-sipnab.md +++ b/website/content/docs/tfps-sipnab.md @@ -454,6 +454,19 @@ The `curl` commands from the same-machine section work unchanged. HTTP token has the `full` scope. Mint one with `--token-scope actions`. - **`/v1/tfps/dropped` answers `502`.** Expected: no TFPS build has `dropped` yet. +- **A ban answers `502` with `answered something this sipnab cannot read`.** + - sipnab reads the JSON of + [`e7ee57a`](https://github.com/sippulse/tfps/commit/e7ee57a) and later, + where every time is epoch seconds. A `tfps_ctl` built from the draft of + that JSON gives `expires` as an [RFC 3339](https://www.rfc-editor.org/rfc/rfc3339.html) date string instead, and sipnab + cannot read it. + - TFPS may still have applied the ban, so sipnab holds it in doubt. + `sipnab --journal-show` lists it, and no other action runs until it is + settled. + - Unban the address with the same token, or with a token with scope + `admin`. TFPS's answer settles it. + - Then build TFPS from master, as in the "Check that TFPS answers in JSON" + step. - **`installed: false`, and the reason says `tfps_ctl` was not found on `PATH`.** - The service runs without `--tfps-ctl`, and its `PATH` has no `tfps_ctl`. diff --git a/website/static/llms-full.txt b/website/static/llms-full.txt index 5347fabea..66dfaae3e 100644 --- a/website/static/llms-full.txt +++ b/website/static/llms-full.txt @@ -8924,10 +8924,24 @@ That is the answer TFPS gave, not an error: `tfps_ctl` missing or failing is `502`, with its reason in `detail`. +- If `tfps_ctl` could not start, TFPS did nothing, and the journal records + the ban as failed. +- If it ran and sipnab could not read its answer, TFPS may have banned the + address. That covers a timeout, a non-zero exit with no result, and an + answer that is not the JSON TFPS has given since + [`e7ee57a`](https://github.com/sippulse/tfps/commit/e7ee57a). In that case: + - the journal holds the ban **in doubt**, and `detail` says so; + - `sipnab --journal-show` lists it with its address, its caller and its + time; + - the caller who asked, or a token with scope `admin`, can lift it with + `POST /v1/tfps/unban`, and TFPS's answer settles it. + `503` means no action may run just now. Either: - sipnab cannot use the journal, or -- actions a crash left unfinished are waiting for TFPS to answer for them. +- actions are in doubt: sipnab stopped before TFPS answered for them, or + sipnab asked TFPS and could not read its answer. They wait for TFPS + to answer for them, or for an unban of their address. --- @@ -8936,7 +8950,8 @@ That is the answer TFPS gave, not an error: Ask TFPS to release a source **sipnab banned**. Off by default, behind the same two locks, rules and limits as the ban. -sipnab lifts only a ban it placed and still holds, as its journal records. +sipnab lifts only a ban it placed and still holds, as its journal records, +or one it asked for and holds in doubt. - A ban TFPS placed itself, or anyone else did, answers `409`. TFPS is not asked. @@ -41988,9 +42003,20 @@ it: } ``` -`refusal` is `rate` (with `retry_after_secs`), `in_doubt` (actions a crash -left unfinished are waiting for TFPS to answer for them), `journal` (sipnab -cannot use the journal), or `tfps` (sipnab could not ask TFPS). +`refusal` is `rate` (with `retry_after_secs`), `in_doubt`, `journal` (sipnab +cannot use the journal), or `tfps`. + +- `in_doubt`: actions wait for TFPS to answer for them, or for an unban of + their address. Either sipnab stopped before TFPS answered, or TFPS was + asked and sipnab could not read its answer. +- `tfps`: either sipnab could not ask TFPS, or it asked and could not read + the answer. + - In the second case TFPS may have acted. That covers a timeout, a + non-zero exit with no result, and an answer that is not the JSON TFPS + has given since [`e7ee57a`](https://github.com/sippulse/tfps/commit/e7ee57a). + - The journal then holds the ban in doubt, and the message says so. + - The caller who asked, or a token with the `admin` scope, can lift it + with `tfps_unban`, and TFPS's answer settles it. The tool is `readOnlyHint: false` and `destructiveHint: true`, so a host that confirms destructive calls asks before this one. `idempotentHint: true`, @@ -42001,8 +42027,8 @@ because banning a banned source changes nothing. Ask TFPS to release a source **sipnab banned**. Off by default, behind the same enabling, rules and limits as `tfps_ban`. -sipnab lifts only a ban it placed and still holds, as its journal records. A -ban TFPS placed itself, or anyone else did, is an error result with `refusal` +sipnab lifts only a ban it placed and still holds, as its journal records, +or one it asked for and holds in doubt. A ban TFPS placed itself, or anyone else did, is an error result with `refusal` `not_owned`, and sipnab does not ask TFPS. A ban belongs to the caller that placed it: another caller gets `refusal` `not_yours`, and the message names the owner, unless its token holds the `admin` scope. Because an address rests for a @@ -57484,6 +57510,19 @@ The `curl` commands from the same-machine section work unchanged. HTTP token has the `full` scope. Mint one with `--token-scope actions`. - **`/v1/tfps/dropped` answers `502`.** Expected: no TFPS build has `dropped` yet. +- **A ban answers `502` with `answered something this sipnab cannot read`.** + - sipnab reads the JSON of + [`e7ee57a`](https://github.com/sippulse/tfps/commit/e7ee57a) and later, + where every time is epoch seconds. A `tfps_ctl` built from the draft of + that JSON gives `expires` as an [RFC 3339](https://www.rfc-editor.org/rfc/rfc3339.html) date string instead, and sipnab + cannot read it. + - TFPS may still have applied the ban, so sipnab holds it in doubt. + `sipnab --journal-show` lists it, and no other action runs until it is + settled. + - Unban the address with the same token, or with a token with scope + `admin`. TFPS's answer settles it. + - Then build TFPS from master, as in the "Check that TFPS answers in JSON" + step. - **`installed: false`, and the reason says `tfps_ctl` was not found on `PATH`.** - The service runs without `--tfps-ctl`, and its `PATH` has no `tfps_ctl`. From 73d8b418729b05c90d71304f65fd920301ff5f41 Mon Sep 17 00:00:00 2001 From: Norm Brandinger Date: Tue, 3 Nov 2026 13:04:54 -0500 Subject: [PATCH 2/6] Report each sipnab instance's build, binary and configuration as a fleet inventory Fleet management phase P0, slice 1 of docs/design/fleet-management.local.md: read-only inventory, pulled from a static hosts file, behind a new least-privilege token scope. What it adds: - GET /v1/inventory and the MCP tool fleet_inventory, built by one function (output::inventory::current): build, SHA-256 of the running binary, signature state (always unverified), configuration digests, privilege, uptime, interface, capture and HEP counters, HEP senders. - Configuration digests only: file_sha256 of the bytes read, parsed_sha256 of the parsed configuration with sorted keys, argv_sha256 of the command line, and effective_sha256 over the last two. No value appears, and a secret given as a value changes effective_sha256. - Token scope inventory: reaches GET /v1/inventory and nothing else. full, actions and admin reach it too. Refused at mint time for MCP. - sipnab fleet inventory --hosts FILE [--json] [--timeout SECS]: asks every listed instance at once, marks unreachable, refused and incompatible instances, reports version_mismatch and unlisted_hep_sender, exits 1 when any instance did not answer or there is a finding. The how-to is docs/fleet.md ("Manage many sipnab instances"): purpose, prerequisites (API TLS, a token per instance, the hosts file, network reachability, no change on other systems), steps, verification and troubleshooting (unreachable, untrusted CA, 401, 404 from an older build, a token file other users can read, a mixed HEP HMAC version, unlisted_hep_sender, a newer schema_version). Its commands and output come from a run on 2026-11-03: four instances in containers on 192.0.2.0/24, three on this build over TLS and one on release 0.5.130, with the controller on the host. The newer schema_version output came from a test server, since no build answers schema_version 2. It is linked from docs/README.md, the "Manage the fleet" section of docs/mcp-estate.md, and the reference rows in docs/rest-api.md, docs/cli-reference.md, docs/auth.md, docs/mcp-tools-server.md and docs/errors.md. Changes from the earlier draft of this slice: - effective_sha256 is new. The design names an effective configuration digest; the draft compared parsed_sha256 only, so two instances with one file and different flags showed one CONFIG. The table's CONFIG column now shows effective_sha256. - fleet_inventory joins SOURCE_INDEPENDENT_TOOLS. The MCP dispatch stamped source_exhausted and source_stopped_early onto its answer, so MCP and REST returned different documents. The draft's parity test called the tool method directly and never passed through the dispatch; the_mcp_dispatch_answers_the_rest_document drives a real sipnab --mcp and failed on exactly those two members before the change. - The tool description is shorter: the server bundle measured 6,347 bytes with the draft's description and 5,976 with this one; the output schema already lists every field. - LoadedConfig's new field was joined onto the line of the field before it; split. Tests (each seen failing for its fault): - effective digest, REST/MCP dispatch parity, secret rotation and build-versus---version checks were written before their code and failed for the missing behavior first. - 24 guarded mutations, one per fault in the design's test table and the extra faults above, each turned at least one test red; each restore was proved with cmp -s. - tests/fleet_inventory_test.rs spawns two sipnab --api instances with different node names and config files; sipnab fleet inventory lists both with different effective digests, marks a third address unreachable, still lists the other two and exits 1. Ratchets moved, attributed per file: - tests/docs_drift_test.rs registered MCP tools 76 -> 77: fleet_inventory (src/mcp/tools/inventory.rs). - tests/docs_drift_test.rs EXPECTED_MARKDOWN_FILES 322 -> 324: docs/fleet.md and website/content/docs/fleet.md. - tests/docs_drift_test.rs EXPECTED_TABLES 1311 -> 1318: docs/fleet.md +3, website/content/docs/fleet.md +3 and docs/design/surface-capability-matrix.md +1 (the "Fleet inventory" row); no other changed page gained or lost a table against origin/main. - src/mcp/server.rs byte budget server 5,500 -> 6,000 (measured 5,976) and full 90,000 -> 90,500 (measured 90,456): fleet_inventory. - tests/link_integrity_test.rs EXPECTED_WIKI_LINKS 2079 -> 2097: docs/fleet.md +4; docs/README.md and docs/auth.md +1 each; docs/cli-reference.md, docs/errors.md, docs/mcp-estate.md, docs/mcp-tools-server.md, docs/mcp-tools.md and docs/rest-api.md +2 each. Counted per page against origin/main. - tests/link_integrity_test.rs EXPECTED_DOCS_PAGES 100 -> 101: docs/fleet.md. - tests/error_reference_test.rs EXPECTED_TEMPLATES 542 -> 546: four messages in src/app/fleet.rs, each with a docs/errors.md entry. - tests/mcp_stdio_test.rs tool count base 69 -> 70: fleet_inventory. Counts in prose moved for the 77th tool, 64 of them read-only: README.md, docs/mcp.md, docs/mcp-protocol.md, docs/mcp-tools.md, docs/design/mcp-write-back.md and website/templates/index.html. docs/fleet.md is registered as a site page (weight 66), a wiki page, a DOCS_TO_SITE entry and a docs nav entry. sipnab fleet's own flags are registered for the two documented-flag gates the way sipnab g729's are, its documented commands are never run by doc_commands_run_test (it contacts other hosts; tests/fleet_inventory_test.rs runs it), its hosts file is foreign to the sipnab-config sample gate and is parsed by its own parser in the_documented_hosts_files_parse, /v1/inventory is exempt from capture_identity with its MCP counterpart checked over the wire, fleet_inventory has an output-schema probe and a row in docs/design/surface-capability-matrix.md, and doc_jq_fields_test checks a sipnab fleet recipe against a report built by the command's own merge over a real inventory record. Two new bounds are documented: MAX_HEP_PEERS (rest-api.md) and MAX_HEAD_BYTES (fleet.md). docs/design/implementation-plan-phases-8-10.md's citation of get_stats is re-pointed by scripts/check-line-drift.py. man/sipnab.1 and docs/design/surface-capability-inventory.md are regenerated. docs/design/testing-matrix.md, website/static/openapi.json and the site mirror are regenerated by their scripts. Not covered: a secret held in a file (a key, token or password file named by path) is digested by its path only, so rotating its contents changes no digest. docs/fleet.md says so. (cherry picked from commit 4a59fe6c5ec875be652599e74cc03283934312a3) --- CHANGELOG.md | 5 + Cargo.toml | 8 + README.md | 6 +- docs/README.md | 3 + docs/auth.md | 3 +- docs/cli-reference.md | 12 +- .../design/implementation-plan-phases-8-10.md | 2 +- docs/design/mcp-write-back.md | 2 +- docs/design/surface-capability-inventory.md | 8 +- docs/design/surface-capability-matrix.md | 11 + docs/design/testing-matrix.md | 110 +- docs/errors.md | 49 + docs/fleet.md | 328 ++++++ docs/mcp-estate.md | 36 +- docs/mcp-protocol.md | 6 +- docs/mcp-tools-server.md | 66 +- docs/mcp-tools.md | 33 +- docs/mcp.md | 2 +- docs/rest-api.md | 100 +- man/sipnab.1 | 4 +- scripts/build-site-internals.py | 1 + scripts/build-site-pages.py | 12 + scripts/build-wiki.py | 3 +- src/app/bootstrap.rs | 10 + src/app/fleet.rs | 709 +++++++++++++ src/app/https_client.rs | 5 + src/app/mod.rs | 4 +- src/auth.rs | 50 + src/cli.rs | 11 +- src/cli/fleet_args.rs | 50 + src/config.rs | 59 +- src/main.rs | 41 + src/mcp/completeness.rs | 8 +- src/mcp/server.rs | 11 +- src/mcp/tools/inventory.rs | 111 ++ src/mcp/tools/mod.rs | 1 + src/mcp_profile.rs | 1 + src/output/api.rs | 201 ++++ src/output/inventory.rs | 949 ++++++++++++++++++ src/output/mod.rs | 2 + src/tls_files.rs | 8 +- tests/api_capture_identity_test.rs | 8 +- tests/cli_flag_behavior_test.rs | 36 +- tests/config_examples_test.rs | 6 + tests/config_wiring_test.rs | 26 + tests/doc_commands_run_test.rs | 12 + tests/doc_jq_fields_test.rs | 44 +- tests/docs_drift_test.rs | 36 +- tests/documented_flag_claims_test.rs | 13 + tests/error_reference_test.rs | 5 +- tests/fleet_inventory_test.rs | 757 ++++++++++++++ tests/homepage_link_landing_test.rs | 2 +- tests/link_integrity_test.rs | 10 +- tests/mcp_protocol_features_test.rs | 2 + tests/mcp_stdio_test.rs | 4 +- website/config.toml | 1 + website/content/docs/api.md | 100 +- website/content/docs/auth.md | 3 +- website/content/docs/cli.md | 12 +- website/content/docs/errors.md | 49 + website/content/docs/fleet.md | 336 +++++++ website/content/docs/mcp-estate.md | 34 +- website/content/docs/mcp-protocol.md | 6 +- website/content/docs/mcp-tools-server.md | 66 +- website/content/docs/mcp-tools.md | 33 +- website/content/docs/mcp.md | 2 +- website/static/llms-full.txt | 638 +++++++++++- website/static/llms.txt | 1 + website/static/openapi.json | 478 +++++++++ website/templates/index.html | 4 +- 70 files changed, 5585 insertions(+), 180 deletions(-) create mode 100644 docs/fleet.md create mode 100644 src/app/fleet.rs create mode 100644 src/cli/fleet_args.rs create mode 100644 src/mcp/tools/inventory.rs create mode 100644 src/output/inventory.rs create mode 100644 tests/fleet_inventory_test.rs create mode 100644 website/content/docs/fleet.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 328d126d9..b6116fb16 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,11 @@ entry that carries them. - **A `[[vcon.rule]]` writes its containers on a run that also serves `--api` or `--mcp`.** The persistence gate both doors share took its ceiling from the export flags alone (`--export-vcon`, `--export-vcon-when`), so on a run whose only writer was a configured rule the gate was never authorized. The exporter consults that gate, so the rule wrote nothing and printed nothing, and `GET /v1/persistence` answered `{"authorized":false,"enabled":false}`; the same config and capture without a door wrote the call. `Cli::persists_content` now reads the config file as well as the flags, and it is the one answer the gate and the `--redact` refusal both use. A rule authorizes the gate as a flag does, so `POST /v1/persistence` can close and reopen it. Found during the 2026-11-03 documentation review. - **A ban TFPS applied but whose answer sipnab could not read is held in doubt, and can be lifted.** Once `tfps_ctl` has run, TFPS may have acted, whatever became of its answer: a timeout, a non-zero exit with no result, or an answer that is not the JSON contract. sipnab journaled each of those as `failed`, which its ledger reads as "did not happen". So the ban stood in TFPS, the journal held nothing to act on, and every unban answered `409` "sipnab holds no ban on this address", with an `admin` token too. The outcome is now journaled as `unknown`, and the ledger keeps the action in doubt. `sipnab --journal-show` lists it with its address, caller and time. The caller who asked, or a token with the `admin` scope, can unban the address over REST or MCP. The unban goes to TFPS, and TFPS's answer settles the doubt: it lifted a ban, or it held none. Only a `tfps_ctl` that never started is still `failed`. The `502` still says what went wrong, now as "TFPS was asked, and sipnab could not read what it did" rather than "TFPS could not be asked", and an answer sipnab cannot read names the JSON contract it reads: sippulse/tfps `e7ee57a` and later, where every time is epoch seconds. A revert whose answer is lost is held in doubt the same way. Found against a real TFPS whose `tfps_ctl` gave `expires` as an [RFC 3339](https://www.rfc-editor.org/rfc/rfc3339.html) string. - **A release build stays reproducible on a host that configures a compiler wrapper.** With sccache set as `build.rustc-wrapper`, the `cc` crate routed mimalloc's C compile through the sccache server, whose clock, not `SOURCE_DATE_EPOCH`, set `__DATE__` and `__TIME__`, so two builds of one commit differed (run 41514782899). `scripts/reproducible-build.sh build` now runs cargo with `RUSTC_WRAPPER`, `RUSTC_WORKSPACE_WRAPPER` and their `CARGO_BUILD_` forms set to the empty string, which overrides a configured wrapper. +### Added + +- **A fleet inventory: which build, binary and configuration each sipnab instance runs.** `GET /v1/inventory` on the REST API and the MCP tool `fleet_inventory` return one record, from one builder: the build (version, git commit and tag, dirty flag, target, compiled features, the `--version` line), SHA-256 of the running binary (`/proc/self/exe` on Linux), the configuration as digests only (`file_sha256` of the bytes read, `parsed_sha256` of the parsed configuration with its keys sorted, `argv_sha256` of the command line, and `effective_sha256` over the last two), the user id and privilege drop, pid, start time and uptime, the capture interfaces' error and drop counters, capture and HEP export counters, and the HEP listener's sender addresses. No configuration value and no command-line argument is in the answer, so a secret in either changes a digest and never appears. `signature.state` is always `unverified`: a process cannot prove which binary it is, so compare `binary.sha256` with the release's attested `SHA256SUMS.txt`. The MCP answer carries no `source_exhausted` or `source_stopped_early`, so the two surfaces return the same document. +- **`sipnab fleet inventory` reads the inventory of every instance in a hosts file.** `--hosts ` names one `[[host]]` table per instance (`name`, `url`, `token_file`, optional `ca_file`), `--json` prints the whole report, and `--timeout ` (1 to 300, default 10) bounds each instance. It asks the instances at once, prints one row per instance with its state (`ok`, `refused`, `unreachable` or `incompatible`), and reports the findings `version_mismatch` and `unlisted_hep_sender` (a HEP sender a listed collector receives from whose address no listed instance has). It exits `0` when every instance answered and there is no finding, `1` otherwise, and `2` when the hosts file cannot be used. An instance that is down is listed as `unreachable`, and the others are still reported. [Manage many sipnab instances](docs/fleet.md) is the how-to. +- **A token scope for the inventory: `--token-scope inventory`.** It reaches `GET /v1/inventory` and nothing else, so a fleet controller does not hold a `full` token, which reads every call. `full`, `actions` and `admin` reach the inventory too. It is a REST API scope; sipnab refuses to mint it for MCP, where a `read` token reaches `fleet_inventory`. ## [0.5.232] - 2026-11-03 diff --git a/Cargo.toml b/Cargo.toml index 028722293..10851022d 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -478,6 +478,14 @@ wasm-opt = false [profile.dev] debug = "line-tables-only" +# SHA-256 at opt-level 0 is slow enough to fail a test: `/v1/inventory` and +# `fleet_inventory` hash the running binary on first use, and a debug sipnab +# is about 200 MB. Measured on the aarch64 build host: the first request took +# 1.97 s unoptimized; on CI's ubuntu runner it outlasted the tests' 10 s read +# timeout. Only sha2 is raised; tests/config_wiring_test.rs holds the setting. +[profile.dev.package.sha2] +opt-level = 3 + # `strip = true` holds for every build, the release included. The release # workflow appends `-C strip=none` to RUSTFLAGS (from # `scripts/split-debuginfo.sh --rustflags`) so the line tables below survive diff --git a/README.md b/README.md index 2c6327f59..3dc8a3da7 100644 --- a/README.md +++ b/README.md @@ -218,7 +218,7 @@ sipnab's analysis as tools. sipnab --mcp -N -I sip-problem-call.pcap --mcp-tools core ``` -The server offers 76 tools. Every tool costs the agent context before it asks +The server offers 77 tools. Every tool costs the agent context before it asks anything. - `--mcp-tools core` loads a small set that still answers a whole call. @@ -388,8 +388,8 @@ Next: - A capture started as root drops to `--user` after opening its devices. - It can confine itself with a Landlock sandbox (`--sandbox`) and a seccomp allowlist (`--seccomp`). -- **MCP server.** 76 tools over stdio or HTTP. - - 63 only read. +- **MCP server.** 77 tools over stdio or HTTP. + - 64 only read. - The 13 that write each say what they change. - File writes stay inside `--mcp-file-root`. - These stay off until you enable them: diff --git a/docs/README.md b/docs/README.md index 415a35014..873868157 100644 --- a/docs/README.md +++ b/docs/README.md @@ -80,6 +80,9 @@ Each answers one "how do I" question and assumes you already know what you want. means. [API TLS](rest-api.md#api-tls) serves it over HTTPS, and [Call the REST API from code](https://sipnab.com/docs/api-clients/) has clients in Python, TypeScript, Rust and Go, and curl one-liners. +- **[Manage many sipnab instances](fleet.md)**: one table of every + instance's build, binary and configuration digests, privilege and uptime, + and which instances did not answer. - **[Set up authentication](auth.md)**: signed bearer tokens for the API and MCP, with lifetimes, key rotation and revocation. - **[Export a call as a vCon](vcon.md)**: write one observed dialog as a diff --git a/docs/auth.md b/docs/auth.md index 95681ec7f..316603ee6 100644 --- a/docs/auth.md +++ b/docs/auth.md @@ -84,7 +84,7 @@ Distribute the printed token to clients. ### Scope: what a token may reach `--token-scope` sets what a token may reach. It takes `full` (the default), -`metrics`, `read`, `actions`, `admin` or `audio`: +`metrics`, `read`, `actions`, `admin`, `audio` or `inventory`: | Scope | Surface | What it reaches | |---|---|---| @@ -94,6 +94,7 @@ Distribute the printed token to clients. | `actions` | either | Everything `full` reaches, plus the actions that change another system, such as a TFPS ban, where the server enables them with `--allow-action`. It lifts only the bans it placed. | | `admin` | either | Everything `actions` reaches, and it also lifts a ban another caller placed, for example one placed by a token since lost. | | `audio` | REST API | The live-audio routes under `/v1/calls/`, and nothing else. No other scope reaches them. | +| `inventory` | REST API | `GET /v1/inventory`, and nothing else. `full`, `actions` and `admin` reach it too. For [`sipnab fleet inventory`](fleet.md). | Give each client the narrowest scope that does its job: diff --git a/docs/cli-reference.md b/docs/cli-reference.md index 695ab29d8..f9c4c61ce 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -2395,7 +2395,7 @@ This applies to root runs and to unprivileged runs set up with | `--syslog` | -- | off | 0.3.0 | Send alerts to syslog. Config: `[security] alert` | | `--mint-token` | -- | off | 0.4.3 | Mint a signed bearer token from the first configured signing key (API or MCP), print it to stdout, and exit (no capture/servers). See [`auth.md`](./auth.md). | | `--token-id` | `` | -- | 0.4.3 | Token id (`jti`) for `--mint-token`, used for revocation. Defaults to a generated id. | -| `--token-scope` | `` | `full` | 0.5.59 | Scope for `--mint-token`. `metrics` reaches `GET /metrics` and returns `401` everywhere else — mint one for a scrape job rather than a credential that also reads `/v1/dialogs` and the message bodies underneath. `read` is the MCP counterpart: it reaches the tools annotated read-only and refuses every tool that writes or replaces state (`export_capture`, `open_capture`, `shutdown_server`, `tfps_ban` and the rest; `tools/list` marks each one with `readOnlyHint: false`). `actions` reaches everything `full` does plus the actions that change another system, such as TFPS ban and unban, where the server enabled them with [`--allow-action`](#security); a `full` token never reaches an action. A ban belongs to the caller that placed it, so an `actions` token lifts only its own bans; `admin` reaches everything `actions` does and also lifts a ban another caller placed. `audio` reaches the REST API's live call audio routes (`GET /v1/calls/live`, `GET /v1/calls/live/events` and `GET /v1/calls/{call_id}/audio`) and nothing else, and no other scope reaches them. A cross-surface mint fails at mint time, so sipnab refuses `metrics` and `audio` with MCP and `read` with the REST API rather than issuing them and then rejecting them. | +| `--token-scope` | `` | `full` | 0.5.59 | Scope for `--mint-token`. `metrics` reaches `GET /metrics` and returns `401` everywhere else — mint one for a scrape job rather than a credential that also reads `/v1/dialogs` and the message bodies underneath. `read` is the MCP counterpart: it reaches the tools annotated read-only and refuses every tool that writes or replaces state (`export_capture`, `open_capture`, `shutdown_server`, `tfps_ban` and the rest; `tools/list` marks each one with `readOnlyHint: false`). `actions` reaches everything `full` does plus the actions that change another system, such as TFPS ban and unban, where the server enabled them with [`--allow-action`](#security); a `full` token never reaches an action. A ban belongs to the caller that placed it, so an `actions` token lifts only its own bans; `admin` reaches everything `actions` does and also lifts a ban another caller placed. `audio` reaches the REST API's live call audio routes (`GET /v1/calls/live`, `GET /v1/calls/live/events` and `GET /v1/calls/{call_id}/audio`) and nothing else, and no other scope reaches them. `inventory` reaches the REST API's `GET /v1/inventory` and nothing else, for [`sipnab fleet inventory`](fleet.md). A cross-surface mint fails at mint time, so sipnab refuses `metrics`, `audio` and `inventory` with MCP and `read` with the REST API rather than issuing them and then rejecting them. | **Examples** - `sipnab -N -I capture.pcap --no-hep-parse --report` — read HEP datagrams as plain UDP when your config file sets `[capture] hep_parse = true` @@ -3057,6 +3057,16 @@ each recording the container names by `url`, and captures nothing. - `sipnab --vcon-audio-fetch fetched/0158a120-0392-891a-9455-bc2e6fad94df.vcon.json --vcon-audio-fetch-out audio --vcon-key recipient.key --vcon-audio-fetch-ca media-ca.pem` — decrypt the audio of an encrypted vCon from storage whose certificate only that CA issued - `sipnab --vcon-audio-fetch fetched/0158a120-0392-891a-9455-bc2e6fad94df.vcon.json --vcon-audio-fetch-out audio --vcon-key recipient.key --vcon-audio-fetch-extra-ca media-ca.pem --vcon-audio-fetch-max-size 1073741824` — the same, trusting the host's bundle too, for a recording up to 1 GiB +## Fleet inventory (`sipnab fleet`) + +`sipnab fleet inventory` asks every sipnab instance in a hosts file for its inventory (`GET /v1/inventory`) and prints one row per instance and the findings across them. It opens no capture. [Manage many sipnab instances](fleet.md) is the full how-to. + +- `--hosts `: the hosts file, one `[[host]]` table per instance with `name`, `url`, `token_file` and an optional `ca_file`. +- `--json`: print the report as JSON instead of a table. +- `--timeout `: seconds to wait for each instance to connect and answer, 1 to 300. Default `10`. + +It exits `0` when every instance answered and there is no finding, `1` when an instance did not answer or there is a finding, and `2` when it cannot use the hosts file. + ## G.729 tables (`sipnab g729`) `sipnab g729` installs and checks the G.729 codebook tables that `--g729-tables` reads. A codebook table is the set of numeric values the G.729 decoder needs. sipnab does not ship them. [G.729 audio](g729.md) is the full how-to. diff --git a/docs/design/implementation-plan-phases-8-10.md b/docs/design/implementation-plan-phases-8-10.md index 385c05175..159b40eb0 100644 --- a/docs/design/implementation-plan-phases-8-10.md +++ b/docs/design/implementation-plan-phases-8-10.md @@ -1682,7 +1682,7 @@ For implementers picking this up, the bridge from each MCP tool to existing func | `tail_dialogs` | `DialogStore::iter` filtered by `updated_at > cursor` | | `security_findings` | `security::AlertEngine` history (extend with ring buffer) | | `snapshot_pcap` | `capture::PcapWriter` + filter on captured packets | -| `stats` | Mirrors `GET /v1/stats` from `output::api::get_stats` ([`src/output/api.rs:5468`](https://github.com/NormB/sipnab/blob/main/src/output/api.rs#L5468)) | +| `stats` | Mirrors `GET /v1/stats` from `output::api::get_stats` ([`src/output/api.rs:5520`](https://github.com/NormB/sipnab/blob/main/src/output/api.rs#L5520)) | | Phase 8 infra | Reuses | |---|---| diff --git a/docs/design/mcp-write-back.md b/docs/design/mcp-write-back.md index 7548b66e2..b5d015c3f 100644 --- a/docs/design/mcp-write-back.md +++ b/docs/design/mcp-write-back.md @@ -85,7 +85,7 @@ raw material and none of the interpretation. This is a real dead end, and it is the strongest argument on this page. **The read surface is rich enough that the asymmetry is conspicuous.** There are -76 tools ([`server.rs`](../../src/mcp/server.rs), `#[tool(` at `:938` through +77 tools ([`server.rs`](../../src/mcp/server.rs), `#[tool(` at `:938` through `:2260`), and several of them do genuine analysis rather than projection: `triage_call` (`:1927`), `check_codec_negotiation` (`:1999`), `diagnose_registration` (`:2085`), `compare_dialogs` (`:1722`). An agent that diff --git a/docs/design/surface-capability-inventory.md b/docs/design/surface-capability-inventory.md index 2e4a4abef..750151637 100644 --- a/docs/design/surface-capability-inventory.md +++ b/docs/design/surface-capability-inventory.md @@ -11,7 +11,7 @@ reachable three ways and not four is visible only once these four are set beside one another. [`tests/capability_matrix_test.rs`](https://github.com/NormB/sipnab/blob/main/tests/capability_matrix_test.rs) keeps this current and requires the matrix to account for every row here. -Totals: CLI 396, TUI 25, REST 47, MCP 76. +Totals: CLI 396, TUI 25, REST 48, MCP 77. ## CLI flags (396) @@ -440,7 +440,7 @@ Totals: CLI 396, TUI 25, REST 47, MCP 76. - `Talkers` - `TfpsObserve` -## REST routes (47) +## REST routes (48) - `/health` - `/metrics` @@ -464,6 +464,7 @@ Totals: CLI 396, TUI 25, REST 47, MCP 76. - `/v1/dialogs/{call_id}/vcon` - `/v1/endpoints` - `/v1/hep/senders` +- `/v1/inventory` - `/v1/persistence` - `/v1/relay/compare/{call_id}` - `/v1/relay/holdings` @@ -490,7 +491,7 @@ Totals: CLI 396, TUI 25, REST 47, MCP 76. - `/v1/vcon/spool` - `/v1/vcon/validate` -## MCP tools (76) +## MCP tools (77) - `actions_revert` - `aggregate_dialogs` @@ -519,6 +520,7 @@ Totals: CLI 396, TUI 25, REST 47, MCP 76. - `find_in_captures` - `find_in_captures_status` - `find_problems` +- `fleet_inventory` - `generate_fail2ban_rule` - `generate_repro` - `generate_wireshark_filter` diff --git a/docs/design/surface-capability-matrix.md b/docs/design/surface-capability-matrix.md index 7b1f143f8..cee9955f2 100644 --- a/docs/design/surface-capability-matrix.md +++ b/docs/design/surface-capability-matrix.md @@ -611,6 +611,17 @@ What did this build compile in, and what did the operator turn on? | REST | `/v1/capabilities` | | MCP | `server_capabilities` | +### Fleet inventory + +Which build, binary and configuration is this process running, as one record a fleet controller compares across instances? + +| Surface | Detail | +|---|---| +| CLI | decision: `sipnab fleet inventory` reads this record from each instance's REST API, and an instance has no reason to print its own when `--version` names its build | +| TUI | decision: a fleet controller is a program that polls many instances, not a human at one instance's screen | +| REST | `/v1/inventory` | +| MCP | `fleet_inventory` | + ### Remote lifecycle stop Stop this process, or remove the uprobes, from a caller with no OS signal. diff --git a/docs/design/testing-matrix.md b/docs/design/testing-matrix.md index a824af17e..dd0150f3e 100644 --- a/docs/design/testing-matrix.md +++ b/docs/design/testing-matrix.md @@ -49,15 +49,15 @@ was driving all of them. | Surface | Rows | `e2e` | `parsed` | `referenced` | `none` | |---|---|---|---|---|---| -| CLI flags | 396 | 319 | 34 | 43 | 0 | -| HTTP routes | 47 | 47 | -- | 0 | 0 | -| MCP tools | 76 | 76 | -- | 0 | 0 | +| CLI flags | 396 | 321 | 33 | 42 | 0 | +| HTTP routes | 48 | 48 | -- | 0 | 0 | +| MCP tools | 77 | 77 | -- | 0 | 0 | **No row can ever say `none`, and that is the point.** `flag_coverage_test` already requires every flag's `--name` token to appear somewhere in the test corpus, and it defines "referenced" as exactly that. A coverage metric built on mentions therefore reports 100% for this project no matter what is actually exercised -- which is what a yes/no "tested" column would have shown. The rows at `referenced` are the ones that gate passes and this document does not. ## What a person found that the detector could not -The generator understates. Of the 43 flags it could only call +The generator understates. Of the 42 flags it could only call `referenced`, a read of the tests found 76 with a real behavior test -- evidence that arrives through a config-file equivalent sharing the flag's resolver, through a golden file, or through a library-level test, none of @@ -84,9 +84,9 @@ behind them. | Flag | Short | Value | Group | Detected | Where | Audited | What a person found | |---|---|---|---|---|---|---|---| | `--help` | `-h` | | Options | e2e | `tests/cli_help_test.rs`, `tests/cli_options_test.rs` +6 | | | -| `--version` | `-V` | | Options | e2e | `tests/cli_options_test.rs`, `tests/cli_test.rs` +2 | | | +| `--version` | `-V` | | Options | e2e | `tests/cli_options_test.rs`, `tests/cli_test.rs` +3 | | | | `--device` | `-d` | `IFACE` | Capture | e2e | `src/app/tui_mode.rs`, `tests/capture_probe_test.rs` +5 | | | -| `--input` | `-I` | | Capture | e2e | `src/app/tui_mode.rs`, `tests/accused_sources_test.rs` +146 | | | +| `--input` | `-I` | | Capture | e2e | `src/app/tui_mode.rs`, `tests/accused_sources_test.rs` +150 | | | | `--recursive` | | | Capture | e2e | `tests/input_set_accounting_test.rs`, `tests/multi_input_test.rs` | | | | `--input-name` | | `GLOB` | Capture | e2e | `tests/multi_input_test.rs` | | | | `--vcon-key` | | `FILE` | Capture | e2e | `tests/vcon_audio_fetch_test.rs`, `tests/vcon_directory_report_test.rs` +2 | | | @@ -101,7 +101,7 @@ behind them. | `--limitlen` | `-S` | `BYTES` | Capture | parsed | `src/cli.rs` | | | | `--no-reassembly` | | | Capture | parsed | `src/cli.rs` | | | | `--quiet-bad-parse` | `-x` | | Capture | e2e | `tests/tui_e2e_test.rs` | | | -| `--portrange` | | `RANGE` | Capture | e2e | `tests/accused_sources_test.rs`, `tests/analyze_test.rs` +17 | | | +| `--portrange` | | `RANGE` | Capture | e2e | `tests/accused_sources_test.rs`, `tests/analyze_test.rs` +18 | | | | `--ws-portrange` | | `RANGE` | Capture | e2e | `tests/config_cli_defects_test.rs` | | | | `--multi-device` | | | Capture | parsed | `src/cli.rs` | | | | `--no-rtp` | | | Capture | e2e | `tests/cli_options_test.rs`, `tests/cli_test.rs` | | | @@ -110,23 +110,23 @@ behind them. | `--bpf-file` | | `FILE` | Capture | e2e | `tests/cli_flag_behavior_test.rs` | | | | `--capture-tunnels` | | | Capture | parsed | `src/cli.rs` | | | | `--count` | `-n` | `N` | Capture | e2e | `tests/cli_flag_behavior_test.rs`, `tests/cli_options_test.rs` +6 | | | -| `--duration` | | `DURATION` | Capture | e2e | `tests/capture_probe_test.rs`, `tests/cli_flag_behavior_test.rs` +2 | | | +| `--duration` | | `DURATION` | Capture | e2e | `tests/capture_probe_test.rs`, `tests/cli_flag_behavior_test.rs` +3 | | | | `--autostop` | | `CONDITION` | Capture | e2e | `tests/parse_path_test.rs` | | | | `--split` | | `CONDITION` | Capture | e2e | `tests/cli_flag_behavior_test.rs`, `tests/output_behavior_test.rs` +1 | | | | `--split-keep` | | `N` | Capture | e2e | `tests/cli_flag_behavior_test.rs` | | | | `--replay` | | | Capture | e2e | `tests/capture_clock_test.rs`, `tests/cores_refusal_test.rs` +1 | | | | `--pcapng` | | | Capture | e2e | `tests/cli_flag_behavior_test.rs`, `tests/integration_test.rs` +3 | | | -| `--archive-password-file` | | `FILE` | Archives | e2e | `tests/archive_password_prompt_test.rs`, `tests/archive_password_test.rs` +1 | | | +| `--archive-password-file` | | `FILE` | Archives | e2e | `tests/archive_password_prompt_test.rs`, `tests/archive_password_test.rs` +2 | | | | `--archive-password-command` | | `CMD` | Archives | e2e | `tests/archive_password_test.rs` | | | | `--archive-password-stdin` | | | Archives | e2e | `tests/archive_password_test.rs` | | | | `--archive-password` | | `PASSWORD` | Archives | e2e | `tests/archive_password_test.rs` | | | | `--archive-password-encoding` | | `ENC` | Archives | e2e | `tests/archive_password_test.rs` | | | | `--no-password-prompt` | | | Archives | referenced | `tests/archive_password_prompt_test.rs` | | | -| `--no-tui` | `-N` | | Mode | e2e | `tests/accused_sources_test.rs`, `tests/actions_journal_mcp_test.rs` +135 | | | +| `--no-tui` | `-N` | | Mode | e2e | `tests/accused_sources_test.rs`, `tests/actions_journal_mcp_test.rs` +140 | | | | `--calls-only` | `-c` | | Mode | e2e | `tests/calls_only_dialogs_test.rs`, `tests/cli_flag_behavior_test.rs` +3 | | | | `--telephone-event` | `-t` | | Mode | e2e | `tests/cli_options_test.rs`, `tests/cores_refusal_test.rs` +5 | | | | `--dtmf-cleartext` | | | Mode | e2e | `tests/decryption_wrapper_matrix_test.rs`, `tests/dtmf_dynamic_payload_type_test.rs` +1 | **behavior** | dtmf_cleartext_emits_the_digit_value_at_debug_level (tests/dtmf_masking_test.rs), with an anti-vacuity guard | -| `--quiet` | `-q` | | Mode | e2e | `tests/app_servers_wiring_test.rs`, `tests/capture_clock_test.rs` +66 | | | +| `--quiet` | `-q` | | Mode | e2e | `tests/app_servers_wiring_test.rs`, `tests/capture_clock_test.rs` +68 | | | | `--resolve` | | | Name resolution | e2e | `tests/integration_test.rs` | | | | `--no-resolve` | | | Name resolution | referenced | `src/cli.rs` | | | | `--reverse-dns` | | | Name resolution | parsed | `src/cli.rs` | | | @@ -155,9 +155,9 @@ behind them. | `--short-calls` | | | Diagnostic aliases | e2e | `tests/cli_options_test.rs`, `tests/filter_corpus_test.rs` +2 | **parse-only** | short_calls_filter asserts only count <= 7; the alias-equivalence test is vacuous (0 of 1334 dialogs selected on its fixture) | | `--one-way` | | | Diagnostic aliases | e2e | `tests/cli_options_test.rs`, `tests/filter_corpus_test.rs` +1 | **behavior** | one_way_filter plus one_way_output_carries_the_stun_versus_sdp_finding (tests/stun_test.rs); both directions pinned | | `--nat-issues` | | | Diagnostic aliases | e2e | `tests/cli_options_test.rs`, `tests/filter_corpus_test.rs` +1 | **behavior** | nat_issues_filter plus the_nat_issues_alias_selects_the_rewritten_call (tests/media_diagnosis_wiring_test.rs) | -| `--json` | | | Output | e2e | `tests/annotate_cli_test.rs`, `tests/archive_input_test.rs` +36 | | | +| `--json` | | | Output | e2e | `tests/annotate_cli_test.rs`, `tests/archive_input_test.rs` +38 | | | | `--json-pretty` | | | Output | e2e | `tests/cli_options_test.rs`, `tests/json_schema_test.rs` +2 | | | -| `--json-dialogs` | | | Output | e2e | `tests/archive_input_test.rs`, `tests/archive_password_prompt_test.rs` +43 | | | +| `--json-dialogs` | | | Output | e2e | `tests/archive_input_test.rs`, `tests/archive_password_prompt_test.rs` +44 | | | | `--plugin` | | `PATH` | Output | e2e | `tests/partial_run_exit_code_test.rs`, `tests/plugin_example_test.rs` | | | | `--report` | | | Output | e2e | `tests/archive_input_test.rs`, `tests/batch_run_paths_test.rs` +35 | | | | `--stun` | | | Output | e2e | `tests/cores_refusal_test.rs`, `tests/stun_test.rs` +1 | | | @@ -165,7 +165,7 @@ behind them. | `--analyze` | | | Output | e2e | `tests/analyze_test.rs`, `tests/cores_refusal_test.rs` | | | | `--json-analyze` | | | Output | e2e | `tests/analyze_test.rs`, `tests/cli_time_window_test.rs` +7 | | | | `--yang-analyze` | | | Output | e2e | `tests/analyze_test.rs`, `tests/cores_refusal_test.rs` +1 | | | -| `--print-yang-module` | | | Output | e2e | `tests/yang_module_test.rs` | | | +| `--print-yang-module` | | | Output | e2e | `tests/startup_refusal_stderr_test.rs`, `tests/yang_module_test.rs` | | | | `--call-report` | | | Output | e2e | `tests/call_report_siprec_test.rs`, `tests/cli_flag_behavior_test.rs` +6 | | | | `--export-vcon` | | | Output | e2e | `tests/batch_run_paths_test.rs`, `tests/cli_flag_behavior_test.rs` +14 | | | | `--export-vcon-when` | | `EXPR` | Output | e2e | `tests/api_test.rs`, `tests/batch_run_paths_test.rs` +11 | | | @@ -190,7 +190,7 @@ behind them. | `--color` | | `WHEN` | Output | e2e | `tests/cli_options_test.rs`, `tests/config_wiring_test.rs` | | | | `--payload-limit` | | `BYTES` | Output | e2e | `tests/cli_options_test.rs` | **behavior** | payload_limit_truncates_raw_dump: [truncated] appears, User-Agent disappears, against a no-flag baseline | | `--text-dump` | `-T` | | Output | e2e | `tests/cli_flag_behavior_test.rs`, `tests/cli_options_test.rs` +1 | | | -| `--no-cli-print` | | | Output | e2e | `tests/analyze_test.rs`, `tests/archive_input_test.rs` +77 | | | +| `--no-cli-print` | | | Output | e2e | `tests/analyze_test.rs`, `tests/archive_input_test.rs` +80 | | | | `--wireshark` | | | Output | e2e | `tests/batch_run_paths_test.rs`, `tests/closed_stdout_test.rs` +1 | **behavior** | golden tests/cli/out/wireshark.trycmd adds two lines absent from the flagless golden | | `--lint` | | | Output | e2e | `tests/cli_options_test.rs`, `tests/config_cli_flag_values_test.rs` +4 | | | | `--lint-fail-on` | | `SEVERITY` | Output | e2e | `tests/cli_options_test.rs` | **parse-only** | EXIT 3 IS NEVER OBSERVED. The assertion is guarded behind `if findings > 0` and no checked-in capture produces a lint finding | @@ -202,7 +202,7 @@ behind them. | `--group-by` | | `FIELD` | Output | e2e | `tests/cli_options_test.rs`, `tests/config_cli_flag_values_test.rs` +2 | | | | `--max-groups` | | `N` | Output | referenced | `src/cli.rs`, `src/output/group.rs` | **behavior** | via config key: probe_max_groups (tests/config_wiring_test.rs); flag and key share one resolver. The flag itself is never passed | | `--max-grouped-messages` | | `N` | Output | referenced | `src/cli.rs`, `src/output/group.rs` | **behavior** | via config key: probe_max_grouped_messages; same shared resolver, same caveat | -| `--node-name` | | `NAME` | Output | e2e | `tests/api_capture_identity_test.rs`, `tests/node_identity_test.rs` +3 | **behavior** | the_node_name_flag_lands_in_capture_identity (tests/node_identity_test.rs); read back through MCP | +| `--node-name` | | `NAME` | Output | e2e | `tests/api_capture_identity_test.rs`, `tests/fleet_inventory_test.rs` +4 | **behavior** | the_node_name_flag_lands_in_capture_identity (tests/node_identity_test.rs); read back through MCP | | `--limit` | `-l` | `N` | Dialog | e2e | `src/app/tui_mode.rs`, `tests/cli_flag_behavior_test.rs` +4 | | | | `--rotate` | `-R` | | Dialog | e2e | `tests/cli_flag_behavior_test.rs`, `tests/cli_options_test.rs` | | | | `--no-rotate` | | | Dialog | e2e | `tests/cli_flag_behavior_test.rs` | | | @@ -248,7 +248,7 @@ behind them. | `--reg-flood-window` | | `SECS` | Security | e2e | `tests/threshold_wiring_test.rs` | **behavior** | reg_flood_window_secs_decides_how_concentrated_refusals_must_be (tests/threshold_wiring_test.rs); an inert flag fails it | | `--reg-flood-transaction-timeout` | | `MS` | Security | e2e | `tests/threshold_wiring_test.rs` | **behavior** | reg_flood_transaction_timeout_decides_whether_a_late_challenge_counts (tests/threshold_wiring_test.rs); an inert flag fails it | | `--kill-rate-limit` | | `N` | Security | parsed | `src/cli.rs` | | | -| `--business-hours` | | | Security | e2e | `tests/threshold_wiring_test.rs` | **behavior** | business_hours_make_the_off_hours_detection_reachable | +| `--business-hours` | | | Security | e2e | `tests/startup_refusal_stderr_test.rs`, `tests/threshold_wiring_test.rs` | **behavior** | business_hours_make_the_off_hours_detection_reachable | | `--fraud-short-call` | | `SECS` | Security | e2e | `tests/review_regressions_test.rs`, `tests/threshold_wiring_test.rs` | **behavior** | fraud_short_call_secs_decides_which_calls_are_short | | `--fraud-wangiri-calls` | | `N` | Security | e2e | `tests/threshold_wiring_test.rs` | **behavior** | fraud_wangiri_calls_decides_how_many_short_calls_are_a_lure | | `--fraud-sequential-calls` | | `N` | Security | e2e | `tests/threshold_wiring_test.rs` | **behavior** | fraud_sequential_calls_decides_how_long_a_scan_must_be | @@ -270,7 +270,7 @@ behind them. | `--stir-shaken` | | | Security | e2e | `tests/batch_run_paths_test.rs`, `tests/cli_options_test.rs` +1 | | | | `--syslog` | | | Security | e2e | `tests/cores_refusal_test.rs` | **mention-only** | No occurrence anywhere outside comments. Acknowledged in flag_coverage_test's KNOWN_UNTESTED ("requires a syslog daemon") | | `--alert-json` | | | Security | e2e | `tests/cli_options_test.rs` | **parse-only** | asserts exit 0 on a fixture where no alert fires; set_json_output has one caller and no test | -| `--run-provenance-file` | | `FILE` | Security | e2e | `tests/doc_commands_run_test.rs`, `tests/run_provenance_test.rs` | | | +| `--run-provenance-file` | | `FILE` | Security | e2e | `tests/doc_commands_run_test.rs`, `tests/run_provenance_test.rs` +1 | | | | `--tui-audit-file` | | `FILE` | Security | e2e | `src/app/tui_mode.rs`, `tests/doc_commands_run_test.rs` +1 | | | | `--recommend-block` | | `DIALECT` | Security | e2e | `tests/block_recommendation_test.rs`, `tests/closed_stdout_test.rs` +1 | **behavior** | a_swept_source_gets_a_rule_naming_it and an_established_source_carries_its_counter_evidence (tests/block_recommendation_test.rs) drive the real binary and assert the emitted rule; an_unaccused_capture_recommends_nothing is the anti-vacuity half | | `--on-dialog-exec` | | `CMD` | Event execution | e2e | `tests/cli_flag_behavior_test.rs`, `tests/config_wiring_test.rs` +2 | | | @@ -282,14 +282,14 @@ behind them. | `--metrics-auth-file` | | `FILE` | Network listeners | parsed | `src/cli.rs` | | | | `--metrics-tls-cert` | | `FILE` | Network listeners | e2e | `tests/config_cli_flag_values_test.rs`, `tests/listener_tls_test.rs` | **behavior** | metrics_tls_flags_serve_https (tests/listener_tls_test.rs): /metrics 200 in the Prometheus text format over HTTPS; plain HTTP not served; an untrusting client fails the handshake; a silent client does not block another; Basic auth 401/200 over HTTPS; the plaintext-credential warning fires only without TLS; one flag alone, missing file, world-readable key, mismatched key refused at startup; refused without --metrics; a flag replaces [metrics] tls_cert (config_keys::a_flag_replaces_its_own_key) | | `--metrics-tls-key` | | `FILE` | Network listeners | e2e | `tests/config_cli_flag_values_test.rs` | **behavior** | same tests; metrics_a_world_readable_key_is_refused_naming_it and metrics_a_key_that_is_not_the_certificates_is_refused name the key file | -| `--api` | | `ADDR` | Network listeners | e2e | `tests/actions_journal_rest_test.rs`, `tests/app_servers_wiring_test.rs` +8 | | | -| `--api-key` | | `KEY` | Network listeners | e2e | `tests/actions_optin_rest_test.rs`, `tests/api_test.rs` +6 | **behavior** | auth_accepts_correct_bearer_and_rejects_everything_else (tests/api_test.rs): 200 vs 401 across five shapes | -| `--api-signing-key` | | `KEY` | Network listeners | e2e | `tests/actions_cli_test.rs`, `tests/actions_journal_rest_test.rs` +3 | **behavior** | seven tests in tests/api_token_test.rs: expiry, forgery, tampering, rotation, scope | -| `--api-signing-key-file` | | `FILE` | Network listeners | e2e | `tests/cli_flag_behavior_test.rs` | | | +| `--api` | | `ADDR` | Network listeners | e2e | `tests/actions_journal_rest_test.rs`, `tests/app_servers_wiring_test.rs` +11 | | | +| `--api-key` | | `KEY` | Network listeners | e2e | `tests/actions_optin_rest_test.rs`, `tests/api_test.rs` +8 | **behavior** | auth_accepts_correct_bearer_and_rejects_everything_else (tests/api_test.rs): 200 vs 401 across five shapes | +| `--api-signing-key` | | `KEY` | Network listeners | e2e | `tests/actions_cli_test.rs`, `tests/actions_journal_rest_test.rs` +4 | **behavior** | seven tests in tests/api_token_test.rs: expiry, forgery, tampering, rotation, scope | +| `--api-signing-key-file` | | `FILE` | Network listeners | e2e | `tests/cli_flag_behavior_test.rs`, `tests/startup_refusal_stderr_test.rs` | | | | `--api-revoked-file` | | `FILE` | Network listeners | e2e | `tests/actions_journal_rest_test.rs`, `tests/api_token_test.rs` | **behavior** | revoked_id_is_rejected_via_denylist_file: denylisted 401, fresh 200, both with valid tokens | | `--api-token-ttl` | | `SECS` | Network listeners | e2e | `tests/cli_flag_behavior_test.rs` | | | -| `--api-tls-cert` | | `FILE` | Network listeners | e2e | `tests/api_test.rs`, `tests/config_cli_flag_values_test.rs` | **behavior** | tls_flags_serve_https: HTTPS /health 200 with a CA-issued cert; plain HTTP to the port not served; an untrusting client fails the handshake; one_tls_flag_alone_fails_fast_naming_it; a_missing_tls_file_fails_fast_naming_it; unit api_tls_refuses_* cover no-certificate, mismatch and missing-file | -| `--api-tls-key` | | `FILE` | Network listeners | e2e | `tests/api_test.rs`, `tests/config_cli_flag_values_test.rs` | **behavior** | same tests; unit api_tls_refuses_a_world_readable_key and api_tls_refuses_a_key_file_with_no_private_key; one_tls_flag_alone_fails_fast_naming_it covers the key without the cert | +| `--api-tls-cert` | | `FILE` | Network listeners | e2e | `tests/api_test.rs`, `tests/config_cli_flag_values_test.rs` +2 | **behavior** | tls_flags_serve_https: HTTPS /health 200 with a CA-issued cert; plain HTTP to the port not served; an untrusting client fails the handshake; one_tls_flag_alone_fails_fast_naming_it; a_missing_tls_file_fails_fast_naming_it; unit api_tls_refuses_* cover no-certificate, mismatch and missing-file | +| `--api-tls-key` | | `FILE` | Network listeners | e2e | `tests/api_test.rs`, `tests/config_cli_flag_values_test.rs` +2 | **behavior** | same tests; unit api_tls_refuses_a_world_readable_key and api_tls_refuses_a_key_file_with_no_private_key; one_tls_flag_alone_fails_fast_naming_it covers the key without the cert | | `--api-max-conn` | | `N` | Network listeners | e2e | `tests/api_test.rs` | **parse-only** | DoS BOUND. Asserts the server still serves with the flag set; the 503 saturation path is untested | | `--api-allowed-host` | | `HOST` | Network listeners | e2e | `tests/api_test.rs`, `tests/doc_commands_run_test.rs` | | | | `--metrics-max-conn` | | `N` | Network listeners | referenced | `src/cli.rs` | **parse-only** | DoS BOUND (SN-02). Resolver precedence tested, ConnGate tested, the join between them is not | @@ -353,8 +353,8 @@ behind them. | `--mos-bad` | | `MOS` | Analysis | referenced | `src/cli.rs` | **behavior** | same test, case mos_bad: MOS 3.5 moves Warning -> Bad | | `--rtt-warn-ms` | | `MS` | Analysis | referenced | `src/cli.rs` | **behavior** | same test, case rtt_warn_ms | | `--rtt-bad-ms` | | `MS` | Analysis | referenced | `src/cli.rs` | **behavior** | same test, case rtt_bad_ms | -| `--hep-listen` | `-L` | `ADDR` | HEP | e2e | `src/app/tui_mode.rs`, `tests/cli_flag_behavior_test.rs` +9 | | | -| `--hep-send` | `-H` | `ADDR` | HEP | e2e | `tests/batch_run_paths_test.rs`, `tests/config_cli_flag_values_test.rs` +6 | | | +| `--hep-listen` | `-L` | `ADDR` | HEP | e2e | `src/app/tui_mode.rs`, `tests/cli_flag_behavior_test.rs` +10 | | | +| `--hep-send` | `-H` | `ADDR` | HEP | e2e | `tests/batch_run_paths_test.rs`, `tests/config_cli_flag_values_test.rs` +7 | | | | `--hep-send-transport` | | | HEP | e2e | `tests/config_cli_flag_values_test.rs`, `tests/hep_test.rs` +1 | | | | `--hep-send-queue` | | `MIB` | HEP | parsed | `src/cli.rs` | | | | `--hep-send-timeout` | | `SECS` | HEP | parsed | `src/cli.rs` | | | @@ -370,18 +370,18 @@ behind them. | `--hep-hmac-window` | | `SECS` | HEP | e2e | `tests/hep_test.rs` | | | | `--hep-silence-warn` | | `SECS` | HEP | e2e | `tests/hep_test.rs` | | | | `--hep-senders` | | | HEP | e2e | `tests/hep_test.rs` | | | -| `--hep-parse` | `-E` | | HEP | e2e | `src/app/tui_mode.rs`, `tests/app_servers_wiring_test.rs` +6 | | | +| `--hep-parse` | `-E` | | HEP | e2e | `src/app/tui_mode.rs`, `tests/app_servers_wiring_test.rs` +7 | | | | `--no-hep-parse` | | | HEP | parsed | `src/cli.rs` | | | -| `--hep-allow` | | `ADDR` | HEP | e2e | `tests/config_wiring_test.rs`, `tests/doc_commands_run_test.rs` +1 | | | +| `--hep-allow` | | `ADDR` | HEP | e2e | `tests/config_wiring_test.rs`, `tests/doc_commands_run_test.rs` +2 | | | | `--hep-rate-limit` | | `N` | HEP | e2e | `tests/hep_test.rs` | | | | `--hep-rate-limit-per-peer` | | | HEP | e2e | `tests/config_wiring_test.rs` | | | -| `--tls-key` | `-k` | `FILE` | TLS / Decryption | e2e | `tests/config_cli_defects_test.rs`, `tests/cores_refusal_test.rs` | **behavior** | DECRYPTION MATERIAL. A real TLS 1.2 GCM record decrypts back to a REGISTER. The flag's file-load hop (RsaKey::from_pem_file) is untested | -| `--keylog` | | `FILE` | TLS / Decryption | e2e | `tests/batch_run_paths_test.rs`, `tests/config_cli_defects_test.rs` +5 | | | -| `--keylog-fd` | | `N` | TLS / Decryption | parsed | `src/cli.rs` | | | -| `--keylog-watch` | | | TLS / Decryption | referenced | `src/app/batch.rs`, `src/app/bootstrap.rs` | **mention-only** | DECRYPTION. The live keylog poll and the late-decrypt hold have no test tying them to the flag. CHANGELOG records a prior regression here | +| `--tls-key` | `-k` | `FILE` | TLS / Decryption | e2e | `tests/config_cli_defects_test.rs`, `tests/cores_refusal_test.rs` +1 | **behavior** | DECRYPTION MATERIAL. A real TLS 1.2 GCM record decrypts back to a REGISTER. The flag's file-load hop (RsaKey::from_pem_file) is untested | +| `--keylog` | | `FILE` | TLS / Decryption | e2e | `tests/batch_run_paths_test.rs`, `tests/config_cli_defects_test.rs` +6 | | | +| `--keylog-fd` | | `N` | TLS / Decryption | e2e | `tests/feature_gate_test.rs` | | | +| `--keylog-watch` | | | TLS / Decryption | e2e | `tests/feature_gate_test.rs` | **mention-only** | DECRYPTION. The live keylog poll and the late-decrypt hold have no test tying them to the flag. CHANGELOG records a prior regression here | | `--tls-lockon-window` | | `RECORDS` | TLS / Decryption | parsed | `src/cli.rs`, `tests/cli_defaults_test.rs` | | | -| `--dtls-keylog` | | `FILE` | TLS / Decryption | e2e | `src/app/tui_mode.rs`, `tests/batch_run_paths_test.rs` +3 | **behavior** | DECRYPTION MATERIAL. Split proof: parse -> extract -> an SRTP packet decrypts. The file-load hop is untested | -| `--srtp-keys` | | `FILE` | TLS / Decryption | e2e | `src/app/tui_mode.rs`, `tests/batch_run_paths_test.rs` +3 | **behavior** | DECRYPTION MATERIAL. Key-file parsing and decryption both tested, with a wrong-key negative. The file-load hop is untested | +| `--dtls-keylog` | | `FILE` | TLS / Decryption | e2e | `src/app/tui_mode.rs`, `tests/batch_run_paths_test.rs` +4 | **behavior** | DECRYPTION MATERIAL. Split proof: parse -> extract -> an SRTP packet decrypts. The file-load hop is untested | +| `--srtp-keys` | | `FILE` | TLS / Decryption | e2e | `src/app/tui_mode.rs`, `tests/batch_run_paths_test.rs` +4 | **behavior** | DECRYPTION MATERIAL. Key-file parsing and decryption both tested, with a wrong-key negative. The file-load hop is untested | | `--pcap-export-mode` | | `MODE` | TLS / Decryption | e2e | `tests/cli_flag_behavior_test.rs`, `tests/decrypted_export_test.rs` +1 | **parse-only** | Three tests assert the string->enum mapping. No test writes a file in two modes and compares | | `--allow-coredump` | | | TLS / Decryption | e2e | `tests/archive_password_test.rs`, `tests/cli_options_test.rs` | **parse-only** | SAFETY SWITCH. Its smoke test runs on a capture with NO key material, so the code it guards never executes either way | | `--uprobe-tls` | | | TLS / Decryption | parsed | `tests/uprobe_cli_test.rs` | | | @@ -401,12 +401,12 @@ behind them. | `--max-tcp-buffer` | | `BYTES` | Resource limits | parsed | `src/cli.rs` | | | | `--max-metadata-file-bytes` | | `BYTES` | Resource limits | referenced | `src/capture/pcapng_meta.rs`, `src/cli.rs` | **behavior** | probe_max_metadata_file_bytes turns "stripped" into "refused" at a 10-byte cap | | `--max-gunzip-bytes` | | `BYTES` | Resource limits | e2e | `tests/archive_input_test.rs` | **behavior** | probe_max_gunzip_bytes turns "stripped" into "refused" at a 100-byte ceiling | -| `--cores` | | `N` | Resource limits | e2e | `tests/archive_input_test.rs`, `tests/batch_run_paths_test.rs` +29 | | | -| `--mint-token` | | | Token minting | e2e | `tests/cli_flag_behavior_test.rs`, `tests/mcp_token_test.rs` | | | +| `--cores` | | `N` | Resource limits | e2e | `tests/archive_input_test.rs`, `tests/batch_run_paths_test.rs` +30 | | | +| `--mint-token` | | | Token minting | e2e | `tests/cli_flag_behavior_test.rs`, `tests/feature_gate_test.rs` +1 | | | | `--token-id` | | `ID` | Token minting | e2e | `tests/cli_flag_behavior_test.rs`, `tests/mcp_token_test.rs` | | | | `--token-scope` | | `SCOPE` | Token minting | e2e | `tests/cli_flag_behavior_test.rs` | | | -| `--vcon-forward` | | `SPOOL_DIR` | vCon forwarder | e2e | `tests/config_cli_defects_test.rs`, `tests/config_cli_flag_values_test.rs` +2 | | | -| `--vcon-spool-status` | | `SPOOL_DIR` | vCon forwarder | e2e | `tests/vcon_spool_status_cli_test.rs` | | | +| `--vcon-forward` | | `SPOOL_DIR` | vCon forwarder | e2e | `tests/config_cli_defects_test.rs`, `tests/config_cli_flag_values_test.rs` +3 | | | +| `--vcon-spool-status` | | `SPOOL_DIR` | vCon forwarder | e2e | `tests/feature_gate_test.rs`, `tests/startup_refusal_stderr_test.rs` +1 | | | | `--vcon-forward-url` | | `URL` | vCon forwarder | e2e | `tests/config_cli_flag_values_test.rs`, `tests/doc_commands_run_test.rs` +2 | | | | `--vcon-forward-auth-file` | | `FILE` | vCon forwarder | e2e | `tests/config_cli_flag_values_test.rs`, `tests/vcon_forward_test.rs` +1 | | | | `--vcon-forward-auth` | | `HEADER` | vCon forwarder | referenced | `src/app/bootstrap.rs`, `src/app/https_client.rs` +9 | | | @@ -430,7 +430,7 @@ behind them. | `--vcon-forward-scitt-sign-key` | | `FILE` | vCon forwarder | e2e | `tests/vcon_scitt_test.rs` | | | | `--vcon-forward-scitt-sign-cert` | | `FILE` | vCon forwarder | e2e | `tests/vcon_scitt_test.rs` | | | | `--vcon-forward-scitt-issuer` | | `URI` | vCon forwarder | e2e | `tests/vcon_scitt_test.rs` | | | -| `--vcon-fetch` | | `UUID` | vCon fetcher | e2e | `tests/config_cli_flag_values_test.rs`, `tests/vcon_fetch_test.rs` | | | +| `--vcon-fetch` | | `UUID` | vCon fetcher | e2e | `tests/config_cli_flag_values_test.rs`, `tests/feature_gate_test.rs` +1 | | | | `--vcon-fetch-url` | | `URL` | vCon fetcher | e2e | `tests/config_cli_flag_values_test.rs`, `tests/doc_commands_run_test.rs` +1 | | | | `--vcon-fetch-auth-file` | | `FILE` | vCon fetcher | e2e | `tests/config_cli_flag_values_test.rs`, `tests/vcon_fetch_test.rs` | | | | `--vcon-fetch-kind` | | `KIND` | vCon fetcher | e2e | `tests/vcon_fetch_test.rs` | | | @@ -444,17 +444,17 @@ behind them. | `--vcon-fetch-decrypt-key` | | `FILE` | vCon fetcher | e2e | `tests/config_cli_flag_values_test.rs`, `tests/vcon_fetch_test.rs` | | | | `--vcon-fetch-signer-ca` | | | vCon fetcher | e2e | `tests/vcon_fetch_test.rs` | | | | `--vcon-fetch-signer-sha256` | | `HEX` | vCon fetcher | e2e | `tests/vcon_fetch_test.rs` | | | -| `--vcon-audio-fetch` | | `FILE` | vCon fetcher | e2e | `tests/config_cli_flag_values_test.rs`, `tests/vcon_audio_fetch_test.rs` | | | -| `--vcon-audio-fetch-out` | | `DIR` | vCon fetcher | e2e | `tests/config_cli_flag_values_test.rs`, `tests/vcon_audio_fetch_test.rs` | | | +| `--vcon-audio-fetch` | | `FILE` | vCon fetcher | e2e | `tests/config_cli_flag_values_test.rs`, `tests/feature_gate_test.rs` +1 | | | +| `--vcon-audio-fetch-out` | | `DIR` | vCon fetcher | e2e | `tests/config_cli_flag_values_test.rs`, `tests/feature_gate_test.rs` +1 | | | | `--vcon-audio-fetch-local` | | | vCon fetcher | e2e | `tests/vcon_audio_fetch_test.rs` | | | | `--vcon-audio-fetch-ca` | | `FILE` | vCon fetcher | e2e | `tests/vcon_audio_fetch_test.rs` | | | | `--vcon-audio-fetch-extra-ca` | | `FILE` | vCon fetcher | e2e | `tests/vcon_audio_fetch_test.rs` | | | | `--vcon-audio-fetch-max-size` | | `BYTES` | vCon fetcher | e2e | `tests/vcon_audio_fetch_test.rs` | | | -| `--vcon-resign` | | `FILE` | vCon archive | e2e | `tests/config_cli_flag_values_test.rs`, `tests/vcon_resign_test.rs` | | | -| `--vcon-resign-key` | | `FILE` | vCon archive | e2e | `tests/config_cli_flag_values_test.rs`, `tests/vcon_resign_test.rs` | | | -| `--vcon-resign-cert` | | `FILE` | vCon archive | e2e | `tests/config_cli_flag_values_test.rs`, `tests/vcon_resign_test.rs` | | | -| `--vcon-resign-out` | | `FILE` | vCon archive | e2e | `tests/config_cli_flag_values_test.rs`, `tests/vcon_resign_test.rs` | | | -| `--config` | `-f` | `FILE` | Config | e2e | `tests/actions_journal_mcp_test.rs`, `tests/actions_journal_rest_test.rs` +44 | | | +| `--vcon-resign` | | `FILE` | vCon archive | e2e | `tests/config_cli_flag_values_test.rs`, `tests/feature_gate_test.rs` +1 | | | +| `--vcon-resign-key` | | `FILE` | vCon archive | e2e | `tests/config_cli_flag_values_test.rs`, `tests/feature_gate_test.rs` +1 | | | +| `--vcon-resign-cert` | | `FILE` | vCon archive | e2e | `tests/config_cli_flag_values_test.rs`, `tests/feature_gate_test.rs` +1 | | | +| `--vcon-resign-out` | | `FILE` | vCon archive | e2e | `tests/config_cli_flag_values_test.rs`, `tests/feature_gate_test.rs` +1 | | | +| `--config` | `-f` | `FILE` | Config | e2e | `tests/actions_journal_mcp_test.rs`, `tests/actions_journal_rest_test.rs` +47 | | | | `--no-config` | `-F` | | Config | e2e | `tests/annotate_cli_test.rs`, `tests/call_report_siprec_test.rs` +35 | | | | `--dump-config` | `-D` | | Config | e2e | `tests/cli_flag_behavior_test.rs`, `tests/cli_options_test.rs` +6 | | | | `--completions` | | `SHELL` | Config | e2e | `tests/cli_help_test.rs` | **behavior** | completions_emit_scripts_for_each_shell runs the real binary for bash/zsh/fish; unknown shell exits 2 | @@ -465,13 +465,13 @@ behind them. | `--prefix-digits` | | `N` | Queries | e2e | `tests/cli_talkers_test.rs` | | | | `--top` | | `N` | Queries | e2e | `tests/cli_compare_captures_test.rs`, `tests/cli_count_by_test.rs` +3 | | | | `--call-tree` | | | Queries | e2e | `tests/cli_call_tree_test.rs` | | | -| `--correlate` | | | Queries | e2e | `tests/cli_correlate_test.rs` | | | +| `--correlate` | | | Queries | e2e | `tests/cli_correlate_test.rs`, `tests/th_callid_test.rs` | | | | `--compare-dialogs` | | | Queries | e2e | `tests/cli_compare_dialogs_test.rs` | | | | `--endpoint-ip` | | `IP` | Queries | e2e | `tests/cli_endpoint_test.rs` | | | | `--endpoint-user` | | `USER` | Queries | e2e | `tests/cli_endpoint_test.rs` | | | | `--export-audio` | | | Queries | e2e | `tests/cli_export_audio_test.rs`, `tests/config_cli_flag_values_test.rs` | | | | `--export-audio-out` | | `FILE` | Queries | e2e | `tests/cli_export_audio_test.rs` | | | -| `--vcon-validate` | | `FILE` | Queries | e2e | `tests/cli_vcon_validate_test.rs` | | | +| `--vcon-validate` | | `FILE` | Queries | e2e | `tests/cli_vcon_validate_test.rs`, `tests/feature_gate_test.rs` | | | | `--compare-captures` | | `A` | Queries | e2e | `tests/cli_compare_captures_test.rs`, `tests/config_cli_flag_values_test.rs` | | | | `--compare-dimensions` | | `DIMENSIONS` | Queries | e2e | `tests/cli_compare_captures_test.rs` | | | | `--find-in-files` | | `PATH` | Queries | e2e | `tests/cli_find_in_files_test.rs` | | | @@ -499,13 +499,14 @@ behind them. | `/v1/dialogs/tail` | exercised | `tests/api_capture_identity_test.rs`, `tests/api_test.rs` | | `/v1/dialogs/{call_id}` | exercised | `tests/api_capture_identity_test.rs`, `tests/api_operator_flows_test.rs` +2 | | `/v1/dialogs/{call_id}/audio` | exercised | `tests/api_capture_identity_test.rs`, `tests/api_test.rs` | -| `/v1/dialogs/{call_id}/correlated` | exercised | `tests/api_capture_identity_test.rs`, `tests/api_test.rs` | +| `/v1/dialogs/{call_id}/correlated` | exercised | `tests/api_capture_identity_test.rs`, `tests/api_test.rs` +2 | | `/v1/dialogs/{call_id}/lint` | exercised | `tests/api_capture_identity_test.rs`, `tests/api_test.rs` | | `/v1/dialogs/{call_id}/report` | exercised | `tests/api_capture_identity_test.rs`, `tests/api_test.rs` | -| `/v1/dialogs/{call_id}/tree` | exercised | `tests/api_capture_identity_test.rs`, `tests/api_test.rs` | +| `/v1/dialogs/{call_id}/tree` | exercised | `tests/api_capture_identity_test.rs`, `tests/api_test.rs` +1 | | `/v1/dialogs/{call_id}/vcon` | exercised | `tests/api_capture_identity_test.rs`, `tests/api_test.rs` +1 | | `/v1/endpoints` | exercised | `tests/api_capture_identity_test.rs`, `tests/api_test.rs` +1 | -| `/v1/hep/senders` | exercised | `tests/api_capture_identity_test.rs`, `tests/api_token_test.rs` +1 | +| `/v1/hep/senders` | exercised | `tests/api_capture_identity_test.rs`, `tests/api_token_test.rs` +2 | +| `/v1/inventory` | exercised | `tests/api_capture_identity_test.rs`, `tests/fleet_inventory_test.rs` | | `/v1/persistence` | exercised | `tests/api_capture_identity_test.rs`, `tests/api_test.rs` +1 | | `/v1/relay/compare/{call_id}` | exercised | `tests/api_capture_identity_test.rs`, `tests/relay_rest_test.rs` | | `/v1/relay/holdings` | exercised | `tests/api_capture_identity_test.rs`, `tests/api_test.rs` +1 | @@ -516,7 +517,7 @@ behind them. | `/v1/report` | exercised | `tests/api_capture_identity_test.rs`, `tests/api_test.rs` +1 | | `/v1/runtime` | exercised | `tests/api_capture_identity_test.rs`, `tests/api_test.rs` +2 | | `/v1/security/findings` | exercised | `tests/api_capture_identity_test.rs`, `tests/api_test.rs` | -| `/v1/stats` | exercised | `tests/api_capture_identity_test.rs`, `tests/api_harness_readiness_test.rs` +5 | +| `/v1/stats` | exercised | `tests/api_capture_identity_test.rs`, `tests/api_harness_readiness_test.rs` +6 | | `/v1/streams` | exercised | `tests/api_capture_identity_test.rs`, `tests/api_operator_flows_test.rs` +4 | | `/v1/streams/{id}` | exercised | `tests/api_capture_identity_test.rs`, `tests/api_operator_flows_test.rs` +1 | | `/v1/talkers` | exercised | `tests/api_capture_identity_test.rs`, `tests/api_test.rs` +1 | @@ -559,14 +560,15 @@ behind them. | `export_capture` | exercised | `tests/mcp_completeness_test.rs`, `tests/mcp_diagnostic_tools_test.rs` +1 | | `export_vcon` | exercised | `tests/app_servers_wiring_test.rs`, `tests/mcp_completeness_test.rs` +4 | | `fetch_vcon` | exercised | `tests/mcp_protocol_features_test.rs`, `tests/mcp_stdio_test.rs` | -| `find_correlated` | exercised | `tests/api_capture_identity_test.rs`, `tests/leg_correlation_window_test.rs` +6 | +| `find_correlated` | exercised | `tests/api_capture_identity_test.rs`, `tests/leg_correlation_window_test.rs` +7 | | `find_in_captures` | exercised | `tests/cli_find_in_files_test.rs`, `tests/config_wiring_test.rs` +2 | | `find_in_captures_status` | exercised | `tests/config_wiring_test.rs`, `tests/mcp_protocol_features_test.rs` +1 | | `find_problems` | exercised | `tests/mcp_completeness_test.rs`, `tests/mcp_completion_test.rs` +5 | +| `fleet_inventory` | exercised | `tests/api_capture_identity_test.rs`, `tests/fleet_inventory_test.rs` +2 | | `generate_fail2ban_rule` | exercised | `tests/mcp_completeness_test.rs`, `tests/mcp_expectations_test.rs` +1 | | `generate_repro` | exercised | `tests/mcp_completeness_test.rs`, `tests/mcp_expectations_test.rs` +1 | | `generate_wireshark_filter` | exercised | `tests/mcp_completeness_test.rs`, `tests/mcp_expectations_test.rs` +1 | -| `get_call_tree` | exercised | `tests/api_capture_identity_test.rs`, `tests/mcp_completeness_test.rs` +3 | +| `get_call_tree` | exercised | `tests/api_capture_identity_test.rs`, `tests/mcp_completeness_test.rs` +4 | | `get_capture_report` | exercised | `tests/api_capture_identity_test.rs`, `tests/mcp_completeness_test.rs` +3 | | `get_dialog` | exercised | `tests/api_capture_identity_test.rs`, `tests/header_support_e2e_test.rs` +6 | | `get_dialog_report` | exercised | `tests/api_capture_identity_test.rs`, `tests/mcp_completeness_test.rs` +2 | diff --git a/docs/errors.md b/docs/errors.md index d7c7da0fd..501af705d 100644 --- a/docs/errors.md +++ b/docs/errors.md @@ -4908,6 +4908,55 @@ install it with `--from`. **Exit code:** 1. +## Fleet inventory (`sipnab fleet`) + +`sipnab fleet inventory` prints these messages. +[Manage many sipnab instances](fleet.md) describes the command. A problem with +the hosts file stops the command with exit code 2. The command prints its +name, the hosts file's path and then the message. A problem with one instance goes in that +instance's row as `unreachable`, and the command still asks every other +instance. + +### `host '{}' has an empty url` + +The `[[host]]` table named in the message has `url = ""`. + +**What to do:** Give the instance's REST API URL, such as +`url = "https://192.0.2.11:8443"`. + +**Exit code:** 2. + +### `host '{}' is named twice` + +Two `[[host]]` tables have the same `name`. The report shows one row per name, +so a repeated name would hide an instance. + +**What to do:** Give each instance its own `name`. + +**Exit code:** 2. + +### `token_file '{}' does not hold one token on one line` + +The instance's token file is empty, or holds whitespace inside the token, such +as two tokens or a token split over lines. The row shows the message, and the +file's path in place of the braces. The token itself is never printed. + +**What to do:** Put the one token that `sipnab --mint-token --token-scope +inventory` printed on that instance in the file, and nothing else. + +**Exit code:** 1. The other instances are still in the report. + +### `the answer is larger than {MAX_ANSWER_BYTES} bytes` + +The instance's answer to `GET /v1/inventory` is longer than +`{MAX_ANSWER_BYTES}`, 4194304 bytes (4 MiB). sipnab stops reading it. An +inventories in the [fleet how-to](fleet.md)'s run measured under 2 KB each, +so the URL most likely names something that is not a sipnab REST API. + +**What to do:** Check the instance's `url` in the hosts file. + +**Exit code:** 1. The other instances are still in the report. + ## Startup, privileges and the process ### `--allow-coredump: a core dump would contain the archive password this run holds` diff --git a/docs/fleet.md b/docs/fleet.md new file mode 100644 index 000000000..fd9cd6c32 --- /dev/null +++ b/docs/fleet.md @@ -0,0 +1,328 @@ +# Manage many sipnab instances + +`sipnab fleet inventory` asks every sipnab instance you list for its +inventory, and prints one table of the fleet. Use it to answer "which build +and which configuration does each instance run, and which ones are down?" +before and after an upgrade or a configuration change. + +The table shows, for each instance: + +- the build it runs, and the SHA-256 of its binary, +- the digest of its effective configuration: the file it loaded and its + command line, +- the user it runs as, and how long it has been up. + +Below the table it names the instances that did not answer, the instances on +a different build from the rest, and the HEP senders a collector receives from +that are not in your list. It reports. It changes nothing on any instance. + +Some upgrades need every sender and its collector on the same build, such as +the HEP HMAC version 2 change in +[Upgrade a fleet that uses HMAC authentication](config-reference.md#upgrade-a-fleet-that-uses-hmac-authentication). +The report shows which instances are still on the old build. + +The commands and output on this page come from a run on 2026-11-03: four +instances in containers on the network `192.0.2.0/24`, and the controller on +the host. The instances ran a development build of 0.5.232 with this feature, +and one ran release 0.5.130. + +## How it works + +- Each instance serves `GET /v1/inventory` on its REST API (`--api`). See + [GET /v1/inventory](rest-api.md#get-v1inventory) for every field. +- `sipnab fleet inventory` runs on one machine, the controller. It reads a + hosts file and asks every instance in it at once. The instances never + connect to the controller. +- Each request carries a bearer token with scope `inventory`. That token reads + the inventory and nothing else: no dialogs, no messages, no audio. +- The answer holds no configuration value and no command-line argument, only + their SHA-256 digests. +- The command reads at most 16 KiB of each answer's status line and headers + (`MAX_HEAD_BYTES`) and at most 4 MiB of its body (`MAX_ANSWER_BYTES`). The + inventories in this page's run were under 2 KB each. + +## Before you start + +- **sipnab on each instance.** A build whose `sipnab --version` lists the + `api` feature, and that serves `GET /v1/inventory`. An older build answers + `404` there, and the report marks it `refused`. +- **sipnab on the controller.** Any build with the `api` feature. It needs no + capture privilege: it opens no capture. +- **TLS for each instance's API.** A certificate whose subject alternative + name is the address or name in the instance's URL, its key readable by the + sipnab user only, and the certificate of the issuing CA on the controller. + Plain `http://` works too, and sends the token in clear text. +- **A signing key on each instance**, in a file only the sipnab user can read. + [Set up authentication](auth.md) shows how to create one. +- **Network.** The controller reaches each instance's API port over TCP, 8443 + in this page's examples. Open that port to the controller's address only. + The instances need no route to the controller. +- **Other systems.** No change. OpenSIPS, Kamailio, Homer and a HEP collector + keep their configuration. A collector that runs sipnab joins the list like + any other instance. + +## Step 1: run each instance with the REST API over TLS + +- **[each instance]** Add `--api`, the TLS certificate and key, the signing key + and a node name to the instance's command line. This is the instance + `edge-a` from the run, which reads a capture file and sends HEP to the + collector `hub`: + + ```bash + sipnab -N -I /var/lib/sipnab/sample.pcap --config /etc/sipnab/edge-a.toml \ + --hep-send 192.0.2.10:9060 --hep-auth-mode hmac --hep-auth-file /etc/sipnab/hep.key \ + --api 192.0.2.11:8443 --api-tls-cert /etc/sipnab/edge-a.crt --api-tls-key /etc/sipnab/edge-a.key \ + --api-signing-key-file /etc/sipnab/edge-a.signing --node-name edge-a + ``` + + The collector ran with the same API flags and its HEP listener: + + ```bash + sipnab -N --config /etc/sipnab/hub.toml \ + --hep-listen 192.0.2.10:9060 --hep-auth-mode hmac --hep-auth-file /etc/sipnab/hep.key \ + --api 192.0.2.10:8443 --api-tls-cert /etc/sipnab/hub.crt --api-tls-key /etc/sipnab/hub.key \ + --api-signing-key-file /etc/sipnab/hub.signing --node-name hub + ``` + + Each instance logs `REST API listening on
` when its API is up. + +## Step 2: mint an inventory token on each instance + +- **[each instance]** Mint a token with that instance's signing key: + + ```bash + sipnab --mint-token --token-scope inventory --token-id fleet-controller \ + --api-signing-key-file /etc/sipnab/edge-a.signing --api-token-ttl 2592000 + ``` + + It prints one token, starting `s2.`, and exits. `--api-token-ttl 2592000` + makes it valid for 30 days. + +- **[controller]** Save each token in its own file, readable by the + controller's user only: + + ```bash + install -m 600 /dev/null /etc/sipnab/fleet/edge-a.token + ``` + + Then paste the token into it. sipnab refuses a token file that other users + can read. + +## Step 3: write the hosts file + +- **[controller]** Write `/etc/sipnab/fleet/fleet.toml`, one `[[host]]` table + per instance. This is the file from the run: + + ```toml + [[host]] + name = "hub" + url = "https://192.0.2.10:8443" + token_file = "/etc/sipnab/fleet/hub.token" + ca_file = "/etc/sipnab/fleet/internal-ca.pem" + + [[host]] + name = "edge-a" + url = "https://192.0.2.11:8443" + token_file = "/etc/sipnab/fleet/edge-a.token" + ca_file = "/etc/sipnab/fleet/internal-ca.pem" + + [[host]] + name = "edge-b" + url = "https://192.0.2.12:8443" + token_file = "/etc/sipnab/fleet/edge-b.token" + ca_file = "/etc/sipnab/fleet/internal-ca.pem" + ``` + + - `name` is the name the report shows. Each name appears once. + - `url` is the instance's REST API, `https://` or `http://`. + - `token_file` holds the instance's `inventory` token on one line. + - `ca_file` is optional: a PEM CA file to trust instead of the host's CA + bundle. + + sipnab refuses an unknown key, an empty name, an empty URL and a name used + twice, with exit `2`. + +## Step 4: read the fleet + +- **[controller]** Run: + + ```bash + sipnab fleet inventory --hosts /etc/sipnab/fleet/fleet.toml + ``` + + Output from the run: + + ```text + NAME STATE VERSION COMMIT BINARY CONFIG UID UPTIME_S + hub ok 0.5.232 29204db7 6bba53867ecd ec09522b49b4 1000 54 + edge-a ok 0.5.232 29204db7 6bba53867ecd 6a434df0c5e2 1000 40 + edge-b ok 0.5.232 29204db7 6bba53867ecd f63dfdce2c9d 1000 39 + ``` + + It exited `0`. + + - `BINARY` and `CONFIG` are the first 12 hex digits of `binary.sha256` and + `config.effective_sha256`. The three instances run one binary and three + configurations: each has its own config file and node name. + - `--json` prints the whole report, with every instance's full inventory. + - `--timeout ` sets how long each instance has to answer. Default `10`. + +## Verify + +- **Every instance is `ok`, and there is no `finding` line.** That is exit + `0`. +- **Two instances you expect to match show one `CONFIG`.** Two instances with + the same `CONFIG` loaded the same configuration values and run the same + command line. A different `--node-name` alone gives a different `CONFIG`. +- **The exit status** says the same for a script: + + | Code | Meaning | + |---|---| + | `0` | Every instance answered and there is no finding. | + | `1` | An instance did not answer, refused the token, or sent an inventory this build cannot read; or there is a finding. | + | `2` | sipnab cannot read or use the hosts file. | + +## Troubleshoot + +Each output below comes from the same run, with one thing changed. + +- **`unreachable` and `connection timed out`.** Nothing answered at that + address within `--timeout`: the instance is down, the URL names the wrong + address or port, or a firewall drops the controller's packets. The other + instances are still listed: + + ```text + edge-c unreachable 192.0.2.14:8443: connection timed out + ``` + + `Connection refused` instead means the host answered and nothing listens on + that port: check `--api` on the instance. + +- **`unreachable` and `invalid peer certificate: UnknownIssuer`.** The + controller does not trust the CA that issued the instance's certificate. Set + `ca_file` for that host: + + ```text + hub unreachable sending: invalid peer certificate: UnknownIssuer + ``` + +- **`refused` with HTTP `401`.** The instance did not accept the token: it was + minted with another instance's signing key, it expired, someone revoked it, or + its scope is not `inventory`, `full`, `actions` or `admin`. Mint a new one + on that instance (step 2). This run gave `edge-b` the hub's token: + + ```text + edge-b refused the instance answered HTTP 401 + ``` + +- **`refused` with HTTP `404`.** The instance runs a build without + `GET /v1/inventory`. Upgrade it. Release 0.5.130 answered: + + ```text + edge-old refused the instance answered HTTP 404 + ``` + + Release 0.5.130 also refuses `--token-scope inventory` at mint time, with + `invalid value 'inventory' for '--token-scope '`. + +- **`unreachable` and `can be read or written by other users`.** The token + file's mode lets other users read it. Run `chmod 600` on it: + + ```text + edge-a unreachable token_file '/etc/sipnab/fleet/open.token' can be read or written by other users (mode 0644); chmod 600 it + ``` + +- **A mixed HEP HMAC version.** A sender on 0.5.130 or earlier signs HEP with + HMAC version 1, and a collector on 0.5.131 or later drops every packet it + sends. In the run, `edge-old` (0.5.130) sent to `hub`. Listed in the hosts + file, `edge-old`'s row is `refused` with HTTP `404`, as above. The collector + logs `HEP HMAC auth is REFUSING version-1 tokens` once, and counts the + dropped packets. Read the count from the JSON report: + + ```bash + sipnab fleet inventory --hosts /etc/sipnab/fleet/fleet.toml --json \ + | jq -c '.hosts[] | {name, refused: .inventory.hep_listener.packets_refused, senders: [.inventory.hep_listener.senders[].peer]}' + ``` + + ```text + {"name":"hub","refused":10,"senders":["192.0.2.11","192.0.2.12"]} + {"name":"edge-a","refused":0,"senders":[]} + {"name":"edge-b","refused":0,"senders":[]} + ``` + + The hub refused 10 packets, and `192.0.2.13` is not among its senders. The + report itself is `ok` when the old sender is not in the hosts file: a + refused packet is not a finding. Upgrade the sender, as + [Upgrade a fleet that uses HMAC authentication](config-reference.md#upgrade-a-fleet-that-uses-hmac-authentication) + describes. + +- **`finding unlisted_hep_sender`.** A collector in the list receives HEP + from an address that no listed instance's URL resolves to. Add that sender + to the hosts file, or find out why it sends. With `edge-b` left out of the + file: + + ```text + finding unlisted_hep_sender: hub receives HEP from 192.0.2.12, which is no instance in the hosts file + ``` + + The check compares addresses only. A sender whose packets leave from a + different address than its API URL names shows up as unlisted. + +- **`incompatible` and a `schema_version`.** The instance answered an + inventory format newer than the controller reads. Upgrade the controller's + sipnab. No release answers a newer format yet, so this output comes from a + test server that answered `schema_version` 2: + + ```text + edge-next incompatible the instance answered inventory schema_version 2; this sipnab reads up to 1 + ``` + +## States and findings + +**State of each instance:** + +| State | Meaning | +|---|---| +| `ok` | It answered with an inventory. | +| `refused` | It answered with an HTTP status other than 200, such as `401` for a wrong, expired or narrower token, or `404` for a build without the inventory. | +| `unreachable` | No answer: no connection, a timeout, a TLS failure, a token file sipnab would not read, or an answer that is not JSON. | +| `incompatible` | Its inventory has a `schema_version` newer than this build reads, or none. Upgrade the controller's sipnab. | + +**Findings across the fleet:** + +| Finding | Meaning | +|---|---| +| `version_mismatch` | The instances that answered do not all run the same build. The finding names each build and the instances on it. | +| `unlisted_hep_sender` | A HEP collector in the list receives from an address that no listed instance's URL resolves to. | + +## Check a binary against its release + +Every instance reports `signature.state` as `unverified`. A process cannot +prove which binary it is, and one that someone had tampered with would report +"verified" too. + +- Download `SHA256SUMS.txt` from the release the instance claims, and verify + it: + + ```bash + gh attestation verify SHA256SUMS.txt --repo NormB/sipnab + ``` + +- Compare the instance's `binary.sha256` with the line for its artifact. On + Linux the instance hashes `/proc/self/exe`, so after an upgrade replaces the + file the digest still names the binary that is running. + +## What it cannot tell you + +- Every field is the instance's own claim. An attacker who controls a host can make + it report anything. +- For a secret held in a file, the digests cover its path only. The configuration and + the command line name key, token and password files by path, so replacing + the contents of such a file, for example rotating the key in + `--api-signing-key-file`, changes no digest. A secret given as a value, such + as `--api-signing-key` or a `${NAME}` the configuration expands, changes + `config.effective_sha256`. +- It reads only the instances in your hosts file. An instance nobody listed is + invisible to it, unless it sends HEP to a listed collector. +- It changes nothing. Pushing a configuration, upgrading and restarting are not + part of it. diff --git a/docs/mcp-estate.md b/docs/mcp-estate.md index d2c4fa22c..96b4c5d59 100644 --- a/docs/mcp-estate.md +++ b/docs/mcp-estate.md @@ -7,9 +7,11 @@ is what comes after that: - an agent that has to reach it from somewhere else, - one agent holding several capture hosts at once, - the hardest of them: following a single call as it crosses an SBC, a proxy - and a PBX that each see a different Call-ID. + and a PBX that each see a different Call-ID, +- managing the sipnab instances themselves: which build and configuration + each one runs, and which ones do not answer. -These four build on each other in that order. Each one names the SINGLE +The first four build on each other in that order. Each one names the SINGLE sipnab it starts from. So you can stop at whichever answers your question. Every scenario assumes you have already done [Install sipnab](mcp-deploy.md#install-sipnab-every-server-once). @@ -708,4 +710,32 @@ To centralize, see [Collect captures from several SIP servers in one place](#collect-captures-from-several-sip-servers-in-one-place). sipnab can run beside a Homer installation, as [Coexisting with Homer](#coexisting-with-homer-or-any-observability-host) -describes. \ No newline at end of file +describes. + +--- + +## Manage the fleet + +Federation and the central collector both answer questions about calls. +Neither says which build each node runs or which configuration it loaded. +`sipnab fleet inventory` does. [Manage many sipnab instances](fleet.md) is the +full how-to. + +- **[each node]** Run the REST API (`--api`) and mint a token with + `--token-scope inventory`. That token reads `GET /v1/inventory` and nothing + else: no dialogs, no messages, no audio. +- **[controller]** List the nodes in a hosts file and run + `sipnab fleet inventory --hosts fleet.toml`. It prints one row per node: the + build, the SHA-256 of the binary, the effective configuration digest, the + user id and the uptime. It marks a node that did not answer and exits `1`. +- **[controller]** Include the central collector in the hosts file. The + report then names every HEP sender the collector receives from whose + address is not in the hosts file, as the finding `unlisted_hep_sender`. +- **[agent]** An agent connected to one node asks the MCP tool + `fleet_inventory`. It returns the same record as `GET /v1/inventory` on + that node. + +Every field is the node's own claim. `signature.state` is always `unverified`: +compare `binary.sha256` with the release's attested `SHA256SUMS.txt`, as +[Check a binary against its release](fleet.md#check-a-binary-against-its-release) +shows. diff --git a/docs/mcp-protocol.md b/docs/mcp-protocol.md index 6c02ef8fa..9ccb632c1 100644 --- a/docs/mcp-protocol.md +++ b/docs/mcp-protocol.md @@ -163,8 +163,8 @@ lines on stdin, the way a client does. - **No prompt-injection cooperation.** Tool descriptions never instruct the LLM to "trust" or "act on" returned content. They describe what the tool returns and stop there. -- **Every tool declares what it does.** All 76 carry MCP annotations, so a host - can decide what to call without asking. Sixty-three are `readOnlyHint: true`. +- **Every tool declares what it does.** All 77 carry MCP annotations, so a host + can decide what to call without asking. Sixty-four are `readOnlyHint: true`. [What the write verbs do](#what-the-write-verbs-do) names the thirteen that are not. @@ -618,7 +618,7 @@ sipnab negotiates the 2025-06-18 and 2025-11-25 revisions. There, ## What the write verbs do -Sixty-three of the 76 tools are `readOnlyHint: true`. These thirteen are not. +Sixty-four of the 77 tools are `readOnlyHint: true`. These thirteen are not. Each declares what kind of change it makes, so a host can decide which need confirmation: diff --git a/docs/mcp-tools-server.md b/docs/mcp-tools-server.md index 037c5f31a..46271b8fe 100644 --- a/docs/mcp-tools-server.md +++ b/docs/mcp-tools-server.md @@ -15,6 +15,7 @@ the other bundles and finds any tool by name. |---|---|---| | [`runtime_stats`](#runtime_stats) | `sample_seconds?` | What sipnab is doing and what it costs the host: its own memory, threads, descriptors and CPU; the host's totals and the basis they came from; its share of them and whether that share is load-bearing; per-interface counters; store occupancy against the caps | | [`capture_health`](#capture_health) | `sample_seconds` | Capture-path counters read twice: run totals, deltas across the window, `undecoded_fraction`, and undecodable frames by reason | +| [`fleet_inventory`](#fleet_inventory) | -- | What this process is, for a fleet controller: build, binary SHA-256, signature state, configuration digests, privilege, uptime, interface, capture and HEP counters, and HEP sender addresses | | [`hep_senders`](#hep_senders) | `limit?` | Who feeds this run's HEP listener: each sender, its count, when it was last heard and whether it went silent, and every address the listener refused, by reason | | [`server_capabilities`](#server_capabilities) | -- | sipnab version, the optional features this binary carries, and the libpcap it runs | | [`shutdown_server`](#shutdown_server) | `dry_run?`, `save_to?`, `discard_unsaved?` | **Destructive.** Stops the process. Needs `--mcp-allow-shutdown`; dry-run by default | @@ -432,6 +433,69 @@ roster. The MCP answer differs in two ways: its keys are in alphabetical order, and it adds `source_exhausted` and `source_stopped_early`. `--hep-senders` prints the roster at the end of a headless run. +### `fleet_inventory` + +What this sipnab process is, for a fleet controller that manages many +instances: its build, the SHA-256 of its binary, its configuration as digests, +its privilege, its uptime and its counters. Starts nothing and waits for +nothing. + +**Parameters:** none. + +The example ran on 2026-11-03 with a development build, against +[`tests/pcap-samples/sip-rtp-g711.pcap`](https://github.com/NormB/sipnab/raw/main/tests/pcap-samples/sip-rtp-g711.pcap) +with `--node-name edge-a --no-config`: + +```jsonc +// fleet_inventory {} +{ + "schema_version": 1, + "node_name": "edge-a", + "build": { + "version": "0.5.232", "commit": "29204db7", "tag": "", "dirty": true, + "target": "aarch64-unknown-linux-gnu", + "features": ["api", "archive", "audio", "g729", "hep", "mcp", "mcp-http", "metrics", + "native", "opus", "plugins", "tls", "tui", "vcon"], + "version_line": "0.5.232 (29204db7-dirty) features: native,tui,audio,tls,hep,api,mcp,mcp-http,metrics,plugins,vcon,opus,archive,g729" + }, + "binary": { + "sha256": "3242a3324b1c3ac762ba15d4f2eec3789a995533540b8aee56ec087afdc51ea8", + "error": null + }, + "signature": { + "state": "unverified", + "note": "sipnab does not verify its own binary. Compare binary.sha256 with SHA256SUMS.txt of the release, after `gh attestation verify SHA256SUMS.txt --repo NormB/sipnab`." + }, + "config": { + "recorded": true, "source": null, "shadowed": [], "file_sha256": null, + "parsed_sha256": "afa5e308cbb51683fe7ac0ca79e2a33f601530f33439211dd29586c9db0c7d75", + "argv_sha256": "cb86e74e2f7d2f54c2e75c3cb01976de06c9658406112c318c5f290cf606418d", + "effective_sha256": "aee2e58c08b314be6538cf9ccf1dfddb992992c4e7336a4d256373b67be72130" + }, + "privilege": { "uid": 1000, "root": false, "dropped_to": null }, + "process": { "pid": 3985285, "started": "2026-11-03T13:26:30Z", "uptime_seconds": 1 }, + "interfaces": [], + "counters": { + "capture_packets_total": 852, "capture_sources_refused_total": 0, + "capture_backpressure_blocks_total": 0, "hep_export_failures_total": null, + "hep_export_dropped_total": null, "hep_export_queue_depth": null + }, + "hep_listener": { "listening": false, "packets_refused": 0, "senders": [] } +} +``` + +- No configuration value and no command-line argument is in the answer, only + their SHA-256 digests. A secret in either changes a digest and never appears. +- `signature.state` is always `unverified`. Compare `binary.sha256` with the + release's attested `SHA256SUMS.txt`. +- Every field is the process's own claim. + +`GET /v1/inventory` returns the same document, from the same builder. Unlike +most tools, the answer carries no `source_exhausted` or `source_stopped_early`: +it describes the process, not a result read from the capture. A `read` token +reaches this tool. [Manage many +sipnab instances](fleet.md) reads the REST route from every instance. + ### `server_capabilities` What this binary can do, what this server permits and loaded, the limits it @@ -483,7 +547,7 @@ opt-in flags: "evaluate_expectations", "explain_attribution", "explain_response_code", "explain_rule", "export_audio", "export_capture", "export_vcon", "fetch_vcon", "find_correlated", "find_in_captures", "find_in_captures_status", "find_problems", - "generate_fail2ban_rule", "generate_repro", "generate_wireshark_filter", + "fleet_inventory", "generate_fail2ban_rule", "generate_repro", "generate_wireshark_filter", "get_call_tree", "get_capture_report", "get_dialog", "get_dialog_report", "get_message", "get_sdp_timeline", "group_dialogs", "hep_senders", "lint_dialog", "list_captures", "list_dialogs", "list_tls_libraries", "media_diagnostics", diff --git a/docs/mcp-tools.md b/docs/mcp-tools.md index 71267ba3e..325b315b1 100644 --- a/docs/mcp-tools.md +++ b/docs/mcp-tools.md @@ -52,7 +52,7 @@ A bundle is a named set of tools defined in sipnab's code. | [`media`](mcp-tools-media.md) | 4 | Where a call's media endpoints came from, why a stream has no dialog, the facts under a quality score, and audio export. | | [`relay`](mcp-tools-relay.md) | 4 | Questions put to the configured media relay, and decoding the relay control messages sipnab captured. | | [`tfps`](mcp-tools-tfps.md) | 7 | The toll-fraud prevention peer (TFPS): what it blocks and logs, and the operator's ban, unban and revert actions. | -| [`server`](mcp-tools-server.md) | 5 | The sipnab process and its capture path: capture health, HEP senders, resource use, build capabilities, and shutdown. | +| [`server`](mcp-tools-server.md) | 6 | The sipnab process and its capture path: capture health, HEP senders, resource use, build capabilities, the fleet inventory record, and shutdown. | | [`vcon`](mcp-tools-vcon.md) | 7 | vCon conversation containers, exported, validated and analyzed, one at a time or a directory at once, read from the operator's store, the vCon export spool's counts, and the SIPREC recording metadata a call carried. | | [`tls`](mcp-tools-tls.md) | 3 | Reading SIP inside TLS without keys through kernel uprobes, and which TLS libraries this host runs. | @@ -126,22 +126,22 @@ resource. | Bundle | Tools | Bytes (default) | Bytes with output schemas | |---|---|---|---| -| core | 9 | 12,316 | 54,624 | -| signaling | 20 | 22,474 | 58,619 | -| captures | 11 | 12,723 | 30,211 | -| security | 6 | 9,954 | 9,954 | -| media | 4 | 3,480 | 16,048 | -| relay | 4 | 4,199 | 19,988 | -| tfps | 7 | 5,563 | 15,947 | -| server | 5 | 5,394 | 27,452 | -| vcon | 7 | 10,714 | 33,024 | -| tls | 3 | 2,850 | 4,943 | -| full | 76 | 89,667 | 270,810 | +| core | 9 | 12,308 | 58,240 | +| signaling | 20 | 22,452 | 66,162 | +| captures | 11 | 12,709 | 30,189 | +| security | 6 | 9,944 | 9,944 | +| media | 4 | 3,478 | 16,042 | +| relay | 4 | 4,199 | 20,607 | +| tfps | 7 | 5,717 | 16,101 | +| server | 6 | 5,972 | 35,126 | +| vcon | 7 | 10,710 | 34,428 | +| tls | 3 | 2,899 | 4,990 | +| full | 77 | 90,388 | 291,829 | Bytes are the compact JSON of each tool's `tools/list` entry, summed over the bundle. -> **Measured on sipnab 0.5.222** on a build with every feature. +> **Measured on sipnab 0.5.232** (a development build of this change, 2026-11-03) on a build with every feature. > > - Sizes, tool counts and bundle membership may change in later releases. > - A build without a feature does not register that feature's tools, so its @@ -181,6 +181,7 @@ links to its section on the bundle's page. | [`find_in_captures`](mcp-tools-captures.md#find_in_captures) | [captures](mcp-tools-captures.md) | Which capture files hold dialogs matching a filter, without opening any of them; a sweep that outlasts its wait returns a job | | [`find_in_captures_status`](mcp-tools-captures.md#find_in_captures_status) | [captures](mcp-tools-captures.md) | A `find_in_captures` job's progress, or its result once it finishes | | [`find_problems`](mcp-tools-core.md#find_problems) | [core](mcp-tools-core.md) | A page of dialogs matching one or more diagnostic alias names | +| [`fleet_inventory`](mcp-tools-server.md#fleet_inventory) | [server](mcp-tools-server.md) | What this process is, for a fleet controller: build, binary SHA-256, signature state, configuration digests, privilege, uptime and counters | | [`generate_fail2ban_rule`](mcp-tools-security.md#generate_fail2ban_rule) | [security](mcp-tools-security.md) | A fail2ban filter and jail derived from ONE recorded finding, with that finding attached as the evidence | | [`generate_repro`](mcp-tools-signaling.md#generate_repro) | [signaling](mcp-tools-signaling.md) | A SIPp scenario replaying one call, with the hypothesis as an input: `pin` holds the suspected cause fixed, `vary` regenerates identity | | [`generate_wireshark_filter`](mcp-tools-signaling.md#generate_wireshark_filter) | [signaling](mcp-tools-signaling.md) | A Wireshark display filter selecting one call's signaling and its RTP by SSRC, plus the tshark line that applies it | @@ -452,13 +453,15 @@ When `source_exhausted` is `false`, `total_matched` counts only what had loaded. Repeat the call, or poll `capture_status` until `source_exhausted` reads `true`, before treating a count as final. -Twenty-one tools are the exception, because their answer cannot move with +Twenty-two tools are the exception, because their answer cannot move with the load. How much of the capture sipnab has read says nothing about what their sources hold: -- Twelve read something other than the capture store: +- Thirteen read something other than the capture store: - `explain_response_code`, `explain_rule`, `decode_evidence`, `decode_ng`, `show_evidence`, `list_captures`, `list_tls_libraries`, `server_capabilities`; + - `fleet_inventory`, which describes the process and returns the same + document as the REST inventory route; - `vcon_spool_status`, which counts the vCon export spool that the forwarder drains; - `compare_captures`, which reads two files and never the loaded capture; diff --git a/docs/mcp.md b/docs/mcp.md index 08ab3f093..134b9d3c8 100644 --- a/docs/mcp.md +++ b/docs/mcp.md @@ -306,7 +306,7 @@ it does these things: It does for this surface what does for the [REST API](rest-api.md). -sipnab registers 76 MCP tools. That is more than anyone reads in a table. The +sipnab registers 77 MCP tools. That is more than anyone reads in a table. The Tools tab is the fastest way to find the one you want. The MCP project maintains Inspector, not sipnab. When a tool diff --git a/docs/rest-api.md b/docs/rest-api.md index 2638f72d8..e82e2ab4d 100644 --- a/docs/rest-api.md +++ b/docs/rest-api.md @@ -216,7 +216,7 @@ only. | `--api-token-ttl ` | Lifetime of a minted token. Default `3600`. | | `--mint-token` | Sign a token with the first configured key, print it, and exit. Starts no capture and no server. | | `--token-id ` | The token's `id` (`jti`), used later for revocation. Defaults to a generated id. | -| `--token-scope ` | `full` (default), `metrics`, `actions`, `admin` or `audio` for the REST API. See **Scope** below. | +| `--token-scope ` | `full` (default), `metrics`, `actions`, `admin`, `audio` or `inventory` for the REST API. See **Scope** below. | | `--api-revoked-file ` | Denylist of revoked token ids, one per line (blanks and `#` comments ignored). | **Scope.** Mint each client a token for the routes it needs: @@ -228,6 +228,7 @@ only. | `actions` | Everything `full` reaches, plus the action routes that ask TFPS to ban or unban a source. The run must also enable them with `--allow-action tfps:rest`. It lifts only the bans it placed. | The system that decides to block a source. | | `admin` | Everything `actions` reaches, and it also lifts a ban another caller placed. | An operator who must be able to lift any ban. | | `audio` | The live-audio routes a `live_audio` vCon rule serves, and nothing else. | A speech-to-text consumer. | +| `inventory` | `GET /v1/inventory` only. Every other route returns `401`. `full` reaches the inventory too. | A fleet controller running [`sipnab fleet inventory`](fleet.md). | The `read` scope is for MCP tokens. Minting it with `--api-signing-key` refuses. @@ -3222,6 +3223,103 @@ The MCP tool `server_capabilities` returns the same feature set and the same `libpcap` block. The second line of `sipnab --version` prints the same library banner. +### GET /v1/inventory + +What this sipnab process is, for managing many instances as one fleet: + +- its build: version, git commit and tag, target and compiled features, +- SHA-256 of the binary it is running, +- its configuration, as digests only, +- its user id and privilege drop, its pid, start time and uptime, +- its capture interfaces' error and drop counters, its capture and HEP export + counters, and the addresses its HEP listener receives from. + +It needs a token with scope `inventory`, or `full`. [`sipnab fleet +inventory`](fleet.md) reads it from every instance in a hosts file. + +**curl:** + +```bash +curl -s -H "Authorization: Bearer $SIPNAB_INVENTORY_TOKEN" http://127.0.0.1:8080/v1/inventory | jq . +``` + +**Response** (from a run on 2026-11-03 of a development build, with the config path replaced by `/etc/sipnab/sipnab.toml`): + +```json +{ + "schema_version": 1, + "node_name": "edge-a", + "build": { + "version": "0.5.232", + "commit": "29204db7", + "tag": "", + "dirty": true, + "target": "aarch64-unknown-linux-gnu", + "features": ["api", "archive", "audio", "g729", "hep", "mcp", "mcp-http", "metrics", + "native", "opus", "plugins", "tls", "tui", "vcon"], + "version_line": "0.5.232 (29204db7-dirty) features: native,tui,audio,tls,hep,api,mcp,mcp-http,metrics,plugins,vcon,opus,archive,g729" + }, + "binary": { + "sha256": "3242a3324b1c3ac762ba15d4f2eec3789a995533540b8aee56ec087afdc51ea8", + "error": null + }, + "signature": { + "state": "unverified", + "note": "sipnab does not verify its own binary. Compare binary.sha256 with SHA256SUMS.txt of the release, after `gh attestation verify SHA256SUMS.txt --repo NormB/sipnab`." + }, + "config": { + "recorded": true, + "source": "/etc/sipnab/sipnab.toml", + "shadowed": [], + "file_sha256": "703493b6b6bdef18b62090d6452a2bd9fc72de264d4b97822cb0b7763697fca6", + "parsed_sha256": "1849a6ee687ec0b1394f2d305198c638bca159e7aa032911156074b8d11f4044", + "argv_sha256": "d5c72de2bcf085f7d8ca9884bed30027d7504215d507b04f6be16ecc063194e0", + "effective_sha256": "31e67effe25101b07a2c7942f05b83d568816bf81a1769d65ab8396c8e89705a" + }, + "privilege": { "uid": 1000, "root": false, "dropped_to": null }, + "process": { "pid": 3983974, "started": "2026-11-03T13:26:05Z", "uptime_seconds": 17 }, + "interfaces": [], + "counters": { + "capture_packets_total": 852, + "capture_sources_refused_total": 0, + "capture_backpressure_blocks_total": 0, + "hep_export_failures_total": null, + "hep_export_dropped_total": null, + "hep_export_queue_depth": null + }, + "hep_listener": { "listening": false, "packets_refused": 0, "senders": [] } +} +``` + +**No configuration value is in the answer.** + +- `file_sha256` is SHA-256 of the config file's bytes as this run read them. +- `parsed_sha256` is SHA-256 of the configuration as sipnab parsed it, after + `${NAME}` expansion, with every object's keys sorted. Two files that differ + only in comments, whitespace, key order or an unknown key give one value. +- `argv_sha256` is SHA-256 of the command line, without the program path. +- `effective_sha256` is SHA-256 over `parsed_sha256` and `argv_sha256`. A flag + overrides the file, so this is the digest to compare across instances. + `sipnab fleet inventory` shows it as `CONFIG`. +- A secret in the file or on the command line changes a digest, and its value + never appears. A secret held in a file the configuration names, such as + `--api-signing-key-file`, reaches the digests by its path only. + +**`signature.state` is always `unverified`.** A process cannot prove which +binary it is. Compare `binary.sha256` with the release's `SHA256SUMS.txt`, +after verifying that file's attestation with `gh attestation verify`. +sipnab reads `binary.sha256` from `/proc/self/exe` on Linux, so after an upgrade +replaces the file it still names the binary that is running. + +**Every field is the process's own claim.** An attacker who controls a host +can report anything. + +`hep_listener.senders` lists at most `MAX_HEP_PEERS` (256) senders. + +`config.recorded` is `false`, and every digest `null`, for a server that did not +record its configuration at start. The MCP tool `fleet_inventory` returns the +same fields and values. + ### GET /v1/stats Aggregate statistics across all dialogs and streams, including PDD percentiles. diff --git a/man/sipnab.1 b/man/sipnab.1 index 6d7eac650..3b6f90d78 100644 --- a/man/sipnab.1 +++ b/man/sipnab.1 @@ -1072,8 +1072,8 @@ Mint a signed bearer token from the first configured signing key, print it to st Token id (jti) for \-\-mint\-token. Defaults to a derived unique id. .TP \fB\-\-token\-scope\fR \fISCOPE\fR -Scope for \-\-mint\-token: \fBfull\fR (default), \fBmetrics\fR, \fBread\fR, \fBactions\fR or \fBaudio\fR. -Values: \fBfull\fR, \fBmetrics\fR, \fBread\fR, \fBactions\fR, \fBadmin\fR, \fBaudio\fR. +Scope for \-\-mint\-token: \fBfull\fR (default), \fBmetrics\fR, \fBread\fR, \fBactions\fR, \fBadmin\fR, \fBaudio\fR or \fBinventory\fR. +Values: \fBfull\fR, \fBmetrics\fR, \fBread\fR, \fBactions\fR, \fBadmin\fR, \fBaudio\fR, \fBinventory\fR. Default: \fBfull\fR. .SS vCon forwarder .TP diff --git a/scripts/build-site-internals.py b/scripts/build-site-internals.py index f9eabae71..0d1f7b720 100755 --- a/scripts/build-site-internals.py +++ b/scripts/build-site-internals.py @@ -273,6 +273,7 @@ "mcp.md": "mcp.md", "mcp-deploy.md": "mcp-deploy.md", "mcp-estate.md": "mcp-estate.md", + "fleet.md": "fleet.md", "mcp-tools.md": "mcp-tools.md", "mcp-tools-core.md": "mcp-tools-core.md", "mcp-tools-signaling.md": "mcp-tools-signaling.md", diff --git a/scripts/build-site-pages.py b/scripts/build-site-pages.py index 35b31fc4f..c57a86c53 100644 --- a/scripts/build-site-pages.py +++ b/scripts/build-site-pages.py @@ -300,6 +300,18 @@ "outside the network, one agent holding many capture hosts, and " "following one call across an SBC, a proxy and a PBX.", ), + ( + "docs/fleet.md", + "fleet.md", + "Manage many sipnab instances", + "Manage many sipnab instances", + # 66: the first weight free on the SITE side, which + # `docs_page_weights_are_unique_and_descriptions_present` reads. + 66, + "One table of every sipnab instance's build, binary and " + "configuration digests, privilege and uptime, the instances that " + "did not answer, and HEP senders nobody listed.", + ), ( "docs/mcp-tools.md", "mcp-tools.md", diff --git a/scripts/build-wiki.py b/scripts/build-wiki.py index fa5130992..1871f51a6 100755 --- a/scripts/build-wiki.py +++ b/scripts/build-wiki.py @@ -95,6 +95,7 @@ "uprobe-walkthrough.md": "Uprobe-Walkthrough", "mcp-deploy.md": "MCP-Deployment", "mcp-estate.md": "MCP-Across-an-Estate", + "fleet.md": "Manage-Many-Instances", "mcp-tools.md": "MCP-Tools", "mcp-tools-core.md": "MCP-Tools-Core", "mcp-tools-signaling.md": "MCP-Tools-Signaling", @@ -158,7 +159,7 @@ "mos-and-codecs.md"]), ("Configuration", ["config-reference.md"]), ("Integrations (API & MCP)", ["rest-api.md", "prometheus-metrics.md", "auth.md", "mcp.md", - "mcp-deploy.md", "mcp-estate.md", "mcp-tools.md", + "mcp-deploy.md", "mcp-estate.md", "fleet.md", "mcp-tools.md", "mcp-tools-core.md", "mcp-tools-signaling.md", "mcp-tools-captures.md", "mcp-tools-security.md", "mcp-tools-media.md", "mcp-tools-relay.md", "mcp-tools-tfps.md", "mcp-tools-server.md", "mcp-tools-vcon.md", diff --git a/src/app/bootstrap.rs b/src/app/bootstrap.rs index b56f284ec..2b80ebbe6 100644 --- a/src/app/bootstrap.rs +++ b/src/app/bootstrap.rs @@ -7284,6 +7284,16 @@ fn mint_token(cli: &Cli) -> Result { .to_string(), ); } + // Nor an inventory route: MCP serves `fleet_inventory` to a `read` token. + if cli.token_args.token_scope == crate::auth::SCOPE_INVENTORY + && audience == crate::auth::AUDIENCE_MCP + { + return Err( + "--token-scope inventory applies to the REST API only; on the MCP surface \ + a `read` token reaches the fleet_inventory tool" + .to_string(), + ); + } // And the REST API has no read-only scope: its routes are one trust // domain apart from /metrics, so a `read` API token would verify and then // be refused by every scope check — worse than failing at mint time, diff --git a/src/app/fleet.rs b/src/app/fleet.rs new file mode 100644 index 000000000..6b510105e --- /dev/null +++ b/src/app/fleet.rs @@ -0,0 +1,709 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// SPDX-FileCopyrightText: The sipnab contributors + +//! `sipnab fleet inventory`: ask every sipnab instance in a hosts file for +//! its inventory, and report the fleet. +//! +//! # Pull, from a static list +//! +//! The controller asks; the instances never call in. Each instance already +//! serves an authenticated HTTP API, so pulling needs no listener here and no +//! new trust on the instances, and an instance that does not answer is +//! itself the finding. The list is a file the operator keeps, so an instance +//! is managed only when someone named it. +//! +//! # What is pure and what is not +//! +//! [`parse_hosts`] and [`merge`] take their inputs as arguments and are where +//! every rule lives: which answer is usable, which instances disagree, which +//! HEP sender nobody listed. [`fetch`] is the one function that opens a +//! connection, and it decides nothing. + +use std::net::IpAddr; +use std::path::PathBuf; + +use serde::{Deserialize, Serialize}; + +/// Wire-format version of [`FleetReport`]. +pub const REPORT_SCHEMA_VERSION: u32 = 1; + +/// The highest inventory `schema_version` this build reads. +pub const INVENTORY_SCHEMA_VERSION: u64 = 1; + +/// The most bytes of one inventory answer this command reads. +const MAX_ANSWER_BYTES: usize = 4 * 1024 * 1024; + +/// The most bytes of one answer's status line and headers. +const MAX_HEAD_BYTES: usize = 16 * 1024; + +/// The hosts file. +#[derive(Debug, Clone, Deserialize, PartialEq, Eq)] +#[serde(deny_unknown_fields)] +struct HostsFile { + /// One table per instance. + #[serde(default)] + host: Vec, +} + +/// One instance in the hosts file. +#[derive(Debug, Clone, Deserialize, PartialEq, Eq)] +#[serde(deny_unknown_fields)] +pub struct HostEntry { + /// The name the report shows; unique in the file. + pub name: String, + /// The instance's REST API base URL, `http://` or `https://`. + pub url: String, + /// A file holding the bearer token for the instance, readable by this + /// user only. + pub token_file: PathBuf, + /// A PEM CA file to trust instead of the host's bundle, for `https://`. + #[serde(default)] + pub ca_file: Option, +} + +/// Parse a hosts file. +/// +/// # Errors +/// +/// Not TOML, an unknown key, no `[[host]]` table, an empty or repeated +/// name, or an empty URL. The message names the problem. +pub fn parse_hosts(text: &str) -> Result, String> { + let file: HostsFile = toml::from_str(text).map_err(|e| e.message().to_string())?; + if file.host.is_empty() { + return Err("the hosts file names no [[host]]".to_string()); + } + let mut seen = std::collections::BTreeSet::new(); + for h in &file.host { + if h.name.trim().is_empty() { + return Err("a [[host]] has an empty name".to_string()); + } + if h.url.trim().is_empty() { + return Err(format!("host '{}' has an empty url", h.name)); + } + if !seen.insert(h.name.as_str()) { + return Err(format!("host '{}' is named twice", h.name)); + } + } + Ok(file.host) +} + +/// What one instance answered. +#[derive(Debug, Clone, PartialEq)] +pub enum Answer { + /// A 200 with a JSON body. + Inventory(serde_json::Value), + /// An HTTP status other than 200. + Refused(u16), + /// No answer: no connection, a timeout, or a body that is not JSON. + Unreachable(String), +} + +/// One instance's result. +#[derive(Debug, Clone, PartialEq)] +pub struct HostResult { + /// Its name in the hosts file. + pub name: String, + /// Its URL. + pub url: String, + /// The addresses its URL's host resolved to, for matching HEP senders. + pub addresses: Vec, + /// What it answered. + pub answer: Answer, +} + +/// The fleet report. +#[derive(Debug, Clone, Serialize, PartialEq)] +pub struct FleetReport { + /// Wire-format version, [`REPORT_SCHEMA_VERSION`]. + pub schema_version: u32, + /// Whether every instance answered and there is no finding. + pub ok: bool, + /// One row per instance, in hosts-file order. + pub hosts: Vec, + /// What the fleet as a whole needs looked at. + pub findings: Vec, +} + +/// One instance in the report. +#[derive(Debug, Clone, Serialize, PartialEq)] +pub struct HostRow { + /// Its name in the hosts file. + pub name: String, + /// Its URL. + pub url: String, + /// `ok`, `refused`, `unreachable` or `incompatible`. + pub state: String, + /// Why it is not `ok`. + pub error: Option, + /// The inventory it returned, when `ok`. + pub inventory: Option, +} + +/// One finding across the fleet. +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +pub struct Finding { + /// `version_mismatch` or `unlisted_hep_sender`. + pub kind: String, + /// What was found, naming the instances. + pub detail: String, +} + +/// The report for `results`. +#[must_use] +pub fn merge(results: Vec) -> FleetReport { + let listed: std::collections::BTreeSet = results + .iter() + .flat_map(|r| r.addresses.iter().copied()) + .collect(); + let mut hosts = Vec::with_capacity(results.len()); + let mut versions: std::collections::BTreeMap> = Default::default(); + let mut findings = Vec::new(); + for r in results { + let (state, error, inventory) = match r.answer { + Answer::Refused(code) => ( + "refused", + Some(format!("the instance answered HTTP {code}")), + None, + ), + Answer::Unreachable(why) => ("unreachable", Some(why), None), + Answer::Inventory(v) => match v["schema_version"].as_u64() { + Some(n) if (1..=INVENTORY_SCHEMA_VERSION).contains(&n) => ("ok", None, Some(v)), + Some(n) => ( + "incompatible", + Some(format!( + "the instance answered inventory schema_version {n}; this sipnab \ + reads up to {INVENTORY_SCHEMA_VERSION}" + )), + None, + ), + None => ( + "incompatible", + Some("the answer carries no inventory schema_version".to_string()), + None, + ), + }, + }; + if let Some(v) = &inventory { + let line = v["build"]["version_line"].as_str().unwrap_or("unknown"); + versions + .entry(line.to_string()) + .or_default() + .push(r.name.clone()); + let senders = v["hep_listener"]["senders"].as_array().map(Vec::as_slice); + for peer in senders.unwrap_or_default() { + let Some(text) = peer["peer"].as_str() else { + continue; + }; + let known = text.parse::().is_ok_and(|ip| listed.contains(&ip)); + if !known { + findings.push(Finding { + kind: "unlisted_hep_sender".to_string(), + detail: format!( + "{} receives HEP from {text}, which is no instance in the \ + hosts file", + r.name + ), + }); + } + } + } + hosts.push(HostRow { + name: r.name, + url: r.url, + state: state.to_string(), + error, + inventory, + }); + } + if versions.len() > 1 { + let detail = versions + .iter() + .map(|(line, names)| format!("{} run {line}", names.join(", "))) + .collect::>() + .join("; "); + findings.insert( + 0, + Finding { + kind: "version_mismatch".to_string(), + detail, + }, + ); + } + let ok = findings.is_empty() && hosts.iter().all(|h| h.state == "ok"); + FleetReport { + schema_version: REPORT_SCHEMA_VERSION, + ok, + hosts, + findings, + } +} + +/// The report as a table, one line per instance, then the findings. +#[must_use] +pub fn render_table(report: &FleetReport) -> String { + use std::fmt::Write as _; + /// The first twelve characters of a digest, or `-`. + fn short(v: &serde_json::Value) -> String { + v.as_str() + .map_or_else(|| "-".to_string(), |d| d.chars().take(12).collect()) + } + let mut out = String::new(); + let _ = writeln!( + out, + "{:<16} {:<12} {:<12} {:<12} {:<12} {:<12} {:>6} {:>10}", + "NAME", "STATE", "VERSION", "COMMIT", "BINARY", "CONFIG", "UID", "UPTIME_S" + ); + for h in &report.hosts { + match &h.inventory { + Some(v) => { + let _ = writeln!( + out, + "{:<16} {:<12} {:<12} {:<12} {:<12} {:<12} {:>6} {:>10}", + h.name, + h.state, + v["build"]["version"].as_str().unwrap_or("-"), + short(&v["build"]["commit"]), + short(&v["binary"]["sha256"]), + short(&v["config"]["effective_sha256"]), + v["privilege"]["uid"] + .as_u64() + .map_or_else(|| "-".to_string(), |u| u.to_string()), + v["process"]["uptime_seconds"] + .as_u64() + .map_or_else(|| "-".to_string(), |u| u.to_string()), + ); + } + None => { + let _ = writeln!( + out, + "{:<16} {:<12} {}", + h.name, + h.state, + h.error.as_deref().unwrap_or("") + ); + } + } + } + for f in &report.findings { + let _ = writeln!(out, "finding {}: {}", f.kind, f.detail); + } + out +} + +/// Ask `entry` for its inventory, waiting at most `timeout` for each step. +/// +/// The token is read from `entry.token_file` and goes nowhere but the +/// `Authorization` header: no error this returns quotes it. +#[must_use] +pub fn fetch(entry: &HostEntry, timeout: std::time::Duration) -> HostResult { + let mut addresses = Vec::new(); + let answer = ask(entry, timeout, &mut addresses).unwrap_or_else(Answer::Unreachable); + HostResult { + name: entry.name.clone(), + url: entry.url.clone(), + addresses, + answer, + } +} + +/// [`fetch`]'s work, with every failure as the message for +/// [`Answer::Unreachable`]. `addresses` receives what the URL's host +/// resolved to, even when the request then fails. +fn ask( + entry: &HostEntry, + timeout: std::time::Duration, + addresses: &mut Vec, +) -> Result { + use super::https_client::{Endpoint, UrlUse}; + use std::io::Read as _; + use std::net::ToSocketAddrs as _; + + let base = Endpoint::parse(&entry.url, UrlUse::FLEET)?; + if let Ok(found) = (base.host.as_str(), base.port).to_socket_addrs() { + addresses.extend(found.map(|a| a.ip())); + } + let mut token = String::new(); + crate::privilege::open_private_file(&entry.token_file, "token_file")? + .read_to_string(&mut token) + .map_err(|e| format!("token_file '{}': {e}", entry.token_file.display()))?; + let token = token.trim(); + if token.is_empty() || token.chars().any(|c| c.is_whitespace() || c.is_control()) { + return Err(format!( + "token_file '{}' does not hold one token on one line", + entry.token_file.display() + )); + } + let tls = if base.tls { + Some(super::https_client::client_config( + entry.ca_file.as_deref(), + None, + &crate::tls_files::TrustNames { + ca: "ca_file", + extra: "ca_file", + issuer: "the instance's issuer", + }, + )?) + } else { + None + }; + let path = format!("{}/v1/inventory", base.target.trim_end_matches('/')); + let request = format!( + "GET {path} HTTP/1.1\r\nHost: {}\r\nAuthorization: Bearer {token}\r\n\ + Accept: application/json\r\nConnection: close\r\n\r\n", + base.host_header() + ); + super::https_client::exchange(&base, timeout, tls.as_ref(), request.as_bytes(), |reader| { + let head = super::https_client::read_head(reader, MAX_HEAD_BYTES, "the answer's headers")?; + let status = super::https_client::status_of(&head)?; + if status != 200 { + return Ok(Answer::Refused(status)); + } + let (body, more) = super::https_client::read_body(reader, &head, MAX_ANSWER_BYTES); + if more { + return Err(format!( + "the answer is larger than {MAX_ANSWER_BYTES} bytes" + )); + } + serde_json::from_slice(&body) + .map(Answer::Inventory) + .map_err(|e| format!("the answer is not JSON: {e}")) + }) +} + +/// `sipnab fleet ...`, with `args` as its whole command line, the first +/// being the command's own name. Returns the exit code. +pub fn main(args: impl IntoIterator) -> i32 { + use clap::Parser as _; + use std::io::Write as _; + let cli = match crate::cli::fleet_args::FleetCli::try_parse_from(args) { + Ok(cli) => cli, + Err(e) => { + let _ = e.print(); + return if e.use_stderr() { 2 } else { 0 }; + } + }; + let crate::cli::fleet_args::FleetCommand::Inventory { + hosts, + json, + timeout, + } = cli.command; + let mut err = std::io::stderr(); + let entries = match std::fs::read_to_string(&hosts) + .map_err(|e| e.to_string()) + .and_then(|t| parse_hosts(&t)) + { + Ok(entries) => entries, + Err(e) => { + let _ = writeln!(err, "sipnab fleet: --hosts '{}': {e}", hosts.display()); + return 2; + } + }; + let timeout = std::time::Duration::from_secs(timeout); + // One thread per instance, so one slow host costs one timeout, not one + // per host after it. The order of the report is the file's order. + let results: Vec = std::thread::scope(|scope| { + let handles: Vec<_> = entries + .iter() + .map(|e| scope.spawn(move || fetch(e, timeout))) + .collect(); + handles + .into_iter() + .zip(&entries) + .map(|(h, e)| { + h.join().unwrap_or_else(|_| HostResult { + name: e.name.clone(), + url: e.url.clone(), + addresses: Vec::new(), + answer: Answer::Unreachable("the request thread failed".to_string()), + }) + }) + .collect() + }); + let report = merge(results); + let text = if json { + match serde_json::to_string_pretty(&report) { + Ok(t) => t + "\n", + Err(e) => { + let _ = writeln!(err, "sipnab fleet: the report did not serialize: {e}"); + return 1; + } + } + } else { + render_table(&report) + }; + if std::io::stdout().write_all(text.as_bytes()).is_err() { + return 1; + } + i32::from(!report.ok) +} + +#[cfg(test)] +mod tests { + use super::*; + + type TestError = Box; + + fn inv(node: &str, version_line: &str, senders: &[&str]) -> serde_json::Value { + let senders: Vec = senders + .iter() + .map(|p| serde_json::json!({"peer": p, "capture_id": 7, "silent": false})) + .collect(); + serde_json::json!({ + "schema_version": 1, + "node_name": node, + "build": {"version_line": version_line, "version": "0.0.0", "commit": "abc"}, + "binary": {"sha256": "00"}, + "config": { + "parsed_sha256": "parsedparsedparsed", + "file_sha256": "filefilefilefile", + "effective_sha256": format!("eff0{node}0123456789"), + }, + "privilege": {"uid": 1000, "root": false}, + "process": {"uptime_seconds": 5}, + "hep_listener": {"listening": !senders.is_empty(), "senders": senders}, + }) + } + + fn host(name: &str, ip: &str, answer: Answer) -> Result { + Ok(HostResult { + name: name.to_string(), + url: format!("http://{ip}:8080"), + addresses: vec![ip.parse()?], + answer, + }) + } + + /// One instance down: the report still lists the other, says which one + /// failed and why, and is not ok. + #[test] + fn one_unreachable_host_is_reported_and_the_rest_listed() -> Result<(), TestError> { + let r = merge(vec![ + host( + "edge-a", + "192.0.2.10", + Answer::Inventory(inv("edge-a", "v1", &[])), + )?, + host( + "edge-b", + "192.0.2.11", + Answer::Unreachable("connection refused".into()), + )?, + host("edge-c", "192.0.2.12", Answer::Refused(401))?, + ]); + assert!(!r.ok, "two of three hosts failed, so the fleet is not ok"); + let states: Vec<&str> = r.hosts.iter().map(|h| h.state.as_str()).collect(); + assert_eq!( + states, + ["ok", "unreachable", "refused"], + "one row per host, in hosts-file order, each with its own state" + ); + assert_eq!( + r.hosts[1].error.as_deref(), + Some("connection refused"), + "the unreachable row carries the reason the request failed" + ); + assert!( + r.hosts[2] + .error + .as_deref() + .is_some_and(|e| e.contains("401")), + "the refused row names the HTTP status" + ); + assert_eq!( + r.hosts[0] + .inventory + .as_ref() + .map(|v| v["node_name"].clone()), + Some(serde_json::json!("edge-a")), + "the answering host's inventory is kept" + ); + assert_eq!( + r.hosts[1].inventory, None, + "no inventory for a host that did not answer" + ); + Ok(()) + } + + /// All answering, same build, no stray sender: ok, no findings. + #[test] + fn a_healthy_fleet_is_ok() -> Result<(), TestError> { + let r = merge(vec![ + host( + "edge-a", + "192.0.2.10", + Answer::Inventory(inv("edge-a", "v1", &[])), + )?, + host( + "hub", + "192.0.2.1", + Answer::Inventory(inv("hub", "v1", &["192.0.2.10"])), + )?, + ]); + assert!(r.ok, "{r:?}"); + assert!(r.findings.is_empty(), "{:?}", r.findings); + assert_eq!(r.hosts.len(), 2, "both answering hosts are listed"); + Ok(()) + } + + /// A newer inventory schema, or none, is reported as incompatible and + /// not read as zeros. + #[test] + fn an_unknown_schema_is_incompatible_not_misread() -> Result<(), TestError> { + let mut newer = inv("edge-a", "v1", &[]); + newer["schema_version"] = serde_json::json!(2); + let r = merge(vec![ + host("edge-a", "192.0.2.10", Answer::Inventory(newer))?, + host( + "edge-b", + "192.0.2.11", + Answer::Inventory(serde_json::json!({"x": 1})), + )?, + ]); + assert!(!r.ok, "an incompatible answer makes the fleet not ok"); + for row in &r.hosts { + assert_eq!(row.state, "incompatible", "{row:?}"); + assert_eq!( + row.inventory, None, + "an unreadable record was passed on: {row:?}" + ); + } + assert!( + r.hosts[0].error.as_deref().is_some_and(|e| e.contains('2')), + "the error names the schema_version the host sent" + ); + Ok(()) + } + + /// A HEP sender whose address is no listed instance's is a finding, + /// naming the collector and the address; a listed one is not. + #[test] + fn an_unlisted_hep_sender_is_a_finding() -> Result<(), TestError> { + let r = merge(vec![ + host( + "edge-a", + "192.0.2.10", + Answer::Inventory(inv("edge-a", "v1", &[])), + )?, + host( + "hub", + "192.0.2.1", + Answer::Inventory(inv("hub", "v1", &["192.0.2.10", "198.51.100.66"])), + )?, + ]); + assert!(!r.ok, "an unlisted HEP sender makes the fleet not ok"); + assert_eq!(r.findings.len(), 1, "{:?}", r.findings); + assert_eq!( + r.findings[0].kind, "unlisted_hep_sender", + "the finding is the unlisted sender" + ); + assert!( + r.findings[0].detail.contains("198.51.100.66"), + "{:?}", + r.findings + ); + assert!(r.findings[0].detail.contains("hub"), "{:?}", r.findings); + Ok(()) + } + + /// Two builds in one fleet is a finding naming both. + #[test] + fn two_builds_in_one_fleet_is_a_finding() -> Result<(), TestError> { + let r = merge(vec![ + host( + "edge-a", + "192.0.2.10", + Answer::Inventory(inv("edge-a", "0.5.130", &[])), + )?, + host( + "edge-b", + "192.0.2.11", + Answer::Inventory(inv("edge-b", "0.5.131", &[])), + )?, + ]); + assert!(!r.ok, "two builds make the fleet not ok"); + assert_eq!(r.findings.len(), 1, "{:?}", r.findings); + assert_eq!( + r.findings[0].kind, "version_mismatch", + "the finding is the build mismatch" + ); + assert!( + r.findings[0].detail.contains("0.5.130") && r.findings[0].detail.contains("0.5.131"), + "the finding names both builds" + ); + Ok(()) + } + + /// The hosts file: names, URLs and token files; unknown keys, repeated + /// names and an empty list refused. + #[test] + fn the_hosts_file_is_parsed_and_checked() { + let good = "[[host]]\nname = \"edge-a\"\nurl = \"https://edge-a.example.net:8443\"\n\ + token_file = \"/etc/sipnab/fleet/edge-a.token\"\n\ + [[host]]\nname = \"hub\"\nurl = \"http://192.0.2.1:8080\"\n\ + token_file = \"hub.token\"\nca_file = \"ca.pem\"\n"; + let hosts = parse_hosts(good); + assert_eq!(hosts.as_ref().map(Vec::len), Ok(2), "{hosts:?}"); + let refused = [ + ("", "no host"), + ( + "[[host]]\nname = \"a\"\nurl = \"http://192.0.2.1\"\ntoken_file = \"t\"\nport = 1\n", + "unknown", + ), + ( + "[[host]]\nname = \"a\"\nurl = \"http://192.0.2.1\"\ntoken_file = \"t\"\n\ + [[host]]\nname = \"a\"\nurl = \"http://192.0.2.2\"\ntoken_file = \"t\"\n", + "twice", + ), + ( + "[[host]]\nname = \"\"\nurl = \"http://192.0.2.1\"\ntoken_file = \"t\"\n", + "empty name", + ), + ]; + for (text, why) in refused { + assert!(parse_hosts(text).is_err(), "accepted ({why}): {text:?}"); + } + } + + /// The table names every instance and its state, and every finding. + #[test] + fn the_table_shows_every_host_and_finding() -> Result<(), TestError> { + let r = merge(vec![ + host( + "edge-a", + "192.0.2.10", + Answer::Inventory(inv("edge-a", "0.5.130", &[])), + )?, + host( + "edge-b", + "192.0.2.11", + Answer::Inventory(inv("edge-b", "0.5.131", &[])), + )?, + host( + "edge-c", + "192.0.2.12", + Answer::Unreachable("timed out".into()), + )?, + ]); + let t = render_table(&r); + for want in [ + "edge-a", + "edge-b", + "edge-c", + "unreachable", + "timed out", + "version_mismatch", + ] { + assert!(t.contains(want), "{want} missing from:\n{t}"); + } + // CONFIG is the effective digest, per instance: the one that also + // moves when a flag does. + let row = |name: &str| t.lines().find(|l| l.starts_with(name)).unwrap_or_default(); + assert!(row("edge-a").contains("eff0edge-a01"), "{t}"); + assert!(row("edge-b").contains("eff0edge-b01"), "{t}"); + assert!( + !t.contains("parsedparsed"), + "the parsed digest is not CONFIG:\n{t}" + ); + Ok(()) + } +} diff --git a/src/app/https_client.rs b/src/app/https_client.rs index 1b00b5a6a..bf1d9a028 100644 --- a/src/app/https_client.rs +++ b/src/app/https_client.rs @@ -59,6 +59,11 @@ impl UrlUse { client: "the enricher", credentials: Some("[[vcon.enrich]] auth_file"), }; + /// An instance URL in a `sipnab fleet` hosts file. + pub const FLEET: Self = Self { + client: "sipnab fleet", + credentials: Some("the host's token_file"), + }; /// A URL sipnab reads without credentials: one named inside a vCon, a /// transparency service's `Location`, or the G.729 package download. pub const DOWNLOAD: Self = Self { diff --git a/src/app/mod.rs b/src/app/mod.rs index 924cc2f98..7ba2af7fd 100644 --- a/src/app/mod.rs +++ b/src/app/mod.rs @@ -8,9 +8,11 @@ pub mod batch; pub mod bootstrap; +#[cfg(feature = "api")] +pub mod fleet; #[cfg(feature = "g729")] pub mod g729_install; -#[cfg(any(feature = "vcon", feature = "g729"))] +#[cfg(any(feature = "vcon", feature = "g729", feature = "api"))] pub mod https_client; #[cfg(all(unix, any(feature = "api", feature = "mcp")))] pub mod journal_cli; diff --git a/src/auth.rs b/src/auth.rs index ed3922afc..3192e40d4 100644 --- a/src/auth.rs +++ b/src/auth.rs @@ -130,6 +130,17 @@ pub const SCOPE_ADMIN: &str = "admin"; /// dashboard is not. pub const SCOPE_AUDIO: &str = "audio"; +/// The fleet inventory: `GET /v1/inventory` on the REST API and nothing +/// else. +/// +/// A fleet controller polls every instance it manages. Before this scope it +/// had to hold a `full` token for each, which reads `/v1/dialogs` and the +/// message bodies of every call the instance sees. The inventory carries +/// digests, versions and counters and no call content, so the credential +/// that reads it reaches nothing more. `full` reaches it too, by the rule +/// every narrow scope follows. +pub const SCOPE_INVENTORY: &str = "inventory"; + /// Whether a token holding `held` may reach something that requires `required`. /// /// One rule for every surface. `admin` satisfies everything; `actions` @@ -655,6 +666,45 @@ mod tests { } } + /// An `inventory` token reaches the inventory and nothing else; `full`, + /// `actions` and `admin` reach the inventory; no other narrow scope does. + #[test] + fn only_inventory_and_the_broad_scopes_reach_the_inventory() { + assert_eq!( + SCOPE_INVENTORY, "inventory", + "the scope's wire name is what --token-scope inventory mints" + ); + assert!( + scope_satisfies(SCOPE_INVENTORY, SCOPE_INVENTORY), + "an inventory token must reach the inventory" + ); + for required in [ + SCOPE_FULL, + SCOPE_ACTIONS, + SCOPE_READ, + SCOPE_METRICS, + SCOPE_ADMIN, + SCOPE_AUDIO, + ] { + assert!( + !scope_satisfies(SCOPE_INVENTORY, required), + "inventory reaches {required}" + ); + } + for held in [SCOPE_FULL, SCOPE_ACTIONS, SCOPE_ADMIN] { + assert!( + scope_satisfies(held, SCOPE_INVENTORY), + "{held} misses inventory" + ); + } + for held in [SCOPE_READ, SCOPE_METRICS, SCOPE_AUDIO] { + assert!( + !scope_satisfies(held, SCOPE_INVENTORY), + "{held} reaches inventory" + ); + } + } + type TestError = Box; /// First test signing key. diff --git a/src/cli.rs b/src/cli.rs index 383942d35..480fc312f 100644 --- a/src/cli.rs +++ b/src/cli.rs @@ -13,6 +13,7 @@ use clap::Parser; mod aggregate_args; mod call_query_args; mod file_job_args; +pub mod fleet_args; pub mod g729_args; mod gate_args; pub mod one_shot; @@ -4879,8 +4880,8 @@ pub struct TokenArgs { #[arg(help_heading = "Token minting", long = "token-id", value_name = "ID")] pub token_id: Option, - /// Scope for --mint-token: `full` (default), `metrics`, `read`, `actions` - /// or `audio`. + /// Scope for --mint-token: `full` (default), `metrics`, `read`, `actions`, + /// `admin`, `audio` or `inventory`. /// /// A `metrics` token reaches `GET /metrics` and nothing else — mint one for /// a scrape job rather than handing it a credential that also reads @@ -4904,12 +4905,16 @@ pub struct TokenArgs { /// /v1/calls/live`, `GET /v1/calls/live/events` and `GET /// /v1/calls/{call_id}/audio`, and nothing else; no other scope reaches /// them. REST API tokens only. + /// + /// An `inventory` token reaches `GET /v1/inventory` and nothing else — + /// mint one for `sipnab fleet inventory` rather than handing a fleet + /// controller a credential that reads every call. REST API tokens only. #[arg( help_heading = "Token minting", long = "token-scope", value_name = "SCOPE", default_value = "full", - value_parser = ["full", "metrics", "read", "actions", "admin", "audio"] + value_parser = ["full", "metrics", "read", "actions", "admin", "audio", "inventory"] )] pub token_scope: String, } diff --git a/src/cli/fleet_args.rs b/src/cli/fleet_args.rs new file mode 100644 index 000000000..03669dc55 --- /dev/null +++ b/src/cli/fleet_args.rs @@ -0,0 +1,50 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// SPDX-FileCopyrightText: The sipnab contributors + +//! `sipnab fleet inventory`: the command line of the command that reads the +//! inventory of many sipnab instances (docs/fleet.md). +//! +//! A command of its own, not flags of [`super::Cli`], for the reason +//! `sipnab g729` is one: it shares none of a capture run's flags, and +//! `src/main.rs` dispatches it on the first argument before the capture +//! command line is parsed. It opens no capture. + +use std::path::PathBuf; + +/// `sipnab fleet`: manage many sipnab instances as one fleet. +#[derive(clap::Parser, Debug)] +#[command( + name = "sipnab fleet", + about = "Read the inventory of many sipnab instances (docs/fleet.md)" +)] +pub struct FleetCli { + /// What to do. + #[command(subcommand)] + pub command: FleetCommand, +} + +/// The `sipnab fleet` commands. +#[derive(clap::Subcommand, Debug)] +pub enum FleetCommand { + /// Ask every instance in a hosts file for its inventory, and report the + /// fleet: each instance's build, binary and configuration digests, + /// privilege and health, the instances that did not answer, and the + /// findings across them. + /// + /// Exits 0 when every instance answered and there is no finding, 1 when + /// an instance did not answer or there is a finding, and 2 when the + /// hosts file cannot be used. + Inventory { + /// The hosts file: one `[[host]]` table per instance, with `name`, + /// `url` and `token_file`, and optionally `ca_file`. + #[arg(long, value_name = "FILE")] + hosts: PathBuf, + /// Print the report as JSON instead of a table. + #[arg(long)] + json: bool, + /// Seconds to wait for each instance to connect and answer. + #[arg(long, value_name = "SECS", default_value_t = 10, + value_parser = clap::value_parser!(u64).range(1..=300))] + timeout: u64, + }, +} diff --git a/src/config.rs b/src/config.rs index 7efdaaa01..88ddeb177 100644 --- a/src/config.rs +++ b/src/config.rs @@ -3793,6 +3793,20 @@ pub fn parse_keycode(s: &str) -> Option { } } +/// The text of the config file at `path`. +/// +/// One reader for every load, so the text a run records is the text it +/// parsed. +/// +/// # Errors +/// `crate::Error::ConfigRead` when `path` cannot be read. +fn read_config_text(path: &Path) -> Result { + std::fs::read_to_string(path).map_err(|e| crate::Error::ConfigRead { + path: path.display().to_string(), + source: e, + }) +} + /// The path from which a config was loaded. #[derive(Debug, Clone)] pub struct LoadedConfig { @@ -3804,6 +3818,10 @@ pub struct LoadedConfig { /// an earlier one in the search order was. Empty for `--config`, /// `$SIPNAB_CONFIG` and `--no-config`, which are choices. pub shadowed: Vec, + /// The text of `source` exactly as it was read, so a digest of it + /// describes the bytes this run parsed and not a later edit of the file. + /// `None` when no file was read. + pub file_text: Option, } impl LoadedConfig { @@ -3813,6 +3831,7 @@ impl LoadedConfig { config: Config::default(), source: None, shadowed: Vec::new(), + file_text: None, } } } @@ -4031,7 +4050,7 @@ impl Config { tracing::debug!("No config file found, using defaults"); return Ok(LoadedConfig::defaults()); }; - let config = Self::load_file(&p)?; + let (config, text) = Self::load_file(&p)?; if !shadowed.is_empty() { tracing::warn!("{}", shadowing_note(&p, &shadowed)); } @@ -4039,6 +4058,7 @@ impl Config { config, source: Some(p), shadowed, + file_text: Some(text), }) } @@ -4048,11 +4068,12 @@ impl Config { /// # Errors /// As for `load_file`. fn load_unshadowed(p: PathBuf) -> Result { - let config = Self::load_file(&p)?; + let (config, text) = Self::load_file(&p)?; Ok(LoadedConfig { config, source: Some(p), shadowed: Vec::new(), + file_text: Some(text), }) } @@ -4093,8 +4114,11 @@ impl Config { /// # Side effects /// Reads `path` from the filesystem; unknown keys are logged as /// warnings via `parse_toml_with_env`. - fn load_file(path: &Path) -> Result { - Self::load_file_with_env(path, &|name| std::env::var(name).ok()) + fn load_file(path: &Path) -> Result<(Config, String), crate::Error> { + let content = read_config_text(path)?; + let config = + Self::parse_toml_with_env(&content, Some(path), &|name| std::env::var(name).ok())?; + Ok((config, content)) } /// `load_file`, with the environment supplied rather than read. @@ -4124,11 +4148,7 @@ impl Config { path: &Path, lookup: &dyn Fn(&str) -> Option, ) -> Result { - let content = std::fs::read_to_string(path).map_err(|e| crate::Error::ConfigRead { - path: path.display().to_string(), - source: e, - })?; - + let content = read_config_text(path)?; Self::parse_toml_with_env(&content, Some(path), lookup) } @@ -5111,6 +5131,27 @@ filter = "/" Ok(()) } + /// Every way of finding a file records the text it parsed, byte for + /// byte, and reading no file records none. The fleet inventory's file + /// digest is taken over this text, so it must be what was parsed. + #[test] + fn every_loaded_file_records_the_text_it_parsed() -> Result<(), TestError> { + let dir = tempfile::tempdir()?; + let body = "# comment\n[capture]\ndevice = \"lo\"\n"; + let path = dir.path().join("sipnab.toml"); + std::fs::write(&path, body)?; + let explicit = Config::load(Some(path.to_str().ok_or("utf-8 path")?), false)?; + assert_eq!(explicit.file_text.as_deref(), Some(body), "--config"); + let env = Config::load_from_env(Some(path.display().to_string()))? + .ok_or("an existing file is loaded")?; + assert_eq!(env.file_text.as_deref(), Some(body), "$SIPNAB_CONFIG"); + let default = Config::load_default_location(std::slice::from_ref(&path))?; + assert_eq!(default.file_text.as_deref(), Some(body), "default location"); + let none = Config::load(None, true)?; + assert_eq!(none.file_text, None, "--no-config"); + Ok(()) + } + /// `$SIPNAB_CONFIG` naming a file that exists loads it, shadowing /// nothing; naming one that does not exist lets the search continue. #[test] diff --git a/src/main.rs b/src/main.rs index 21b747ba7..c286f6828 100644 --- a/src/main.rs +++ b/src/main.rs @@ -79,6 +79,13 @@ fn main() { std::process::exit(run_g729_command()); } + // 0c. `sipnab fleet inventory`: read the inventory of many instances. + // Dispatched before the capture command line is parsed, for the + // reason `g729` is. It opens no capture and exits. + if std::env::args_os().nth(1).is_some_and(|a| a == "fleet") { + std::process::exit(run_fleet_command()); + } + // 1. Parse CLI arguments and set up logging. let cli = Cli::parse_args(); bootstrap::init_logging(&cli); @@ -94,6 +101,19 @@ fn main() { Err(e) => e.exit(), }; + // 5a. What this run loaded, as digests, for the fleet inventory + // (`GET /v1/inventory`, `fleet_inventory`). Taken once, here, so the + // answer describes the configuration this run applied. + #[cfg(any(feature = "api", feature = "mcp"))] + { + let argv: Vec = std::env::args_os().collect(); + sipnab::output::inventory::record_run(sipnab::output::inventory::RunRecord::of( + &loaded, + &argv, + chrono::Utc::now(), + )); + } + // 5b-6b. The crash policy, and the commands that need only the config. run_config_steps(&cli, &loaded); @@ -141,6 +161,27 @@ fn main() { sipnab::capture::archive::release_run(); } +/// Step 0c of [`main`]: `sipnab fleet ...`, with `sipnab fleet` as the +/// command's own name in its help and errors. +fn run_fleet_command() -> i32 { + #[cfg(feature = "api")] + { + let args = std::iter::once(std::ffi::OsString::from("sipnab fleet")) + .chain(std::env::args_os().skip(2)); + sipnab::app::fleet::main(args) + } + #[cfg(not(feature = "api"))] + { + use std::io::Write as _; + // `writeln!`, not `eprintln!`: a closed stderr must not panic. + let _ = writeln!( + std::io::stderr(), + "sipnab fleet needs a build with the api feature" + ); + 1 + } +} + /// Step 0b of [`main`]: `sipnab g729 ...`, with `sipnab g729` as the /// command's own name in its help and errors. fn run_g729_command() -> i32 { diff --git a/src/mcp/completeness.rs b/src/mcp/completeness.rs index c3379ed06..91732356a 100644 --- a/src/mcp/completeness.rs +++ b/src/mcp/completeness.rs @@ -115,7 +115,8 @@ const COMPLETE_KEY: &str = "complete"; /// of the file root, likewise. /// - `list_captures` lists files on disk, and `vcon_spool_status` counts /// the files in the vCon export spool. -/// - `server_capabilities` describes the binary. +/// - `server_capabilities` describes the binary, and `fleet_inventory` the +/// process. /// /// `mcp_completeness_test.rs` derives the set of store-reading tools from the /// source and fails if any of them appears here. @@ -132,6 +133,11 @@ pub const SOURCE_INDEPENDENT_TOOLS: &[&str] = &[ "decode_ng", "explain_response_code", "explain_rule", + // Describes the process: its build, binary and configuration digests, + // privilege and cumulative counters. It reads no capture store and claims + // no completeness, and `GET /v1/inventory` returns the same document from + // the same builder; a stamp here would make the two surfaces differ. + "fleet_inventory", // Reads one container from the operator's vCon store. How much of the // capture has been read says nothing about what another system holds. "fetch_vcon", diff --git a/src/mcp/server.rs b/src/mcp/server.rs index 118f0f8c4..c7a0656e4 100644 --- a/src/mcp/server.rs +++ b/src/mcp/server.rs @@ -375,7 +375,8 @@ impl SipnabMcp { + Self::endpoints_router() + Self::relay_router() + Self::tfps_router() - + Self::await_condition_router(); + + Self::await_condition_router() + + Self::inventory_router(); // Shadowed rather than a `+ vcon_router()` term, because the module // itself is absent without the feature. A mismatch between the two is // a compile error rather than a tool that lists and cannot run. @@ -13257,7 +13258,9 @@ mod tests { ("media", 3_500), ("relay", 4_500), ("tfps", 6_000), - ("server", 5_500), + // 5_500 -> 6_000: `fleet_inventory`, the fleet inventory record, + // joins the server bundle; server measured 5,976 over six tools. + ("server", 6_000), // 5_000 -> 7_000 and full 81_000 -> 83_500 by `vcon_spool_status` // and `export_vcon`'s `cursor`, `media` and `next_cursor` (vcon // measured 6,627, full 83,052). @@ -13296,7 +13299,9 @@ mod tests { // `filter` parameters, once bare filter values parse (measured: // full 89,667; core 12,316, signaling 22,474 and security 9,954 // stay under theirs). - ("full", 90_000), + // full 90_000 -> 90_500: `fleet_inventory`, as the server row + // above says; full measured 90,456 over 77 tools. + ("full", 90_500), ]; let sizes: std::collections::BTreeMap = empty_server() .with_output_schemas(false) diff --git a/src/mcp/tools/inventory.rs b/src/mcp/tools/inventory.rs new file mode 100644 index 000000000..e08df01bc --- /dev/null +++ b/src/mcp/tools/inventory.rs @@ -0,0 +1,111 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// SPDX-FileCopyrightText: The sipnab contributors + +//! `fleet_inventory`: what this sipnab process is, for a fleet controller. +//! +//! [`crate::output::inventory::Inventory`], built by the one function +//! `GET /v1/inventory` calls, so the two surfaces cannot describe one process +//! differently. It holds versions, digests and counters and no byte of any +//! packet, so, like `runtime_stats`, it goes out without the untrusted content +//! note. + +use crate::mcp::server::SipnabMcp; +use rmcp::handler::server::tool::schema_for_output; +use rmcp::model::{CallToolResult, ContentBlock}; +use rmcp::{tool, tool_router}; + +#[tool_router(router = inventory_router, vis = "pub(crate)")] +impl SipnabMcp { + /// This process's fleet inventory record. + /// + /// # Errors + /// + /// Only if the record fails to serialize, which a struct of strings and + /// integers cannot. + #[tool( + name = "fleet_inventory", + description = "Returns what this sipnab process is, for managing many \ + instances as one fleet: build, SHA-256 of the running \ + binary, signature state (always unverified), \ + configuration digests (never a value), uid and \ + privilege drop, uptime, counters and HEP sender \ + addresses. Every field is the process's own claim. \ + Same document as GET /v1/inventory.", + output_schema = schema_for_output::(), + annotations(read_only_hint = true, open_world_hint = false) + )] + pub async fn fleet_inventory(&self) -> Result { + // The builder `GET /v1/inventory` calls, over this server's meter, + // interfaces and start time. Hashing the binary reads a file, so it + // runs off the async workers. + let meter = self.capture_meter.clone(); + let interfaces = self.capture_interfaces.clone(); + let uptime = self.started_at.elapsed().as_secs(); + let record = tokio::task::spawn_blocking(move || { + crate::output::inventory::current(meter.as_ref(), &interfaces, uptime) + }) + .await + .map_err(|e| { + rmcp::ErrorData::internal_error(format!("the inventory did not build: {e}"), None) + })?; + Ok(CallToolResult::success(vec![ContentBlock::json(record)?])) + } +} + +#[cfg(test)] +mod tests { + use std::sync::Arc; + + use parking_lot::RwLock; + + use super::*; + use crate::rtp::stream_store::StreamStore; + use crate::sip::dialog_store::DialogStore; + type TestError = Box; + + /// Registered, and annotated read-only, so a `read`-scoped MCP token + /// reaches it. + #[test] + fn fleet_inventory_is_registered_and_annotated_read_only() -> Result<(), TestError> { + let router = SipnabMcp::inventory_router(); + let tool = router + .get("fleet_inventory") + .ok_or("fleet_inventory must be registered")?; + let annotations = tool.annotations.as_ref().ok_or("annotations")?; + assert_eq!( + annotations.read_only_hint, + Some(true), + "a read token reaches only tools annotated read-only" + ); + Ok(()) + } + + /// The tool answers with the shared record, for this server's interfaces. + #[tokio::test] + async fn fleet_inventory_returns_the_shared_record() -> Result<(), TestError> { + let srv = SipnabMcp::new( + Arc::new(RwLock::new(DialogStore::new(100, false))), + Arc::new(RwLock::new(StreamStore::new(100))), + ) + .with_capture_interfaces(vec!["lo".to_string()]); + let result = srv + .fleet_inventory() + .await + .map_err(|e| format!("the tool answers: {e:?}"))?; + let text = result + .content + .iter() + .find_map(|c| c.as_text()) + .map(|t| t.text.clone()) + .ok_or("a JSON block")?; + let v: serde_json::Value = serde_json::from_str(&text)?; + assert_eq!(v["schema_version"], 1, "{v}"); + assert_eq!( + v["build"]["version_line"], + crate::cli::build_version(), + "{v}" + ); + assert_eq!(v["interfaces"][0]["name"], "lo", "{v}"); + Ok(()) + } +} diff --git a/src/mcp/tools/mod.rs b/src/mcp/tools/mod.rs index a9fa5dd21..f0ca361dc 100644 --- a/src/mcp/tools/mod.rs +++ b/src/mcp/tools/mod.rs @@ -28,6 +28,7 @@ pub mod expectations; #[cfg(all(feature = "mcp", feature = "hep"))] pub mod hep; pub mod inspect; +pub mod inventory; pub mod provenance; pub mod relay; pub mod tfps; diff --git a/src/mcp_profile.rs b/src/mcp_profile.rs index 8491d70de..3d4c45b4e 100644 --- a/src/mcp_profile.rs +++ b/src/mcp_profile.rs @@ -165,6 +165,7 @@ pub const BUNDLES: &[(&str, &[&str])] = &[ "server", &[ "capture_health", + "fleet_inventory", "hep_senders", "runtime_stats", "server_capabilities", diff --git a/src/output/api.rs b/src/output/api.rs index ed76d9056..52aaef52e 100644 --- a/src/output/api.rs +++ b/src/output/api.rs @@ -1103,6 +1103,7 @@ pub fn build_router(state: ApiState) -> Router { .route("/v1/relay/holdings/{call_id}", get(get_relay_holdings_call)) .route("/v1/runtime", get(get_runtime)) .route("/v1/capabilities", get(get_capabilities)) + .route("/v1/inventory", get(get_inventory)) .route("/metrics", get(get_metrics)) .with_state(state) // Request hardening on every route. @@ -1785,6 +1786,57 @@ async fn health_check() -> &'static str { "ok" } +/// `GET /v1/inventory` — what this sipnab process is, for a fleet controller. +/// +/// # Arguments +/// +/// * `state` — Shared application state (for the meter, the interfaces and +/// the guard). +/// * `addr` — Client socket address used for rate limiting. +/// * `headers` — Request headers (auth). +/// +/// # Returns +/// +/// 200 with the [`crate::output::inventory::Inventory`] record; 401/503 from +/// the guard. +/// +/// # Side effects +/// +/// The first call hashes the running binary; every call reads the capture +/// interfaces' counters. Mutates the rate limiter. +#[utoipa::path( + get, + path = "/v1/inventory", + tag = "operations", + summary = "Fleet inventory of this process", + description = "What this sipnab process is, for managing many instances as one fleet: the build (version, git commit and tag, dirty flag, target, compiled features, the `--version` line), SHA-256 of the running binary, the signature state, the configuration as digests only, the user id and privilege drop, pid, start time and uptime, the capture interfaces' error and drop counters, capture and HEP export counters, and the HEP listener's sender addresses.\n\nNo configuration value and no command-line argument is in the answer, only their SHA-256 digests, so a secret in either changes a digest and never appears. `signature.state` is always `unverified`: compare `binary.sha256` with the release's attested `SHA256SUMS.txt`. Every field is the process's own claim.\n\nReached by a token with scope `inventory`, or `full` and the broader scopes. The MCP tool `fleet_inventory` returns the same fields and values.", + responses( + (status = 200, description = "The inventory record.", body = crate::output::inventory::Inventory), + (status = 401, description = "No bearer credential, or one this server does not accept.", body = schema::ProblemJson, content_type = "application/problem+json"), + (status = 500, description = "The task that builds the record failed.", body = schema::ProblemJson, content_type = "application/problem+json"), + (status = 503, description = "Over the per-source-IP rate limit of 100 requests per second.", body = schema::ProblemJson, content_type = "application/problem+json"), + ) +)] +async fn get_inventory( + State(state): State, + ConnectInfo(addr): ConnectInfo, + headers: HeaderMap, +) -> Result { + guard_scoped(&state, &headers, addr.ip(), crate::auth::SCOPE_INVENTORY)?; + // The builder the MCP tool calls, so the two surfaces cannot disagree. + // Hashing the binary reads a file, so it runs off the async workers. + let inventory = tokio::task::spawn_blocking(move || { + crate::output::inventory::current( + state.capture_meter.as_ref(), + &state.capture_interfaces, + state.started_at.elapsed().as_secs(), + ) + }) + .await + .map_err(|e| Problem::detailed(StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?; + Ok(Json(inventory)) +} + /// `GET /v1/capabilities` — what this build can do and what the operator turned /// on, so a client can discover the surface before a refusal it could have /// predicted reads as a dead end (PAR3: the machine contract REST had lacked). @@ -7572,10 +7624,22 @@ impl utoipa::Modify for BearerAuth { get_relay_holdings_call, get_runtime, get_capabilities, + get_inventory, get_metrics, ), components(schemas( schema::ProblemJson, + crate::output::inventory::Inventory, + crate::output::inventory::BuildInfo, + crate::output::inventory::BinaryInfo, + crate::output::inventory::SignatureInfo, + crate::output::inventory::ConfigInfo, + crate::output::inventory::PrivilegeInfo, + crate::output::inventory::ProcessInfo, + crate::output::inventory::InterfaceCounters, + crate::output::inventory::Counters, + crate::output::inventory::HepListenerInfo, + crate::output::inventory::HepPeer, schema::Capabilities, schema::CapabilitiesRuntime, schema::Libpcap, @@ -12247,6 +12311,143 @@ mod tests { Ok(()) } + /// The status `uri` answers a bearer token minted with `key` and `scope`, + /// expiring `ttl` seconds from now, on a fresh router over `state`. + async fn inventory_status( + state: ApiState, + key: &[u8], + uri: &str, + scope: &str, + ttl: i64, + ) -> Result { + let token = crate::auth::mint( + key, + "fleet-test", + chrono::Utc::now().timestamp() + ttl, + crate::auth::AUDIENCE_API, + scope, + ); + let req = test_request_with_header(uri, "Authorization", &format!("Bearer {token}"))?; + let resp = build_router(state) + .oneshot(req) + .await + .map_err(|e| format!("oneshot: {e:?}"))?; + Ok(resp.status()) + } + + /// An `inventory` token reads `/v1/inventory` and nothing that holds + /// call content or process internals; `full` reads it too; `metrics` + /// does not. + #[tokio::test] + async fn the_inventory_scope_reaches_the_inventory_and_nothing_else() -> Result<(), TestError> { + let key = crate::test_material::key_bytes("rest-router-signing"); + let s = || make_state_with_signing_key(key); + let inv = crate::auth::SCOPE_INVENTORY; + assert_eq!( + inventory_status(s(), key, "/v1/inventory", inv, 3600).await?, + StatusCode::OK, + "an inventory token must read the inventory" + ); + for uri in ["/v1/dialogs", "/v1/runtime", "/v1/streams", "/metrics"] { + assert_eq!( + inventory_status(s(), key, uri, inv, 3600).await?, + StatusCode::UNAUTHORIZED, + "an inventory token reached {uri}" + ); + } + assert_eq!( + inventory_status(s(), key, "/v1/inventory", crate::auth::SCOPE_FULL, 3600).await?, + StatusCode::OK, + "a full token reaches every narrow read scope, the inventory included" + ); + for scope in [crate::auth::SCOPE_METRICS, crate::auth::SCOPE_AUDIO] { + assert_eq!( + inventory_status(s(), key, "/v1/inventory", scope, 3600).await?, + StatusCode::UNAUTHORIZED, + "a {scope} token reached the inventory" + ); + } + Ok(()) + } + + /// An expired token is refused, and a source over its rate is refused + /// before its credential is looked at: the second wrong-token request + /// from one address, on a limiter of one request per second, is 503 and + /// not 401. + #[tokio::test] + async fn the_inventory_refuses_expired_tokens_and_rate_limits_first() -> Result<(), TestError> { + let key = crate::test_material::key_bytes("rest-router-signing"); + let inv = crate::auth::SCOPE_INVENTORY; + assert_eq!( + inventory_status( + make_state_with_signing_key(key), + key, + "/v1/inventory", + inv, + -1 + ) + .await?, + StatusCode::UNAUTHORIZED, + "an expired token is refused" + ); + let mut state = make_state_with_signing_key(key); + state.rate_limiter = Arc::new(Mutex::new(RateLimiter::new(1, 1024))); + let wrong = crate::test_material::key_bytes("rest-router-not-the-signing-key"); + let first = inventory_status(state.clone(), wrong, "/v1/inventory", inv, 3600).await?; + let second = inventory_status(state, wrong, "/v1/inventory", inv, 3600).await?; + assert_eq!( + first, + StatusCode::UNAUTHORIZED, + "a wrong token under the rate is refused as unauthenticated" + ); + assert_eq!( + second, + StatusCode::SERVICE_UNAVAILABLE, + "rate limit after auth" + ); + Ok(()) + } + + /// The route's body is the shared builder's record, with this build's + /// version line. + #[tokio::test] + async fn the_inventory_route_returns_the_shared_record() -> Result<(), TestError> { + let key = crate::test_material::key_bytes("rest-router-signing"); + let token = crate::auth::mint( + key, + "fleet-test", + chrono::Utc::now().timestamp() + 3600, + crate::auth::AUDIENCE_API, + crate::auth::SCOPE_INVENTORY, + ); + let req = + test_request_with_header("/v1/inventory", "Authorization", &format!("Bearer {token}"))?; + let resp = build_router(make_state_with_signing_key(key)) + .oneshot(req) + .await + .map_err(|e| format!("oneshot: {e:?}"))?; + assert_eq!( + resp.status(), + StatusCode::OK, + "an inventory token must read the inventory" + ); + let bytes = resp + .into_body() + .collect() + .await + .map_err(|e| format!("body: {e:?}"))? + .to_bytes(); + let v: Value = serde_json::from_slice(&bytes)?; + assert_eq!(v["schema_version"], 1, "{v}"); + assert_eq!( + v["build"]["version_line"], + crate::cli::build_version(), + "{v}" + ); + assert_eq!(v["signature"]["state"], "unverified", "{v}"); + Ok(()) + } + /// A signed token whose expiry is in the past is rejected with 401. #[tokio::test] async fn auth_expired_signed_token_returns_401() -> Result<(), TestError> { diff --git a/src/output/inventory.rs b/src/output/inventory.rs new file mode 100644 index 000000000..a1c52e04f --- /dev/null +++ b/src/output/inventory.rs @@ -0,0 +1,949 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// SPDX-FileCopyrightText: The sipnab contributors + +//! What this sipnab process is: its build, its binary, its configuration, +//! its privilege and its health, as one record a fleet controller reads. +//! +//! # Why this module exists +//! +//! An estate of sipnab instances had no way to say which build each one runs +//! or which configuration it loaded. The upgrade note for HEP HMAC version 2 +//! (`docs/config-reference.md`, "Upgrade a fleet that uses HMAC +//! authentication") could only tell the operator to run `sipnab --version` +//! on every host by hand. `GET /v1/inventory`, the MCP `fleet_inventory` tool +//! and `sipnab fleet inventory` read this one record, so the three cannot +//! describe one process differently. +//! +//! # What it never carries +//! +//! No configuration value and no command-line argument. Each is reported as a +//! SHA-256 digest only, so a secret in either changes the digest and never +//! appears in the answer. That is also why the record needs only the +//! `inventory` token scope ([`crate::auth::SCOPE_INVENTORY`]) and no scope +//! that reads call content. +//! +//! # What it cannot prove +//! +//! Every field is the process's own claim. A compromised host can report +//! anything, and nothing here detects that. In particular the record never +//! says its binary is a verified release: [`SIGNATURE_STATE`] is always +//! `unverified`, and the controller compares [`BinaryInfo::sha256`] with the +//! release's attested `SHA256SUMS.txt`. + +use std::sync::OnceLock; + +use serde::Serialize; +use sha2::{Digest, Sha256}; + +/// Wire-format version of [`Inventory`]. +pub const SCHEMA_VERSION: u32 = 1; + +/// The only signature state an instance reports about itself. +/// +/// A process cannot verify the binary it is running against a release +/// attestation without the attestation's trust root, and a process that +/// could be tampered with would report "verified" as easily as anything +/// else. The check belongs to whoever holds the release's `SHA256SUMS.txt`. +pub const SIGNATURE_STATE: &str = "unverified"; + +/// The most HEP senders one inventory lists. +pub const MAX_HEP_PEERS: usize = 256; + +/// The lowercase hexadecimal SHA-256 digest of `bytes`. +#[must_use] +pub fn sha256_hex(bytes: &[u8]) -> String { + hex(&Sha256::digest(bytes)) +} + +/// The digest of a parsed configuration. +/// +/// Taken over the configuration as sipnab parsed it, after `${NAME}` +/// expansion, so two files that differ only in key order, comments or +/// whitespace give one digest, and a changed value always gives another. The +/// serialization is canonical: `serde_json::Value` objects are sorted maps, +/// so struct field order and map insertion order cannot reach the digest. +/// +/// # Errors +/// +/// When the configuration does not serialize, which a configuration that +/// deserialized cannot do; the message is returned rather than a digest of +/// nothing. +pub fn parsed_config_digest(config: &crate::config::Config) -> Result { + let value = serde_json::to_value(config).map_err(|e| e.to_string())?; + let canonical = canonical_json(&value); + Ok(sha256_hex(canonical.as_bytes())) +} + +/// `value` serialized with every object's keys sorted, whatever map type +/// `serde_json` was built with. +fn canonical_json(value: &serde_json::Value) -> String { + match value { + serde_json::Value::Object(map) => { + let mut keys: Vec<&String> = map.keys().collect(); + keys.sort(); + let body: Vec = keys + .into_iter() + .map(|k| { + let key = serde_json::Value::String(k.clone()).to_string(); + format!("{key}:{}", canonical_json(&map[k])) + }) + .collect(); + format!("{{{}}}", body.join(",")) + } + serde_json::Value::Array(items) => { + let body: Vec = items.iter().map(canonical_json).collect(); + format!("[{}]", body.join(",")) + } + other => other.to_string(), + } +} + +/// The digest of a command line. +/// +/// Each argument is length-prefixed, so `["ab", "c"]` and `["a", "bc"]` +/// differ. The first argument, the program path, is left out: the same build +/// installed at two paths runs the same command. +#[must_use] +pub fn argv_digest(argv: &[std::ffi::OsString]) -> String { + let mut hasher = Sha256::new(); + for arg in argv.iter().skip(1) { + let bytes = arg.as_encoded_bytes(); + hasher.update((bytes.len() as u64).to_be_bytes()); + hasher.update(bytes); + } + hex(&hasher.finalize()) +} + +/// The digest of everything that decides this run's effective +/// configuration: the parsed configuration and the command line. +/// +/// A command-line flag overrides the file, so two instances with one file +/// and different flags run differently; this digest tells them apart, where +/// [`parsed_config_digest`] alone would not. Each part is length-prefixed and +/// labeled. `None` when the parsed configuration has no digest. +#[must_use] +pub fn effective_digest(parsed_sha256: Option<&str>, argv_sha256: &str) -> Option { + let parsed = parsed_sha256?; + let mut hasher = Sha256::new(); + for (label, part) in [("parsed", parsed), ("argv", argv_sha256)] { + hasher.update(label.as_bytes()); + hasher.update((part.len() as u64).to_be_bytes()); + hasher.update(part.as_bytes()); + } + Some(hex(&hasher.finalize())) +} + +/// `bytes` as lowercase hexadecimal. +fn hex(bytes: &[u8]) -> String { + let mut out = String::with_capacity(bytes.len() * 2); + for b in bytes { + use std::fmt::Write as _; + let _ = write!(out, "{b:02x}"); + } + out +} + +/// What this run recorded about its configuration when it started. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct RunRecord { + /// The file the configuration was read from, if any. + pub source: Option, + /// Default-location files present but not read. + pub shadowed: Vec, + /// SHA-256 of the bytes of `source`, as read. + pub file_sha256: Option, + /// [`parsed_config_digest`] of the configuration, or `None` when it did + /// not serialize. + pub parsed_sha256: Option, + /// [`argv_digest`] of the command line. + pub argv_sha256: String, + /// [`effective_digest`] of the two above. + pub effective_sha256: Option, + /// When the record was taken, RFC 3339. + pub started: String, +} + +impl RunRecord { + /// The record for `loaded`, run with `argv`, at `started`. + #[must_use] + pub fn of( + loaded: &crate::config::LoadedConfig, + argv: &[std::ffi::OsString], + started: chrono::DateTime, + ) -> Self { + let parsed_sha256 = parsed_config_digest(&loaded.config).ok(); + let argv_sha256 = argv_digest(argv); + Self { + source: loaded.source.as_ref().map(|p| p.display().to_string()), + shadowed: loaded + .shadowed + .iter() + .map(|p| p.display().to_string()) + .collect(), + file_sha256: loaded + .file_text + .as_deref() + .map(|t| sha256_hex(t.as_bytes())), + effective_sha256: effective_digest(parsed_sha256.as_deref(), &argv_sha256), + parsed_sha256, + argv_sha256, + started: started.to_rfc3339_opts(chrono::SecondsFormat::Secs, true), + } + } +} + +/// This process's record, set once by [`record_run`]. +static RUN: OnceLock = OnceLock::new(); + +/// Record this run's configuration, once. A second call changes nothing, so +/// the answer is stable for the life of the process. +pub fn record_run(record: RunRecord) { + let _ = RUN.set(record); +} + +/// The record [`record_run`] set, if it ran. +#[must_use] +pub fn recorded() -> Option<&'static RunRecord> { + RUN.get() +} + +/// SHA-256 of the file at `path`, read in blocks. +/// +/// # Errors +/// +/// When the file cannot be opened or read; the message names the path. +pub fn file_sha256(path: &std::path::Path) -> Result { + use std::io::Read as _; + let mut file = std::fs::File::open(path).map_err(|e| format!("'{}': {e}", path.display()))?; + let mut hasher = Sha256::new(); + let mut buf = vec![0u8; 64 * 1024]; + loop { + let n = file + .read(&mut buf) + .map_err(|e| format!("'{}': {e}", path.display()))?; + if n == 0 { + break; + } + hasher.update(&buf[..n]); + } + Ok(hex(&hasher.finalize())) +} + +/// The file this process is running from. +/// +/// `/proc/self/exe` on Linux, which opens the image the process runs even +/// after an upgrade replaced or removed the file at its path, so the digest +/// describes what runs rather than what is installed. Elsewhere, the path +/// the operating system reports. +fn running_image() -> Result { + if cfg!(target_os = "linux") { + return Ok(std::path::PathBuf::from("/proc/self/exe")); + } + std::env::current_exe().map_err(|e| e.to_string()) +} + +/// SHA-256 of the running binary, computed on first use and kept. +fn binary_digest() -> &'static Result { + static DIGEST: OnceLock> = OnceLock::new(); + DIGEST.get_or_init(|| running_image().and_then(|p| file_sha256(&p))) +} + +/// One inventory record. +#[derive(Debug, Clone, Serialize, PartialEq)] +#[cfg_attr(feature = "mcp", derive(rmcp::schemars::JsonSchema))] +#[cfg_attr(feature = "mcp", schemars(crate = "rmcp::schemars"))] +#[cfg_attr(feature = "api", derive(utoipa::ToSchema))] +pub struct Inventory { + /// Wire-format version of this envelope, [`SCHEMA_VERSION`]. + pub schema_version: u32, + /// The node name this process reports (`--node-name`, + /// `[capture] node_name`, else the host name). + pub node_name: String, + /// The build. + pub build: BuildInfo, + /// The running binary. + pub binary: BinaryInfo, + /// What this process can say about its release signature. + pub signature: SignatureInfo, + /// The configuration this run loaded, as digests. + pub config: ConfigInfo, + /// The privilege the process holds. + pub privilege: PrivilegeInfo, + /// The process. + pub process: ProcessInfo, + /// The capture interfaces and their error and drop counters. + pub interfaces: Vec, + /// Capture-path and HEP export counters. + pub counters: Counters, + /// The HEP listener, when this run has one. + pub hep_listener: HepListenerInfo, +} + +/// The build, as `sipnab --version` reports it. +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +#[cfg_attr(feature = "mcp", derive(rmcp::schemars::JsonSchema))] +#[cfg_attr(feature = "mcp", schemars(crate = "rmcp::schemars"))] +#[cfg_attr(feature = "api", derive(utoipa::ToSchema))] +pub struct BuildInfo { + /// The crate version. + pub version: String, + /// The git commit the build recorded; empty when it recorded none. + pub commit: String, + /// The git tag at that commit; empty when there was none. + pub tag: String, + /// Whether the working tree had uncommitted changes. + pub dirty: bool, + /// The target triple. + pub target: String, + /// The compiled feature set, sorted. + pub features: Vec, + /// The whole `--version` line. + pub version_line: String, +} + +/// The running binary. +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +#[cfg_attr(feature = "mcp", derive(rmcp::schemars::JsonSchema))] +#[cfg_attr(feature = "mcp", schemars(crate = "rmcp::schemars"))] +#[cfg_attr(feature = "api", derive(utoipa::ToSchema))] +pub struct BinaryInfo { + /// SHA-256 of the running image, lowercase hex; absent when it could + /// not be read. + pub sha256: Option, + /// Why it could not be read. + pub error: Option, +} + +/// The release signature, as far as the process can speak to it. +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +#[cfg_attr(feature = "mcp", derive(rmcp::schemars::JsonSchema))] +#[cfg_attr(feature = "mcp", schemars(crate = "rmcp::schemars"))] +#[cfg_attr(feature = "api", derive(utoipa::ToSchema))] +pub struct SignatureInfo { + /// Always [`SIGNATURE_STATE`]. + pub state: String, + /// How to verify the binary. + pub note: String, +} + +/// The configuration, as digests only. +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +#[cfg_attr(feature = "mcp", derive(rmcp::schemars::JsonSchema))] +#[cfg_attr(feature = "mcp", schemars(crate = "rmcp::schemars"))] +#[cfg_attr(feature = "api", derive(utoipa::ToSchema))] +pub struct ConfigInfo { + /// Whether this run recorded its configuration. `false` means every + /// digest below is unknown, not empty. + pub recorded: bool, + /// The file read, if any. + pub source: Option, + /// Default-location files present but not read. + pub shadowed: Vec, + /// SHA-256 of the file's bytes. + pub file_sha256: Option, + /// SHA-256 of the parsed configuration, canonically serialized. + pub parsed_sha256: Option, + /// SHA-256 of the command line, without the program path. + pub argv_sha256: Option, + /// SHA-256 over `parsed_sha256` and `argv_sha256`: what decides the + /// effective configuration. The one configuration digest to compare + /// across instances. + pub effective_sha256: Option, +} + +/// The privilege the process holds. +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +#[cfg_attr(feature = "mcp", derive(rmcp::schemars::JsonSchema))] +#[cfg_attr(feature = "mcp", schemars(crate = "rmcp::schemars"))] +#[cfg_attr(feature = "api", derive(utoipa::ToSchema))] +pub struct PrivilegeInfo { + /// The user id the process runs as. + pub uid: Option, + /// Whether that is root. + pub root: bool, + /// The user a privilege drop switched to; absent when none happened. + pub dropped_to: Option, +} + +/// The process. +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +#[cfg_attr(feature = "mcp", derive(rmcp::schemars::JsonSchema))] +#[cfg_attr(feature = "mcp", schemars(crate = "rmcp::schemars"))] +#[cfg_attr(feature = "api", derive(utoipa::ToSchema))] +pub struct ProcessInfo { + /// The process id. + pub pid: u32, + /// When the configuration was recorded, RFC 3339; absent when it was + /// not. + pub started: Option, + /// Seconds this process has been serving. + pub uptime_seconds: u64, +} + +/// One capture interface's error and drop counters, read from the interface. +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +#[cfg_attr(feature = "mcp", derive(rmcp::schemars::JsonSchema))] +#[cfg_attr(feature = "mcp", schemars(crate = "rmcp::schemars"))] +#[cfg_attr(feature = "api", derive(utoipa::ToSchema))] +pub struct InterfaceCounters { + /// The interface name. + pub name: String, + /// Link state; absent when not readable on this platform. + pub operstate: Option, + /// Receive errors. + pub rx_errors: Option, + /// Packets the kernel dropped on receive. + pub rx_dropped: Option, + /// Packets the NIC missed for lack of ring space. + pub rx_missed_errors: Option, +} + +/// Capture-path and HEP export counters, cumulative. +#[derive(Debug, Clone, Default, Serialize, PartialEq, Eq)] +#[cfg_attr(feature = "mcp", derive(rmcp::schemars::JsonSchema))] +#[cfg_attr(feature = "mcp", schemars(crate = "rmcp::schemars"))] +#[cfg_attr(feature = "api", derive(utoipa::ToSchema))] +pub struct Counters { + /// Packets the capture path received. + pub capture_packets_total: u64, + /// Capture sources refused past the source-table cap. + pub capture_sources_refused_total: u64, + /// Times the capture path blocked on a full queue; absent with no queue. + pub capture_backpressure_blocks_total: Option, + /// HEP export failures of every kind; absent with no exporter. + pub hep_export_failures_total: Option, + /// HEP export packets dropped of every kind; absent with no exporter. + pub hep_export_dropped_total: Option, + /// HEP export packets queued now; absent with no exporter. + pub hep_export_queue_depth: Option, +} + +/// The HEP listener's senders, by address only. +#[derive(Debug, Clone, Default, Serialize, PartialEq, Eq)] +#[cfg_attr(feature = "mcp", derive(rmcp::schemars::JsonSchema))] +#[cfg_attr(feature = "mcp", schemars(crate = "rmcp::schemars"))] +#[cfg_attr(feature = "api", derive(utoipa::ToSchema))] +pub struct HepListenerInfo { + /// Whether this run has a HEP listener. + pub listening: bool, + /// Packets the listener refused, for any reason. + pub packets_refused: u64, + /// The senders it tracks, at most [`MAX_HEP_PEERS`]. + pub senders: Vec, +} + +/// One HEP sender. +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +#[cfg_attr(feature = "mcp", derive(rmcp::schemars::JsonSchema))] +#[cfg_attr(feature = "mcp", schemars(crate = "rmcp::schemars"))] +#[cfg_attr(feature = "api", derive(utoipa::ToSchema))] +pub struct HepPeer { + /// The address its packets came from. + pub peer: String, + /// The capture id it claims; never proven. + pub capture_id: Option, + /// Whether it has been silent past the warning threshold. + pub silent: bool, +} + +/// The build half of the record, from the one `--version` source. +#[must_use] +pub fn build_info() -> BuildInfo { + let mut features: Vec = crate::cli::compiled_features() + .iter() + .map(|s| (*s).to_string()) + .collect(); + features.sort(); + BuildInfo { + version: env!("CARGO_PKG_VERSION").to_string(), + commit: env!("SIPNAB_GIT_COMMIT").to_string(), + tag: env!("SIPNAB_GIT_TAG").to_string(), + dirty: !env!("SIPNAB_GIT_DIRTY").is_empty(), + target: env!("SIPNAB_TARGET").to_string(), + features, + version_line: crate::cli::build_version(), + } +} + +/// The configuration half of the record, from `record`. +#[must_use] +pub fn config_info(record: Option<&RunRecord>) -> ConfigInfo { + match record { + Some(r) => ConfigInfo { + recorded: true, + source: r.source.clone(), + shadowed: r.shadowed.clone(), + file_sha256: r.file_sha256.clone(), + parsed_sha256: r.parsed_sha256.clone(), + argv_sha256: Some(r.argv_sha256.clone()), + effective_sha256: r.effective_sha256.clone(), + }, + None => ConfigInfo { + recorded: false, + source: None, + shadowed: Vec::new(), + file_sha256: None, + parsed_sha256: None, + argv_sha256: None, + effective_sha256: None, + }, + } +} + +/// The signature half of the record. Never anything but +/// [`SIGNATURE_STATE`]. +#[must_use] +pub fn signature_info() -> SignatureInfo { + SignatureInfo { + state: SIGNATURE_STATE.to_string(), + note: "sipnab does not verify its own binary. Compare binary.sha256 \ + with SHA256SUMS.txt of the release, after `gh attestation \ + verify SHA256SUMS.txt --repo NormB/sipnab`." + .to_string(), + } +} + +/// The HEP half of the record, from the listener's roster when there is one. +#[must_use] +pub fn hep_listener_info(meter: Option<&crate::capture::channel::CaptureMeter>) -> HepListenerInfo { + #[cfg(feature = "hep")] + { + let report = crate::capture::hep_roster::senders_report( + meter.and_then(crate::capture::channel::CaptureMeter::hep_roster), + MAX_HEP_PEERS, + ); + HepListenerInfo { + listening: report.listening, + packets_refused: report.packets_refused, + senders: report + .senders + .into_iter() + .map(|row| HepPeer { + peer: row.peer, + capture_id: row.capture_id, + silent: row.silent, + }) + .collect(), + } + } + #[cfg(not(feature = "hep"))] + { + let _ = meter; + HepListenerInfo::default() + } +} + +/// The counters half of the record. +#[must_use] +pub fn counters(meter: Option<&crate::capture::channel::CaptureMeter>) -> Counters { + let exporter = meter + .and_then(crate::capture::channel::CaptureMeter::hep_export) + .map(crate::output::runtime::HepExportStats::of); + Counters { + capture_packets_total: crate::capture::captured_packets(), + capture_sources_refused_total: crate::capture::packet::capture_source_counts().refused, + capture_backpressure_blocks_total: meter + .map(crate::capture::channel::CaptureMeter::backpressure_blocks), + hep_export_failures_total: exporter.as_ref().map(|e| e.failures.values().sum()), + hep_export_dropped_total: exporter.as_ref().map(|e| e.dropped.values().sum()), + hep_export_queue_depth: exporter.as_ref().map(|e| e.queue_depth), + } +} + +/// This process's inventory record, now. +/// +/// The one builder `GET /v1/inventory` and the MCP `fleet_inventory` tool +/// call. +/// +/// # Arguments +/// +/// * `meter` — the capture meter, for the queue, exporter and roster. +/// * `interfaces` — the interfaces sipnab was asked to capture on. +/// * `uptime_seconds` — how long this process has been serving. +#[must_use] +pub fn current( + meter: Option<&crate::capture::channel::CaptureMeter>, + interfaces: &[String], + uptime_seconds: u64, +) -> Inventory { + let record = recorded(); + let binary = match binary_digest() { + Ok(d) => BinaryInfo { + sha256: Some(d.clone()), + error: None, + }, + Err(e) => BinaryInfo { + sha256: None, + error: Some(e.clone()), + }, + }; + #[cfg(unix)] + let uid = Some(crate::privilege::current_uid()); + #[cfg(not(unix))] + let uid = None; + Inventory { + schema_version: SCHEMA_VERSION, + node_name: crate::provenance::node_name().to_string(), + build: build_info(), + binary, + signature: signature_info(), + config: config_info(record), + privilege: PrivilegeInfo { + uid, + root: uid == Some(0), + dropped_to: crate::privilege::dropped_to().map(str::to_string), + }, + process: ProcessInfo { + pid: std::process::id(), + started: record.map(|r| r.started.clone()), + uptime_seconds, + }, + interfaces: interfaces + .iter() + .map(|n| { + let s = crate::output::runtime::interface_stats(n); + InterfaceCounters { + name: s.name, + operstate: s.operstate, + rx_errors: s.rx_errors, + rx_dropped: s.rx_dropped, + rx_missed_errors: s.rx_missed_errors, + } + }) + .collect(), + counters: counters(meter), + hep_listener: hep_listener_info(meter), + } +} + +#[cfg(test)] +mod tests { + use super::*; + + type TestError = Box; + + /// An independent SHA-256, so a test never checks the module's digest + /// against the module's own digest. + fn reference_hex(bytes: &[u8]) -> String { + Sha256::digest(bytes) + .iter() + .map(|b| format!("{b:02x}")) + .collect() + } + + fn parse(text: &str) -> Result { + Ok(toml::from_str(text)?) + } + + /// FIPS 180-2's "abc" vector, so the hex encoding is checked against a + /// published value and not only against itself. + #[test] + fn sha256_hex_matches_the_published_vector() { + assert_eq!( + sha256_hex(b"abc"), + "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad", + "\"abc\" must hash to the FIPS 180-2 vector" + ); + } + + /// The parsed digest ignores key order, whitespace and comments, and + /// changes when one value changes. + #[test] + fn parsed_digest_follows_values_not_layout() -> Result<(), TestError> { + let a = parse("[capture]\nnode_name = \"edge-a\"\nsnaplen = 1500\n")?; + let b = parse("# a comment\n[capture]\n snaplen = 1500\nnode_name=\"edge-a\"\n")?; + let c = parse("[capture]\nnode_name = \"edge-a\"\nsnaplen = 1501\n")?; + let da = parsed_config_digest(&a)?; + assert_eq!(da.len(), 64, "a SHA-256 in hex: {da:?}"); + assert_eq!(da, parsed_config_digest(&b)?, "layout reached the digest"); + assert_ne!( + da, + parsed_config_digest(&c)?, + "a changed value kept the digest" + ); + assert_eq!( + da, + reference_hex(canonical_json(&serde_json::to_value(&a)?).as_bytes()), + "the digest is SHA-256 of the canonical serialization" + ); + Ok(()) + } + + /// What a run records follows values, not layout: two files that differ + /// only in key order, whitespace and comments give one parsed and one + /// effective digest, and two file digests. + #[test] + fn the_run_record_digests_values_not_layout() -> Result<(), TestError> { + let record = |text: &str| -> Result { + let loaded = crate::config::LoadedConfig { + config: parse(text)?, + source: None, + shadowed: Vec::new(), + file_text: Some(text.to_string()), + }; + let argv: Vec = vec!["sipnab".into(), "-N".into()]; + let at = chrono::DateTime::from_timestamp(1_790_000_000, 0).ok_or("a time")?; + Ok(RunRecord::of(&loaded, &argv, at)) + }; + let a = record("[capture]\nnode_name = \"edge-a\"\nsnaplen = 1500\n")?; + let b = record("# a comment\n[capture]\n snaplen = 1500\nnode_name=\"edge-a\"\n")?; + assert_ne!( + a.file_sha256, b.file_sha256, + "the files differ byte for byte" + ); + assert_eq!( + a.parsed_sha256, b.parsed_sha256, + "layout reached the parsed digest" + ); + assert_eq!( + a.effective_sha256, b.effective_sha256, + "layout reached the effective digest" + ); + assert!( + a.effective_sha256.is_some(), + "a parsed configuration has an effective digest" + ); + Ok(()) + } + + /// A secret a configuration takes from the environment with `${NAME}` + /// changes the parsed digest, though the file's bytes are the same: the + /// digest is over the configuration after expansion. + #[test] + fn an_expanded_secret_changes_the_parsed_digest() -> Result<(), TestError> { + let dir = tempfile::tempdir()?; + let path = dir.path().join("sipnab.toml"); + std::fs::write( + &path, + "[vcon_forward]\nauth_file = \"/run/example/${FLEET_TEST_SECRET}\"\n", + )?; + let digest = |value: &'static str| -> Result { + let config = crate::config::Config::load_file_with_env(&path, &move |name| { + (name == "FLEET_TEST_SECRET").then(|| value.to_string()) + })?; + Ok(parsed_config_digest(&config)?) + }; + assert_ne!( + digest("first-value")?, + digest("second-value")?, + "the expanded value must reach the parsed digest" + ); + Ok(()) + } + + /// Canonical JSON sorts every object's keys, at every depth. + #[test] + fn canonical_json_sorts_keys_at_every_depth() -> Result<(), TestError> { + let v: serde_json::Value = + serde_json::from_str(r#"{"b":{"y":1,"x":[{"d":2,"c":3}]},"a":0}"#)?; + assert_eq!( + canonical_json(&v), + r#"{"a":0,"b":{"x":[{"c":3,"d":2}],"y":1}}"#, + "the digest is SHA-256 of the canonical serialization" + ); + Ok(()) + } + + /// A secret in the configuration or on the command line changes the + /// digests, and its value is in no field of the record. + #[test] + fn a_secret_changes_the_digest_and_never_appears() -> Result<(), TestError> { + const SECRET: &str = "s3cr3t-marker-0b1f0e2d"; + let text = |v: &str| format!("[vcon_forward]\nauth_file = \"/run/example/{v}\"\n"); + let loaded = |v: &str| -> Result { + Ok(crate::config::LoadedConfig { + config: parse(&text(v))?, + source: Some(std::path::PathBuf::from("/etc/sipnab/sipnab.toml")), + shadowed: Vec::new(), + file_text: Some(text(v)), + }) + }; + let argv = |v: &str| -> Vec { + ["sipnab", "--api-key", v].iter().map(Into::into).collect() + }; + let at = chrono::DateTime::from_timestamp(1_790_000_000, 0).ok_or("a time")?; + let with = RunRecord::of(&loaded(SECRET)?, &argv(SECRET), at); + let other = RunRecord::of(&loaded("other-value")?, &argv("other-value"), at); + assert_ne!(with.file_sha256, other.file_sha256, "file digest"); + assert_ne!(with.parsed_sha256, other.parsed_sha256, "parsed digest"); + assert_ne!(with.argv_sha256, other.argv_sha256, "argv digest"); + assert_ne!( + with.effective_sha256, other.effective_sha256, + "effective digest" + ); + assert!(with.effective_sha256.is_some(), "an effective digest"); + assert!( + with.file_sha256.is_some() && with.parsed_sha256.is_some(), + "a loaded file has a file digest and a parsed digest" + ); + let json = serde_json::to_string(&config_info(Some(&with)))?; + assert!( + !json.contains(SECRET), + "the secret reached the record: {json}" + ); + Ok(()) + } + + /// The effective digest is SHA-256 of the two labeled, length-prefixed + /// parts; it follows each part, cannot be forged by moving bytes from one + /// part to the other, and is absent without a parsed digest. + #[test] + fn the_effective_digest_covers_the_file_and_the_command_line() -> Result<(), TestError> { + let (p, a) = ("1".repeat(64), "2".repeat(64)); + let got = effective_digest(Some(&p), &a).ok_or("a digest")?; + let mut want = Vec::new(); + for (label, part) in [("parsed", &p), ("argv", &a)] { + want.extend_from_slice(label.as_bytes()); + want.extend_from_slice(&(part.len() as u64).to_be_bytes()); + want.extend_from_slice(part.as_bytes()); + } + assert_eq!( + got, + reference_hex(&want), + "the effective digest is SHA-256 of the labeled, length-prefixed parts" + ); + assert_ne!( + Some(got.clone()), + effective_digest(Some(&"3".repeat(64)), &a), + "parsed" + ); + assert_ne!( + Some(got.clone()), + effective_digest(Some(&p), &"3".repeat(64)), + "argv" + ); + assert_ne!( + effective_digest(Some("ab"), "c"), + effective_digest(Some("a"), "bc"), + "a byte moved between the parts" + ); + assert_eq!( + effective_digest(None, &a), + None, + "no parsed digest, no effective digest" + ); + Ok(()) + } + + /// Arguments are length-prefixed and the program path is left out. + #[test] + fn argv_digest_separates_arguments_and_skips_the_program() { + let v = + |args: &[&str]| -> Vec { args.iter().map(Into::into).collect() }; + let ab_c = argv_digest(&v(&["sipnab", "ab", "c"])); + assert_eq!(ab_c.len(), 64, "{ab_c:?}"); + assert_ne!( + ab_c, + argv_digest(&v(&["sipnab", "a", "bc"])), + "moving a byte between arguments must change the digest" + ); + assert_eq!( + ab_c, + argv_digest(&v(&["/usr/local/bin/sipnab", "ab", "c"])), + "the program path is left out of the digest" + ); + } + + /// The file digest is SHA-256 of the file's bytes. + #[test] + fn file_sha256_is_the_digest_of_the_bytes() -> Result<(), TestError> { + let dir = tempfile::tempdir()?; + let path = dir.path().join("blob"); + let bytes: Vec = (0..200_000u32).map(|i| (i % 251) as u8).collect(); + std::fs::write(&path, &bytes)?; + assert_eq!( + file_sha256(&path)?, + reference_hex(&bytes), + "the file digest is SHA-256 of the file's bytes" + ); + assert!( + file_sha256(&dir.path().join("absent")).is_err(), + "a missing file is an error, not a digest" + ); + Ok(()) + } + + /// The binary digest is SHA-256 of the image this process runs. + #[test] + fn the_binary_digest_is_the_running_image() -> Result<(), TestError> { + let exe = std::env::current_exe()?; + let inv = current(None, &[], 0); + assert_eq!(inv.binary.error, None, "the running image must be readable"); + assert_eq!( + inv.binary.sha256, + Some(reference_hex(&std::fs::read(exe)?)), + "the binary digest is SHA-256 of the running image" + ); + Ok(()) + } + + /// The process never claims its own signature is verified. + #[test] + fn an_instance_reports_its_signature_unverified() { + assert_eq!( + signature_info().state, + "unverified", + "an instance never reports its signature verified" + ); + assert_eq!( + current(None, &[], 0).signature.state, + "unverified", + "the record carries the unverified state" + ); + } + + /// The build fields come from the one `--version` source. + #[test] + fn build_info_is_the_version_line_split() { + let b = build_info(); + assert_eq!( + b.version_line, + crate::cli::build_version(), + "the version line is the one --version prints" + ); + assert_eq!( + b.version, + env!("CARGO_PKG_VERSION"), + "the version is the crate version" + ); + assert_eq!( + b.commit, + env!("SIPNAB_GIT_COMMIT"), + "the commit is the one the build recorded" + ); + assert_eq!( + b.target, + env!("SIPNAB_TARGET"), + "the target is the one the build recorded" + ); + assert_eq!( + b.dirty, + !env!("SIPNAB_GIT_DIRTY").is_empty(), + "the dirty flag is the one the build recorded" + ); + let mut want: Vec = crate::cli::compiled_features() + .iter() + .map(|s| (*s).to_string()) + .collect(); + want.sort(); + assert!(!want.is_empty(), "this build compiles some feature"); + assert_eq!( + b.features, want, + "the features are the compiled set, sorted" + ); + } + + /// No record means unknown, said as `recorded: false`, never as digests + /// of nothing. + #[test] + fn an_unrecorded_run_says_so() { + let c = config_info(None); + assert!(!c.recorded, "no record means not recorded"); + assert_eq!(c.parsed_sha256, None, "no record has no parsed digest"); + assert_eq!(c.argv_sha256, None, "no record has no command-line digest"); + assert_eq!( + c.effective_sha256, None, + "no record has no effective digest" + ); + } +} diff --git a/src/output/mod.rs b/src/output/mod.rs index 78311f7f5..cfb334e3f 100644 --- a/src/output/mod.rs +++ b/src/output/mod.rs @@ -56,6 +56,8 @@ pub mod event_exec; pub mod fail2ban; pub mod group; pub mod hexdump; +#[cfg(any(feature = "api", feature = "mcp"))] +pub mod inventory; pub mod json; #[cfg(feature = "vcon")] pub mod live_rtp; diff --git a/src/tls_files.rs b/src/tls_files.rs index c1a2bfa8c..ab46efb82 100644 --- a/src/tls_files.rs +++ b/src/tls_files.rs @@ -39,7 +39,7 @@ pub(crate) fn provider() -> std::sync::Arc { /// issuer to the system store does not also have to name it here, and a host /// whose bundle sipnab cannot find is told to pass a CA file rather than /// silently trusting nothing. -#[cfg(any(feature = "hep", feature = "vcon", feature = "g729"))] +#[cfg(any(feature = "hep", feature = "vcon", feature = "g729", feature = "api"))] const SYSTEM_CA_BUNDLES: &[&str] = &[ // Debian, Ubuntu, Arch "/etc/ssl/certs/ca-certificates.crt", @@ -55,7 +55,7 @@ const SYSTEM_CA_BUNDLES: &[&str] = &[ /// The host's CA bundle: `$SSL_CERT_FILE` if it names a file, else the first /// of [`SYSTEM_CA_BUNDLES`] that exists, else `None`. -#[cfg(any(feature = "hep", feature = "vcon", feature = "g729"))] +#[cfg(any(feature = "hep", feature = "vcon", feature = "g729", feature = "api"))] pub(crate) fn host_ca_bundle() -> Option { std::env::var_os("SSL_CERT_FILE") .map(std::path::PathBuf::from) @@ -70,7 +70,7 @@ pub(crate) fn host_ca_bundle() -> Option { /// What a trust rule's refusals call its two settings and the certificate /// the operator is told to name. -#[cfg(any(feature = "hep", feature = "vcon", feature = "g729"))] +#[cfg(any(feature = "hep", feature = "vcon", feature = "g729", feature = "api"))] pub(crate) struct TrustNames<'a> { /// The setting whose CA file replaces the trust store (`--hep-tls-ca`). pub ca: &'a str, @@ -100,7 +100,7 @@ pub(crate) struct TrustNames<'a> { /// Both files named; a named file that cannot be read, holds no /// certificate, or holds one rustls refuses; or no host bundle where one is /// needed, the message naming `names.ca` as the way out. -#[cfg(any(feature = "hep", feature = "vcon", feature = "g729"))] +#[cfg(any(feature = "hep", feature = "vcon", feature = "g729", feature = "api"))] pub(crate) fn trust_roots( ca: Option<&std::path::Path>, extra: Option<&std::path::Path>, diff --git a/tests/api_capture_identity_test.rs b/tests/api_capture_identity_test.rs index 44200bbc8..0538be0e4 100644 --- a/tests/api_capture_identity_test.rs +++ b/tests/api_capture_identity_test.rs @@ -163,6 +163,11 @@ const WITHOUT_IDENTITY: &[(&str, &str)] = &[ bytes: the listener's roster, not the capture's contents; checked \ below", ), + ( + "/v1/inventory", + "the MCP `fleet_inventory` tool is source-independent and answers the \ + same document: what this process is, not what the capture holds", + ), ( "/v1/capabilities", "the MCP `server_capabilities` tool is source-independent: what this \ @@ -259,7 +264,8 @@ const WITHOUT_IDENTITY: &[(&str, &str)] = &[ /// The MCP counterparts [`WITHOUT_IDENTITY`] cites as carrying no identity, /// with arguments that make each answer. #[cfg(feature = "hep")] -const COUNTERPARTS_WITHOUT: &[(&str, &str)] = &[("hep_senders", r#"{}"#)]; +const COUNTERPARTS_WITHOUT: &[(&str, &str)] = + &[("hep_senders", r#"{}"#), ("fleet_inventory", r#"{}"#)]; /// Every GET path the published document declares, as the router serves it. fn documented_gets() -> Result, TestError> { diff --git a/tests/cli_flag_behavior_test.rs b/tests/cli_flag_behavior_test.rs index f91adabb4..cb20ddb25 100644 --- a/tests/cli_flag_behavior_test.rs +++ b/tests/cli_flag_behavior_test.rs @@ -2468,20 +2468,44 @@ fn token_scope_flag_mints_a_scope_the_verifier_honors() -> Result<(), TestError> "no other scope reaches call audio" ); } + + let inventory = mint("inventory")?; + assert!( + verifier.verify(&inventory, now, sipnab::auth::SCOPE_INVENTORY), + "--token-scope inventory must mint a token accepted for the inventory" + ); + assert!( + !verifier.verify(&inventory, now, sipnab::auth::SCOPE_FULL), + "and one that reads nothing else" + ); + assert!( + verifier.verify(&full, now, sipnab::auth::SCOPE_INVENTORY), + "a full token reads the inventory too" + ); Ok(()) } -/// `--token-scope audio` is refused for the MCP surface at mint time. +/// `--token-scope audio` and `--token-scope inventory` are refused for the +/// MCP surface at mint time. /// -/// MCP has no call audio routes, so the token could never authenticate -/// anything there. Failing at mint beats handing the operator a token that -/// silently works nowhere. +/// MCP has no call audio routes and no inventory route, so the token could +/// never authenticate anything there. Failing at mint beats handing the +/// operator a token that silently works nowhere. #[test] -fn token_scope_audio_is_refused_for_the_mcp_surface() -> Result<(), TestError> { +fn rest_only_scopes_are_refused_for_the_mcp_surface() -> Result<(), TestError> { + for scope in ["audio", "inventory"] { + rest_only_scope_is_refused_for_mcp(scope)?; + } + Ok(()) +} + +/// `--mint-token --token-scope ` for the MCP surface fails, saying the +/// scope applies to the REST API only. The MCP surface has no call audio +/// routes, and it serves the inventory tool to a `read` token. +fn rest_only_scope_is_refused_for_mcp(scope: &str) -> Result<(), TestError> { let dir = tempfile::tempdir()?; let key_path = dir.path().join("mcp.key"); std::fs::File::create(&key_path)?.write_all(b"mcp-scope-signing-key-0123456789")?; - let scope = "audio"; let out = std::process::Command::new(env!("CARGO_BIN_EXE_sipnab")) .args([ "--mint-token", diff --git a/tests/config_examples_test.rs b/tests/config_examples_test.rs index d2bada73b..7b671d09a 100644 --- a/tests/config_examples_test.rs +++ b/tests/config_examples_test.rs @@ -116,6 +116,12 @@ const FOREIGN_TOML: &[(&str, &str, &str)] = &[ "mcp_servers", "Codex CLI: its MCP server table", ), + ( + "docs/fleet.md", + "host", + "the `sipnab fleet inventory` hosts file, which \ + tests/fleet_inventory_test.rs parses with the command's own parser", + ), ( "docs/mcp-tools-security.md", "rules", diff --git a/tests/config_wiring_test.rs b/tests/config_wiring_test.rs index f8b94bfe8..3433e7c94 100644 --- a/tests/config_wiring_test.rs +++ b/tests/config_wiring_test.rs @@ -449,6 +449,32 @@ fn pcap_export_mode_all_variants_round_trip() -> Result<(), TestError> { Ok(()) } +/// Verify debug and test builds compile `sha2` optimized. +/// +/// `/v1/inventory` and the MCP `fleet_inventory` tool hash the running binary +/// on first use. A debug sipnab is about 200 MB, and unoptimized SHA-256 took +/// 1.97 s over it on the aarch64 build host (a release build hashes its own +/// image in milliseconds). On CI's ubuntu runner, under eight test binaries +/// at once, the first request outlasted the 10 s read timeout of +/// tests/support/server.rs, and four tests in tests/fleet_inventory_test.rs +/// failed with `WouldBlock`. Read from the `[profile.dev.package.sha2]` +/// section itself. +#[test] +fn cargo_toml_dev_profile_optimizes_sha2() -> Result<(), TestError> { + let cargo = std::fs::read_to_string("Cargo.toml")?; + let section = cargo + .split("\n[profile.dev.package.sha2]\n") + .nth(1) + .and_then(|rest| rest.split("\n[").next()) + .ok_or("Cargo.toml has a [profile.dev.package.sha2] section")?; + assert!( + section.lines().any(|l| l.trim() == "opt-level = 3"), + "debug builds must compile sha2 at opt-level 3 so the inventory's \ + binary digest answers within a test's read timeout: {section}" + ); + Ok(()) +} + /// Verify the release profile unwinds on panic, and is optimized and stripped. /// /// `unwind`, not `abort`: `crash::contain` catches a panic in a parser of diff --git a/tests/doc_commands_run_test.rs b/tests/doc_commands_run_test.rs index 0fa701c77..dda9ce842 100644 --- a/tests/doc_commands_run_test.rs +++ b/tests/doc_commands_run_test.rs @@ -313,6 +313,13 @@ fn classify(cmd: &str) -> Result { if g729_command.is_match(cmd) { return Ok(Plan::SideEffects); } + // `sipnab fleet inventory` connects to every instance its hosts file + // names, on other hosts. `tests/fleet_inventory_test.rs` runs it against + // instances of its own. + let fleet_command = regex::Regex::new(r"(^|\s)sipnab\s+fleet(\s|$)")?; + if fleet_command.is_match(cmd) { + return Ok(Plan::SideEffects); + } // `$(…)`, `$VAR`, or a line that opens a loop: the shell is doing the work. let shell_var = regex::Regex::new(r"\$[A-Za-z_(]")?; // Process substitution, `<(…)`, and a numbered redirection, `3< file`, are @@ -1604,6 +1611,11 @@ fn privilege_escalation_is_never_run_but_printing_a_filter_is() -> Result<(), Te "{cmd:?} would be RUN by this gate, and it acts on the host" ); } + assert_eq!( + classify("sipnab fleet inventory --hosts /etc/sipnab/fleet/fleet.toml")?, + Plan::SideEffects, + "sipnab fleet connects to other hosts and is never run by this gate" + ); // An ordinary read beside them is not swept up. assert_eq!(classify("sipnab -N -I capture.pcap --report")?, Plan::Reads); assert_eq!( diff --git a/tests/doc_jq_fields_test.rs b/tests/doc_jq_fields_test.rs index 050f40c1a..9b58f8dbc 100644 --- a/tests/doc_jq_fields_test.rs +++ b/tests/doc_jq_fields_test.rs @@ -56,6 +56,9 @@ enum Mode { Stun, /// `--json-analyze`: one object for the whole capture analysis. Analyze, + /// `sipnab fleet inventory --json`: one report over many instances, a + /// different document from a capture run's `--json`. + Fleet, } /// Fields that are legitimately absent from a flat key scan. @@ -131,7 +134,9 @@ fn recipes() -> Result, TestError> { .split_whitespace() .any(|w| w.split('=').next() == Some(*q)) }); - let mode = if let Some(flag) = query { + let mode = if block.contains("sipnab fleet") { + Mode::Fleet + } else if let Some(flag) = query { Mode::Query(flag) } else if block.contains("--json-dialogs") { Mode::Dialogs @@ -321,6 +326,7 @@ fn harvest(v: &serde_json::Value, depth: usize, keys: &mut BTreeSet) { fn keys_for(mode: Mode) -> Result, TestError> { let flag = match mode { Mode::Query(flag) => return Err(format!("{flag}: a query runs its own command").into()), + Mode::Fleet => return fleet_keys(), Mode::Messages => "--json", Mode::Dialogs => "--json-dialogs", Mode::Stun => "--json-stun", @@ -331,6 +337,7 @@ fn keys_for(mode: Mode) -> Result, TestError> { let fixtures: &[&str] = match mode { Mode::Messages | Mode::Dialogs | Mode::Query(_) => FIXTURES, Mode::Stun | Mode::Analyze => STUN_FIXTURES, + Mode::Fleet => return fleet_keys(), }; let mut keys = BTreeSet::new(); for pcap in fixtures { @@ -377,6 +384,34 @@ fn keys_for(mode: Mode) -> Result, TestError> { Ok(keys) } +/// The keys of a `sipnab fleet inventory --json` report: the command's own +/// `merge` over this process's real inventory record, the one +/// `GET /v1/inventory` returns. A live report needs instances on the network; +/// `tests/fleet_inventory_test.rs` runs the command against real ones, and +/// this builds the same document from the same two functions. Walked to depth +/// 6, which reaches `hosts[].inventory.hep_listener.senders[].peer`. +#[cfg(feature = "api")] +fn fleet_keys() -> Result, TestError> { + use sipnab::app::fleet::{Answer, HostResult, merge}; + let inventory = serde_json::to_value(sipnab::output::inventory::current(None, &[], 0))?; + let report = merge(vec![HostResult { + name: "edge-a".into(), + url: "https://192.0.2.11:8443".into(), + addresses: vec!["192.0.2.11".parse()?], + answer: Answer::Inventory(inventory), + }]); + let mut keys = BTreeSet::new(); + harvest(&serde_json::to_value(&report)?, 6, &mut keys); + Ok(keys) +} + +/// Without the `api` feature there is no `sipnab fleet`, and so no key a +/// fleet recipe could read. +#[cfg(not(feature = "api"))] +fn fleet_keys() -> Result, TestError> { + Ok(BTreeSet::new()) +} + /// Every documented `jq` field exists in the output its own block produces. #[test] fn documented_jq_fields_exist_in_the_output() -> Result<(), TestError> { @@ -392,9 +427,14 @@ fn documented_jq_fields_exist_in_the_output() -> Result<(), TestError> { let dialogs = keys_for(Mode::Dialogs)?; let stun = keys_for(Mode::Stun)?; let analyze = keys_for(Mode::Analyze)?; + let fleet = keys_for(Mode::Fleet)?; let mut bad = Vec::new(); for r in &recipes { + // A build without `api` has no `sipnab fleet` to check against. + if r.mode == Mode::Fleet && !cfg!(feature = "api") { + continue; + } let own; let keys = match r.mode { Mode::Query(flag) => { @@ -409,6 +449,7 @@ fn documented_jq_fields_exist_in_the_output() -> Result<(), TestError> { Mode::Dialogs => &dialogs, Mode::Stun => &stun, Mode::Analyze => &analyze, + Mode::Fleet => &fleet, }; for f in &r.fields { if !keys.contains(f) { @@ -421,6 +462,7 @@ fn documented_jq_fields_exist_in_the_output() -> Result<(), TestError> { Mode::Dialogs => "--json-dialogs", Mode::Stun => "--json-stun", Mode::Analyze => "--json-analyze", + Mode::Fleet => "sipnab fleet inventory --json", } )); } diff --git a/tests/docs_drift_test.rs b/tests/docs_drift_test.rs index 9bfd75c29..e12de5d93 100644 --- a/tests/docs_drift_test.rs +++ b/tests/docs_drift_test.rs @@ -1141,6 +1141,14 @@ const FOREIGN_FLAGS: &[(&str, &[&str])] = &[ "SECURITY.md", "docs/install.md", "website/content/docs/install.md", + // `gh attestation verify --repo`, in the fleet inventory's + // signature note and its how-to. + "docs/fleet.md", + "website/content/docs/fleet.md", + "docs/rest-api.md", + "website/content/docs/api.md", + "docs/mcp-tools-server.md", + "website/content/docs/mcp-tools-server.md", ], ), // `cosign verify-blob --bundle --certificate-identity-regexp @@ -1966,6 +1974,16 @@ fn cli_long_flags() -> BTreeSet { } } } + // `sipnab fleet inventory` likewise: its own clap definition, dispatched + // on the first argument, documented in docs/fleet.md and the CLI reference. + let fleet = sipnab::cli::fleet_args::FleetCli::command(); + for sub in std::iter::once(&fleet).chain(fleet.get_subcommands()) { + for arg in sub.get_arguments() { + if let Some(long) = arg.get_long() { + flags.insert(long.to_string()); + } + } + } // clap provides these automatically; get_arguments() doesn't list them. flags.insert("help".to_string()); flags.insert("version".to_string()); @@ -4210,9 +4228,13 @@ fn mcp_tool_table_lists_every_registered_tool() -> Result<(), TestError> { // directory of the file root, reads files and changes nothing, so the // split moves to 63 read-only of 76 and the thirteen write-capable // tools are unchanged. + // 76 -> 77 by `fleet_inventory` (src/mcp/tools/inventory.rs), the + // process's fleet inventory record, the same document as GET + // /v1/inventory. It reads and changes nothing, so the split moves to 64 + // read-only of 77 and the thirteen write-capable tools are unchanged. assert_eq!( registered.len(), - 76, + 77, "found only {} #[tool(name = ...)] entries under src/mcp/ — the \ attribute shape changed and this test is no longer reading the \ registry: {registered:?}", @@ -4958,13 +4980,17 @@ fn no_documentation_table_repeats_a_row() -> Result<(), TestError> { // scripts/build-site-internals.py. Attributed with // `git ls-files -o --exclude-standard` before they were staged. // 320 -> 321: website/brand/README.md, the logo's files and terms of use. + // 321 -> 323: docs/fleet.md, the fleet inventory how-to, and its site + // mirror website/content/docs/fleet.md. // Attributed against the staged diff: `--diff-filter=A` lists it as the // one markdown file added. // 321 -> 322: docs/integrations.md, the event-hook and alert page that // was hand-written for the site alone and is now its source. Attributed // against the staged diff: `--diff-filter=A` lists it as the one // markdown file added. - const EXPECTED_MARKDOWN_FILES: usize = 322; + // 322 -> 324: docs/fleet.md, the fleet inventory how-to, and its site + // mirror website/content/docs/fleet.md. + const EXPECTED_MARKDOWN_FILES: usize = 324; /// How many tables this gate expects to walk. /// /// Named rather than written twice. The count and the failure message @@ -5707,7 +5733,11 @@ fn no_documentation_table_repeats_a_row() -> Result<(), TestError> { // plugin load failures, and its site mirror website/content/docs/plugins.md // +1. Attributed per file against origin/main ffd06bfb1: no other changed // page gained or lost a table. - const EXPECTED_TABLES: usize = 1311; + // 1311 -> 1318 (the fleet inventory): docs/fleet.md +3 (exit status, + // host states, findings), its site mirror website/content/docs/fleet.md + // +3, and docs/design/surface-capability-matrix.md +1, the "Fleet + // inventory" capability row. Attributed per file against origin/main. + const EXPECTED_TABLES: usize = 1318; let repo = std::path::Path::new(env!("CARGO_MANIFEST_DIR")); let out = std::process::Command::new("git") diff --git a/tests/documented_flag_claims_test.rs b/tests/documented_flag_claims_test.rs index d69005be4..2a167950d 100644 --- a/tests/documented_flag_claims_test.rs +++ b/tests/documented_flag_claims_test.rs @@ -67,6 +67,16 @@ fn clap_long_flags() -> BTreeSet { } } flags.insert("g729 --help".into()); + // `sipnab fleet inventory` likewise, keyed `fleet `. + let fleet = sipnab::cli::fleet_args::FleetCli::command(); + for cmd in std::iter::once(&fleet).chain(fleet.get_subcommands()) { + for a in cmd.get_arguments() { + if let Some(l) = a.get_long() { + flags.insert(format!("fleet --{l}")); + } + } + } + flags.insert("fleet --help".into()); flags } @@ -111,6 +121,9 @@ fn flags_in(text: &str) -> BTreeSet { let command = if tokens.peek() == Some(&"g729") { tokens.next(); "g729 " + } else if tokens.peek() == Some(&"fleet") { + tokens.next(); + "fleet " } else { "" }; diff --git a/tests/error_reference_test.rs b/tests/error_reference_test.rs index bc9feb85b..ce0b0aa4e 100644 --- a/tests/error_reference_test.rs +++ b/tests/error_reference_test.rs @@ -878,7 +878,10 @@ const EXPECTED_UNRESOLVED: usize = 84; // 540 -> 542: src/config.rs, the refusals of 0 for `[sip] th_mapping_max` // and `[sip] th_mapping_ttl_secs`, the OpenSIPS E_SIPNAB_TH_MAP table's // bound and lifetime (#617). Nothing removed. -const EXPECTED_TEMPLATES: usize = 542; +// 542 -> 546: src/app/fleet.rs, `sipnab fleet inventory`: the hosts file's +// empty url and repeated name, a token file that does not hold one token, +// and an answer over MAX_ANSWER_BYTES. Nothing removed. +const EXPECTED_TEMPLATES: usize = 546; // --------------------------------------------------------------------------- // The page gates diff --git a/tests/fleet_inventory_test.rs b/tests/fleet_inventory_test.rs new file mode 100644 index 000000000..f23ccf120 --- /dev/null +++ b/tests/fleet_inventory_test.rs @@ -0,0 +1,757 @@ +// SPDX-License-Identifier: MIT OR Apache-2.0 +// SPDX-FileCopyrightText: The sipnab contributors + +//! The fleet inventory end to end: real `sipnab --api` processes, the +//! `GET /v1/inventory` route, the MCP `fleet_inventory` tool, and +//! `sipnab fleet inventory` reading two instances and one that is down. +//! +//! In `tests/` because it needs the built binary and more than one process. +#![cfg(all(feature = "api", feature = "mcp"))] + +use std::io::Write as _; +use std::process::Command; + +use server::TestError; + +#[path = "support/server.rs"] +mod server; + +#[path = "support/mcp.rs"] +mod mcp; + +#[path = "support/ports.rs"] +mod ports; + +use server::ApiServer; + +/// The signing key every instance here verifies tokens with. +const KEY: &str = "fleet-inventory-e2e-signing-key-0123456789"; + +/// A value that stands for a secret: it is put in a configuration file and +/// on a command line, and must appear in no inventory output. +const SECRET: &str = "s3cr3t-fleet-marker-5f1c2a9e"; + +/// An independent SHA-256 in hex. +fn sha256_hex(bytes: &[u8]) -> String { + use sha2::Digest as _; + sha2::Sha256::digest(bytes) + .iter() + .map(|b| format!("{b:02x}")) + .collect() +} + +/// A token for the inventory, from [`KEY`]. +fn inventory_token() -> String { + sipnab::auth::mint( + KEY.as_bytes(), + "fleet-e2e", + chrono::Utc::now().timestamp() + 3600, + sipnab::auth::AUDIENCE_API, + sipnab::auth::SCOPE_INVENTORY, + ) +} + +/// A config file in `dir` whose one value is `value`. +fn config_file( + dir: &std::path::Path, + name: &str, + value: &str, +) -> Result<(std::path::PathBuf, String), TestError> { + let text = format!("# fleet test\n[vcon_forward]\nauth_file = \"/run/example/{value}\"\n"); + let path = dir.join(name); + std::fs::write(&path, &text)?; + Ok((path, text)) +} + +/// An instance with node name `node` and config `config`. +fn instance(node: &str, config: &std::path::Path) -> Result { + let config = config.to_str().ok_or("utf-8 path")?; + ApiServer::spawn(&[ + "--api-signing-key", + KEY, + "--node-name", + node, + "--config", + config, + ]) +} + +/// A token file in `dir`, readable by this user only. +fn token_file( + dir: &std::path::Path, + name: &str, + token: &str, +) -> Result { + let path = dir.join(name); + let mut f = std::fs::File::create(&path)?; + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt as _; + f.set_permissions(std::fs::Permissions::from_mode(0o600))?; + } + writeln!(f, "{token}")?; + Ok(path) +} + +/// `sipnab fleet inventory --json` over `hosts`: exit code, stdout, stderr. +fn fleet(hosts: &std::path::Path, json: bool) -> Result<(i32, String, String), TestError> { + let mut cmd = Command::new(env!("CARGO_BIN_EXE_sipnab")); + cmd.args(["fleet", "inventory", "--hosts"]).arg(hosts); + cmd.args(["--timeout", "5"]); + if json { + cmd.arg("--json"); + } + let out = cmd.output()?; + Ok(( + out.status.code().unwrap_or(-1), + String::from_utf8(out.stdout)?, + String::from_utf8(out.stderr)?, + )) +} + +/// The route reports this build, the running binary's digest, the +/// configuration's digests and the node name; and the signature as +/// unverified. +#[test] +fn the_route_reports_build_binary_and_config_digests() -> Result<(), TestError> { + let dir = tempfile::tempdir()?; + let (path, text) = config_file(dir.path(), "a.toml", "edge-a.cred")?; + let srv = instance("edge-a", &path)?; + let resp = srv.get_bearer("/v1/inventory", &inventory_token())?; + assert_eq!(resp.status, 200, "{}", resp.body); + let v = resp.json()?; + assert_eq!(v["schema_version"], 1, "{v}"); + assert_eq!(v["node_name"], "edge-a", "{v}"); + assert_eq!( + v["build"]["version_line"], + sipnab::cli::build_version(), + "{v}" + ); + let exe = std::fs::read(env!("CARGO_BIN_EXE_sipnab"))?; + assert_eq!(v["binary"]["sha256"], sha256_hex(&exe), "{v}"); + assert_eq!(v["signature"]["state"], "unverified", "{v}"); + assert_eq!(v["config"]["recorded"], true, "{v}"); + assert_eq!(v["config"]["source"], path.display().to_string(), "{v}"); + assert_eq!( + v["config"]["file_sha256"], + sha256_hex(text.as_bytes()), + "{v}" + ); + let parsed: sipnab::config::Config = toml::from_str(&text)?; + assert_eq!( + v["config"]["parsed_sha256"], + sipnab::output::inventory::parsed_config_digest(&parsed)?, + "{v}" + ); + assert_eq!(v["process"]["pid"], srv.pid(), "{v}"); + let argv = v["config"]["argv_sha256"].as_str().ok_or("argv digest")?; + assert_eq!( + v["config"]["effective_sha256"].as_str(), + sipnab::output::inventory::effective_digest(v["config"]["parsed_sha256"].as_str(), argv) + .as_deref(), + "{v}" + ); + + // The build fields agree with what the same binary prints for + // `--version`, a source the route does not share with this test. + let out = Command::new(env!("CARGO_BIN_EXE_sipnab")) + .arg("--version") + .output()?; + let printed = String::from_utf8(out.stdout)?; + let first = printed.lines().next().ok_or("a --version line")?; + let line = v["build"]["version_line"].as_str().ok_or("version_line")?; + assert_eq!(first, format!("sipnab {line}"), "{v}"); + let version = v["build"]["version"].as_str().ok_or("version")?; + assert!( + first.starts_with(&format!("sipnab {version}")), + "{first} vs {v}" + ); + // `--version` prints `X.Y.Z (TAG COMMIT[-dirty]) features: ...`, or no + // parenthesis when the build recorded no commit. + let printed_commit = first + .split_once(" (") + .and_then(|(_, rest)| rest.split_once(')')) + .and_then(|(inside, _)| inside.split(' ').next_back()) + .map_or("", |c| c.trim_end_matches("-dirty")); + assert_eq!(v["build"]["commit"], printed_commit, "{first} vs {v}"); + assert_eq!( + v["build"]["dirty"], + first.contains("-dirty)"), + "{first} vs {v}" + ); + let features: Vec<&str> = v["build"]["features"] + .as_array() + .ok_or("features")? + .iter() + .filter_map(serde_json::Value::as_str) + .collect(); + let mut printed_features: Vec<&str> = first + .split(" features: ") + .nth(1) + .ok_or("a features part")? + .split(',') + .collect(); + printed_features.sort_unstable(); + assert_eq!(features, printed_features, "{first} vs {v}"); + Ok(()) +} + +/// The REST route and the MCP tool return the same record for one process +/// state. +#[tokio::test] +async fn rest_and_mcp_return_the_same_record() -> Result<(), TestError> { + use axum::body::Body; + use axum::extract::ConnectInfo; + use axum::http::Request; + use http_body_util::BodyExt as _; + use parking_lot::{Mutex, RwLock}; + use std::net::{IpAddr, SocketAddr}; + use std::sync::Arc; + use tower::ServiceExt as _; + + let dialogs = Arc::new(RwLock::new(sipnab::sip::dialog_store::DialogStore::new( + 100, false, + ))); + let streams = Arc::new(RwLock::new(sipnab::rtp::stream_store::StreamStore::new( + 100, + ))); + // A start well in the past, so each door's uptime shows it read the + // start it was given and not its own construction time. + let started = std::time::Instant::now() + .checked_sub(std::time::Duration::from_secs(100)) + .ok_or("a start 100 s ago")?; + let interfaces = vec!["lo".to_string()]; + // The first record of a process hashes its binary, which under a loaded + // test run takes seconds. Taken here, so neither door's read waits on it + // and the two reads are close enough in time to compare. + let _ = sipnab::output::inventory::current(None, &[], 0); + // One capture meter for both doors, carrying a HEP roster where the + // build has a listener, so the sender list is compared too. + #[cfg(feature = "hep")] + let meter = Some(roster("192.0.2.10", "198.51.100.66")?); + #[cfg(not(feature = "hep"))] + let meter: Option = None; + let state = sipnab::output::api::ApiState { + relay_query: Default::default(), + alias_thresholds: Default::default(), + dialog_store: dialogs.clone(), + stream_store: streams.clone(), + verifier: Arc::new(sipnab::auth::TokenVerifier::new( + sipnab::auth::VerifierConfig { + static_keys: vec![KEY.to_string()], + ..Default::default() + }, + )), + rate_limiter: Arc::new(Mutex::new(sipnab::output::api::RateLimiter::new( + 1000, 1024, + ))), + max_inline_media_bytes: None, + content_deny_header: None, + max_rows: 1000, + capture: None, + source_exhausted: None, + capture_interfaces: interfaces.clone(), + capture_meter: meter.clone(), + started_at: started, + persistence_gate: Arc::new(sipnab::output::persistence::PersistenceGate::new(false)), + tfps: Default::default(), + actions: Default::default(), + alert_engine: None, + armed_detections: Vec::new(), + scanner_response: Default::default(), + file_root: None, + vcon_spools: Vec::new(), + vcon_fetch: Default::default(), + live_audio: Arc::default(), + pipeline_options: Default::default(), + archive: sipnab::output::api::ArchivePasswordPolicy::default(), + }; + let mut req = Request::builder() + .uri("/v1/inventory") + .header("Authorization", format!("Bearer {KEY}")) + .body(Body::empty())?; + req.extensions_mut().insert(ConnectInfo(SocketAddr::new( + IpAddr::V4(std::net::Ipv4Addr::LOCALHOST), + 40000, + ))); + let resp = sipnab::output::api::build_router(state) + .oneshot(req) + .await?; + assert_eq!( + resp.status(), + 200, + "the REST route answers an inventory token" + ); + let rest: serde_json::Value = + serde_json::from_slice(&resp.into_body().collect().await?.to_bytes())?; + + let mut srv = sipnab::mcp::server::SipnabMcp::new(dialogs, streams) + .with_capture_interfaces(interfaces) + .with_capture_meter(meter); + srv.started_at = started; + let result = srv + .fleet_inventory() + .await + .map_err(|e| format!("the tool answers: {e:?}"))?; + let text = result + .content + .iter() + .find_map(|c| c.as_text()) + .map(|t| t.text.clone()) + .ok_or("a JSON block")?; + let mut mcp: serde_json::Value = serde_json::from_str(&text)?; + // Uptime is the one field that moves between the two reads, which can + // straddle a second boundary. Both must come from `started`, and every + // other field must be equal. + let mut rest = rest; + let up = |v: &mut serde_json::Value| v["process"]["uptime_seconds"].take().as_u64(); + let (rest_up, mcp_up) = ( + up(&mut rest).ok_or("REST uptime")?, + up(&mut mcp).ok_or("MCP uptime")?, + ); + assert!((100..=110).contains(&rest_up), "REST uptime {rest_up}"); + assert!( + mcp_up >= rest_up && mcp_up - rest_up <= 2, + "MCP uptime {mcp_up} vs {rest_up}" + ); + assert_eq!(rest, mcp, "one process, two doors, two answers"); + assert_eq!(rest["interfaces"][0]["name"], "lo", "{rest}"); + assert_eq!(rest["schema_version"], 1, "{rest}"); + #[cfg(feature = "hep")] + assert_eq!( + rest["hep_listener"]["senders"].as_array().map(Vec::len), + Some(2), + "{rest}" + ); + Ok(()) +} + +/// Every JSON pointer in `v` that names an object member, sorted. +fn member_paths(v: &serde_json::Value, at: &str, out: &mut Vec) { + match v { + serde_json::Value::Object(map) => { + for (k, inner) in map { + let path = format!("{at}/{k}"); + out.push(path.clone()); + member_paths(inner, &path, out); + } + } + serde_json::Value::Array(items) => { + for (i, inner) in items.iter().enumerate() { + member_paths(inner, &format!("{at}/{i}"), out); + } + } + _ => {} + } +} + +/// The real MCP dispatch, not only the tool method: `tools/call +/// fleet_inventory` on a running `sipnab --mcp` answers the document a +/// running `sipnab --api` answers on `GET /v1/inventory`, member for member, +/// and with equal values wherever the two processes are the same (build, +/// binary, configuration file, node name, privilege, signature). Whatever the +/// MCP server adds to every answer on the way out would show here as a +/// member REST does not have. +#[test] +fn the_mcp_dispatch_answers_the_rest_document() -> Result<(), TestError> { + let dir = tempfile::tempdir()?; + let (cfg, _) = config_file(dir.path(), "m.toml", "edge-m.cred")?; + let cfg_s = cfg.to_str().ok_or("utf-8 path")?; + let srv = instance("edge-m", &cfg)?; + let rest = srv.get_bearer("/v1/inventory", &inventory_token())?; + assert_eq!(rest.status, 200, "{}", rest.body); + let rest = rest.json()?; + + let pcap = format!( + "{}/tests/fixtures/sip_call.pcap", + env!("CARGO_MANIFEST_DIR") + ); + let mut session = mcp::McpSession::start(&pcap, &["--config", cfg_s, "--node-name", "edge-m"])?; + let reply = session.call("fleet_inventory", serde_json::json!({}))?; + let _ = session.stop(); + let text = reply["result"]["content"][0]["text"] + .as_str() + .ok_or_else(|| format!("no text block in {reply}"))?; + let mcp: serde_json::Value = serde_json::from_str(text)?; + assert_eq!( + reply["result"]["structuredContent"], mcp, + "the structured copy is the text block" + ); + + let (mut rest_paths, mut mcp_paths) = (Vec::new(), Vec::new()); + member_paths(&rest, "", &mut rest_paths); + member_paths(&mcp, "", &mut mcp_paths); + rest_paths.sort(); + mcp_paths.sort(); + assert_eq!(mcp_paths, rest_paths, "REST {rest}\nMCP {mcp}"); + for same in [ + "/schema_version", + "/node_name", + "/build", + "/binary", + "/signature", + "/config/source", + "/config/file_sha256", + "/config/parsed_sha256", + "/config/recorded", + "/privilege", + "/hep_listener", + ] { + assert_eq!( + rest.pointer(same), + mcp.pointer(same), + "{same}: REST {rest}\nMCP {mcp}" + ); + } + Ok(()) +} + +/// Two instances with different configurations, and one that is down: +/// `sipnab fleet inventory` lists all three, gives each answering instance +/// the digests that instance reports for itself, marks the third +/// unreachable, and exits 1. +#[test] +fn the_fleet_command_reads_two_instances_and_reports_one_down() -> Result<(), TestError> { + let dir = tempfile::tempdir()?; + let (a_cfg, _) = config_file(dir.path(), "a.toml", "edge-a.cred")?; + let (b_cfg, _) = config_file(dir.path(), "b.toml", "edge-b.cred")?; + let a = instance("edge-a", &a_cfg)?; + let b = instance("edge-b", &b_cfg)?; + let token = inventory_token(); + let tok = token_file(dir.path(), "fleet.token", &token)?; + let own = |s: &ApiServer| -> Result { + s.get_bearer("/v1/inventory", &token)?.json() + }; + let (a_own, b_own) = (own(&a)?, own(&b)?); + assert_ne!( + a_own["config"]["parsed_sha256"], b_own["config"]["parsed_sha256"], + "the two instances must differ for the comparison to prove anything" + ); + assert_ne!( + a_own["config"]["effective_sha256"], b_own["config"]["effective_sha256"], + "two configurations, two effective digests" + ); + + // Held for the whole test, so no parallel test can be handed the number + // and listen on it. + let down_port = ports::refused_tcp_port()?; + let down = down_port.port(); + let hosts = dir.path().join("hosts.toml"); + let tok = tok.display(); + std::fs::write( + &hosts, + format!( + "[[host]]\nname = \"edge-a\"\nurl = \"http://{}\"\ntoken_file = \"{tok}\"\n\ + [[host]]\nname = \"edge-b\"\nurl = \"http://{}\"\ntoken_file = \"{tok}\"\n\ + [[host]]\nname = \"edge-down\"\nurl = \"http://127.0.0.1:{down}\"\ntoken_file = \"{tok}\"\n", + a.addr, b.addr + ), + )?; + let (code, out, err) = fleet(&hosts, true)?; + assert_eq!(code, 1, "one host is down: stdout {out}\nstderr {err}"); + let report: serde_json::Value = serde_json::from_str(&out)?; + let rows = report["hosts"].as_array().ok_or("hosts")?; + assert_eq!(rows.len(), 3, "{report}"); + assert_eq!(rows[0]["name"], "edge-a", "rows keep the hosts-file order"); + assert_eq!(rows[0]["state"], "ok", "{report}"); + assert_eq!(rows[1]["state"], "ok", "{report}"); + assert_eq!( + rows[2]["name"], "edge-down", + "the host that is down is still listed" + ); + assert_eq!(rows[2]["state"], "unreachable", "{report}"); + assert_eq!(report["ok"], false, "a host down makes the report not ok"); + for (row, own) in [(&rows[0], &a_own), (&rows[1], &b_own)] { + let inv = &row["inventory"]; + assert_eq!(inv["node_name"], own["node_name"], "{report}"); + assert_eq!( + inv["config"]["parsed_sha256"], own["config"]["parsed_sha256"], + "each row's parsed digest is the one that instance reports itself" + ); + assert_eq!( + inv["config"]["effective_sha256"], own["config"]["effective_sha256"], + "each row's effective digest is the one that instance reports itself" + ); + assert_eq!( + inv["config"]["file_sha256"], own["config"]["file_sha256"], + "each row's file digest is the one that instance reports itself" + ); + assert_eq!( + inv["binary"]["sha256"], own["binary"]["sha256"], + "each row's binary digest is the one that instance reports itself" + ); + } + assert_eq!( + rows[0]["inventory"]["node_name"], "edge-a", + "the first row is edge-a's inventory" + ); + assert_eq!( + rows[1]["inventory"]["node_name"], "edge-b", + "the second row is edge-b's inventory" + ); + + // The two that answer alone are a healthy fleet: exit 0, no finding. + std::fs::write( + &hosts, + format!( + "[[host]]\nname = \"edge-a\"\nurl = \"http://{}\"\ntoken_file = \"{tok}\"\n\ + [[host]]\nname = \"edge-b\"\nurl = \"http://{}\"\ntoken_file = \"{tok}\"\n", + a.addr, b.addr + ), + )?; + let (code, out, err) = fleet(&hosts, false)?; + assert_eq!(code, 0, "stdout {out}\nstderr {err}"); + // The table's CONFIG column is each instance's own effective digest. + for (name, own) in [("edge-a", &a_own), ("edge-b", &b_own)] { + let row = out + .lines() + .find(|l| l.starts_with(name)) + .ok_or_else(|| format!("no {name} row in\n{out}"))?; + let digest = own["config"]["effective_sha256"].as_str().ok_or("digest")?; + assert!(row.contains(&digest[..12]), "{name}: {digest} not in {row}"); + } + Ok(()) +} + +/// A secret in an instance's configuration and on its command line, and +/// the controller's own token, appear in no output: not the REST body, not +/// the MCP tool's answer, not the fleet command's stdout or stderr in either +/// format, not the instance's log. A refused instance's error does not quote +/// the token either. +#[test] +fn no_secret_appears_in_any_inventory_output() -> Result<(), TestError> { + let dir = tempfile::tempdir()?; + let (cfg, _) = config_file(dir.path(), "s.toml", SECRET)?; + let cfg_s = cfg.to_str().ok_or("utf-8 path")?; + let api_key = format!("{SECRET}-api-key"); + let srv = ApiServer::spawn(&[ + "--api-signing-key", + KEY, + "--api-key", + &api_key, + "--node-name", + "edge-s", + "--config", + cfg_s, + ])?; + let token = inventory_token(); + let mut outputs: Vec<(String, String)> = Vec::new(); + + let rest = srv.get_bearer("/v1/inventory", &token)?; + assert_eq!(rest.status, 200, "{}", rest.body); + outputs.push(("REST body".into(), rest.body.clone())); + outputs.push(("instance log".into(), srv.startup_log.clone())); + + let pcap = format!( + "{}/tests/fixtures/sip_call.pcap", + env!("CARGO_MANIFEST_DIR") + ); + let mut session = mcp::McpSession::start(&pcap, &["--config", cfg_s])?; + let tool = session.call("fleet_inventory", serde_json::json!({}))?; + outputs.push(("MCP answer".into(), tool.to_string())); + assert!(tool["result"]["isError"] != true, "{tool}"); + let _ = session.stop(); + + let tok = token_file(dir.path(), "fleet.token", &token)?; + let bad = token_file(dir.path(), "bad.token", &format!("{SECRET}-wrong-token"))?; + let hosts = dir.path().join("hosts.toml"); + std::fs::write( + &hosts, + format!( + "[[host]]\nname = \"edge-s\"\nurl = \"http://{}\"\ntoken_file = \"{}\"\n\ + [[host]]\nname = \"edge-refused\"\nurl = \"http://{}\"\ntoken_file = \"{}\"\n", + srv.addr, + tok.display(), + srv.addr, + bad.display() + ), + )?; + for json in [true, false] { + let (code, out, err) = fleet(&hosts, json)?; + assert_eq!(code, 1, "one host refuses: {out}\n{err}"); + assert!(out.contains("refused"), "{out}"); + outputs.push((format!("fleet stdout json={json}"), out)); + outputs.push((format!("fleet stderr json={json}"), err)); + } + + let inv: serde_json::Value = serde_json::from_str(&rest.body)?; + assert!( + inv["config"]["parsed_sha256"].is_string(), + "the digest is there: {inv}" + ); + + // The same instance with only the secret changed reports a different + // effective digest, and the same parsed one. + let rotated_key = format!("{SECRET}-rotated"); + let rotated = ApiServer::spawn(&[ + "--api-signing-key", + KEY, + "--api-key", + &rotated_key, + "--node-name", + "edge-s", + "--config", + cfg_s, + ])?; + let rotated_body = rotated.get_bearer("/v1/inventory", &token)?; + assert_eq!(rotated_body.status, 200, "{}", rotated_body.body); + outputs.push(("rotated REST body".into(), rotated_body.body.clone())); + let rot: serde_json::Value = serde_json::from_str(&rotated_body.body)?; + assert_eq!( + rot["config"]["parsed_sha256"], inv["config"]["parsed_sha256"], + "the collector is listening with a roster" + ); + assert_ne!( + rot["config"]["effective_sha256"], inv["config"]["effective_sha256"], + "a changed secret kept the effective digest" + ); + assert!(rot["config"]["effective_sha256"].is_string(), "{rot}"); + for (name, text) in &outputs { + assert!(!text.contains(SECRET), "{name} carries the secret:\n{text}"); + assert!( + !text.contains(&token), + "{name} carries the controller's token:\n{text}" + ); + } + Ok(()) +} + +/// The fleet command refuses a token file other users can read, without +/// quoting it, and a hosts file it cannot use, with exit 2. +#[test] +fn the_fleet_command_refuses_unsafe_or_unusable_input() -> Result<(), TestError> { + let dir = tempfile::tempdir()?; + let hosts = dir.path().join("hosts.toml"); + std::fs::write(&hosts, "[[host]]\nname = \"a\"\n")?; + let (code, _, err) = fleet(&hosts, false)?; + assert_eq!(code, 2, "{err}"); + + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt as _; + let closed = ports::refused_tcp_port()?; + let open = dir.path().join("open.token"); + std::fs::write(&open, format!("{SECRET}\n"))?; + std::fs::set_permissions(&open, std::fs::Permissions::from_mode(0o644))?; + std::fs::write( + &hosts, + format!( + "[[host]]\nname = \"a\"\nurl = \"http://127.0.0.1:{}\"\ntoken_file = \"{}\"\n", + closed.port(), + open.display() + ), + )?; + let (code, out, err) = fleet(&hosts, true)?; + assert_eq!(code, 1, "{out}\n{err}"); + let report: serde_json::Value = serde_json::from_str(&out)?; + let error = report["hosts"][0]["error"].as_str().unwrap_or_default(); + assert!(error.contains("open.token"), "{report}"); + assert!( + !out.contains(SECRET) && !err.contains(SECRET), + "{out}\n{err}" + ); + } + Ok(()) +} + +/// A HEP roster with one sender at `listed`, one at `stray`, both admitted +/// on a frozen clock. +#[cfg(feature = "hep")] +fn roster(listed: &str, stray: &str) -> Result { + use sipnab::capture::hep_roster::{HepRoster, RosterState, SenderTrust, hep_source_label}; + let t = std::time::Instant::now(); + let wall = chrono::DateTime::from_timestamp(1_790_000_000, 0).ok_or("a time")?; + let mut state = RosterState::new( + SenderTrust::SharedSecretPlain, + 4096, + std::time::Duration::from_secs(30), + t, + wall, + ); + for (id, peer) in [(7, listed), (9, stray)] { + let peer: std::net::IpAddr = peer.parse()?; + state.admitted(Some(id), peer, &hep_source_label(Some(id), peer), t); + } + let frozen = t + std::time::Duration::from_secs(1); + let (_tx, rx) = sipnab::capture::channel::packet_channel(8); + let meter = rx.meter(); + if !meter.attach_hep_roster(HepRoster::with_clock( + state, + std::sync::Arc::new(move || frozen), + )) { + return Err("a fresh meter takes a roster".into()); + } + Ok(meter) +} + +/// The cross-check end to end, from the listener's roster: a collector's +/// inventory, built by the one builder both surfaces call, lists its HEP +/// senders; `merge` flags the sender no listed instance's address +/// accounts for, naming the collector and the address, and not the one +/// that is listed. +#[cfg(feature = "hep")] +#[test] +fn a_roster_sender_missing_from_the_inventory_is_flagged() -> Result<(), TestError> { + use sipnab::app::fleet::{Answer, HostResult, merge}; + let meter = roster("192.0.2.10", "198.51.100.66")?; + let hub = sipnab::output::inventory::current(Some(&meter), &[], 5); + let hub = serde_json::to_value(&hub)?; + assert_eq!(hub["hep_listener"]["listening"], true, "{hub}"); + let edge = serde_json::to_value(sipnab::output::inventory::current(None, &[], 5))?; + let report = merge(vec![ + HostResult { + name: "edge-a".into(), + url: "http://192.0.2.10:8080".into(), + addresses: vec!["192.0.2.10".parse()?], + answer: Answer::Inventory(edge), + }, + HostResult { + name: "hub".into(), + url: "http://192.0.2.1:8080".into(), + addresses: vec!["192.0.2.1".parse()?], + answer: Answer::Inventory(hub), + }, + ]); + assert!(!report.ok, "{report:?}"); + let stray: Vec<_> = report + .findings + .iter() + .filter(|f| f.kind == "unlisted_hep_sender") + .collect(); + assert_eq!(stray.len(), 1, "{:?}", report.findings); + assert!(stray[0].detail.contains("198.51.100.66"), "{:?}", stray[0]); + assert!(stray[0].detail.contains("hub"), "{:?}", stray[0]); + assert!(!stray[0].detail.contains("192.0.2.10"), "{:?}", stray[0]); + Ok(()) +} + +/// Every hosts file `docs/fleet.md` shows parses with the command's own +/// parser, so a reader who copies one gets a file `sipnab fleet inventory` +/// accepts. +#[test] +fn the_documented_hosts_files_parse() -> Result<(), TestError> { + let page = std::fs::read_to_string(concat!(env!("CARGO_MANIFEST_DIR"), "/docs/fleet.md"))?; + let mut parsed = 0; + let mut rest = page.as_str(); + while let Some(at) = rest.find("```toml\n") { + let body_start = at + "```toml\n".len(); + let body_len = rest[body_start..] + .find("```") + .ok_or("an unterminated toml fence")?; + let body: String = rest[body_start..body_start + body_len] + .lines() + .map(str::trim_start) + .collect::>() + .join("\n"); + rest = &rest[body_start + body_len..]; + if !body.contains("[[host]]") { + continue; + } + let hosts = sipnab::app::fleet::parse_hosts(&body) + .map_err(|e| format!("docs/fleet.md hosts file does not parse: {e}\n{body}"))?; + assert!(!hosts.is_empty(), "a documented hosts file names hosts"); + parsed += 1; + } + assert!(parsed >= 1, "docs/fleet.md shows a hosts file"); + Ok(()) +} diff --git a/tests/homepage_link_landing_test.rs b/tests/homepage_link_landing_test.rs index 5b78261a9..6af42c85a 100644 --- a/tests/homepage_link_landing_test.rs +++ b/tests/homepage_link_landing_test.rs @@ -82,7 +82,7 @@ const LANDINGS: &[Landing] = &[ ), ( "@/docs/mcp-tools.md", - "76 MCP tools", + "77 MCP tools", &["tool", "lint", "evidence", "correlat"], ), // Trust. diff --git a/tests/link_integrity_test.rs b/tests/link_integrity_test.rs index 598bf960a..5b73a9de1 100644 --- a/tests/link_integrity_test.rs +++ b/tests/link_integrity_test.rs @@ -1578,7 +1578,12 @@ fn wiki_intra_docs_links_resolve() -> Result<(), TestError> { // the site alone; docs/plugins.md +2, docs/README.md +1, docs/examples.md // +1, docs/fail2ban-sipnab.md +1 and docs/library.md +1. Counted per page // against origin/main ffd06bfb1; every other page held its count. - const EXPECTED_WIKI_LINKS: usize = 2079; + // 2079 -> 2097 (the fleet inventory): docs/fleet.md +4, its new how-to; + // README.md, auth.md +1 each and cli-reference.md, errors.md, + // mcp-estate.md, mcp-tools-server.md, mcp-tools.md and rest-api.md +2 + // each, links to fleet.md and its sections or, in mcp-tools.md, to the + // new tool's section. Counted per page against origin/main. + const EXPECTED_WIKI_LINKS: usize = 2097; // Raised 459 -> 460 when SRC1 stage 1 shipped: docs/cli-reference.md's // `--hep-listen` row now points at cookbook recipe "Take signaling from HEP and media off the wire, in one process" in docs/examples.md // rather than restating how to pair `-L` with `-d`. Attributed per file @@ -2835,7 +2840,8 @@ fn every_docs_page_is_linked_from_the_index() -> Result<(), TestError> { // operator-guide-notes.md. // 99 -> 100: docs/integrations.md, reached through the How-to list in // docs/README.md. - const EXPECTED_DOCS_PAGES: usize = 100; + // 100 -> 101: docs/fleet.md, reached from docs/README.md. + const EXPECTED_DOCS_PAGES: usize = 101; // Links are extracted from PROSE, not from the file's bytes. A raw // `contains("](backers.md")` counted a link that had been wrapped in an // HTML comment: the substring was still there, the page was reachable from diff --git a/tests/mcp_protocol_features_test.rs b/tests/mcp_protocol_features_test.rs index 990910a65..fa6798b64 100644 --- a/tests/mcp_protocol_features_test.rs +++ b/tests/mcp_protocol_features_test.rs @@ -318,6 +318,8 @@ fn schema_probes(call_id: &str) -> Vec<(&'static str, Value)> { // `note` the schema must describe, since a live roster never sends // them absent. ("hep_senders", json!({})), + // The test server's own process record. + ("fleet_inventory", json!({})), ("find_correlated", json!({"call_id": call_id})), // One second, and the filter is allowed to match nothing: the // deadline path is a SUCCESSFUL answer, so it exercises the same diff --git a/tests/mcp_stdio_test.rs b/tests/mcp_stdio_test.rs index a13e275b7..210ef541d 100644 --- a/tests/mcp_stdio_test.rs +++ b/tests/mcp_stdio_test.rs @@ -741,6 +741,7 @@ fn stdio_mcp_full_tool_set_and_remaining_tools() -> Result<(), TestError> { "find_in_captures", "find_in_captures_status", "find_problems", + "fleet_inventory", "generate_fail2ban_rule", "generate_repro", "generate_wireshark_filter", @@ -810,7 +811,8 @@ fn stdio_mcp_full_tool_set_and_remaining_tools() -> Result<(), TestError> { } expected.sort(); assert_eq!(names, expected, "MCP tool set drifted"); - let want = 69 + if cfg!(feature = "vcon") { 6 } else { 0 } + usize::from(cfg!(feature = "hep")); + // 69 -> 70: `fleet_inventory`, in every build with the MCP server. + let want = 70 + if cfg!(feature = "vcon") { 6 } else { 0 } + usize::from(cfg!(feature = "hep")); assert_eq!(names.len(), want, "expected exactly {want} MCP tools"); // Wait for the replay to be read to its end. Until then the store-reading diff --git a/website/config.toml b/website/config.toml index 1ad79e973..cd5f0ef7e 100644 --- a/website/config.toml +++ b/website/config.toml @@ -401,6 +401,7 @@ pages = [ { path = "docs/mcp-tools-tls.md", menu = false }, { path = "docs/mcp-deploy.md", menu = false }, { path = "docs/mcp-estate.md", menu = false }, + { path = "docs/fleet.md", menu = false }, { path = "docs/mcp-protocol.md", menu = false }, { path = "docs/vcon-harness.md", menu = false }, { path = "docs/g729.md", menu = false }, diff --git a/website/content/docs/api.md b/website/content/docs/api.md index 4bb3ccf1d..cb59cfd8a 100644 --- a/website/content/docs/api.md +++ b/website/content/docs/api.md @@ -224,7 +224,7 @@ only. | `--api-token-ttl ` | Lifetime of a minted token. Default `3600`. | | `--mint-token` | Sign a token with the first configured key, print it, and exit. Starts no capture and no server. | | `--token-id ` | The token's `id` (`jti`), used later for revocation. Defaults to a generated id. | -| `--token-scope ` | `full` (default), `metrics`, `actions`, `admin` or `audio` for the REST API. See **Scope** below. | +| `--token-scope ` | `full` (default), `metrics`, `actions`, `admin`, `audio` or `inventory` for the REST API. See **Scope** below. | | `--api-revoked-file ` | Denylist of revoked token ids, one per line (blanks and `#` comments ignored). | **Scope.** Mint each client a token for the routes it needs: @@ -236,6 +236,7 @@ only. | `actions` | Everything `full` reaches, plus the action routes that ask TFPS to ban or unban a source. The run must also enable them with `--allow-action tfps:rest`. It lifts only the bans it placed. | The system that decides to block a source. | | `admin` | Everything `actions` reaches, and it also lifts a ban another caller placed. | An operator who must be able to lift any ban. | | `audio` | The live-audio routes a `live_audio` vCon rule serves, and nothing else. | A speech-to-text consumer. | +| `inventory` | `GET /v1/inventory` only. Every other route returns `401`. `full` reaches the inventory too. | A fleet controller running [`sipnab fleet inventory`](@/docs/fleet.md). | The `read` scope is for MCP tokens. Minting it with `--api-signing-key` refuses. @@ -3230,6 +3231,103 @@ The MCP tool `server_capabilities` returns the same feature set and the same `libpcap` block. The second line of `sipnab --version` prints the same library banner. +### GET /v1/inventory + +What this sipnab process is, for managing many instances as one fleet: + +- its build: version, git commit and tag, target and compiled features, +- SHA-256 of the binary it is running, +- its configuration, as digests only, +- its user id and privilege drop, its pid, start time and uptime, +- its capture interfaces' error and drop counters, its capture and HEP export + counters, and the addresses its HEP listener receives from. + +It needs a token with scope `inventory`, or `full`. [`sipnab fleet +inventory`](@/docs/fleet.md) reads it from every instance in a hosts file. + +**curl:** + +```bash +curl -s -H "Authorization: Bearer $SIPNAB_INVENTORY_TOKEN" http://127.0.0.1:8080/v1/inventory | jq . +``` + +**Response** (from a run on 2026-11-03 of a development build, with the config path replaced by `/etc/sipnab/sipnab.toml`): + +```json +{ + "schema_version": 1, + "node_name": "edge-a", + "build": { + "version": "0.5.232", + "commit": "29204db7", + "tag": "", + "dirty": true, + "target": "aarch64-unknown-linux-gnu", + "features": ["api", "archive", "audio", "g729", "hep", "mcp", "mcp-http", "metrics", + "native", "opus", "plugins", "tls", "tui", "vcon"], + "version_line": "0.5.232 (29204db7-dirty) features: native,tui,audio,tls,hep,api,mcp,mcp-http,metrics,plugins,vcon,opus,archive,g729" + }, + "binary": { + "sha256": "3242a3324b1c3ac762ba15d4f2eec3789a995533540b8aee56ec087afdc51ea8", + "error": null + }, + "signature": { + "state": "unverified", + "note": "sipnab does not verify its own binary. Compare binary.sha256 with SHA256SUMS.txt of the release, after `gh attestation verify SHA256SUMS.txt --repo NormB/sipnab`." + }, + "config": { + "recorded": true, + "source": "/etc/sipnab/sipnab.toml", + "shadowed": [], + "file_sha256": "703493b6b6bdef18b62090d6452a2bd9fc72de264d4b97822cb0b7763697fca6", + "parsed_sha256": "1849a6ee687ec0b1394f2d305198c638bca159e7aa032911156074b8d11f4044", + "argv_sha256": "d5c72de2bcf085f7d8ca9884bed30027d7504215d507b04f6be16ecc063194e0", + "effective_sha256": "31e67effe25101b07a2c7942f05b83d568816bf81a1769d65ab8396c8e89705a" + }, + "privilege": { "uid": 1000, "root": false, "dropped_to": null }, + "process": { "pid": 3983974, "started": "2026-11-03T13:26:05Z", "uptime_seconds": 17 }, + "interfaces": [], + "counters": { + "capture_packets_total": 852, + "capture_sources_refused_total": 0, + "capture_backpressure_blocks_total": 0, + "hep_export_failures_total": null, + "hep_export_dropped_total": null, + "hep_export_queue_depth": null + }, + "hep_listener": { "listening": false, "packets_refused": 0, "senders": [] } +} +``` + +**No configuration value is in the answer.** + +- `file_sha256` is SHA-256 of the config file's bytes as this run read them. +- `parsed_sha256` is SHA-256 of the configuration as sipnab parsed it, after + `${NAME}` expansion, with every object's keys sorted. Two files that differ + only in comments, whitespace, key order or an unknown key give one value. +- `argv_sha256` is SHA-256 of the command line, without the program path. +- `effective_sha256` is SHA-256 over `parsed_sha256` and `argv_sha256`. A flag + overrides the file, so this is the digest to compare across instances. + `sipnab fleet inventory` shows it as `CONFIG`. +- A secret in the file or on the command line changes a digest, and its value + never appears. A secret held in a file the configuration names, such as + `--api-signing-key-file`, reaches the digests by its path only. + +**`signature.state` is always `unverified`.** A process cannot prove which +binary it is. Compare `binary.sha256` with the release's `SHA256SUMS.txt`, +after verifying that file's attestation with `gh attestation verify`. +sipnab reads `binary.sha256` from `/proc/self/exe` on Linux, so after an upgrade +replaces the file it still names the binary that is running. + +**Every field is the process's own claim.** An attacker who controls a host +can report anything. + +`hep_listener.senders` lists at most `MAX_HEP_PEERS` (256) senders. + +`config.recorded` is `false`, and every digest `null`, for a server that did not +record its configuration at start. The MCP tool `fleet_inventory` returns the +same fields and values. + ### GET /v1/stats Aggregate statistics across all dialogs and streams, including PDD percentiles. diff --git a/website/content/docs/auth.md b/website/content/docs/auth.md index 687f9b0a2..590477b0a 100644 --- a/website/content/docs/auth.md +++ b/website/content/docs/auth.md @@ -92,7 +92,7 @@ Distribute the printed token to clients. ### Scope: what a token may reach `--token-scope` sets what a token may reach. It takes `full` (the default), -`metrics`, `read`, `actions`, `admin` or `audio`: +`metrics`, `read`, `actions`, `admin`, `audio` or `inventory`: | Scope | Surface | What it reaches | |---|---|---| @@ -102,6 +102,7 @@ Distribute the printed token to clients. | `actions` | either | Everything `full` reaches, plus the actions that change another system, such as a TFPS ban, where the server enables them with `--allow-action`. It lifts only the bans it placed. | | `admin` | either | Everything `actions` reaches, and it also lifts a ban another caller placed, for example one placed by a token since lost. | | `audio` | REST API | The live-audio routes under `/v1/calls/`, and nothing else. No other scope reaches them. | +| `inventory` | REST API | `GET /v1/inventory`, and nothing else. `full`, `actions` and `admin` reach it too. For [`sipnab fleet inventory`](@/docs/fleet.md). | Give each client the narrowest scope that does its job: diff --git a/website/content/docs/cli.md b/website/content/docs/cli.md index c4474c059..559762f58 100644 --- a/website/content/docs/cli.md +++ b/website/content/docs/cli.md @@ -2403,7 +2403,7 @@ This applies to root runs and to unprivileged runs set up with | `--syslog` | -- | off | 0.3.0 | Send alerts to syslog. Config: `[security] alert` | | `--mint-token` | -- | off | 0.4.3 | Mint a signed bearer token from the first configured signing key (API or MCP), print it to stdout, and exit (no capture/servers). See [`auth.md`](@/docs/auth.md). | | `--token-id` | `` | -- | 0.4.3 | Token id (`jti`) for `--mint-token`, used for revocation. Defaults to a generated id. | -| `--token-scope` | `` | `full` | 0.5.59 | Scope for `--mint-token`. `metrics` reaches `GET /metrics` and returns `401` everywhere else — mint one for a scrape job rather than a credential that also reads `/v1/dialogs` and the message bodies underneath. `read` is the MCP counterpart: it reaches the tools annotated read-only and refuses every tool that writes or replaces state (`export_capture`, `open_capture`, `shutdown_server`, `tfps_ban` and the rest; `tools/list` marks each one with `readOnlyHint: false`). `actions` reaches everything `full` does plus the actions that change another system, such as TFPS ban and unban, where the server enabled them with [`--allow-action`](#security); a `full` token never reaches an action. A ban belongs to the caller that placed it, so an `actions` token lifts only its own bans; `admin` reaches everything `actions` does and also lifts a ban another caller placed. `audio` reaches the REST API's live call audio routes (`GET /v1/calls/live`, `GET /v1/calls/live/events` and `GET /v1/calls/{call_id}/audio`) and nothing else, and no other scope reaches them. A cross-surface mint fails at mint time, so sipnab refuses `metrics` and `audio` with MCP and `read` with the REST API rather than issuing them and then rejecting them. | +| `--token-scope` | `` | `full` | 0.5.59 | Scope for `--mint-token`. `metrics` reaches `GET /metrics` and returns `401` everywhere else — mint one for a scrape job rather than a credential that also reads `/v1/dialogs` and the message bodies underneath. `read` is the MCP counterpart: it reaches the tools annotated read-only and refuses every tool that writes or replaces state (`export_capture`, `open_capture`, `shutdown_server`, `tfps_ban` and the rest; `tools/list` marks each one with `readOnlyHint: false`). `actions` reaches everything `full` does plus the actions that change another system, such as TFPS ban and unban, where the server enabled them with [`--allow-action`](#security); a `full` token never reaches an action. A ban belongs to the caller that placed it, so an `actions` token lifts only its own bans; `admin` reaches everything `actions` does and also lifts a ban another caller placed. `audio` reaches the REST API's live call audio routes (`GET /v1/calls/live`, `GET /v1/calls/live/events` and `GET /v1/calls/{call_id}/audio`) and nothing else, and no other scope reaches them. `inventory` reaches the REST API's `GET /v1/inventory` and nothing else, for [`sipnab fleet inventory`](@/docs/fleet.md). A cross-surface mint fails at mint time, so sipnab refuses `metrics`, `audio` and `inventory` with MCP and `read` with the REST API rather than issuing them and then rejecting them. | **Examples** - `sipnab -N -I capture.pcap --no-hep-parse --report` — read HEP datagrams as plain UDP when your config file sets `[capture] hep_parse = true` @@ -3065,6 +3065,16 @@ each recording the container names by `url`, and captures nothing. - `sipnab --vcon-audio-fetch fetched/0158a120-0392-891a-9455-bc2e6fad94df.vcon.json --vcon-audio-fetch-out audio --vcon-key recipient.key --vcon-audio-fetch-ca media-ca.pem` — decrypt the audio of an encrypted vCon from storage whose certificate only that CA issued - `sipnab --vcon-audio-fetch fetched/0158a120-0392-891a-9455-bc2e6fad94df.vcon.json --vcon-audio-fetch-out audio --vcon-key recipient.key --vcon-audio-fetch-extra-ca media-ca.pem --vcon-audio-fetch-max-size 1073741824` — the same, trusting the host's bundle too, for a recording up to 1 GiB +## Fleet inventory (`sipnab fleet`) + +`sipnab fleet inventory` asks every sipnab instance in a hosts file for its inventory (`GET /v1/inventory`) and prints one row per instance and the findings across them. It opens no capture. [Manage many sipnab instances](@/docs/fleet.md) is the full how-to. + +- `--hosts `: the hosts file, one `[[host]]` table per instance with `name`, `url`, `token_file` and an optional `ca_file`. +- `--json`: print the report as JSON instead of a table. +- `--timeout `: seconds to wait for each instance to connect and answer, 1 to 300. Default `10`. + +It exits `0` when every instance answered and there is no finding, `1` when an instance did not answer or there is a finding, and `2` when it cannot use the hosts file. + ## G.729 tables (`sipnab g729`) `sipnab g729` installs and checks the G.729 codebook tables that `--g729-tables` reads. A codebook table is the set of numeric values the G.729 decoder needs. sipnab does not ship them. [G.729 audio](@/docs/g729.md) is the full how-to. diff --git a/website/content/docs/errors.md b/website/content/docs/errors.md index 5579191a7..8d391e666 100644 --- a/website/content/docs/errors.md +++ b/website/content/docs/errors.md @@ -4916,6 +4916,55 @@ install it with `--from`. **Exit code:** 1. +## Fleet inventory (`sipnab fleet`) + +`sipnab fleet inventory` prints these messages. +[Manage many sipnab instances](@/docs/fleet.md) describes the command. A problem with +the hosts file stops the command with exit code 2. The command prints its +name, the hosts file's path and then the message. A problem with one instance goes in that +instance's row as `unreachable`, and the command still asks every other +instance. + +### `host '{}' has an empty url` + +The `[[host]]` table named in the message has `url = ""`. + +**What to do:** Give the instance's REST API URL, such as +`url = "https://192.0.2.11:8443"`. + +**Exit code:** 2. + +### `host '{}' is named twice` + +Two `[[host]]` tables have the same `name`. The report shows one row per name, +so a repeated name would hide an instance. + +**What to do:** Give each instance its own `name`. + +**Exit code:** 2. + +### `token_file '{}' does not hold one token on one line` + +The instance's token file is empty, or holds whitespace inside the token, such +as two tokens or a token split over lines. The row shows the message, and the +file's path in place of the braces. The token itself is never printed. + +**What to do:** Put the one token that `sipnab --mint-token --token-scope +inventory` printed on that instance in the file, and nothing else. + +**Exit code:** 1. The other instances are still in the report. + +### `the answer is larger than {MAX_ANSWER_BYTES} bytes` + +The instance's answer to `GET /v1/inventory` is longer than +`{MAX_ANSWER_BYTES}`, 4194304 bytes (4 MiB). sipnab stops reading it. An +inventories in the [fleet how-to](@/docs/fleet.md)'s run measured under 2 KB each, +so the URL most likely names something that is not a sipnab REST API. + +**What to do:** Check the instance's `url` in the hosts file. + +**Exit code:** 1. The other instances are still in the report. + ## Startup, privileges and the process ### `--allow-coredump: a core dump would contain the archive password this run holds` diff --git a/website/content/docs/fleet.md b/website/content/docs/fleet.md new file mode 100644 index 000000000..c71570c05 --- /dev/null +++ b/website/content/docs/fleet.md @@ -0,0 +1,336 @@ ++++ +title = "Manage many sipnab instances" +weight = 66 +description = "One table of every sipnab instance's build, binary and configuration digests, privilege and uptime, the instances that did not answer, and HEP senders nobody listed." + +[extra] +source = "docs/fleet.md" ++++ + + +`sipnab fleet inventory` asks every sipnab instance you list for its +inventory, and prints one table of the fleet. Use it to answer "which build +and which configuration does each instance run, and which ones are down?" +before and after an upgrade or a configuration change. + +The table shows, for each instance: + +- the build it runs, and the SHA-256 of its binary, +- the digest of its effective configuration: the file it loaded and its + command line, +- the user it runs as, and how long it has been up. + +Below the table it names the instances that did not answer, the instances on +a different build from the rest, and the HEP senders a collector receives from +that are not in your list. It reports. It changes nothing on any instance. + +Some upgrades need every sender and its collector on the same build, such as +the HEP HMAC version 2 change in +[Upgrade a fleet that uses HMAC authentication](@/docs/config.md#upgrade-a-fleet-that-uses-hmac-authentication). +The report shows which instances are still on the old build. + +The commands and output on this page come from a run on 2026-11-03: four +instances in containers on the network `192.0.2.0/24`, and the controller on +the host. The instances ran a development build of 0.5.232 with this feature, +and one ran release 0.5.130. + +## How it works + +- Each instance serves `GET /v1/inventory` on its REST API (`--api`). See + [GET /v1/inventory](@/docs/api.md#get-v1-inventory) for every field. +- `sipnab fleet inventory` runs on one machine, the controller. It reads a + hosts file and asks every instance in it at once. The instances never + connect to the controller. +- Each request carries a bearer token with scope `inventory`. That token reads + the inventory and nothing else: no dialogs, no messages, no audio. +- The answer holds no configuration value and no command-line argument, only + their SHA-256 digests. +- The command reads at most 16 KiB of each answer's status line and headers + (`MAX_HEAD_BYTES`) and at most 4 MiB of its body (`MAX_ANSWER_BYTES`). The + inventories in this page's run were under 2 KB each. + +## Before you start + +- **sipnab on each instance.** A build whose `sipnab --version` lists the + `api` feature, and that serves `GET /v1/inventory`. An older build answers + `404` there, and the report marks it `refused`. +- **sipnab on the controller.** Any build with the `api` feature. It needs no + capture privilege: it opens no capture. +- **TLS for each instance's API.** A certificate whose subject alternative + name is the address or name in the instance's URL, its key readable by the + sipnab user only, and the certificate of the issuing CA on the controller. + Plain `http://` works too, and sends the token in clear text. +- **A signing key on each instance**, in a file only the sipnab user can read. + [Set up authentication](@/docs/auth.md) shows how to create one. +- **Network.** The controller reaches each instance's API port over TCP, 8443 + in this page's examples. Open that port to the controller's address only. + The instances need no route to the controller. +- **Other systems.** No change. OpenSIPS, Kamailio, Homer and a HEP collector + keep their configuration. A collector that runs sipnab joins the list like + any other instance. + +## Step 1: run each instance with the REST API over TLS + +- **[each instance]** Add `--api`, the TLS certificate and key, the signing key + and a node name to the instance's command line. This is the instance + `edge-a` from the run, which reads a capture file and sends HEP to the + collector `hub`: + + ```bash + sipnab -N -I /var/lib/sipnab/sample.pcap --config /etc/sipnab/edge-a.toml \ + --hep-send 192.0.2.10:9060 --hep-auth-mode hmac --hep-auth-file /etc/sipnab/hep.key \ + --api 192.0.2.11:8443 --api-tls-cert /etc/sipnab/edge-a.crt --api-tls-key /etc/sipnab/edge-a.key \ + --api-signing-key-file /etc/sipnab/edge-a.signing --node-name edge-a + ``` + + The collector ran with the same API flags and its HEP listener: + + ```bash + sipnab -N --config /etc/sipnab/hub.toml \ + --hep-listen 192.0.2.10:9060 --hep-auth-mode hmac --hep-auth-file /etc/sipnab/hep.key \ + --api 192.0.2.10:8443 --api-tls-cert /etc/sipnab/hub.crt --api-tls-key /etc/sipnab/hub.key \ + --api-signing-key-file /etc/sipnab/hub.signing --node-name hub + ``` + + Each instance logs `REST API listening on
` when its API is up. + +## Step 2: mint an inventory token on each instance + +- **[each instance]** Mint a token with that instance's signing key: + + ```bash + sipnab --mint-token --token-scope inventory --token-id fleet-controller \ + --api-signing-key-file /etc/sipnab/edge-a.signing --api-token-ttl 2592000 + ``` + + It prints one token, starting `s2.`, and exits. `--api-token-ttl 2592000` + makes it valid for 30 days. + +- **[controller]** Save each token in its own file, readable by the + controller's user only: + + ```bash + install -m 600 /dev/null /etc/sipnab/fleet/edge-a.token + ``` + + Then paste the token into it. sipnab refuses a token file that other users + can read. + +## Step 3: write the hosts file + +- **[controller]** Write `/etc/sipnab/fleet/fleet.toml`, one `[[host]]` table + per instance. This is the file from the run: + + ```toml + [[host]] + name = "hub" + url = "https://192.0.2.10:8443" + token_file = "/etc/sipnab/fleet/hub.token" + ca_file = "/etc/sipnab/fleet/internal-ca.pem" + + [[host]] + name = "edge-a" + url = "https://192.0.2.11:8443" + token_file = "/etc/sipnab/fleet/edge-a.token" + ca_file = "/etc/sipnab/fleet/internal-ca.pem" + + [[host]] + name = "edge-b" + url = "https://192.0.2.12:8443" + token_file = "/etc/sipnab/fleet/edge-b.token" + ca_file = "/etc/sipnab/fleet/internal-ca.pem" + ``` + + - `name` is the name the report shows. Each name appears once. + - `url` is the instance's REST API, `https://` or `http://`. + - `token_file` holds the instance's `inventory` token on one line. + - `ca_file` is optional: a PEM CA file to trust instead of the host's CA + bundle. + + sipnab refuses an unknown key, an empty name, an empty URL and a name used + twice, with exit `2`. + +## Step 4: read the fleet + +- **[controller]** Run: + + ```bash + sipnab fleet inventory --hosts /etc/sipnab/fleet/fleet.toml + ``` + + Output from the run: + + ```text + NAME STATE VERSION COMMIT BINARY CONFIG UID UPTIME_S + hub ok 0.5.232 29204db7 6bba53867ecd ec09522b49b4 1000 54 + edge-a ok 0.5.232 29204db7 6bba53867ecd 6a434df0c5e2 1000 40 + edge-b ok 0.5.232 29204db7 6bba53867ecd f63dfdce2c9d 1000 39 + ``` + + It exited `0`. + + - `BINARY` and `CONFIG` are the first 12 hex digits of `binary.sha256` and + `config.effective_sha256`. The three instances run one binary and three + configurations: each has its own config file and node name. + - `--json` prints the whole report, with every instance's full inventory. + - `--timeout ` sets how long each instance has to answer. Default `10`. + +## Verify + +- **Every instance is `ok`, and there is no `finding` line.** That is exit + `0`. +- **Two instances you expect to match show one `CONFIG`.** Two instances with + the same `CONFIG` loaded the same configuration values and run the same + command line. A different `--node-name` alone gives a different `CONFIG`. +- **The exit status** says the same for a script: + + | Code | Meaning | + |---|---| + | `0` | Every instance answered and there is no finding. | + | `1` | An instance did not answer, refused the token, or sent an inventory this build cannot read; or there is a finding. | + | `2` | sipnab cannot read or use the hosts file. | + +## Troubleshoot + +Each output below comes from the same run, with one thing changed. + +- **`unreachable` and `connection timed out`.** Nothing answered at that + address within `--timeout`: the instance is down, the URL names the wrong + address or port, or a firewall drops the controller's packets. The other + instances are still listed: + + ```text + edge-c unreachable 192.0.2.14:8443: connection timed out + ``` + + `Connection refused` instead means the host answered and nothing listens on + that port: check `--api` on the instance. + +- **`unreachable` and `invalid peer certificate: UnknownIssuer`.** The + controller does not trust the CA that issued the instance's certificate. Set + `ca_file` for that host: + + ```text + hub unreachable sending: invalid peer certificate: UnknownIssuer + ``` + +- **`refused` with HTTP `401`.** The instance did not accept the token: it was + minted with another instance's signing key, it expired, someone revoked it, or + its scope is not `inventory`, `full`, `actions` or `admin`. Mint a new one + on that instance (step 2). This run gave `edge-b` the hub's token: + + ```text + edge-b refused the instance answered HTTP 401 + ``` + +- **`refused` with HTTP `404`.** The instance runs a build without + `GET /v1/inventory`. Upgrade it. Release 0.5.130 answered: + + ```text + edge-old refused the instance answered HTTP 404 + ``` + + Release 0.5.130 also refuses `--token-scope inventory` at mint time, with + `invalid value 'inventory' for '--token-scope '`. + +- **`unreachable` and `can be read or written by other users`.** The token + file's mode lets other users read it. Run `chmod 600` on it: + + ```text + edge-a unreachable token_file '/etc/sipnab/fleet/open.token' can be read or written by other users (mode 0644); chmod 600 it + ``` + +- **A mixed HEP HMAC version.** A sender on 0.5.130 or earlier signs HEP with + HMAC version 1, and a collector on 0.5.131 or later drops every packet it + sends. In the run, `edge-old` (0.5.130) sent to `hub`. Listed in the hosts + file, `edge-old`'s row is `refused` with HTTP `404`, as above. The collector + logs `HEP HMAC auth is REFUSING version-1 tokens` once, and counts the + dropped packets. Read the count from the JSON report: + + ```bash + sipnab fleet inventory --hosts /etc/sipnab/fleet/fleet.toml --json \ + | jq -c '.hosts[] | {name, refused: .inventory.hep_listener.packets_refused, senders: [.inventory.hep_listener.senders[].peer]}' + ``` + + ```text + {"name":"hub","refused":10,"senders":["192.0.2.11","192.0.2.12"]} + {"name":"edge-a","refused":0,"senders":[]} + {"name":"edge-b","refused":0,"senders":[]} + ``` + + The hub refused 10 packets, and `192.0.2.13` is not among its senders. The + report itself is `ok` when the old sender is not in the hosts file: a + refused packet is not a finding. Upgrade the sender, as + [Upgrade a fleet that uses HMAC authentication](@/docs/config.md#upgrade-a-fleet-that-uses-hmac-authentication) + describes. + +- **`finding unlisted_hep_sender`.** A collector in the list receives HEP + from an address that no listed instance's URL resolves to. Add that sender + to the hosts file, or find out why it sends. With `edge-b` left out of the + file: + + ```text + finding unlisted_hep_sender: hub receives HEP from 192.0.2.12, which is no instance in the hosts file + ``` + + The check compares addresses only. A sender whose packets leave from a + different address than its API URL names shows up as unlisted. + +- **`incompatible` and a `schema_version`.** The instance answered an + inventory format newer than the controller reads. Upgrade the controller's + sipnab. No release answers a newer format yet, so this output comes from a + test server that answered `schema_version` 2: + + ```text + edge-next incompatible the instance answered inventory schema_version 2; this sipnab reads up to 1 + ``` + +## States and findings + +**State of each instance:** + +| State | Meaning | +|---|---| +| `ok` | It answered with an inventory. | +| `refused` | It answered with an HTTP status other than 200, such as `401` for a wrong, expired or narrower token, or `404` for a build without the inventory. | +| `unreachable` | No answer: no connection, a timeout, a TLS failure, a token file sipnab would not read, or an answer that is not JSON. | +| `incompatible` | Its inventory has a `schema_version` newer than this build reads, or none. Upgrade the controller's sipnab. | + +**Findings across the fleet:** + +| Finding | Meaning | +|---|---| +| `version_mismatch` | The instances that answered do not all run the same build. The finding names each build and the instances on it. | +| `unlisted_hep_sender` | A HEP collector in the list receives from an address that no listed instance's URL resolves to. | + +## Check a binary against its release + +Every instance reports `signature.state` as `unverified`. A process cannot +prove which binary it is, and one that someone had tampered with would report +"verified" too. + +- Download `SHA256SUMS.txt` from the release the instance claims, and verify + it: + + ```bash + gh attestation verify SHA256SUMS.txt --repo NormB/sipnab + ``` + +- Compare the instance's `binary.sha256` with the line for its artifact. On + Linux the instance hashes `/proc/self/exe`, so after an upgrade replaces the + file the digest still names the binary that is running. + +## What it cannot tell you + +- Every field is the instance's own claim. An attacker who controls a host can make + it report anything. +- For a secret held in a file, the digests cover its path only. The configuration and + the command line name key, token and password files by path, so replacing + the contents of such a file, for example rotating the key in + `--api-signing-key-file`, changes no digest. A secret given as a value, such + as `--api-signing-key` or a `${NAME}` the configuration expands, changes + `config.effective_sha256`. +- It reads only the instances in your hosts file. An instance nobody listed is + invisible to it, unless it sends HEP to a listed collector. +- It changes nothing. Pushing a configuration, upgrading and restarting are not + part of it. diff --git a/website/content/docs/mcp-estate.md b/website/content/docs/mcp-estate.md index 2c2dcc389..a4390aebf 100644 --- a/website/content/docs/mcp-estate.md +++ b/website/content/docs/mcp-estate.md @@ -16,9 +16,11 @@ is what comes after that: - an agent that has to reach it from somewhere else, - one agent holding several capture hosts at once, - the hardest of them: following a single call as it crosses an SBC, a proxy - and a PBX that each see a different Call-ID. + and a PBX that each see a different Call-ID, +- managing the sipnab instances themselves: which build and configuration + each one runs, and which ones do not answer. -These four build on each other in that order. Each one names the SINGLE +The first four build on each other in that order. Each one names the SINGLE sipnab it starts from. So you can stop at whichever answers your question. Every scenario assumes you have already done [Install sipnab](@/docs/mcp-deploy.md#install-sipnab-every-server-once). @@ -718,3 +720,31 @@ To centralize, see sipnab can run beside a Homer installation, as [Coexisting with Homer](#coexisting-with-homer-or-any-observability-host) describes. + +--- + +## Manage the fleet + +Federation and the central collector both answer questions about calls. +Neither says which build each node runs or which configuration it loaded. +`sipnab fleet inventory` does. [Manage many sipnab instances](@/docs/fleet.md) is the +full how-to. + +- **[each node]** Run the REST API (`--api`) and mint a token with + `--token-scope inventory`. That token reads `GET /v1/inventory` and nothing + else: no dialogs, no messages, no audio. +- **[controller]** List the nodes in a hosts file and run + `sipnab fleet inventory --hosts fleet.toml`. It prints one row per node: the + build, the SHA-256 of the binary, the effective configuration digest, the + user id and the uptime. It marks a node that did not answer and exits `1`. +- **[controller]** Include the central collector in the hosts file. The + report then names every HEP sender the collector receives from whose + address is not in the hosts file, as the finding `unlisted_hep_sender`. +- **[agent]** An agent connected to one node asks the MCP tool + `fleet_inventory`. It returns the same record as `GET /v1/inventory` on + that node. + +Every field is the node's own claim. `signature.state` is always `unverified`: +compare `binary.sha256` with the release's attested `SHA256SUMS.txt`, as +[Check a binary against its release](@/docs/fleet.md#check-a-binary-against-its-release) +shows. diff --git a/website/content/docs/mcp-protocol.md b/website/content/docs/mcp-protocol.md index 840c08f11..e0364ba6a 100644 --- a/website/content/docs/mcp-protocol.md +++ b/website/content/docs/mcp-protocol.md @@ -171,8 +171,8 @@ lines on stdin, the way a client does. - **No prompt-injection cooperation.** Tool descriptions never instruct the LLM to "trust" or "act on" returned content. They describe what the tool returns and stop there. -- **Every tool declares what it does.** All 76 carry MCP annotations, so a host - can decide what to call without asking. Sixty-three are `readOnlyHint: true`. +- **Every tool declares what it does.** All 77 carry MCP annotations, so a host + can decide what to call without asking. Sixty-four are `readOnlyHint: true`. [What the write verbs do](#what-the-write-verbs-do) names the thirteen that are not. @@ -626,7 +626,7 @@ sipnab negotiates the 2025-06-18 and 2025-11-25 revisions. There, ## What the write verbs do -Sixty-three of the 76 tools are `readOnlyHint: true`. These thirteen are not. +Sixty-four of the 77 tools are `readOnlyHint: true`. These thirteen are not. Each declares what kind of change it makes, so a host can decide which need confirmation: diff --git a/website/content/docs/mcp-tools-server.md b/website/content/docs/mcp-tools-server.md index 112e9f32d..bcc5c2514 100644 --- a/website/content/docs/mcp-tools-server.md +++ b/website/content/docs/mcp-tools-server.md @@ -24,6 +24,7 @@ the other bundles and finds any tool by name. |---|---|---| | [`runtime_stats`](#runtime-stats) | `sample_seconds?` | What sipnab is doing and what it costs the host: its own memory, threads, descriptors and CPU; the host's totals and the basis they came from; its share of them and whether that share is load-bearing; per-interface counters; store occupancy against the caps | | [`capture_health`](#capture-health) | `sample_seconds` | Capture-path counters read twice: run totals, deltas across the window, `undecoded_fraction`, and undecodable frames by reason | +| [`fleet_inventory`](#fleet-inventory) | -- | What this process is, for a fleet controller: build, binary SHA-256, signature state, configuration digests, privilege, uptime, interface, capture and HEP counters, and HEP sender addresses | | [`hep_senders`](#hep-senders) | `limit?` | Who feeds this run's HEP listener: each sender, its count, when it was last heard and whether it went silent, and every address the listener refused, by reason | | [`server_capabilities`](#server-capabilities) | -- | sipnab version, the optional features this binary carries, and the libpcap it runs | | [`shutdown_server`](#shutdown-server) | `dry_run?`, `save_to?`, `discard_unsaved?` | **Destructive.** Stops the process. Needs `--mcp-allow-shutdown`; dry-run by default | @@ -441,6 +442,69 @@ roster. The MCP answer differs in two ways: its keys are in alphabetical order, and it adds `source_exhausted` and `source_stopped_early`. `--hep-senders` prints the roster at the end of a headless run. +### `fleet_inventory` + +What this sipnab process is, for a fleet controller that manages many +instances: its build, the SHA-256 of its binary, its configuration as digests, +its privilege, its uptime and its counters. Starts nothing and waits for +nothing. + +**Parameters:** none. + +The example ran on 2026-11-03 with a development build, against +[`tests/pcap-samples/sip-rtp-g711.pcap`](https://github.com/NormB/sipnab/raw/main/tests/pcap-samples/sip-rtp-g711.pcap) +with `--node-name edge-a --no-config`: + +```jsonc +// fleet_inventory {} +{ + "schema_version": 1, + "node_name": "edge-a", + "build": { + "version": "0.5.232", "commit": "29204db7", "tag": "", "dirty": true, + "target": "aarch64-unknown-linux-gnu", + "features": ["api", "archive", "audio", "g729", "hep", "mcp", "mcp-http", "metrics", + "native", "opus", "plugins", "tls", "tui", "vcon"], + "version_line": "0.5.232 (29204db7-dirty) features: native,tui,audio,tls,hep,api,mcp,mcp-http,metrics,plugins,vcon,opus,archive,g729" + }, + "binary": { + "sha256": "3242a3324b1c3ac762ba15d4f2eec3789a995533540b8aee56ec087afdc51ea8", + "error": null + }, + "signature": { + "state": "unverified", + "note": "sipnab does not verify its own binary. Compare binary.sha256 with SHA256SUMS.txt of the release, after `gh attestation verify SHA256SUMS.txt --repo NormB/sipnab`." + }, + "config": { + "recorded": true, "source": null, "shadowed": [], "file_sha256": null, + "parsed_sha256": "afa5e308cbb51683fe7ac0ca79e2a33f601530f33439211dd29586c9db0c7d75", + "argv_sha256": "cb86e74e2f7d2f54c2e75c3cb01976de06c9658406112c318c5f290cf606418d", + "effective_sha256": "aee2e58c08b314be6538cf9ccf1dfddb992992c4e7336a4d256373b67be72130" + }, + "privilege": { "uid": 1000, "root": false, "dropped_to": null }, + "process": { "pid": 3985285, "started": "2026-11-03T13:26:30Z", "uptime_seconds": 1 }, + "interfaces": [], + "counters": { + "capture_packets_total": 852, "capture_sources_refused_total": 0, + "capture_backpressure_blocks_total": 0, "hep_export_failures_total": null, + "hep_export_dropped_total": null, "hep_export_queue_depth": null + }, + "hep_listener": { "listening": false, "packets_refused": 0, "senders": [] } +} +``` + +- No configuration value and no command-line argument is in the answer, only + their SHA-256 digests. A secret in either changes a digest and never appears. +- `signature.state` is always `unverified`. Compare `binary.sha256` with the + release's attested `SHA256SUMS.txt`. +- Every field is the process's own claim. + +`GET /v1/inventory` returns the same document, from the same builder. Unlike +most tools, the answer carries no `source_exhausted` or `source_stopped_early`: +it describes the process, not a result read from the capture. A `read` token +reaches this tool. [Manage many +sipnab instances](@/docs/fleet.md) reads the REST route from every instance. + ### `server_capabilities` What this binary can do, what this server permits and loaded, the limits it @@ -492,7 +556,7 @@ opt-in flags: "evaluate_expectations", "explain_attribution", "explain_response_code", "explain_rule", "export_audio", "export_capture", "export_vcon", "fetch_vcon", "find_correlated", "find_in_captures", "find_in_captures_status", "find_problems", - "generate_fail2ban_rule", "generate_repro", "generate_wireshark_filter", + "fleet_inventory", "generate_fail2ban_rule", "generate_repro", "generate_wireshark_filter", "get_call_tree", "get_capture_report", "get_dialog", "get_dialog_report", "get_message", "get_sdp_timeline", "group_dialogs", "hep_senders", "lint_dialog", "list_captures", "list_dialogs", "list_tls_libraries", "media_diagnostics", diff --git a/website/content/docs/mcp-tools.md b/website/content/docs/mcp-tools.md index 83e213b95..9859662a1 100644 --- a/website/content/docs/mcp-tools.md +++ b/website/content/docs/mcp-tools.md @@ -60,7 +60,7 @@ A bundle is a named set of tools defined in sipnab's code. | [`media`](@/docs/mcp-tools-media.md) | 4 | Where a call's media endpoints came from, why a stream has no dialog, the facts under a quality score, and audio export. | | [`relay`](@/docs/mcp-tools-relay.md) | 4 | Questions put to the configured media relay, and decoding the relay control messages sipnab captured. | | [`tfps`](@/docs/mcp-tools-tfps.md) | 7 | The toll-fraud prevention peer (TFPS): what it blocks and logs, and the operator's ban, unban and revert actions. | -| [`server`](@/docs/mcp-tools-server.md) | 5 | The sipnab process and its capture path: capture health, HEP senders, resource use, build capabilities, and shutdown. | +| [`server`](@/docs/mcp-tools-server.md) | 6 | The sipnab process and its capture path: capture health, HEP senders, resource use, build capabilities, the fleet inventory record, and shutdown. | | [`vcon`](@/docs/mcp-tools-vcon.md) | 7 | vCon conversation containers, exported, validated and analyzed, one at a time or a directory at once, read from the operator's store, the vCon export spool's counts, and the SIPREC recording metadata a call carried. | | [`tls`](@/docs/mcp-tools-tls.md) | 3 | Reading SIP inside TLS without keys through kernel uprobes, and which TLS libraries this host runs. | @@ -134,22 +134,22 @@ resource. | Bundle | Tools | Bytes (default) | Bytes with output schemas | |---|---|---|---| -| core | 9 | 12,316 | 54,624 | -| signaling | 20 | 22,474 | 58,619 | -| captures | 11 | 12,723 | 30,211 | -| security | 6 | 9,954 | 9,954 | -| media | 4 | 3,480 | 16,048 | -| relay | 4 | 4,199 | 19,988 | -| tfps | 7 | 5,563 | 15,947 | -| server | 5 | 5,394 | 27,452 | -| vcon | 7 | 10,714 | 33,024 | -| tls | 3 | 2,850 | 4,943 | -| full | 76 | 89,667 | 270,810 | +| core | 9 | 12,308 | 58,240 | +| signaling | 20 | 22,452 | 66,162 | +| captures | 11 | 12,709 | 30,189 | +| security | 6 | 9,944 | 9,944 | +| media | 4 | 3,478 | 16,042 | +| relay | 4 | 4,199 | 20,607 | +| tfps | 7 | 5,717 | 16,101 | +| server | 6 | 5,972 | 35,126 | +| vcon | 7 | 10,710 | 34,428 | +| tls | 3 | 2,899 | 4,990 | +| full | 77 | 90,388 | 291,829 | Bytes are the compact JSON of each tool's `tools/list` entry, summed over the bundle. -> **Measured on sipnab 0.5.222** on a build with every feature. +> **Measured on sipnab 0.5.232** (a development build of this change, 2026-11-03) on a build with every feature. > > - Sizes, tool counts and bundle membership may change in later releases. > - A build without a feature does not register that feature's tools, so its @@ -189,6 +189,7 @@ links to its section on the bundle's page. | [`find_in_captures`](@/docs/mcp-tools-captures.md#find-in-captures) | [captures](@/docs/mcp-tools-captures.md) | Which capture files hold dialogs matching a filter, without opening any of them; a sweep that outlasts its wait returns a job | | [`find_in_captures_status`](@/docs/mcp-tools-captures.md#find-in-captures-status) | [captures](@/docs/mcp-tools-captures.md) | A `find_in_captures` job's progress, or its result once it finishes | | [`find_problems`](@/docs/mcp-tools-core.md#find-problems) | [core](@/docs/mcp-tools-core.md) | A page of dialogs matching one or more diagnostic alias names | +| [`fleet_inventory`](@/docs/mcp-tools-server.md#fleet-inventory) | [server](@/docs/mcp-tools-server.md) | What this process is, for a fleet controller: build, binary SHA-256, signature state, configuration digests, privilege, uptime and counters | | [`generate_fail2ban_rule`](@/docs/mcp-tools-security.md#generate-fail2ban-rule) | [security](@/docs/mcp-tools-security.md) | A fail2ban filter and jail derived from ONE recorded finding, with that finding attached as the evidence | | [`generate_repro`](@/docs/mcp-tools-signaling.md#generate-repro) | [signaling](@/docs/mcp-tools-signaling.md) | A SIPp scenario replaying one call, with the hypothesis as an input: `pin` holds the suspected cause fixed, `vary` regenerates identity | | [`generate_wireshark_filter`](@/docs/mcp-tools-signaling.md#generate-wireshark-filter) | [signaling](@/docs/mcp-tools-signaling.md) | A Wireshark display filter selecting one call's signaling and its RTP by SSRC, plus the tshark line that applies it | @@ -460,13 +461,15 @@ When `source_exhausted` is `false`, `total_matched` counts only what had loaded. Repeat the call, or poll `capture_status` until `source_exhausted` reads `true`, before treating a count as final. -Twenty-one tools are the exception, because their answer cannot move with +Twenty-two tools are the exception, because their answer cannot move with the load. How much of the capture sipnab has read says nothing about what their sources hold: -- Twelve read something other than the capture store: +- Thirteen read something other than the capture store: - `explain_response_code`, `explain_rule`, `decode_evidence`, `decode_ng`, `show_evidence`, `list_captures`, `list_tls_libraries`, `server_capabilities`; + - `fleet_inventory`, which describes the process and returns the same + document as the REST inventory route; - `vcon_spool_status`, which counts the vCon export spool that the forwarder drains; - `compare_captures`, which reads two files and never the loaded capture; diff --git a/website/content/docs/mcp.md b/website/content/docs/mcp.md index 67a7169d5..7640bd6f0 100644 --- a/website/content/docs/mcp.md +++ b/website/content/docs/mcp.md @@ -314,7 +314,7 @@ it does these things: It does for this surface what does for the [REST API](@/docs/api.md). -sipnab registers 76 MCP tools. That is more than anyone reads in a table. The +sipnab registers 77 MCP tools. That is more than anyone reads in a table. The Tools tab is the fastest way to find the one you want. The MCP project maintains Inspector, not sipnab. When a tool diff --git a/website/static/llms-full.txt b/website/static/llms-full.txt index 66dfaae3e..ea97709b3 100644 --- a/website/static/llms-full.txt +++ b/website/static/llms-full.txt @@ -6870,7 +6870,7 @@ only. | `--api-token-ttl ` | Lifetime of a minted token. Default `3600`. | | `--mint-token` | Sign a token with the first configured key, print it, and exit. Starts no capture and no server. | | `--token-id ` | The token's `id` (`jti`), used later for revocation. Defaults to a generated id. | -| `--token-scope ` | `full` (default), `metrics`, `actions`, `admin` or `audio` for the REST API. See **Scope** below. | +| `--token-scope ` | `full` (default), `metrics`, `actions`, `admin`, `audio` or `inventory` for the REST API. See **Scope** below. | | `--api-revoked-file ` | Denylist of revoked token ids, one per line (blanks and `#` comments ignored). | **Scope.** Mint each client a token for the routes it needs: @@ -6882,6 +6882,7 @@ only. | `actions` | Everything `full` reaches, plus the action routes that ask TFPS to ban or unban a source. The run must also enable them with `--allow-action tfps:rest`. It lifts only the bans it placed. | The system that decides to block a source. | | `admin` | Everything `actions` reaches, and it also lifts a ban another caller placed. | An operator who must be able to lift any ban. | | `audio` | The live-audio routes a `live_audio` vCon rule serves, and nothing else. | A speech-to-text consumer. | +| `inventory` | `GET /v1/inventory` only. Every other route returns `401`. `full` reaches the inventory too. | A fleet controller running [`sipnab fleet inventory`](fleet.md). | The `read` scope is for MCP tokens. Minting it with `--api-signing-key` refuses. @@ -9876,6 +9877,103 @@ The MCP tool `server_capabilities` returns the same feature set and the same `libpcap` block. The second line of `sipnab --version` prints the same library banner. +### GET /v1/inventory + +What this sipnab process is, for managing many instances as one fleet: + +- its build: version, git commit and tag, target and compiled features, +- SHA-256 of the binary it is running, +- its configuration, as digests only, +- its user id and privilege drop, its pid, start time and uptime, +- its capture interfaces' error and drop counters, its capture and HEP export + counters, and the addresses its HEP listener receives from. + +It needs a token with scope `inventory`, or `full`. [`sipnab fleet +inventory`](fleet.md) reads it from every instance in a hosts file. + +**curl:** + +```bash +curl -s -H "Authorization: Bearer $SIPNAB_INVENTORY_TOKEN" http://127.0.0.1:8080/v1/inventory | jq . +``` + +**Response** (from a run on 2026-11-03 of a development build, with the config path replaced by `/etc/sipnab/sipnab.toml`): + +```json +{ + "schema_version": 1, + "node_name": "edge-a", + "build": { + "version": "0.5.232", + "commit": "29204db7", + "tag": "", + "dirty": true, + "target": "aarch64-unknown-linux-gnu", + "features": ["api", "archive", "audio", "g729", "hep", "mcp", "mcp-http", "metrics", + "native", "opus", "plugins", "tls", "tui", "vcon"], + "version_line": "0.5.232 (29204db7-dirty) features: native,tui,audio,tls,hep,api,mcp,mcp-http,metrics,plugins,vcon,opus,archive,g729" + }, + "binary": { + "sha256": "3242a3324b1c3ac762ba15d4f2eec3789a995533540b8aee56ec087afdc51ea8", + "error": null + }, + "signature": { + "state": "unverified", + "note": "sipnab does not verify its own binary. Compare binary.sha256 with SHA256SUMS.txt of the release, after `gh attestation verify SHA256SUMS.txt --repo NormB/sipnab`." + }, + "config": { + "recorded": true, + "source": "/etc/sipnab/sipnab.toml", + "shadowed": [], + "file_sha256": "703493b6b6bdef18b62090d6452a2bd9fc72de264d4b97822cb0b7763697fca6", + "parsed_sha256": "1849a6ee687ec0b1394f2d305198c638bca159e7aa032911156074b8d11f4044", + "argv_sha256": "d5c72de2bcf085f7d8ca9884bed30027d7504215d507b04f6be16ecc063194e0", + "effective_sha256": "31e67effe25101b07a2c7942f05b83d568816bf81a1769d65ab8396c8e89705a" + }, + "privilege": { "uid": 1000, "root": false, "dropped_to": null }, + "process": { "pid": 3983974, "started": "2026-11-03T13:26:05Z", "uptime_seconds": 17 }, + "interfaces": [], + "counters": { + "capture_packets_total": 852, + "capture_sources_refused_total": 0, + "capture_backpressure_blocks_total": 0, + "hep_export_failures_total": null, + "hep_export_dropped_total": null, + "hep_export_queue_depth": null + }, + "hep_listener": { "listening": false, "packets_refused": 0, "senders": [] } +} +``` + +**No configuration value is in the answer.** + +- `file_sha256` is SHA-256 of the config file's bytes as this run read them. +- `parsed_sha256` is SHA-256 of the configuration as sipnab parsed it, after + `${NAME}` expansion, with every object's keys sorted. Two files that differ + only in comments, whitespace, key order or an unknown key give one value. +- `argv_sha256` is SHA-256 of the command line, without the program path. +- `effective_sha256` is SHA-256 over `parsed_sha256` and `argv_sha256`. A flag + overrides the file, so this is the digest to compare across instances. + `sipnab fleet inventory` shows it as `CONFIG`. +- A secret in the file or on the command line changes a digest, and its value + never appears. A secret held in a file the configuration names, such as + `--api-signing-key-file`, reaches the digests by its path only. + +**`signature.state` is always `unverified`.** A process cannot prove which +binary it is. Compare `binary.sha256` with the release's `SHA256SUMS.txt`, +after verifying that file's attestation with `gh attestation verify`. +sipnab reads `binary.sha256` from `/proc/self/exe` on Linux, so after an upgrade +replaces the file it still names the binary that is running. + +**Every field is the process's own claim.** An attacker who controls a host +can report anything. + +`hep_listener.senders` lists at most `MAX_HEP_PEERS` (256) senders. + +`config.recorded` is `false`, and every digest `null`, for a server that did not +record its configuration at start. The MCP tool `fleet_inventory` returns the +same fields and values. + ### GET /v1/stats Aggregate statistics across all dialogs and streams, including PDD percentiles. @@ -11999,7 +12097,7 @@ it does these things: It does for this surface what does for the [REST API](rest-api.md). -sipnab registers 76 MCP tools. That is more than anyone reads in a table. The +sipnab registers 77 MCP tools. That is more than anyone reads in a table. The Tools tab is the fastest way to find the one you want. The MCP project maintains Inspector, not sipnab. When a tool @@ -20217,7 +20315,7 @@ This applies to root runs and to unprivileged runs set up with | `--syslog` | -- | off | 0.3.0 | Send alerts to syslog. Config: `[security] alert` | | `--mint-token` | -- | off | 0.4.3 | Mint a signed bearer token from the first configured signing key (API or MCP), print it to stdout, and exit (no capture/servers). See [`auth.md`](./auth.md). | | `--token-id` | `` | -- | 0.4.3 | Token id (`jti`) for `--mint-token`, used for revocation. Defaults to a generated id. | -| `--token-scope` | `` | `full` | 0.5.59 | Scope for `--mint-token`. `metrics` reaches `GET /metrics` and returns `401` everywhere else — mint one for a scrape job rather than a credential that also reads `/v1/dialogs` and the message bodies underneath. `read` is the MCP counterpart: it reaches the tools annotated read-only and refuses every tool that writes or replaces state (`export_capture`, `open_capture`, `shutdown_server`, `tfps_ban` and the rest; `tools/list` marks each one with `readOnlyHint: false`). `actions` reaches everything `full` does plus the actions that change another system, such as TFPS ban and unban, where the server enabled them with [`--allow-action`](#security); a `full` token never reaches an action. A ban belongs to the caller that placed it, so an `actions` token lifts only its own bans; `admin` reaches everything `actions` does and also lifts a ban another caller placed. `audio` reaches the REST API's live call audio routes (`GET /v1/calls/live`, `GET /v1/calls/live/events` and `GET /v1/calls/{call_id}/audio`) and nothing else, and no other scope reaches them. A cross-surface mint fails at mint time, so sipnab refuses `metrics` and `audio` with MCP and `read` with the REST API rather than issuing them and then rejecting them. | +| `--token-scope` | `` | `full` | 0.5.59 | Scope for `--mint-token`. `metrics` reaches `GET /metrics` and returns `401` everywhere else — mint one for a scrape job rather than a credential that also reads `/v1/dialogs` and the message bodies underneath. `read` is the MCP counterpart: it reaches the tools annotated read-only and refuses every tool that writes or replaces state (`export_capture`, `open_capture`, `shutdown_server`, `tfps_ban` and the rest; `tools/list` marks each one with `readOnlyHint: false`). `actions` reaches everything `full` does plus the actions that change another system, such as TFPS ban and unban, where the server enabled them with [`--allow-action`](#security); a `full` token never reaches an action. A ban belongs to the caller that placed it, so an `actions` token lifts only its own bans; `admin` reaches everything `actions` does and also lifts a ban another caller placed. `audio` reaches the REST API's live call audio routes (`GET /v1/calls/live`, `GET /v1/calls/live/events` and `GET /v1/calls/{call_id}/audio`) and nothing else, and no other scope reaches them. `inventory` reaches the REST API's `GET /v1/inventory` and nothing else, for [`sipnab fleet inventory`](fleet.md). A cross-surface mint fails at mint time, so sipnab refuses `metrics`, `audio` and `inventory` with MCP and `read` with the REST API rather than issuing them and then rejecting them. | **Examples** - `sipnab -N -I capture.pcap --no-hep-parse --report` — read HEP datagrams as plain UDP when your config file sets `[capture] hep_parse = true` @@ -20879,6 +20977,16 @@ each recording the container names by `url`, and captures nothing. - `sipnab --vcon-audio-fetch fetched/0158a120-0392-891a-9455-bc2e6fad94df.vcon.json --vcon-audio-fetch-out audio --vcon-key recipient.key --vcon-audio-fetch-ca media-ca.pem` — decrypt the audio of an encrypted vCon from storage whose certificate only that CA issued - `sipnab --vcon-audio-fetch fetched/0158a120-0392-891a-9455-bc2e6fad94df.vcon.json --vcon-audio-fetch-out audio --vcon-key recipient.key --vcon-audio-fetch-extra-ca media-ca.pem --vcon-audio-fetch-max-size 1073741824` — the same, trusting the host's bundle too, for a recording up to 1 GiB +## Fleet inventory (`sipnab fleet`) + +`sipnab fleet inventory` asks every sipnab instance in a hosts file for its inventory (`GET /v1/inventory`) and prints one row per instance and the findings across them. It opens no capture. [Manage many sipnab instances](fleet.md) is the full how-to. + +- `--hosts `: the hosts file, one `[[host]]` table per instance with `name`, `url`, `token_file` and an optional `ca_file`. +- `--json`: print the report as JSON instead of a table. +- `--timeout `: seconds to wait for each instance to connect and answer, 1 to 300. Default `10`. + +It exits `0` when every instance answered and there is no finding, `1` when an instance did not answer or there is a finding, and `2` when it cannot use the hosts file. + ## G.729 tables (`sipnab g729`) `sipnab g729` installs and checks the G.729 codebook tables that `--g729-tables` reads. A codebook table is the set of numeric values the G.729 decoder needs. sipnab does not ship them. [G.729 audio](g729.md) is the full how-to. @@ -25936,6 +26044,55 @@ install it with `--from`. **Exit code:** 1. +## Fleet inventory (`sipnab fleet`) + +`sipnab fleet inventory` prints these messages. +[Manage many sipnab instances](fleet.md) describes the command. A problem with +the hosts file stops the command with exit code 2. The command prints its +name, the hosts file's path and then the message. A problem with one instance goes in that +instance's row as `unreachable`, and the command still asks every other +instance. + +### `host '{}' has an empty url` + +The `[[host]]` table named in the message has `url = ""`. + +**What to do:** Give the instance's REST API URL, such as +`url = "https://192.0.2.11:8443"`. + +**Exit code:** 2. + +### `host '{}' is named twice` + +Two `[[host]]` tables have the same `name`. The report shows one row per name, +so a repeated name would hide an instance. + +**What to do:** Give each instance its own `name`. + +**Exit code:** 2. + +### `token_file '{}' does not hold one token on one line` + +The instance's token file is empty, or holds whitespace inside the token, such +as two tokens or a token split over lines. The row shows the message, and the +file's path in place of the braces. The token itself is never printed. + +**What to do:** Put the one token that `sipnab --mint-token --token-scope +inventory` printed on that instance in the file, and nothing else. + +**Exit code:** 1. The other instances are still in the report. + +### `the answer is larger than {MAX_ANSWER_BYTES} bytes` + +The instance's answer to `GET /v1/inventory` is longer than +`{MAX_ANSWER_BYTES}`, 4194304 bytes (4 MiB). sipnab stops reading it. An +inventories in the [fleet how-to](fleet.md)'s run measured under 2 KB each, +so the URL most likely names something that is not a sipnab REST API. + +**What to do:** Check the instance's `url` in the hosts file. + +**Exit code:** 1. The other instances are still in the report. + ## Startup, privileges and the process ### `--allow-coredump: a core dump would contain the archive password this run holds` @@ -34476,9 +34633,11 @@ is what comes after that: - an agent that has to reach it from somewhere else, - one agent holding several capture hosts at once, - the hardest of them: following a single call as it crosses an SBC, a proxy - and a PBX that each see a different Call-ID. + and a PBX that each see a different Call-ID, +- managing the sipnab instances themselves: which build and configuration + each one runs, and which ones do not answer. -These four build on each other in that order. Each one names the SINGLE +The first four build on each other in that order. Each one names the SINGLE sipnab it starts from. So you can stop at whichever answers your question. Every scenario assumes you have already done [Install sipnab](mcp-deploy.md#install-sipnab-every-server-once). @@ -35181,6 +35340,367 @@ describes. --- +## Manage the fleet + +Federation and the central collector both answer questions about calls. +Neither says which build each node runs or which configuration it loaded. +`sipnab fleet inventory` does. [Manage many sipnab instances](fleet.md) is the +full how-to. + +- **[each node]** Run the REST API (`--api`) and mint a token with + `--token-scope inventory`. That token reads `GET /v1/inventory` and nothing + else: no dialogs, no messages, no audio. +- **[controller]** List the nodes in a hosts file and run + `sipnab fleet inventory --hosts fleet.toml`. It prints one row per node: the + build, the SHA-256 of the binary, the effective configuration digest, the + user id and the uptime. It marks a node that did not answer and exits `1`. +- **[controller]** Include the central collector in the hosts file. The + report then names every HEP sender the collector receives from whose + address is not in the hosts file, as the finding `unlisted_hep_sender`. +- **[agent]** An agent connected to one node asks the MCP tool + `fleet_inventory`. It returns the same record as `GET /v1/inventory` on + that node. + +Every field is the node's own claim. `signature.state` is always `unverified`: +compare `binary.sha256` with the release's attested `SHA256SUMS.txt`, as +[Check a binary against its release](fleet.md#check-a-binary-against-its-release) +shows. + +--- + + + +# Manage many sipnab instances + +`sipnab fleet inventory` asks every sipnab instance you list for its +inventory, and prints one table of the fleet. Use it to answer "which build +and which configuration does each instance run, and which ones are down?" +before and after an upgrade or a configuration change. + +The table shows, for each instance: + +- the build it runs, and the SHA-256 of its binary, +- the digest of its effective configuration: the file it loaded and its + command line, +- the user it runs as, and how long it has been up. + +Below the table it names the instances that did not answer, the instances on +a different build from the rest, and the HEP senders a collector receives from +that are not in your list. It reports. It changes nothing on any instance. + +Some upgrades need every sender and its collector on the same build, such as +the HEP HMAC version 2 change in +[Upgrade a fleet that uses HMAC authentication](config-reference.md#upgrade-a-fleet-that-uses-hmac-authentication). +The report shows which instances are still on the old build. + +The commands and output on this page come from a run on 2026-11-03: four +instances in containers on the network `192.0.2.0/24`, and the controller on +the host. The instances ran a development build of 0.5.232 with this feature, +and one ran release 0.5.130. + +## How it works + +- Each instance serves `GET /v1/inventory` on its REST API (`--api`). See + [GET /v1/inventory](rest-api.md#get-v1inventory) for every field. +- `sipnab fleet inventory` runs on one machine, the controller. It reads a + hosts file and asks every instance in it at once. The instances never + connect to the controller. +- Each request carries a bearer token with scope `inventory`. That token reads + the inventory and nothing else: no dialogs, no messages, no audio. +- The answer holds no configuration value and no command-line argument, only + their SHA-256 digests. +- The command reads at most 16 KiB of each answer's status line and headers + (`MAX_HEAD_BYTES`) and at most 4 MiB of its body (`MAX_ANSWER_BYTES`). The + inventories in this page's run were under 2 KB each. + +## Before you start + +- **sipnab on each instance.** A build whose `sipnab --version` lists the + `api` feature, and that serves `GET /v1/inventory`. An older build answers + `404` there, and the report marks it `refused`. +- **sipnab on the controller.** Any build with the `api` feature. It needs no + capture privilege: it opens no capture. +- **TLS for each instance's API.** A certificate whose subject alternative + name is the address or name in the instance's URL, its key readable by the + sipnab user only, and the certificate of the issuing CA on the controller. + Plain `http://` works too, and sends the token in clear text. +- **A signing key on each instance**, in a file only the sipnab user can read. + [Set up authentication](auth.md) shows how to create one. +- **Network.** The controller reaches each instance's API port over TCP, 8443 + in this page's examples. Open that port to the controller's address only. + The instances need no route to the controller. +- **Other systems.** No change. OpenSIPS, Kamailio, Homer and a HEP collector + keep their configuration. A collector that runs sipnab joins the list like + any other instance. + +## Step 1: run each instance with the REST API over TLS + +- **[each instance]** Add `--api`, the TLS certificate and key, the signing key + and a node name to the instance's command line. This is the instance + `edge-a` from the run, which reads a capture file and sends HEP to the + collector `hub`: + + ```bash + sipnab -N -I /var/lib/sipnab/sample.pcap --config /etc/sipnab/edge-a.toml \ + --hep-send 192.0.2.10:9060 --hep-auth-mode hmac --hep-auth-file /etc/sipnab/hep.key \ + --api 192.0.2.11:8443 --api-tls-cert /etc/sipnab/edge-a.crt --api-tls-key /etc/sipnab/edge-a.key \ + --api-signing-key-file /etc/sipnab/edge-a.signing --node-name edge-a + ``` + + The collector ran with the same API flags and its HEP listener: + + ```bash + sipnab -N --config /etc/sipnab/hub.toml \ + --hep-listen 192.0.2.10:9060 --hep-auth-mode hmac --hep-auth-file /etc/sipnab/hep.key \ + --api 192.0.2.10:8443 --api-tls-cert /etc/sipnab/hub.crt --api-tls-key /etc/sipnab/hub.key \ + --api-signing-key-file /etc/sipnab/hub.signing --node-name hub + ``` + + Each instance logs `REST API listening on
` when its API is up. + +## Step 2: mint an inventory token on each instance + +- **[each instance]** Mint a token with that instance's signing key: + + ```bash + sipnab --mint-token --token-scope inventory --token-id fleet-controller \ + --api-signing-key-file /etc/sipnab/edge-a.signing --api-token-ttl 2592000 + ``` + + It prints one token, starting `s2.`, and exits. `--api-token-ttl 2592000` + makes it valid for 30 days. + +- **[controller]** Save each token in its own file, readable by the + controller's user only: + + ```bash + install -m 600 /dev/null /etc/sipnab/fleet/edge-a.token + ``` + + Then paste the token into it. sipnab refuses a token file that other users + can read. + +## Step 3: write the hosts file + +- **[controller]** Write `/etc/sipnab/fleet/fleet.toml`, one `[[host]]` table + per instance. This is the file from the run: + + ```toml + [[host]] + name = "hub" + url = "https://192.0.2.10:8443" + token_file = "/etc/sipnab/fleet/hub.token" + ca_file = "/etc/sipnab/fleet/internal-ca.pem" + + [[host]] + name = "edge-a" + url = "https://192.0.2.11:8443" + token_file = "/etc/sipnab/fleet/edge-a.token" + ca_file = "/etc/sipnab/fleet/internal-ca.pem" + + [[host]] + name = "edge-b" + url = "https://192.0.2.12:8443" + token_file = "/etc/sipnab/fleet/edge-b.token" + ca_file = "/etc/sipnab/fleet/internal-ca.pem" + ``` + + - `name` is the name the report shows. Each name appears once. + - `url` is the instance's REST API, `https://` or `http://`. + - `token_file` holds the instance's `inventory` token on one line. + - `ca_file` is optional: a PEM CA file to trust instead of the host's CA + bundle. + + sipnab refuses an unknown key, an empty name, an empty URL and a name used + twice, with exit `2`. + +## Step 4: read the fleet + +- **[controller]** Run: + + ```bash + sipnab fleet inventory --hosts /etc/sipnab/fleet/fleet.toml + ``` + + Output from the run: + + ```text + NAME STATE VERSION COMMIT BINARY CONFIG UID UPTIME_S + hub ok 0.5.232 29204db7 6bba53867ecd ec09522b49b4 1000 54 + edge-a ok 0.5.232 29204db7 6bba53867ecd 6a434df0c5e2 1000 40 + edge-b ok 0.5.232 29204db7 6bba53867ecd f63dfdce2c9d 1000 39 + ``` + + It exited `0`. + + - `BINARY` and `CONFIG` are the first 12 hex digits of `binary.sha256` and + `config.effective_sha256`. The three instances run one binary and three + configurations: each has its own config file and node name. + - `--json` prints the whole report, with every instance's full inventory. + - `--timeout ` sets how long each instance has to answer. Default `10`. + +## Verify + +- **Every instance is `ok`, and there is no `finding` line.** That is exit + `0`. +- **Two instances you expect to match show one `CONFIG`.** Two instances with + the same `CONFIG` loaded the same configuration values and run the same + command line. A different `--node-name` alone gives a different `CONFIG`. +- **The exit status** says the same for a script: + + | Code | Meaning | + |---|---| + | `0` | Every instance answered and there is no finding. | + | `1` | An instance did not answer, refused the token, or sent an inventory this build cannot read; or there is a finding. | + | `2` | sipnab cannot read or use the hosts file. | + +## Troubleshoot + +Each output below comes from the same run, with one thing changed. + +- **`unreachable` and `connection timed out`.** Nothing answered at that + address within `--timeout`: the instance is down, the URL names the wrong + address or port, or a firewall drops the controller's packets. The other + instances are still listed: + + ```text + edge-c unreachable 192.0.2.14:8443: connection timed out + ``` + + `Connection refused` instead means the host answered and nothing listens on + that port: check `--api` on the instance. + +- **`unreachable` and `invalid peer certificate: UnknownIssuer`.** The + controller does not trust the CA that issued the instance's certificate. Set + `ca_file` for that host: + + ```text + hub unreachable sending: invalid peer certificate: UnknownIssuer + ``` + +- **`refused` with HTTP `401`.** The instance did not accept the token: it was + minted with another instance's signing key, it expired, someone revoked it, or + its scope is not `inventory`, `full`, `actions` or `admin`. Mint a new one + on that instance (step 2). This run gave `edge-b` the hub's token: + + ```text + edge-b refused the instance answered HTTP 401 + ``` + +- **`refused` with HTTP `404`.** The instance runs a build without + `GET /v1/inventory`. Upgrade it. Release 0.5.130 answered: + + ```text + edge-old refused the instance answered HTTP 404 + ``` + + Release 0.5.130 also refuses `--token-scope inventory` at mint time, with + `invalid value 'inventory' for '--token-scope '`. + +- **`unreachable` and `can be read or written by other users`.** The token + file's mode lets other users read it. Run `chmod 600` on it: + + ```text + edge-a unreachable token_file '/etc/sipnab/fleet/open.token' can be read or written by other users (mode 0644); chmod 600 it + ``` + +- **A mixed HEP HMAC version.** A sender on 0.5.130 or earlier signs HEP with + HMAC version 1, and a collector on 0.5.131 or later drops every packet it + sends. In the run, `edge-old` (0.5.130) sent to `hub`. Listed in the hosts + file, `edge-old`'s row is `refused` with HTTP `404`, as above. The collector + logs `HEP HMAC auth is REFUSING version-1 tokens` once, and counts the + dropped packets. Read the count from the JSON report: + + ```bash + sipnab fleet inventory --hosts /etc/sipnab/fleet/fleet.toml --json \ + | jq -c '.hosts[] | {name, refused: .inventory.hep_listener.packets_refused, senders: [.inventory.hep_listener.senders[].peer]}' + ``` + + ```text + {"name":"hub","refused":10,"senders":["192.0.2.11","192.0.2.12"]} + {"name":"edge-a","refused":0,"senders":[]} + {"name":"edge-b","refused":0,"senders":[]} + ``` + + The hub refused 10 packets, and `192.0.2.13` is not among its senders. The + report itself is `ok` when the old sender is not in the hosts file: a + refused packet is not a finding. Upgrade the sender, as + [Upgrade a fleet that uses HMAC authentication](config-reference.md#upgrade-a-fleet-that-uses-hmac-authentication) + describes. + +- **`finding unlisted_hep_sender`.** A collector in the list receives HEP + from an address that no listed instance's URL resolves to. Add that sender + to the hosts file, or find out why it sends. With `edge-b` left out of the + file: + + ```text + finding unlisted_hep_sender: hub receives HEP from 192.0.2.12, which is no instance in the hosts file + ``` + + The check compares addresses only. A sender whose packets leave from a + different address than its API URL names shows up as unlisted. + +- **`incompatible` and a `schema_version`.** The instance answered an + inventory format newer than the controller reads. Upgrade the controller's + sipnab. No release answers a newer format yet, so this output comes from a + test server that answered `schema_version` 2: + + ```text + edge-next incompatible the instance answered inventory schema_version 2; this sipnab reads up to 1 + ``` + +## States and findings + +**State of each instance:** + +| State | Meaning | +|---|---| +| `ok` | It answered with an inventory. | +| `refused` | It answered with an HTTP status other than 200, such as `401` for a wrong, expired or narrower token, or `404` for a build without the inventory. | +| `unreachable` | No answer: no connection, a timeout, a TLS failure, a token file sipnab would not read, or an answer that is not JSON. | +| `incompatible` | Its inventory has a `schema_version` newer than this build reads, or none. Upgrade the controller's sipnab. | + +**Findings across the fleet:** + +| Finding | Meaning | +|---|---| +| `version_mismatch` | The instances that answered do not all run the same build. The finding names each build and the instances on it. | +| `unlisted_hep_sender` | A HEP collector in the list receives from an address that no listed instance's URL resolves to. | + +## Check a binary against its release + +Every instance reports `signature.state` as `unverified`. A process cannot +prove which binary it is, and one that someone had tampered with would report +"verified" too. + +- Download `SHA256SUMS.txt` from the release the instance claims, and verify + it: + + ```bash + gh attestation verify SHA256SUMS.txt --repo NormB/sipnab + ``` + +- Compare the instance's `binary.sha256` with the line for its artifact. On + Linux the instance hashes `/proc/self/exe`, so after an upgrade replaces the + file the digest still names the binary that is running. + +## What it cannot tell you + +- Every field is the instance's own claim. An attacker who controls a host can make + it report anything. +- For a secret held in a file, the digests cover its path only. The configuration and + the command line name key, token and password files by path, so replacing + the contents of such a file, for example rotating the key in + `--api-signing-key-file`, changes no digest. A secret given as a value, such + as `--api-signing-key` or a `${NAME}` the configuration expands, changes + `config.effective_sha256`. +- It reads only the instances in your hosts file. An instance nobody listed is + invisible to it, unless it sends HEP to a listed collector. +- It changes nothing. Pushing a configuration, upgrading and restarting are not + part of it. + +--- + # MCP tool reference @@ -35237,7 +35757,7 @@ A bundle is a named set of tools defined in sipnab's code. | [`media`](mcp-tools-media.md) | 4 | Where a call's media endpoints came from, why a stream has no dialog, the facts under a quality score, and audio export. | | [`relay`](mcp-tools-relay.md) | 4 | Questions put to the configured media relay, and decoding the relay control messages sipnab captured. | | [`tfps`](mcp-tools-tfps.md) | 7 | The toll-fraud prevention peer (TFPS): what it blocks and logs, and the operator's ban, unban and revert actions. | -| [`server`](mcp-tools-server.md) | 5 | The sipnab process and its capture path: capture health, HEP senders, resource use, build capabilities, and shutdown. | +| [`server`](mcp-tools-server.md) | 6 | The sipnab process and its capture path: capture health, HEP senders, resource use, build capabilities, the fleet inventory record, and shutdown. | | [`vcon`](mcp-tools-vcon.md) | 7 | vCon conversation containers, exported, validated and analyzed, one at a time or a directory at once, read from the operator's store, the vCon export spool's counts, and the SIPREC recording metadata a call carried. | | [`tls`](mcp-tools-tls.md) | 3 | Reading SIP inside TLS without keys through kernel uprobes, and which TLS libraries this host runs. | @@ -35311,22 +35831,22 @@ resource. | Bundle | Tools | Bytes (default) | Bytes with output schemas | |---|---|---|---| -| core | 9 | 12,316 | 54,624 | -| signaling | 20 | 22,474 | 58,619 | -| captures | 11 | 12,723 | 30,211 | -| security | 6 | 9,954 | 9,954 | -| media | 4 | 3,480 | 16,048 | -| relay | 4 | 4,199 | 19,988 | -| tfps | 7 | 5,563 | 15,947 | -| server | 5 | 5,394 | 27,452 | -| vcon | 7 | 10,714 | 33,024 | -| tls | 3 | 2,850 | 4,943 | -| full | 76 | 89,667 | 270,810 | +| core | 9 | 12,308 | 58,240 | +| signaling | 20 | 22,452 | 66,162 | +| captures | 11 | 12,709 | 30,189 | +| security | 6 | 9,944 | 9,944 | +| media | 4 | 3,478 | 16,042 | +| relay | 4 | 4,199 | 20,607 | +| tfps | 7 | 5,717 | 16,101 | +| server | 6 | 5,972 | 35,126 | +| vcon | 7 | 10,710 | 34,428 | +| tls | 3 | 2,899 | 4,990 | +| full | 77 | 90,388 | 291,829 | Bytes are the compact JSON of each tool's `tools/list` entry, summed over the bundle. -> **Measured on sipnab 0.5.222** on a build with every feature. +> **Measured on sipnab 0.5.232** (a development build of this change, 2026-11-03) on a build with every feature. > > - Sizes, tool counts and bundle membership may change in later releases. > - A build without a feature does not register that feature's tools, so its @@ -35366,6 +35886,7 @@ links to its section on the bundle's page. | [`find_in_captures`](mcp-tools-captures.md#find_in_captures) | [captures](mcp-tools-captures.md) | Which capture files hold dialogs matching a filter, without opening any of them; a sweep that outlasts its wait returns a job | | [`find_in_captures_status`](mcp-tools-captures.md#find_in_captures_status) | [captures](mcp-tools-captures.md) | A `find_in_captures` job's progress, or its result once it finishes | | [`find_problems`](mcp-tools-core.md#find_problems) | [core](mcp-tools-core.md) | A page of dialogs matching one or more diagnostic alias names | +| [`fleet_inventory`](mcp-tools-server.md#fleet_inventory) | [server](mcp-tools-server.md) | What this process is, for a fleet controller: build, binary SHA-256, signature state, configuration digests, privilege, uptime and counters | | [`generate_fail2ban_rule`](mcp-tools-security.md#generate_fail2ban_rule) | [security](mcp-tools-security.md) | A fail2ban filter and jail derived from ONE recorded finding, with that finding attached as the evidence | | [`generate_repro`](mcp-tools-signaling.md#generate_repro) | [signaling](mcp-tools-signaling.md) | A SIPp scenario replaying one call, with the hypothesis as an input: `pin` holds the suspected cause fixed, `vary` regenerates identity | | [`generate_wireshark_filter`](mcp-tools-signaling.md#generate_wireshark_filter) | [signaling](mcp-tools-signaling.md) | A Wireshark display filter selecting one call's signaling and its RTP by SSRC, plus the tshark line that applies it | @@ -35637,13 +36158,15 @@ When `source_exhausted` is `false`, `total_matched` counts only what had loaded. Repeat the call, or poll `capture_status` until `source_exhausted` reads `true`, before treating a count as final. -Twenty-one tools are the exception, because their answer cannot move with +Twenty-two tools are the exception, because their answer cannot move with the load. How much of the capture sipnab has read says nothing about what their sources hold: -- Twelve read something other than the capture store: +- Thirteen read something other than the capture store: - `explain_response_code`, `explain_rule`, `decode_evidence`, `decode_ng`, `show_evidence`, `list_captures`, `list_tls_libraries`, `server_capabilities`; + - `fleet_inventory`, which describes the process and returns the same + document as the REST inventory route; - `vcon_spool_status`, which counts the vCon export spool that the forwarder drains; - `compare_captures`, which reads two files and never the loaded capture; @@ -42109,6 +42632,7 @@ the other bundles and finds any tool by name. |---|---|---| | [`runtime_stats`](#runtime_stats) | `sample_seconds?` | What sipnab is doing and what it costs the host: its own memory, threads, descriptors and CPU; the host's totals and the basis they came from; its share of them and whether that share is load-bearing; per-interface counters; store occupancy against the caps | | [`capture_health`](#capture_health) | `sample_seconds` | Capture-path counters read twice: run totals, deltas across the window, `undecoded_fraction`, and undecodable frames by reason | +| [`fleet_inventory`](#fleet_inventory) | -- | What this process is, for a fleet controller: build, binary SHA-256, signature state, configuration digests, privilege, uptime, interface, capture and HEP counters, and HEP sender addresses | | [`hep_senders`](#hep_senders) | `limit?` | Who feeds this run's HEP listener: each sender, its count, when it was last heard and whether it went silent, and every address the listener refused, by reason | | [`server_capabilities`](#server_capabilities) | -- | sipnab version, the optional features this binary carries, and the libpcap it runs | | [`shutdown_server`](#shutdown_server) | `dry_run?`, `save_to?`, `discard_unsaved?` | **Destructive.** Stops the process. Needs `--mcp-allow-shutdown`; dry-run by default | @@ -42526,6 +43050,69 @@ roster. The MCP answer differs in two ways: its keys are in alphabetical order, and it adds `source_exhausted` and `source_stopped_early`. `--hep-senders` prints the roster at the end of a headless run. +### `fleet_inventory` + +What this sipnab process is, for a fleet controller that manages many +instances: its build, the SHA-256 of its binary, its configuration as digests, +its privilege, its uptime and its counters. Starts nothing and waits for +nothing. + +**Parameters:** none. + +The example ran on 2026-11-03 with a development build, against +[`tests/pcap-samples/sip-rtp-g711.pcap`](https://github.com/NormB/sipnab/raw/main/tests/pcap-samples/sip-rtp-g711.pcap) +with `--node-name edge-a --no-config`: + +```jsonc +// fleet_inventory {} +{ + "schema_version": 1, + "node_name": "edge-a", + "build": { + "version": "0.5.232", "commit": "29204db7", "tag": "", "dirty": true, + "target": "aarch64-unknown-linux-gnu", + "features": ["api", "archive", "audio", "g729", "hep", "mcp", "mcp-http", "metrics", + "native", "opus", "plugins", "tls", "tui", "vcon"], + "version_line": "0.5.232 (29204db7-dirty) features: native,tui,audio,tls,hep,api,mcp,mcp-http,metrics,plugins,vcon,opus,archive,g729" + }, + "binary": { + "sha256": "3242a3324b1c3ac762ba15d4f2eec3789a995533540b8aee56ec087afdc51ea8", + "error": null + }, + "signature": { + "state": "unverified", + "note": "sipnab does not verify its own binary. Compare binary.sha256 with SHA256SUMS.txt of the release, after `gh attestation verify SHA256SUMS.txt --repo NormB/sipnab`." + }, + "config": { + "recorded": true, "source": null, "shadowed": [], "file_sha256": null, + "parsed_sha256": "afa5e308cbb51683fe7ac0ca79e2a33f601530f33439211dd29586c9db0c7d75", + "argv_sha256": "cb86e74e2f7d2f54c2e75c3cb01976de06c9658406112c318c5f290cf606418d", + "effective_sha256": "aee2e58c08b314be6538cf9ccf1dfddb992992c4e7336a4d256373b67be72130" + }, + "privilege": { "uid": 1000, "root": false, "dropped_to": null }, + "process": { "pid": 3985285, "started": "2026-11-03T13:26:30Z", "uptime_seconds": 1 }, + "interfaces": [], + "counters": { + "capture_packets_total": 852, "capture_sources_refused_total": 0, + "capture_backpressure_blocks_total": 0, "hep_export_failures_total": null, + "hep_export_dropped_total": null, "hep_export_queue_depth": null + }, + "hep_listener": { "listening": false, "packets_refused": 0, "senders": [] } +} +``` + +- No configuration value and no command-line argument is in the answer, only + their SHA-256 digests. A secret in either changes a digest and never appears. +- `signature.state` is always `unverified`. Compare `binary.sha256` with the + release's attested `SHA256SUMS.txt`. +- Every field is the process's own claim. + +`GET /v1/inventory` returns the same document, from the same builder. Unlike +most tools, the answer carries no `source_exhausted` or `source_stopped_early`: +it describes the process, not a result read from the capture. A `read` token +reaches this tool. [Manage many +sipnab instances](fleet.md) reads the REST route from every instance. + ### `server_capabilities` What this binary can do, what this server permits and loaded, the limits it @@ -42577,7 +43164,7 @@ opt-in flags: "evaluate_expectations", "explain_attribution", "explain_response_code", "explain_rule", "export_audio", "export_capture", "export_vcon", "fetch_vcon", "find_correlated", "find_in_captures", "find_in_captures_status", "find_problems", - "generate_fail2ban_rule", "generate_repro", "generate_wireshark_filter", + "fleet_inventory", "generate_fail2ban_rule", "generate_repro", "generate_wireshark_filter", "get_call_tree", "get_capture_report", "get_dialog", "get_dialog_report", "get_message", "get_sdp_timeline", "group_dialogs", "hep_senders", "lint_dialog", "list_captures", "list_dialogs", "list_tls_libraries", "media_diagnostics", @@ -44681,8 +45268,8 @@ lines on stdin, the way a client does. - **No prompt-injection cooperation.** Tool descriptions never instruct the LLM to "trust" or "act on" returned content. They describe what the tool returns and stop there. -- **Every tool declares what it does.** All 76 carry MCP annotations, so a host - can decide what to call without asking. Sixty-three are `readOnlyHint: true`. +- **Every tool declares what it does.** All 77 carry MCP annotations, so a host + can decide what to call without asking. Sixty-four are `readOnlyHint: true`. [What the write verbs do](#what-the-write-verbs-do) names the thirteen that are not. @@ -45136,7 +45723,7 @@ sipnab negotiates the 2025-06-18 and 2025-11-25 revisions. There, ## What the write verbs do -Sixty-three of the 76 tools are `readOnlyHint: true`. These thirteen are not. +Sixty-four of the 77 tools are `readOnlyHint: true`. These thirteen are not. Each declares what kind of change it makes, so a host can decide which need confirmation: @@ -54608,7 +55195,7 @@ Distribute the printed token to clients. ### Scope: what a token may reach `--token-scope` sets what a token may reach. It takes `full` (the default), -`metrics`, `read`, `actions`, `admin` or `audio`: +`metrics`, `read`, `actions`, `admin`, `audio` or `inventory`: | Scope | Surface | What it reaches | |---|---|---| @@ -54618,6 +55205,7 @@ Distribute the printed token to clients. | `actions` | either | Everything `full` reaches, plus the actions that change another system, such as a TFPS ban, where the server enables them with `--allow-action`. It lifts only the bans it placed. | | `admin` | either | Everything `actions` reaches, and it also lifts a ban another caller placed, for example one placed by a token since lost. | | `audio` | REST API | The live-audio routes under `/v1/calls/`, and nothing else. No other scope reaches them. | +| `inventory` | REST API | `GET /v1/inventory`, and nothing else. `full`, `actions` and `admin` reach it too. For [`sipnab fleet inventory`](fleet.md). | Give each client the narrowest scope that does its job: diff --git a/website/static/llms.txt b/website/static/llms.txt index 4e733eea1..ef05c1838 100644 --- a/website/static/llms.txt +++ b/website/static/llms.txt @@ -31,6 +31,7 @@ This file indexes the documentation as plain text. `llms-full.txt` beside it car - [Read TLS without keys (eBPF)](https://sipnab.com/docs/uprobe-walkthrough/): What uprobe and eBPF capture is and is NOT, its security implications, whether your kernel supports it at all, and how to run both backends step by step. - [Deploy the MCP server](https://sipnab.com/docs/mcp-deploy/): Step-by-step MCP deployment scenarios: same-box stdio, remote production servers over SSH or HTTP, HEP capture hosts, TLS endpoints, fleets, and headless automation. - [Use MCP across many servers](https://sipnab.com/docs/mcp-estate/): Several SIP servers feeding one capture host, reaching sipnab from outside the network, one agent holding many capture hosts, and following one call across an SBC, a proxy and a PBX. +- [Manage many sipnab instances](https://sipnab.com/docs/fleet/): One table of every sipnab instance's build, binary and configuration digests, privilege and uptime, the instances that did not answer, and HEP senders nobody listed. - [Look up an MCP tool](https://sipnab.com/docs/mcp-tools/): The MCP tool bundles, every tool by name with the page that documents it, and the rules every tool follows. - [Look up a tool in the MCP core bundle](https://sipnab.com/docs/mcp-tools-core/): The MCP tools in the core bundle. The smallest set that answers what happened on one call, and whether the fault was in signaling or media, from listing the calls to the bytes that prove it. - [Look up a tool in the MCP signaling bundle](https://sipnab.com/docs/mcp-tools-signaling/): The MCP tools in the signaling bundle. Follow-up questions about SIP signaling: single messages, call flows, SDP negotiation, related call legs, RFC conformance, time windows, and reproducing a call. diff --git a/website/static/openapi.json b/website/static/openapi.json index c6c282157..65966b6d6 100644 --- a/website/static/openapi.json +++ b/website/static/openapi.json @@ -155,6 +155,26 @@ ], "type": "object" }, + "BinaryInfo": { + "description": "The running binary.", + "properties": { + "error": { + "description": "Why it could not be read.", + "type": [ + "string", + "null" + ] + }, + "sha256": { + "description": "SHA-256 of the running image, lowercase hex; absent when it could\nnot be read.", + "type": [ + "string", + "null" + ] + } + }, + "type": "object" + }, "BucketDeltaView": { "description": "One value's movement between the two captures. `value` is raw.", "properties": { @@ -186,6 +206,52 @@ ], "type": "object" }, + "BuildInfo": { + "description": "The build, as `sipnab --version` reports it.", + "properties": { + "commit": { + "description": "The git commit the build recorded; empty when it recorded none.", + "type": "string" + }, + "dirty": { + "description": "Whether the working tree had uncommitted changes.", + "type": "boolean" + }, + "features": { + "description": "The compiled feature set, sorted.", + "items": { + "type": "string" + }, + "type": "array" + }, + "tag": { + "description": "The git tag at that commit; empty when there was none.", + "type": "string" + }, + "target": { + "description": "The target triple.", + "type": "string" + }, + "version": { + "description": "The crate version.", + "type": "string" + }, + "version_line": { + "description": "The whole `--version` line.", + "type": "string" + } + }, + "required": [ + "version", + "commit", + "tag", + "dirty", + "target", + "features", + "version_line" + ], + "type": "object" + }, "CallReport": { "additionalProperties": false, "description": "Serialized form of DialogJson (src/output/json.rs): the `--call-report --json` output and GET /v1/dialogs/{id}. The embedded `streams` items mirror stream.schema.json (kept self-contained, no cross-file $ref).", @@ -2220,6 +2286,62 @@ ], "type": "object" }, + "ConfigInfo": { + "description": "The configuration, as digests only.", + "properties": { + "argv_sha256": { + "description": "SHA-256 of the command line, without the program path.", + "type": [ + "string", + "null" + ] + }, + "effective_sha256": { + "description": "SHA-256 over `parsed_sha256` and `argv_sha256`: what decides the\neffective configuration. The one configuration digest to compare\nacross instances.", + "type": [ + "string", + "null" + ] + }, + "file_sha256": { + "description": "SHA-256 of the file's bytes.", + "type": [ + "string", + "null" + ] + }, + "parsed_sha256": { + "description": "SHA-256 of the parsed configuration, canonically serialized.", + "type": [ + "string", + "null" + ] + }, + "recorded": { + "description": "Whether this run recorded its configuration. `false` means every\ndigest below is unknown, not empty.", + "type": "boolean" + }, + "shadowed": { + "description": "Default-location files present but not read.", + "items": { + "type": "string" + }, + "type": "array" + }, + "source": { + "description": "The file read, if any.", + "type": [ + "string", + "null" + ] + } + }, + "required": [ + "recorded", + "shadowed" + ], + "type": "object" + }, "ContainerFinding": { "description": "One schema finding against an invalid container.", "properties": { @@ -2349,6 +2471,64 @@ ], "type": "object" }, + "Counters": { + "description": "Capture-path and HEP export counters, cumulative.", + "properties": { + "capture_backpressure_blocks_total": { + "description": "Times the capture path blocked on a full queue; absent with no queue.", + "format": "int64", + "minimum": 0, + "type": [ + "integer", + "null" + ] + }, + "capture_packets_total": { + "description": "Packets the capture path received.", + "format": "int64", + "minimum": 0, + "type": "integer" + }, + "capture_sources_refused_total": { + "description": "Capture sources refused past the source-table cap.", + "format": "int64", + "minimum": 0, + "type": "integer" + }, + "hep_export_dropped_total": { + "description": "HEP export packets dropped of every kind; absent with no exporter.", + "format": "int64", + "minimum": 0, + "type": [ + "integer", + "null" + ] + }, + "hep_export_failures_total": { + "description": "HEP export failures of every kind; absent with no exporter.", + "format": "int64", + "minimum": 0, + "type": [ + "integer", + "null" + ] + }, + "hep_export_queue_depth": { + "description": "HEP export packets queued now; absent with no exporter.", + "format": "int64", + "minimum": 0, + "type": [ + "integer", + "null" + ] + } + }, + "required": [ + "capture_packets_total", + "capture_sources_refused_total" + ], + "type": "object" + }, "Dialog": { "description": "`GET /v1/dialogs/{call_id}` — the full dialog.\n\nThe projection `output::json::dialog_to_json` builds, which is a\nSUPERSET of [`DialogSummary`] and not the same shape: it carries the\ndisplay names, the SDP timeline, the media diagnosis and the streams\nthemselves. `tests/openapi_contract_test.rs` reads a real one off a\nrunning server and fails if a key here is missing from it, or one of\nits keys is missing from here.\n\nThe nested objects are left open rather than enumerated. Their\ndefinitions live in `output::json`, and repeating them here would be\nthe drift this document exists to remove; the live check is what holds\nthe top level honest.", "properties": { @@ -3040,6 +3220,61 @@ ], "type": "object" }, + "HepListenerInfo": { + "description": "The HEP listener's senders, by address only.", + "properties": { + "listening": { + "description": "Whether this run has a HEP listener.", + "type": "boolean" + }, + "packets_refused": { + "description": "Packets the listener refused, for any reason.", + "format": "int64", + "minimum": 0, + "type": "integer" + }, + "senders": { + "description": "The senders it tracks, at most [`MAX_HEP_PEERS`].", + "items": { + "$ref": "#/components/schemas/HepPeer" + }, + "type": "array" + } + }, + "required": [ + "listening", + "packets_refused", + "senders" + ], + "type": "object" + }, + "HepPeer": { + "description": "One HEP sender.", + "properties": { + "capture_id": { + "description": "The capture id it claims; never proven.", + "format": "int32", + "minimum": 0, + "type": [ + "integer", + "null" + ] + }, + "peer": { + "description": "The address its packets came from.", + "type": "string" + }, + "silent": { + "description": "Whether it has been silent past the warning threshold.", + "type": "boolean" + } + }, + "required": [ + "peer", + "silent" + ], + "type": "object" + }, "HepRefusedSourceRow": { "description": "One address the listener has been turning away.\n\nKeyed by address alone, never by the capture id a refused packet claimed:\na packet that failed authentication has proven nothing, including its id.", "properties": { @@ -3344,6 +3579,121 @@ ], "type": "object" }, + "InterfaceCounters": { + "description": "One capture interface's error and drop counters, read from the interface.", + "properties": { + "name": { + "description": "The interface name.", + "type": "string" + }, + "operstate": { + "description": "Link state; absent when not readable on this platform.", + "type": [ + "string", + "null" + ] + }, + "rx_dropped": { + "description": "Packets the kernel dropped on receive.", + "format": "int64", + "minimum": 0, + "type": [ + "integer", + "null" + ] + }, + "rx_errors": { + "description": "Receive errors.", + "format": "int64", + "minimum": 0, + "type": [ + "integer", + "null" + ] + }, + "rx_missed_errors": { + "description": "Packets the NIC missed for lack of ring space.", + "format": "int64", + "minimum": 0, + "type": [ + "integer", + "null" + ] + } + }, + "required": [ + "name" + ], + "type": "object" + }, + "Inventory": { + "description": "One inventory record.", + "properties": { + "binary": { + "$ref": "#/components/schemas/BinaryInfo", + "description": "The running binary." + }, + "build": { + "$ref": "#/components/schemas/BuildInfo", + "description": "The build." + }, + "config": { + "$ref": "#/components/schemas/ConfigInfo", + "description": "The configuration this run loaded, as digests." + }, + "counters": { + "$ref": "#/components/schemas/Counters", + "description": "Capture-path and HEP export counters." + }, + "hep_listener": { + "$ref": "#/components/schemas/HepListenerInfo", + "description": "The HEP listener, when this run has one." + }, + "interfaces": { + "description": "The capture interfaces and their error and drop counters.", + "items": { + "$ref": "#/components/schemas/InterfaceCounters" + }, + "type": "array" + }, + "node_name": { + "description": "The node name this process reports (`--node-name`,\n`[capture] node_name`, else the host name).", + "type": "string" + }, + "privilege": { + "$ref": "#/components/schemas/PrivilegeInfo", + "description": "The privilege the process holds." + }, + "process": { + "$ref": "#/components/schemas/ProcessInfo", + "description": "The process." + }, + "schema_version": { + "description": "Wire-format version of this envelope, [`SCHEMA_VERSION`].", + "format": "int32", + "minimum": 0, + "type": "integer" + }, + "signature": { + "$ref": "#/components/schemas/SignatureInfo", + "description": "What this process can say about its release signature." + } + }, + "required": [ + "schema_version", + "node_name", + "build", + "binary", + "signature", + "config", + "privilege", + "process", + "interfaces", + "counters", + "hep_listener" + ], + "type": "object" + }, "LadderRow": { "description": "One message of a dialog's ladder.", "properties": { @@ -3612,6 +3962,35 @@ ], "type": "object" }, + "PrivilegeInfo": { + "description": "The privilege the process holds.", + "properties": { + "dropped_to": { + "description": "The user a privilege drop switched to; absent when none happened.", + "type": [ + "string", + "null" + ] + }, + "root": { + "description": "Whether that is root.", + "type": "boolean" + }, + "uid": { + "description": "The user id the process runs as.", + "format": "int32", + "minimum": 0, + "type": [ + "integer", + "null" + ] + } + }, + "required": [ + "root" + ], + "type": "object" + }, "Problem": { "description": "An RFC 9457 `application/problem+json` body: what every 4xx and 5xx\ncarries.\n\nSerialized by [`super::Problem`]'s `IntoResponse`, so this is the wire\ntype itself rather than a description of one.", "properties": { @@ -3647,6 +4026,35 @@ ], "type": "object" }, + "ProcessInfo": { + "description": "The process.", + "properties": { + "pid": { + "description": "The process id.", + "format": "int32", + "minimum": 0, + "type": "integer" + }, + "started": { + "description": "When the configuration was recorded, RFC 3339; absent when it was\nnot.", + "type": [ + "string", + "null" + ] + }, + "uptime_seconds": { + "description": "Seconds this process has been serving.", + "format": "int64", + "minimum": 0, + "type": "integer" + } + }, + "required": [ + "pid", + "uptime_seconds" + ], + "type": "object" + }, "RateGroup": { "description": "One group's carrier metrics, with the population behind them and the\nreason for any metric its population could not support.", "properties": { @@ -4913,6 +5321,24 @@ ], "type": "object" }, + "SignatureInfo": { + "description": "The release signature, as far as the process can speak to it.", + "properties": { + "note": { + "description": "How to verify the binary.", + "type": "string" + }, + "state": { + "description": "Always [`SIGNATURE_STATE`].", + "type": "string" + } + }, + "required": [ + "state", + "note" + ], + "type": "object" + }, "SkippedPort": { "description": "One of the busiest ports the port gate excluded.", "properties": { @@ -8525,6 +8951,58 @@ ] } }, + "/v1/inventory": { + "get": { + "description": "What this sipnab process is, for managing many instances as one fleet: the build (version, git commit and tag, dirty flag, target, compiled features, the `--version` line), SHA-256 of the running binary, the signature state, the configuration as digests only, the user id and privilege drop, pid, start time and uptime, the capture interfaces' error and drop counters, capture and HEP export counters, and the HEP listener's sender addresses.\n\nNo configuration value and no command-line argument is in the answer, only their SHA-256 digests, so a secret in either changes a digest and never appears. `signature.state` is always `unverified`: compare `binary.sha256` with the release's attested `SHA256SUMS.txt`. Every field is the process's own claim.\n\nReached by a token with scope `inventory`, or `full` and the broader scopes. The MCP tool `fleet_inventory` returns the same fields and values.", + "operationId": "get_inventory", + "responses": { + "200": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/Inventory" + } + } + }, + "description": "The inventory record." + }, + "401": { + "content": { + "application/problem+json": { + "schema": { + "$ref": "#/components/schemas/Problem" + } + } + }, + "description": "No bearer credential, or one this server does not accept." + }, + "500": { + "content": { + "application/problem+json": { + "schema": { + "$ref": "#/components/schemas/Problem" + } + } + }, + "description": "The task that builds the record failed." + }, + "503": { + "content": { + "application/problem+json": { + "schema": { + "$ref": "#/components/schemas/Problem" + } + } + }, + "description": "Over the per-source-IP rate limit of 100 requests per second." + } + }, + "summary": "Fleet inventory of this process", + "tags": [ + "operations" + ] + } + }, "/v1/persistence": { "get": { "description": "Whether this capture is writing content to disk, and whether its command line and config file allow it to.\n\nBehind the same guard as every other route: what a capture is keeping is not a public fact, and a reader who can see the answer is one step from a writer who can change it.", diff --git a/website/templates/index.html b/website/templates/index.html index 63b85909e..bb27c53a4 100644 --- a/website/templates/index.html +++ b/website/templates/index.html @@ -234,7 +234,7 @@