Repository navigation
ci(windows): keep the test data off the runner's system drive - #235
Merged
Merged
Conversation
About 11 minutes after a hosted Windows runner boots, its system drive is read for about 45 s, and a flush there takes seconds. A document test that writes then reaches the 10 s limit of one guest call. That failed the windows job in 4 of its first 32 runs. The Tests step now keeps its data below RUNNER_TEMP and prints the boot time of the runner. The stall watch of #232 goes, because the cause is found. TestSpareAdoption compared two readings of a clock that Windows can give as equal. It now starts the adoption only when the clock has passed its reading. The assertion is the same. The panic of the Tailscale library that failed the job once is recorded in docs/network-provider-lifecycle.md, with the fix that the library's development branch has and that no release holds yet. No code changes for it.
The current-state item said that the system drive of the runner was the cause of the four CI events. The boot times of those four runners are not recorded, so the item now says that the events fit the clock of the reads. It gives the count of failed tests in a run, and twelve of twelve measured runners. The place of the log truncation is now listed under "Open questions". The lifecycle text dates its statement on the newest release of the library, and the playbook names the log line with the boot time.
Fifteen of fifteen measured runners showed the reads of the system drive, and one of them started its job about 5 min after its boot.
The temporary drive of a hosted runner keeps no short (8.3) names, so TestExactNameRefusesARealShortAlias skipped there. The package now runs first, with the default temp directory and with -count=1, and that test runs on the system drive again. The open question of the spec now says "flushes again", which is exact.
The third runner check had one runner whose job started 11 min 1 s after its boot. The reads of the system drive came in its setup steps, and its counters began later. The spec item now counts seventeen runners with the reads and one without, and gives the range of the job start for both. The spec item also names the one package that runs with the default temp directory, for the test that needs short names.
The spec item said that the reads came on the runner whose job started 11 min 1 s after its boot. Nothing recorded them there: its counters began later. The item now says that its counters began after the time at which the reads come on the other runners, and recorded none.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
The
windowsCI job failed 11 of its 32 completed runs onmain. Five of those had causes that #220 and #232 removed. The other six have three causes. This change fixes two of them and records the third.Testsstep keeps its data belowRUNNER_TEMP, on the temporary drive, and prints the boot time of the runner. The stall watch of #232 is removed, because the cause is found.tailscale.com v1.104.0dereferences a nil pointer when a status with a network map is still in work as its backend closes.docs/network-provider-lifecycle.md. No code changes.TestSpareAdoptioncompared two readings of a clock that Windows can give as equal.The move of the test data has one side effect that review found: the temporary drive keeps no 8.3 short names, so
TestExactNameRefusesARealShortAliasofinternal/siteskipped there. The step now runs that package first, with the default temp directory and-count=1.No product code changes. The diff holds one workflow, three test files, and documents.
Cause B stays
The job, and a host, can still end with the panic of the Tailscale library, at the rate so far of 1 in 32 runs of the job, until Flats uses a release of the library that has commit
b9d5ff3ae(2026-11-10, "ipn/ipnlocal: don't apply netmap from a replaced control client"). No such release exists on 2026-11-13: v1.104.7 is the newest. The commit removes the sequence of the CI event. A backend that closes while it handles its first network map stays possible on the development branch. I did not reproduce the panic here (480 runs of the retry tests and 720 closes near the first network map).Evidence
C:, the run that met the reads had a guest call of 7.9 to 11.2 s, and one of the three failed two tests with the errors of the four CI events. On three runners that kept the data onD:, the longest guest call of 75 runs was 2.7 s.Testsstep ran on 9 hosted runners with-count=1in three runner checks. The last one is on the tree of the final commit plus a check workflow: 6 900 tests passed on each runner, 0 failed, 0 packages from the test cache, no log line of a later command with the old temp directory, andinternal/privatedirran. The first command passed all sixinternal/sitetests on each runner.TestSpareAdoptionpasses 30 of 30 on Windows and on Linux, and fails 3 of 3 under each of two mutations ofinternal/runtime/manager.go.go vet ./...passes forwindows/amd64,linux/amd64anddarwin/arm64. No Go test ran on macOS.Limits
-count=1. The CI run of this pull request is the first run of the step asci.ymlhas it, with the test cache of the job.docs/spec/windows-support.md. The product is unchanged.Review
Each stage had a review by a different model, in separate sessions. The author models are
claude-fable-5-2and, for the later work,claude-sonnet-5-5(the model of the session changed once). The reviewer isclaude-fable-5-1. Findings were answered one by one, corrected, and re-reviewed. The first line of each result:DESIGN: REJECTDESIGN: REJECTDESIGN: ACCEPTDESIGN: ACCEPTeachIMPLEMENTATION: ACCEPTIMPLEMENTATION: ACCEPTTestExactNameRefusesARealShortAliasskip on every runner. Corrected by runninginternal/sitefirst. 1 minor.IMPLEMENTATION: ACCEPTeachTESTING: ACCEPTTestSpareAdoptionthat the test does not support. The record was corrected.TESTING: ACCEPTThe review packages and receipts are on the author's machine, so they are not a source for a remote reader. This description is the accessible copy of the task report.
Release
This change holds no product code. A release from it would be the program of
v0.17.0with a new version stamp. Whether to release is the maintainer's decision, and no tag is pushed here.