You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 72eedd0
Browse filesBrowse the repository at this point in the historyBrowse files
Mark speed differences that code layout alone can make as level
Rebuilding the same sources, or adding one function that nothing calls,
moved map_int by up to 5%, and the same commit measured 7% apart on
alloc_small depending on which runtime's directory built it. The
tables marked only differences within twice the run-to-run spread as
level, so these read as regressions or wins that the runtime did not
make. A speed ratio within 5% of level is now marked "~" too, and
BENCHMARKS.md says why.
Copy file name to clipboardExpand all lines: BENCHMARKS.md
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -87,6 +87,7 @@ These numbers are from one Windows machine; Linux and macOS results are still to
87
87
-**One benchmark per process.** Each benchmark runs in its own process, so peak memory belongs to that benchmark alone: the peak working set on Windows, the maximum resident set size elsewhere. Each process does 2 untimed warm-up samples and 10 timed samples, and there are 3 processes per benchmark and runtime. The runtimes take turns, so drift on the machine affects both alike.
88
88
-**Same work, checked.** Every benchmark returns a checksum of the work it did, and every process under both runtimes has to produce the same one. A change that skips work or computes the wrong answer fails the run instead of looking faster.
89
89
-**Spread.** "±" is the median absolute deviation of the sample times, as a percentage of the median. A ratio marked "~" is within twice the larger spread of the two runtimes, so it counts as level.
90
+
-**Code layout.** A tight benchmark's speed also moves with where the compiler happens to place code. The same sources built again, or with one function added that nothing calls, measured up to 5% apart on `map_int`, and the same commit measured 7% apart on `alloc_small` depending on which runtime's directory built it. So a speed ratio within 5% of level is marked "~" as well, and a change of that size between two runs of these tables is not a regression on its own.
b.add('\nAbove 1 favours libhxcpp; "~" marks a difference within the run-to-run spread. '
581
+
b.add('\nAbove 1 favours libhxcpp; "~" marks a difference within the run-to-run spread, or for speed within the 5% that code layout alone can move it. '
582
582
+'${report.host}; ${env.compiler}, ${env.pointerBits}-bit; Haxe ${report.haxe}; ${lib.label} at ${report.libhxcpp}; ${report.date}.\n');
0 commit comments