Skip to content

Commit fb7f804

Browse files
Pass the downstream CrossByte suite over its known Int64 test failure
testASmallInt64IsAnInt64WhenWrapped has failed in every downstream run since at least 10-23, against the same CrossByte commit: "expected 16 but it is 18". It expects an int32 when Std.isOfType(boxed, Int) is true for Int64.ofInt(5) held in a Dynamic. hxcpp has answered true for an Int64 in the Int range since 2016, and libhxcpp still does, while CrossByte's BsonWriter.isInt64, which its encoder goes by, sees the Int64 box libhxcpp has made of small values since 01d67a2, and writes an int64, as it should. The test is CrossByte's to fix, by asking BsonWriter.isInt64(boxed) as the encoder does. test/downstream/run-suite.sh runs the native suite and passes it with a warning when every test utest reports as failed is named in KNOWN_FAILURES, which the workflow sets to this one test with the reason beside it. A crash, a timeout, a run with no report, or any other failure fails the step as before. Tried against the logs of the runs on 10-27 (passes with the warning), 04f436b (fails, naming the five other tests), a run with no failures and a crash.
1 parent b9a4b29 commit fb7f804

2 files changed

Lines changed: 51 additions & 2 deletions

File tree

‎.github/workflows/downstream.yml‎

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -48,14 +48,25 @@ jobs:
4848
haxelib set utest 1.13.2 --always
4949
haxelib dev crossbyte "$PWD/crossbyte"
5050
51-
# Run from CrossByte's root: some tests look for src/ above the working directory
51+
# Run from CrossByte's root: some tests look for src/ above the working directory.
52+
# KNOWN_FAILURES are failures of CrossByte's own tests, not of this runtime:
53+
# the step passes with a warning when they are the only ones. Take each out
54+
# once CrossByte's test is fixed.
55+
# - testASmallInt64IsAnInt64WhenWrapped expects an int32 when
56+
# Std.isOfType(boxed, Int) is true. hxcpp has answered true for an Int64
57+
# in the Int range since 2016, and BsonWriter.isInt64, which the encoder
58+
# goes by, sees the Int64 box libhxcpp makes of small values as well
59+
# (01d67a2f), so an int64 is written. The test should ask
60+
# BsonWriter.isInt64(boxed), as the encoder does.
5261
- name: native suite
5362
shell: bash
5463
working-directory: crossbyte
64+
env:
65+
KNOWN_FAILURES: testASmallInt64IsAnInt64WhenWrapped
5566
run: |
5667
haxe -cp src -lib emitter -cp tests -lib utest -D HXCPP_M64 -main NativeSmokeMain --cpp ../cb/native \
5768
--macro "crossbyte._internal.macro.HostPlatform.define()" --macro "crossbyte.test.SuiteCoverage.check()"
58-
timeout 1200 ../cb/native/NativeSmokeMain
69+
bash ../test/downstream/run-suite.sh ../cb/native.log timeout 1200 ../cb/native/NativeSmokeMain
5970
6071
- name: stress suite
6172
if: ${{ !cancelled() }}

‎test/downstream/run-suite.sh‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
#!/bin/bash
2+
# run-suite.sh LOG COMMAND...
3+
#
4+
# Runs a CrossByte utest suite, its output kept in LOG as well, and passes
5+
# it with a warning when every test that failed is named in KNOWN_FAILURES, a
6+
# list of method names separated by spaces. Those are failures of CrossByte's
7+
# own tests, each with its reason where the workflow lists it. A crash, a
8+
# timeout, or a run that printed no report fails as before, and so does any
9+
# failure not on the list.
10+
log=$1
11+
shift
12+
"$@" 2>&1 | tee "$log"
13+
status=${PIPESTATUS[0]}
14+
if [ "$status" -eq 0 ]; then
15+
exit 0
16+
fi
17+
if [ "$status" -ne 1 ] || ! grep -q '^results: SOME TESTS FAILURES' "$log"; then
18+
exit "$status"
19+
fi
20+
21+
# utest's report: a line for each case, then " method: FAILURE ..." for each
22+
# of its methods that did not pass
23+
failed=$(sed -nE 's/^ ([A-Za-z0-9_]+): (FAILURE|ERROR|WARNING).*/\1/p' "$log" | sort -u)
24+
if [ -z "$failed" ]; then
25+
exit "$status"
26+
fi
27+
unknown=""
28+
for name in $failed; do
29+
case " $KNOWN_FAILURES " in
30+
*" $name "*) echo "::warning::$name failed, as it is known to: see KNOWN_FAILURES" ;;
31+
*) unknown="$unknown $name" ;;
32+
esac
33+
done
34+
if [ -n "$unknown" ]; then
35+
echo "::error::Failed:$unknown"
36+
exit 1
37+
fi
38+
exit 0

0 commit comments

Comments
 (0)