Skip to content

Commit 89a7691

Browse files
Fix Date.now after 2038 on Windows and current GC memory, and speed up Array<Int>.join (#440)
Date.now() on Windows keeps the seconds since 1970 in 64 bits, so it no longer goes back to 1902 after January 2038 (#413). Gc.memInfo64(MEM_INFO_CURRENT) counts a block taken from the free list without the lock, the usual way (#414). Joining an Array<Int> with a byte separator writes the digits straight into one allocation: 200 joins of 8,192 integers went from 33 to 38 ms to 20 ms (#415). Fixes #413, fixes #414, fixes #415. # Conflicts: # Changes.md
2 parents 31ccfc1 + ff4d994 commit 89a7691

14 files changed

Lines changed: 218 additions & 2 deletions

File tree

‎Changes.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,9 @@ libhxcpp 6.0.0 (unreleased)
1010
* Fixed cpp.encoding.Utf8.encode writing bytes for a code point that is no Unicode scalar value, a surrogate or one past U+10FFFF, which UTF-8 cannot hold and the UTF-16 encoder refused. Both encoders, and both getByteCount, throw for one before writing anything, at API level 500 (libhxcpp #407)
1111
* Fixed Haxe 5's Bytes.ofString keeping an array of three bytes for each unit of a string that was not all ASCII: cpp.encoding.Utf8.encode allocated for the worst case and only shortened it, so a megabyte of ASCII with one other character held 3 MB. It counts the bytes and allocates them exactly (libhxcpp #405)
1212
* Fixed a malformed cppia file that calls a static function on a host interface crashing the loader: it looked the function up in the interface's script glue, which an interface does not have. The fuzz job found it
13+
* Sped up joining an Array<Int>, which made a String for each element and then joined those. The digits go straight into the result, so joining 8,192 integers 200 times takes 41 ms instead of 53 ms, what writing them with std::to_chars takes (libhxcpp #415)
14+
* Fixed Gc.memInfo64(Gc.MEM_INFO_CURRENT) leaving out most of what was allocated since the last collection: a block taken from the free list without the lock, which is the usual way, was not counted until the next collection. Retaining 2 MB of arrays showed 90 KB more (libhxcpp #414)
15+
* Fixed Date.now() on Windows going back to 1902 after January 2038, x64 builds too: the seconds since 1970 passed through a long, which is 32 bits on Windows (libhxcpp #413)
1316
* Added timedWait to hx::thread::ConditionVariable, which Haxe 5's sys.thread.Condition.timedWait calls: with the Haxe nightly, every program that used Condition failed to compile. It returns true once a signal or broadcast came and false when the time ran out, and it waits out the wakeups MSVC's timed waits give just before the deadline with nothing signalled
1417
* Fixed an exception from blit, setSize or resize under the cppia JIT ending the process instead of reaching the script's catch, on typed arrays and on Array<Dynamic>: their native calls had no exception bridge. The script catches it, as it does interpreted and in compiled code (libhxcpp #404)
1518
* Fixed resize() and cpp.NativeArray.setSize() on an Array<Dynamic> under the cppia JIT evaluating the array expression twice, so a call in it ran twice and the array it returned the second time was the one resized. The JIT evaluates it once (libhxcpp #411)

‎src/Array.cpp‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -674,6 +674,59 @@ String ArrayBase::joinArray(Array_obj<String> *inArray, String inSeparator)
674674
}
675675

676676

677+
static inline int intDigits(int inValue)
678+
{
679+
unsigned int u = inValue<0 ? (unsigned int)0 - (unsigned int)inValue : (unsigned int)inValue;
680+
int digits = inValue<0 ? 2 : 1;
681+
while(u>=10)
682+
{
683+
u /= 10;
684+
digits++;
685+
}
686+
return digits;
687+
}
688+
689+
// The digits go straight into the result, as String(int) writes them: a String
690+
// for each element was an allocation for each value of more than one digit
691+
// (libhxcpp #415)
692+
static String joinInts(ArrayBase *inBase, String inSeparator)
693+
{
694+
int length = inBase->length;
695+
const int *values = (const int *)inBase->GetBase();
696+
long long total = (long long)(length-1) * inSeparator.length;
697+
for(int i=0;i<length;i++)
698+
total += intDigits(values[i]);
699+
if (total > 0x7fffffff-8)
700+
hx::Throw( HX_CSTRING("String too long") );
701+
int len = (int)total;
702+
char *buf = hx::NewString(len);
703+
704+
// A moving collection may have run for the allocation
705+
values = (const int *)inBase->GetBase();
706+
char *pos = buf;
707+
for(int i=0;i<length;i++)
708+
{
709+
if (i && inSeparator.length)
710+
{
711+
memcpy(pos,inSeparator.raw_ptr(),inSeparator.length);
712+
pos += inSeparator.length;
713+
}
714+
int value = values[i];
715+
int digits = intDigits(value);
716+
unsigned int u = value<0 ? (unsigned int)0 - (unsigned int)value : (unsigned int)value;
717+
char *p = pos + digits;
718+
do {
719+
*--p = (char)('0' + (u % 10));
720+
u /= 10;
721+
} while (u);
722+
if (value<0)
723+
*--p = '-';
724+
pos += digits;
725+
}
726+
buf[len] = '\0';
727+
return String(buf,len);
728+
}
729+
677730
// Per thread, as an anonymous object's is: a thread formatting a nested array
678731
// must not use up the depth of another formatting its own
679732
static thread_local int _hx_toString_depth = 0;
@@ -685,6 +738,13 @@ String ArrayBase::joinArray(ArrayBase *inBase, String inSeparator)
685738
if (length==0)
686739
return HX_CSTRING("");
687740

741+
#ifdef HX_SMART_STRINGS
742+
if (inBase->getStoreType()==hx::arrayInt && !inSeparator.isUTF16Encoded())
743+
#else
744+
if (inBase->getStoreType()==hx::arrayInt)
745+
#endif
746+
return joinInts(inBase, inSeparator);
747+
688748
Array<String> stringArray = Array_obj<String>::__new(length, length);
689749
_hx_toString_depth++;
690750
try

‎src/hx/Date.cpp‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -594,7 +594,8 @@ double __hxcpp_date_now()
594594
ularge.LowPart = file_time.dwLowDateTime;
595595
ularge.HighPart = file_time.dwHighDateTime;
596596

597-
return (double)( (long) ((ularge.QuadPart - EPOCH) / 10000000L) ) +
597+
// A long is 32 bits on Windows, x64 too, and the seconds pass it in 2038
598+
return (double)( (__int64) ((ularge.QuadPart - EPOCH) / 10000000L) ) +
598599
system_time.wMilliseconds*0.001;
599600
#elif defined(USE_GETTIMEOFDAY)
600601
struct timeval tv;

‎src/hx/gc/Immix.cpp‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4848,7 +4848,12 @@ class GlobalAllocator
48484848

48494849
// After zero/reclaim, it might be that the hole size is smaller than we thought.
48504850
if (result->mMaxHoleSize>=inRequiredBytes)
4851+
{
4852+
// Assume all will be used, as below: a block taken here was
4853+
// missing from the current memory until the next collection
4854+
mCurrentRowsInUse += result->GetFreeRows();
48514855
return result;
4856+
}
48524857

48534858
repoolReclaimedBlock(result);
48544859
continue;
@@ -7672,7 +7677,8 @@ class GlobalAllocator
76727677

76737678

76747679
size_t mRowsInUse;
7675-
size_t mCurrentRowsInUse;
7680+
// Threads take free blocks without a lock, and count them here
7681+
std::atomic_size_t mCurrentRowsInUse;
76767682
std::atomic_size_t mLargeAllocSpace;
76777683
size_t mLargeAllocForceRefresh;
76787684
// Read unlocked to decide on a collection, changed under mLargeListLock
Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
#pragma once
2+
3+
#if defined(_WIN32) && defined(_MSC_VER)
4+
#ifndef WIN32_LEAN_AND_MEAN
5+
#define WIN32_LEAN_AND_MEAN
6+
#endif
7+
#ifndef NOMINMAX
8+
#define NOMINMAX
9+
#endif
10+
#include <windows.h>
11+
12+
static inline void WINAPI hxcppTestFutureSystemTime(LPSYSTEMTIME result)
13+
{
14+
SYSTEMTIME value = {};
15+
value.wYear = 2040;
16+
value.wMonth = 1;
17+
value.wDay = 1;
18+
value.wMilliseconds = 125;
19+
*result = value;
20+
}
21+
22+
#define GetSystemTime hxcppTestFutureSystemTime
23+
#define HXCPP_TEST_FUTURE_CLOCK 1
24+
#endif
Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
@:buildXml('<files id="runtime"><compilerflag value="/FI${HXCPP}/test/regression/DateNowAfter2038/FutureClock.h" if="isMsvc"/></files><files id="haxe"><compilerflag value="/FI${HXCPP}/test/regression/DateNowAfter2038/FutureClock.h" if="isMsvc"/></files>')
2+
@:cppFileCode('
3+
static bool futureClockActive() {
4+
#ifdef HXCPP_TEST_FUTURE_CLOCK
5+
return true;
6+
#else
7+
return false;
8+
#endif
9+
}
10+
')
11+
class Main {
12+
static function main() {
13+
var exact = true;
14+
var year = true;
15+
var control = true;
16+
if (untyped __cpp__("futureClockActive()")) {
17+
var expected = 2208988800125.0;
18+
var date = Date.now();
19+
exact = date.getTime() == expected;
20+
year = date.getUTCFullYear() == 2040;
21+
control = Date.fromTime(expected).getUTCFullYear() == 2040;
22+
if (Sys.args().length > 0)
23+
Sys.println(haxe.Json.stringify({actual: date.getTime(), expected: expected, utcYear: date.getUTCFullYear()}));
24+
}
25+
Sys.println("Date.now preserves the simulated 2040 timestamp: " + exact);
26+
Sys.println("Date.now returns the simulated UTC year: " + year);
27+
Sys.println("explicit post-2038 dates remain correct: " + control);
28+
}
29+
}
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
-m Main
2+
--cpp bin
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
Date.now preserves the simulated 2040 timestamp: true
2+
Date.now returns the simulated UTC year: true
3+
explicit post-2038 dates remain correct: true
Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
#if !emscripten
2+
@:cppFileCode('
3+
#include <thread>
4+
#include <hx/Native.h>
5+
static void runJoined(Dynamic callback) {
6+
std::thread worker([callback]() mutable { hx::NativeAttach attach; callback(); });
7+
hx::AutoGCFreeZone freeZone;
8+
worker.join();
9+
}
10+
')
11+
#end
12+
class Main {
13+
static var arrays:Array<Array<Int>>;
14+
static function allocate():Void {
15+
arrays = [];
16+
for (i in 0...2048) {
17+
var values:Array<Int> = [];
18+
values.resize(256);
19+
values[0] = i;
20+
values[255] = -i;
21+
arrays.push(values);
22+
}
23+
}
24+
static function main() {
25+
var current = true;
26+
var collected = true;
27+
var intact = true;
28+
#if !emscripten
29+
for (_ in 0...3) cpp.vm.Gc.run(true);
30+
var before = cpp.vm.Gc.memInfo64(cpp.vm.Gc.MEM_INFO_CURRENT);
31+
var liveBefore = cpp.vm.Gc.memInfo64(4);
32+
cpp.vm.Gc.enable(false);
33+
untyped __cpp__("runJoined({0})", allocate);
34+
var after = cpp.vm.Gc.memInfo64(cpp.vm.Gc.MEM_INFO_CURRENT);
35+
cpp.vm.Gc.enable(true);
36+
var payload = 2048.0 * 256 * 4;
37+
current = after - before >= payload;
38+
cpp.vm.Gc.run(true);
39+
var liveAfter = cpp.vm.Gc.memInfo64(4);
40+
var refreshed = cpp.vm.Gc.memInfo64(cpp.vm.Gc.MEM_INFO_CURRENT);
41+
collected = liveAfter - liveBefore >= payload && refreshed - before >= payload;
42+
for (i in 0...arrays.length)
43+
if (arrays[i][0] != i || arrays[i][255] != -i) intact = false;
44+
if (Sys.args().length > 0)
45+
Sys.println(haxe.Json.stringify({payload: payload, before: before, after: after, afterCollection: refreshed, liveGrowth: liveAfter - liveBefore}));
46+
#end
47+
Sys.println("current heap estimate includes the newly allocated payload: " + current);
48+
Sys.println("full collection accounts for the retained payload: " + collected);
49+
Sys.println("all retained arrays remain intact: " + intact);
50+
}
51+
}
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
-m Main
2+
--cpp bin

0 commit comments

Comments
 (0)