feat: spool large request bodies to disk and map them

[martin-rueegg]

Summary

Imaginary reads a request body whole with ioutil.ReadAll, which grows its slice by reallocation:
the heap reaches 2–5 × the body before libvips sees any of it. With a memory limit, as in Nextcloud
AIO's Imaginary container, large images and PDFs get the process OOM-killed.

This PR writes a body larger than 4 MiB (configurable) to a temporary file and maps it read-only. libvips still
gets a []byte, so bimg is unchanged, but the pages are backed by the file and can be reclaimed by
the kernel.

Changes

• readSpooled (spool.go):
  • Up to 4 MiB (-spool-threshold), it reads into memory as before.
  • Beyond that, it writes a temporary file, unlinks it at once (nothing is left behind, even after a
    crash), and maps it with PROT_READ/MAP_SHARED.
• ReleasingImageSource: an optional interface for sources whose buffer must be released.
  imageController releases it once the response is written. GetImage still exists, and returns a
  copy.
• Multipart: a part that net/http has already written to disk (larger than 64 MiB) is mapped
  instead of read back into memory.
• -spool-threshold <bytes>: a new flag for the size above which a body is spooled. The
  default is 4 MiB; 0 spools every body, and a negative value is refused at start.
• -tmp-dir <path>: a new flag for where the temporary files go. It defaults to the system's
  temporary directory.
• -mount is unchanged on purpose:
  • Its files belong to someone else. A mapped file that is truncated while libvips reads it raises
    SIGBUS and takes the whole server down.
  • ReadFile sizes its buffer from stat, so it costs about 1× the file, not ReadAll's 2–5×.
• libvips' operation cache is switched off (VipsCacheSetMax(0), VipsCacheSetMaxMem(0)):
  • libvips keys a loader's cached result by the address of the image buffer.
  • Once a buffer is unmapped, or freed by the GC, a later buffer at the same address could be served
    a result that is not its own.
  • Dropping the cache per request instead (vips_cache_drop_all) is not an option. In libvips 8.18
    it frees the cache's hash table, after which every cached operation leaks (GLib-CRITICAL
    hash_table != NULL). We measured about 60 MB per request.
• Windows: mmap_windows.go reads the file instead of mapping it.

Measurements

Setup: Nextcloud AIO's Imaginary image (Alpine 3.24, libvips 8.18.2), with only the binary
rebuilt from this branch. Memory limit 2500 MB, 2 CPUs, the temporary directory on ZFS, and a
container restart before every request.

The figures are the peak of anonymous memory during the request (mapped file pages not included)
and the time:

Input	Pipeline	master	this PR
JPEG 200 MP, 55 MB	fit	160 MB, 2.2 s	50 MB, 2.3 s
JPEG 200 MP, orientation 6	fit → autorotate → fit	209 MB, 2.6 s	84 MB, 2.6 s
LZW TIFF 200 MP, 389 MB	fit	928 MB, 6.9 s	73 MB, 6.7 s
LZW TIFF 200 MP, 389 MB	fit → autorotate → fit	967 MB, 7.8 s	101 MB, 7.1 s
LZW TIFF 200 MP, 389 MB	autorotate → fit	1998 MB, 8.6 s	1134 MB, 8.6 s
PNG 100 MP, 199 MB	fit	447 MB, 7.0 s	45 MB, 7.3 s
uncompressed TIFF 761 MB, 266 MP	fit	1889 MB, 5.8 s	67 MB, 6.0 s
uncompressed TIFF 761 MB, 266 MP	fit → autorotate → fit	1916 MB, 6.5 s	90 MB, 6.1 s
uncompressed TIFF 761 MB, 266 MP	autorotate → fit	2450 MB, 10.5 s	1510 MB, 8.9 s
PDF 1.7 GB, 1817 pages	convert → fit	OOM-killed after 27 s	1659 MB, 6.0 s

Further results:

• Output: identical in dimensions and orientation for every input. Times are unchanged within
  noise, one run per cell.
• Leak check: the same request ten times without a restart stays flat at 28–32 MB.

Not in this PR:

• -max-allowed-size is not applied to bodies. It limits the URL source today, and applying it
  here would change its meaning.

Tests

• 8 new tests in spool_test.go:
  • small, large and threshold-sized bodies, and the threshold itself;
  • unlinking, mapping, and the body source;
  • the controller with a large body.
• go vet ./... is clean, and go test ./... passes in golang:1.27.1-alpine3.24 with libvips 8.18.
• Windows: the test binary was cross-compiled with mingw-w64 against libvips 8.18.7's official
  Windows build, and run on Windows. The full suite passes, the new tests included, through
  mmap_windows.go and its fallback.
  • That build omits libvips' deprecated API, so bimg v1.1.7 needs vips/vips7compat.h, which isn't
    there. For the test build, a stand-in header gave it an empty im__print_all, the only function
    bimg uses from it. That is bimg's matter, not this PR's.

Context

This came out of Nextcloud previews of large archive files. The Nextcloud side, which shrinks before
rotating, is nextcloud/server#65483. Nextcloud AIO builds Imaginary from this repository at
6a274b4.

Written with AI assistance (Claude), which did the measurements, the code and the tests. They were
reviewed and the PR submitted by the author.