Skip to content

Fix 32-bit line buffer size overflow in ppm_in, yuv_out and raw_out - #424

Open
naaz234 wants to merge 1 commit into
aous72:masterfrom
naaz234:img-io-size-t-buffers
Open

naaz234 wants to merge 1 commit into
aous72:masterfrom
naaz234:img-io-size-t-buffers

Conversation

@naaz234

@naaz234 naaz234 commented Nov 15, 2026

Copy link
Copy Markdown

ASan, ojph_compress -i wide.pgm -o out.j2c -num_decomps 0 -tile_size {1073741824,1} on a 278-byte PGM whose header is P5 2147483649 1 65535:

ERROR: AddressSanitizer: heap-buffer-overflow
WRITE of size 256 at 0x602000000952 thread T0
    #4 ojph::ppm_in::read(ojph::line_buf const*, unsigned int) ojph_img_io.cpp:353
0x602000000952 is located 0 bytes after 2-byte region
    #1 ojph::ppm_in::open(char const*) ojph_img_io.cpp:317

ppm_in::open sizes temp_buf as num_comps * width * bytes_per_sample in ui32, so 1 * 2147483649 * 2 wraps to 2, while fread is asked for num_ele_per_line samples and copies the rest of the file into the 2-byte buffer. For P6 the same wrap in num_ele_per_line also makes the de-interleave loop read past the buffer.

yuv_out::configure and raw_out::configure have the same ui32 product, with width taken from the SIZ marker of the codestream being decoded. A 1x1 16-bit codestream with Xsiz patched to 0x80000001 (XTsiz 0x40000001) decoded to .raw or .yuv with ojph_expand gets a 2-byte buffer and overflows in raw_out::write (ojph_img_io.cpp:1795) / yuv_out::write (ojph_img_io.cpp:1501).

The sizes are now computed in size_t and stored in size_t members, matching pfm_in, ppm_out and tif_out. With the patch, the PGM case fails with "not enough data in file", and both ojph_expand cases write the full 4294967298-byte line. Release build with -DOJPH_BUILD_TESTS=ON: ctest 138/138 passing. There is no regression test because reaching these widths needs multi-GB allocations, which is not practical in CI.

ppm_in::open, yuv_out::configure and raw_out::configure multiplied the
image width by the number of components and/or bytes per sample in
32-bit arithmetic. The width comes from the PGM/PPM header or from the
SIZ marker of the codestream being decoded, so a width above 2^31 wraps
the product, a few bytes are allocated, and the following fread() or
sample loop writes a full line past the end of the buffer.

Do the arithmetic in size_t and keep the sizes in size_t members, as
pfm_in, ppm_out and tif_out already do.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant