[harelb]

v2, after review. The indexer now lives in the Hydra python package as hydra_python.image_store, with its CLI under hydra image-store. Images are read and written with imageio, and hydra_python imports without the compiled bindings (it warns), so pip install hydra/ works without a Hydra build. History is rebuilt on current develop. No C++ changes.

Hydra (#199) and Khronos (MIT-SPARK/Khronos#48) only log plain files. hydra_python.image_store builds one SQLite store per robot from those files, after the run or on first open, and serves the read API.

What this does

Inputs	Keyframe InputData archives (<keyframes>/<ts>.zip), Khronos track logs (<tracks>/<id>/track.json + kept images), frontend/track_bindings.json, backend/merges.json, frontend/backend dsg.json (agent nodes). Contract: image_store/FORMAT.md.
Indexing	ImageStore.open(run) indexes a run that was never indexed, and re-indexes when the raw files changed (fingerprint in meta, under a lock file). hydra image-store index RUN does it explicitly. Re-indexing is additive. An unreadable archive is skipped with one warning and read again only when its mtime or size changes. Track ids from EXTERNAL_TRACK_BASE (1 << 40) up belong to other writers and are never touched.
Frames	Every archive becomes a keyframe (frames.kind 0). Observations go on the keyframe of the same sensor and stamp, else on an observation frame (kind 2). Readers reject any other kind value with a clear error.
Poses	Every frame is anchored to the nearer in time of the agent nodes before and after it. anchor_T_body comes from odometry, and frame_pose composes it with the optimized agent pose, so poses follow loop closures.
Read API	observations(node) follows merges. Also frames_for_agent, agent_frame, frames, frame_pose, calib, mask, canonical, members, unbound_tracks, descriptor(s). schema_version 1.
CLI	`hydra image-store index

Review notes addressed

• In the hydra python package, not a top-level directory.
  • Code: hydra/python/src/hydra_python/image_store/, with README and FORMAT next to it.
  • Tests: hydra/python/tests/image_store/.
  • The colcon install (HYDRA_ENABLE_PYTHON=ON) and the scikit-build wheel both ship the directory as is, schema.sql included.
  • No new dependencies: numpy, pyyaml, imageio and click are already hydra_python dependencies.
• imageio instead of PIL + OpenCV.
  • PNG/JPEG decode and encode, image headers and mask export all use imageio.v3.
  • Depth EXR is read by a small numpy reader covering everything InputData::save writes (none / RLE / ZIP). No imageio backend reads EXR out of the box (the opencv plugin needs OPENCV_IO_ENABLE_OPENEXR before cv2 is imported; freeimage needs a binary download).
  • Other EXR encodings fall back to imageio's opencv plugin.
  • OpenCV is a test-only dependency, used to write archives exactly as input_data_io.cpp does.
• pip install / warn when bindings are missing (separate first commit).
  • import hydra_python without _hydra_bindings warns once (BindingsNotBuiltWarning; hydra_python.BINDINGS_ERROR holds the reason) and skips the binding-dependent modules.
  • A top-level CMake build that cannot find hydra skips the _hydra_bindings target instead of failing.
  • colcon builds are unchanged.

Commits

1. feat(hydra_python): import without the compiled bindings
2. feat(image_store): store reader/writer, codecs and the store schema
3. feat(image_store): read Hydra InputData keyframe archives
4. feat(image_store): indexer for Hydra/Khronos image logs
5. feat(image_store): hydra image-store CLI, check and export
6. test(image_store): store, indexer and CLI tests, including files from the C++ writers
7. docs(image_store): README and the image log format

Tests

• cd hydra/python && python -m pytest tests/image_store: 85 passed.
  • Includes no-OpenCV and no-bindings subprocess tests, the EXR fallback, and the hydra image-store entry.
  • tests/image_store/data/real_cpp holds files written by the Hydra/Khronos gtests themselves (regenerate.sh).
• pre-commit passes.
• pip install --no-deps hydra/ in a clean venv without Hydra: the CMake warning is printed, the wheel contains hydra_python/image_store (with schema.sql), and import hydra_python.image_store works with the warning.

Evidence

Indexing the same raw Penn runs (Spot, courtyard) with the v1 package and with this one:

• Stores are equal: frames with anchors and poses, observations, bindings, aliases, unbound/pending lists.
• Keyframe JPEGs are byte-identical.
• Depth PNGs decode to identical arrays; their bytes differ (zlib settings; level 1 is used, OpenCV's default).
• check --dsg passes.

Live 60 s and 300 s windows match the earlier runs (numbers in the follow-up notes). Earlier evidence for the format (9 full sessions, LC windows, reprojection) is unchanged, since the store content is identical.

Notes

• Node and track ids restart with every Hydra process, so the indexer expects one run directory per Hydra session.
• Follow-up (stacked, separate): hydra image-store tail, which indexes live from Hydra's event logs.

Related PRs

• Save keyframe archives, track bindings and backend merges for image lookup #199: keyframes, bindings, merges (C++)
• Log object tracks for image indexing (track logger) Khronos#48: track logger (C++)
• Backend deformation and merge fixes; process the last frontend packet on stop #219: general backend fixes (independent)

[nathanhhughes]

At a high-level, it would be nice if this ended up in the hydra python bindings / package instead of at the top-level directory (and used imageio for reading images instead of a mix of PIL and a random opencv flavor). I would be okay with making it so that it's possible to pip install hydra and just having hydra warn on import that the bindings aren't built (but also I have it set up now that the python bindings are available when you source your colcon workspace so you might not actually need to). Neither of these are super necessary though.

Will tag Multy as a reviewer to take a second look, but this probably doesn't need a super careful review

[harelb]

Thanks! I did both:

• Moved into the python package. It is now hydra_python.image_store (hydra/python/src/hydra_python/image_store), with tests under hydra/python/tests/image_store. The CLI is a click group on the existing hydra CLI: hydra image-store index|check|info|export. python -m hydra_python.image_store ... runs the same commands without importing the rest of hydra (the timing command pulls in matplotlib/seaborn). The top-level hydra_image_store/ directory and its pyproject.toml are gone. No new dependencies: it needs numpy, pyyaml, imageio and click, which hydra_python already lists.
• Bindings optional. import hydra_python no longer fails without _hydra_bindings. It warns once (BindingsNotBuiltWarning) and skips the modules that need the bindings or their heavy deps (dataloaders/trajectory, pipeline, simulators). hydra_python.BINDINGS_ERROR holds the reason. In python/CMakeLists.txt, a top-level (pip/scikit-build) build that cannot find hydra now skips the _hydra_bindings target with a CMake warning instead of failing. So pip install hydra/ without a Hydra build gives the pure-python package; I checked this in a clean venv. colcon builds are unchanged. This is its own commit (feat(hydra_python): import without the compiled bindings) in case you'd rather take it separately or not at all. The image store never touches the bindings, so it also works from a sourced workspace or with hydra/python/src on PYTHONPATH.
• imageio instead of PIL/OpenCV. All PNG/JPEG reads and writes go through imageio.v3. EXR is the one exception. No imageio backend reads it out of the box: pillow/tifffile can't, freeimage needs a binary download, and the opencv plugin has the same OPENCV_IO_ENABLE_OPENEXR problem (and returns 8-bit unless flags=-1). So depth EXR is still read by the small numpy reader. It covers everything InputData::save writes (none/RLE/ZIP), and only other encodings fall back to imageio's opencv plugin. OpenCV is now only a test dependency, used to write archives the way input_data_io.cpp does. Output is unchanged on real Penn data: the keyframe JPEGs are byte-identical to the old OpenCV path. The 16-bit depth PNGs decode to identical arrays but differ in bytes (zlib settings; I use level 1, OpenCV's default).

History is rebuilt as clean commits on develop. Unknown frame kinds are rejected as plain input validation. The tail mode (following a running Hydra through its event logs) is a separate follow-up stacked on this one, as a draft in #221 (hydra image-store tail).

[MultyXu]

LGTM! I think to me the functionality make sense.

[nathanhhughes]

Last quick comment: I like openexr as a depth format (it helped with saving float32 images if I remember correctly), but also okay with switching the serialization to tiff if it makes this simpler (idk if it's that important to make that switch now, but would save us in rebuilding image stores later)