ImHex Pattern Language (.hexpat) implementation in TypeScript, shipped as a
host-independent library plus an r2js plugin.
--pancake
You need radare2 and r2frida (for the r2frida-compile program) installed.
make
make user-install
r2 -q -i hexpat.r2.js -c 'hexpat demo/test.hexpat' /bin/ls
Commands (all output goes through r2's console, so ~grep and | work):
| command | description |
|---|---|
hexpat [-q] [-I dir] [file] |
evaluate file (replacing the loaded ones) and show its patterns; without file show the loaded ones |
hexpat+ [-I dir] file |
evaluate file and add it to the loaded ones |
hexpat- [n] |
unload all loaded files (or the nth) |
hexpat-v |
show the r2hexpat version |
hexpatj [file] |
patterns as JSON (hexpatj~{} to indent) |
hexpat* [file] |
patterns as r2 commands: flags in the hexpat flagspace, flag colors/comments, Cd/Cs data hints and value comments (.hexpat* applies them) |
hexpat-* |
r2 commands to remove generated flags (.hexpat-* applies them) |
hexpatl[j] |
list loaded files and their top-level patterns |
hexpate expr |
evaluate an expression or assignment in the last loaded file (hexpate sizeof(header), hexpate header.value = 3) |
hexpatp[?+=-ijl] ... |
build partial instances and masked search patterns (hexpatp? for help) |
hexpatm [options] Type |
print a libmagic header decoder (hexpatm? for help) |
hexpatt[j*] [options] [Type [address]|pattern] |
view a type at an address or a loaded pattern as a tree, JSON or r2 commands (hexpatt? for help) |
hexpats [file] |
one-line summary (types with their field count, enums, functions, patterns, size) or the error, meant for status bars |
hexpat? |
help |
Any file argument can be base64:<data> to evaluate the encoded source
directly, so frontends (like iaito) can feed the contents of an editor on every
change and show hexpats base64:... in a status bar for live feedback.
Addresses are r2 addresses: pattern offsets are translated by the binary's
base address (ij.bin.baddr), or #pragma base_address 0x....
hexpate header.value = 3 writes the field's bytes through r2 I/O, using its
declared size and byte order. Compound assignments (+=, |=, etc.), array
entries and bitfield members also work. Open r2 with -w, or enable
e io.cache=true to stage edits in r2's write cache. Assignments print nothing;
query the field again or use p8 to inspect the result. Editing a count or
pointer does not rebuild the pattern layout; reload the pattern file when
its layout changes. Fields with [[transform(...)]] cannot be written because
the transform has no inverse.
Quote the whole r2 command when it contains operators that r2 handles itself,
for example "hexpate header.value |= 8" or "hexpate header.value >= 3".
hexpatt displays a type's fields as a tree with addresses, values, sizes
and evaluated attributes, including color, comment and name:
hexpatt -f types.hexpat Header @ 0x100
hexpat -q types.hexpat
hexpatt Header 0x100
hexpatt header
hexpatt header.inner
hexpatt
A type uses the current r2 seek when no address is given; an explicit address
can be an r2 expression or flag name. -f file supplies types without loading
the file's placements, executing its top-level statements or calling main().
It accepts base64: sources and -I dir for includes. Without -f, types come
from the last loaded file. Type decoding follows ordinary hexpat declarations,
including data-dependent arrays and pointers, and honors the binary base
address or #pragma base_address. These one-shot views preserve loaded files
and search instances. A loaded pattern name or nested path shows its existing
tree; no arguments shows all loaded patterns.
Rows display their color attributes even with e scr.color=0; enabling colors
also colors the rows. Enclosing aggregate colors apply to members that have no
explicit color of their own. hexpattj and hexpatj expose color, comment,
displayName and evaluated attributes in JSON.
hexpat* and hexpatt* emit fc flag=rgb:RRGGBB for colored flags and fC
for pattern comments. The runtime's std::core::set_pattern_color is supported
too. Apply a type's flags directly with .hexpatt* -f types.hexpat Header @ 0x100.
hexpat-* prints a removal script; .hexpat-* applies it. This removes flags
matching hexpat.* in the hexpat flagspace, including their flag metadata,
and restores the previous flagspace. It works after the patterns are unloaded
and preserves other flags, loaded files and search instances. Address comments
and data hints remain. hexpat- (or hexpat- *, with a space) still unloads
the pattern files.
hexpatp instantiates a type without a backing address. All fields start
unset. Set only the fields you know; the rest, including padding, become
wildcards. Its normal output is a single bytes:mask argument for r2's /x,
including bit masks for individual bitfield members:
r2 -q -i hexpat.r2.js \
-c '/x `hexpatp -f demo/player.hexpat -a 8 Player health=100 lives=3`' target.bin-f compiles types and constant globals from a file, resolving includes with
-I dir. It does not decode placements or execute the file's top-level code
or main(). Without -f, the last file loaded with hexpat supplies the types.
The file can also be base64:<data>. One-shot patterns do not create a named
instance or change r2 I/O.
For interactive use, create a named instance and refine it:
hexpatp+ -f demo/player.hexpat -a 8 wanted Player
hexpatpi wanted
hexpatp= wanted health=100 lives=3 role=Role::Rogue position.z=3
hexpatp wanted
/x `hexpatp wanted`
hexpatp- wanted role position
hexpatpi wanted
hexpatpj wanted
hexpatpl
hexpatp- wanted
| command | result |
|---|---|
hexpatp [options] Type [field=expr ...] |
one-shot /x pattern |
hexpatp name |
named instance's /x pattern |
hexpatp+ [options] name Type [field=expr ...] |
create a named instance |
hexpatp= name field=expr ... |
set fields atomically |
hexpatp- name field ... |
unset fields or entire nested aggregates |
hexpatp- name / hexpatp- * |
delete one / all named instances |
hexpatpi name / hexpatpj name |
inspect fields as text / JSON |
hexpatpl / hexpatplj |
list named instances as text / JSON |
Options go before the type or instance name: -f file, -I dir, and
-a N (also --align N). Nested paths and constant array indices work, e.g.
position.z=3 and values[2]=0x1234. Values are hexpat expressions, including
enum names, arithmetic, booleans, and quoted strings with spaces. Multiple
assignments can be separated by whitespace or commas. A zero value is set;
it constrains zero bytes with ff mask bytes. An unset field contributes no
mask bits. JSON reports defined for each leaf and unset, partial or set
for the instance. Overlapping union fields are independent constraints;
conflicting values are rejected, and unsetting one preserves the others.
Quote the whole r2 command when it contains r2 operators or template brackets,
for example "hexpatp -f types.hexpat Array<u16,2> values[1]=1|2".
The default layout is packed (-a 1). -a 2, 4, 8 or 16 aligns each
member to its natural alignment, capped at that number, and includes struct
tail padding in array strides. Power-of-two scalar widths use their width
as their natural alignment; other widths and ImHex bitfields align to one
byte. [[packed]] on a type and [[align(N)]] on a member override this policy.
Explicit padding[N] and aggregate [[fixed_size(N)]] remain wildcarded.
Pointer widths stay explicit (Player *next : u64; in the example).
This is an explicit layout policy: it does not infer a C ABI or implement C
bitfield packing. Files with a non-packed abi pragma require an explicit -a selection.
Choose the layout that matches the target; r2's search.align separately
controls candidate addresses (e.g. e search.align=8). For background, see
Frida's packed and native layout example.
Search types must have fixed, contiguous layouts. Arrays sized by an unset
field, variable-length arrays, conditionals, address-dependent layouts, and
transformed values are rejected instead of guessing their layout. Setting a
length field does not make a dynamic type static. hexpat and hexpate still
decode and modify ordinary placed patterns as before.
hexpatm exports a fixed type as a text magic file for r2's pm or file -m:
hexpatm -f demo/test.hexpat MachHeader > /tmp/macho.magic
pm /tmp/macho.magic @ 0
Without -f, it uses the types from the last file loaded with hexpat.
It accepts the same -f file, -I dir, -a N / --align N options as
hexpatp, including base64: sources, and compiles types without decoding
placements or running top-level statements or main(). Exporting leaves
the loaded files and search instances intact.
The decoder prints nested field paths, fixed array entries and all union
views at their declared offsets and byte order. Integers, enums, booleans,
char16 scalars and pointer addresses print their raw hexadecimal bits;
float and double print numeric values. Fixed char arrays and char
scalars print bounded strings; char16 arrays use libmagic's string16 types.
Padding is omitted. Each field uses an unconditional x test: this decodes
the selected header type rather than identifying its file format by a
signature. The first rule checks that the whole fixed header is available.
Increase r2's block size (b N) when the header exceeds the current block.
Dynamic layouts, transformed fields, bitfields, integer/pointer widths other
than 1, 2, 4 or 8 bytes, and float16 are rejected. Use ordinary hexpat
evaluation for those types. Library callers can export the same decoder
with new SearchPattern(program, "MachHeader").toMagic(); any assigned
search values are ignored.
The library in lib/ never talks to radare2. Embedders implement
HexpatHost (lib/host.ts) to provide the data and resolve includes:
import { runHexpat, HexpatHost } from "./lib/index";
const host: HexpatHost = {
size: () => data.length,
read: (addr, size) => data.subarray(addr, addr + size),
print: (msg) => console.log(msg), // std::print output
resolve: (path, from) => undefined, // #include / import sources
};
const inst = runHexpat(src, host);
inst.dump();toJson(inst) serializes the placed patterns, inst.evaluateExpression(src)
evaluates an expression or assignment in the global scope of an evaluated pattern and
BufferHost is a ready-made host over a Uint8Array. HostHooks adds
functions, types, pragma handlers and in variables, like the C++ runtime's
addFunction/addType/addPragma.
decodeType(program, "Header", host, offset) decodes one type at a host-relative
offset after preparing types and constants only. Its root pattern is named
value (with underscores added if a constant already uses that name).
dumpTree(inst, print, { base, color, maxEntries }) renders a tree
with metadata; patterns can select specific roots.
Hosts can implement write(addr, bytes) for interactive field assignments;
read-only hosts reject those writes. Assignments while evaluating a pattern
file retain ImHex's value overrides and do not write host data.
SearchPattern exposes the same partial instances without r2:
import { compile, SearchPattern } from "./lib/index";
const program = compile(source); // optional host resolves includes
const wanted = new SearchPattern(program, "Player", { alignment: 8 });
wanted.set({ health: 100, lives: 3, position: { z: 3 } });
wanted.isDefined("role"); // false
wanted.get("role"); // undefined
wanted.set("health", 0); // a defined zero
wanted.assign("role=Role::Rogue");
wanted.unset("position"); // clear all position fields
wanted.state(); // "partial"
const { bytes, mask } = wanted.encode(); // independent Uint8Array copies
console.log(wanted.toR2());
console.log(JSON.stringify(wanted));set() merges partial objects and arrays, or sets a single field path;
unset() with no argument clears the whole instance. state(path) reports
aggregate state, fields() returns field offsets and definition status, and
get(path) returns undefined for unset leaves. Integer values are decoded
as bigint; supply bigint for values beyond JavaScript's safe integer range.
A pointer set to null is a defined zero pointer. Strings fill their fixed
field and pad with zero bytes. SearchOptions also accepts maxSize (default
1 MiB) and maxFields (default 65536). The library reads no target bytes while
creating or editing a search instance.
make check # tsc type-check (lint)
make test # everything below
make test-unit # mock-host unit test
make test-upstream # upstream conformance suite under node
make test-r2 # r2 golden tests: tests/*.hexpat vs .golden on tests/sample.bin
make test-upstream-r2 # upstream suite inside r2's QuickJS (one r2 process)
make test-corpus CORPUS=path/to/ImHex-Patterns parses the whole
ImHex-Patterns corpus and
evaluates every pattern that ships sample data.
The conformance suite (tests/upstream/) holds the tests of the reference
C++ implementation, extracted by tests/extract_upstream.py from
third_party/parser_cpp and run against its test_data. Tests expected to
fail are listed in tests/upstream/XFAIL; make test-upstream ARGS=-v
shows errors and ARGS=-u regenerates the list. sh tests/run_r2.sh -u
regenerates the r2 goldens.