Skip to content

Specialize on callbacks that are only passed on - #206

Open
pankgeorg wants to merge 2 commits into
JuliaIO:masterfrom
pankgeorg:trim-specialize-callbacks
Open

pankgeorg wants to merge 2 commits into
JuliaIO:masterfrom
pankgeorg:trim-specialize-callbacks

Conversation

@pankgeorg

@pankgeorg pankgeorg commented Oct 29, 2026 •

Copy link
Copy Markdown

This removes two of the blockers to using Tar in programs compiled with juliac --trim=safe. With JuliaIO/ArgTools.jl#46 and a Base change (below), reading and extracting tarballs verifies cleanly; the change is correct and tested on its own.

Specialize on callbacks that are only passed on. The API functions and their helpers take the predicate or callback as f::Function and pass it on: to the next function, to a do block, or, for every method with keyword arguments, to the method's keyword body. Julia does not specialize a method on a Function argument it only passes on (performance tips), so these methods are compiled for f::Function and calling the callback is a dynamic dispatch. After calling create, list, extract, rewrite and tree_hash with closures, 31 method instances of Tar have Function in their signature on master; with this change, none. The callbacks are now typed f::F where {F <: Function}; the signatures, and so dispatch, are unchanged.

Take a header error's message as the String it is. read_standard_header converted the message of the ErrorException thrown by its header checks with String(err.msg), a dynamic call (the field is an AbstractString). Both checks throw through header_error, which calls error with a String, so the message is asserted to be one instead.

Effect on juliac --trim=safe (JuliaC 0.3.12, Julia master 0d77de6d2c). A program that calls Tar.list (with and without a callback) and Tar.extract with a predicate on a tarball path gets 0 verifier errors with this change, JuliaIO/ArgTools.jl#46 and JuliaLang/julia#64448, so that show of a Symbol does not compile the expression printer (check_header shows a Symbol through repr(hdr)), and 264 with only the first two. On master Tar.list(path) stops at ArgTools (2 errors in open_nolock). Adding create, tree_hash and rewrite to that program leaves 8 errors, all in create, all from repr of a Dict in write_tarball's error message, which is Base's show for Pairs and not changed here.

Cost. More methods are compiled per callback type. First calls in a fresh process, in this order (median of 5, master; this change vs. without): with the default predicate create 167 ms vs. 168, list 137 vs. 151, extract 146 vs. 165, tree_hash 107 vs. 112, rewrite 71 vs. 74; then with new user closures create(pred) 14 vs. 14 and extract(pred) 55 vs. 27, the extract path being compiled for the new closure type.

Tests. A new testset calls the API with closures in a fresh Julia process and checks that no Tar method was compiled with Function in its signature (a fresh process, since inference and reflection elsewhere can compile such instances); it fails on master, listing them. The full test suite passes on Julia 1.13.2 and on master.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HN2L2VxELK7cEP7rLbYHoC

pankgeorg and others added 2 commits October 29, 2026 09:53
The API functions and their helpers take the predicate or callback as
`f::Function` and pass it on: to the next function, to a `do` block, or,
for every method with keyword arguments, to the method's keyword body.
Julia does not specialize a method on a `Function` argument it only
passes on, so these methods are compiled for `f::Function` and calling
the callback is a dynamic dispatch. Under `juliac --trim=safe` this
makes every `Tar.create`, `list`, `extract`, `rewrite` and `tree_hash`
fail verification (together with ArgTools' `arg_read` / `arg_write`,
which have the same problem).

Type the callbacks as `f::F where {F <: Function}`. The signatures, and
so dispatch, are unchanged; the methods are compiled for the callback's
type.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`read_standard_header` rewrites the message of the `ErrorException` that
`check_version_field` or `check_checksum_field` throws. It converted the
message with `String(err.msg)`, but `ErrorException`'s `msg` field is an
`AbstractString`, so that is a dynamic call, which `juliac --trim=safe`
rejects. Both checks throw through `header_error`, which calls `error`
with a `String` (built by `sprint`): assert that instead of converting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@pankgeorg
pankgeorg marked this pull request as ready for review October 29, 2026 18:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant