Repository navigation
Release 2.11.0 - #1273
Merged
Merged
Release 2.11.0#1273
Conversation
Bumps [smol-toml](https://github.com/squirrelchat/smol-toml) from 1.8.0 to 1.9.1. - [Release notes](https://github.com/squirrelchat/smol-toml/releases) - [Commits](squirrelchat/smol-toml@v1.8.0...v1.9.1) --- updated-dependencies: - dependency-name: smol-toml dependency-version: 1.9.1 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps rust from 1.98-bookworm to 1.99-bookworm. --- updated-dependencies: - dependency-name: rust dependency-version: 1.99-bookworm dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>
#1262 (Docker Rust 1.99) was still open when 2.10.0 was tagged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016brRcptooQkxWr6TyWnwAE
chore: sync master (v2.10.0) back into develop
Selective alignment kept refseq.bin on the heap at one byte per base, the largest single allocation of a run, though mapping only reads short windows of it. It now packs the file on load and decodes just the gaps and flanks it aligns; MEM extension compares a word of bases at a time. Bytes two bits cannot hold (N, IUPAC, lowercase) are kept as sorted runs. The bias models still read whole transcripts, so --seqBias and --gcBias keep bytes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bootstrap and Gibbs replicates are conditional on one library's equivalence classes, so they can understate the variance between technical replicate libraries, notably for isoform proportions and the tximport length offsets derived from them. Add checks for diagnosing that instability. Refs #1229 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bumps the website-minor group in /website with 2 updates: [@astrojs/starlight](https://github.com/withastro/starlight/tree/HEAD/packages/starlight) and [astro](https://github.com/withastro/astro/tree/HEAD/packages/astro). Updates `@astrojs/starlight` from 0.42.5 to 0.42.8 - [Release notes](https://github.com/withastro/starlight/releases) - [Changelog](https://github.com/withastro/starlight/blob/main/packages/starlight/CHANGELOG.md) - [Commits](https://github.com/withastro/starlight/commits/@astrojs/starlight@0.42.8/packages/starlight) Updates `astro` from 7.3.5 to 7.4.4 - [Release notes](https://github.com/withastro/astro/releases) - [Changelog](https://github.com/withastro/astro/blob/main/packages/astro/CHANGELOG.md) - [Commits](https://github.com/withastro/astro/commits/astro@7.4.4/packages/astro) --- updated-dependencies: - dependency-name: "@astrojs/starlight" dependency-version: 0.42.8 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: website-minor - dependency-name: astro dependency-version: 7.4.4 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: website-minor ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps the cargo-patch group with 8 updates: | Package | From | To | | --- | --- | --- | | [thiserror](https://github.com/dtolnay/thiserror) | `2.0.20` | `2.0.21` | | [serde_json](https://github.com/serde-rs/json) | `1.0.151` | `1.0.154` | | [xxhash-rust](https://github.com/DoumanAsh/xxhash-rust) | `0.8.18` | `0.8.19` | | [sha2](https://github.com/RustCrypto/hashes) | `0.11.0` | `0.11.1` | | [libc](https://github.com/rust-lang/libc) | `0.2.189` | `0.2.191` | | [realfft](https://github.com/HEnquist/realfft) | `3.5.0` | `3.5.1` | | [ensure_simd](https://github.com/RagnarGrootKoerkamp/ensure_simd) | `0.1.0` | `0.1.1` | | [jiff](https://github.com/BurntSushi/jiff) | `0.2.37` | `0.2.38` | Updates `thiserror` from 2.0.20 to 2.0.21 - [Release notes](https://github.com/dtolnay/thiserror/releases) - [Commits](dtolnay/thiserror@2.0.20...2.0.21) Updates `serde_json` from 1.0.151 to 1.0.154 - [Release notes](https://github.com/serde-rs/json/releases) - [Commits](serde-rs/json@v1.0.151...v1.0.154) Updates `xxhash-rust` from 0.8.18 to 0.8.19 - [Commits](https://github.com/DoumanAsh/xxhash-rust/commits) Updates `sha2` from 0.11.0 to 0.11.1 - [Commits](RustCrypto/hashes@sha2-v0.11.0...sha2-v0.11.1) Updates `libc` from 0.2.189 to 0.2.191 - [Release notes](https://github.com/rust-lang/libc/releases) - [Changelog](https://github.com/rust-lang/libc/blob/0.2.191/CHANGELOG.md) - [Commits](rust-lang/libc@0.2.189...0.2.191) Updates `realfft` from 3.5.0 to 3.5.1 - [Release notes](https://github.com/HEnquist/realfft/releases) - [Commits](HEnquist/realfft@v3.5.0...v3.5.1) Updates `ensure_simd` from 0.1.0 to 0.1.1 - [Changelog](https://github.com/RagnarGrootKoerkamp/ensure_simd/blob/master/CHANGELOG.md) - [Commits](RagnarGrootKoerkamp/ensure_simd@v0.1.0...v0.1.1) Updates `jiff` from 0.2.37 to 0.2.38 - [Release notes](https://github.com/BurntSushi/jiff/releases) - [Changelog](https://github.com/BurntSushi/jiff/blob/master/CHANGELOG.md) - [Commits](BurntSushi/jiff@jiff-static-0.2.37...jiff-static-0.2.38) --- updated-dependencies: - dependency-name: thiserror dependency-version: 2.0.21 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: cargo-patch - dependency-name: serde_json dependency-version: 1.0.154 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: cargo-patch - dependency-name: xxhash-rust dependency-version: 0.8.19 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: cargo-patch - dependency-name: sha2 dependency-version: 0.11.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: cargo-patch - dependency-name: libc dependency-version: 0.2.191 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: cargo-patch - dependency-name: realfft dependency-version: 3.5.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: cargo-patch - dependency-name: ensure_simd dependency-version: 0.1.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: cargo-patch - dependency-name: jiff dependency-version: 0.2.38 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: cargo-patch ... Signed-off-by: dependabot[bot] <support@github.com>
`salmon quant` could not take a FIFO or process substitution: the FASTQ
preflight sniffed every input, which failed the rewind on a plain stream
("Illegal seek") and left the mapper a gzip stream with no header. It now
checks regular files only.
`salmon index` reads each reference twice, so stream inputs are copied into
the intermediate directory for the length of the build and removed afterwards.
scripts/quant_ena_stream.sh quantifies a public study straight from ENA,
validating each run against the read count ENA announces and resuming.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A reference with any run of bytes two bits cannot hold (every chromosome of a genome decoy has a few N runs) lost the word-at-a-time extension, and every base it compared looked its runs up again: a genome-decoy run was 5-8% slower in wall time and about 10% higher in CPU than with byte references. match_right/match_left now compare whole words up to each run, then the run's bytes one at a time (N never matching), so they always take the fast path on packed references; decoding a window looks up only the runs that overlap it. Also fixes --online --posBias, which panicked with packed references (and, on develop already, with --sketch): the effective-length correction read the transcript sequence only for its length. It now takes the length, and the sequence travels with the sequence-bias term, the only one that reads it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016brRcptooQkxWr6TyWnwAE
A build opened every input up to three times: once to sniff its compression, once to preprocess it, and once more on a separate thread to hash it. A pipe can be read only once, so the spool copied a stream to disk first. - open_fasta opens anything but a regular file once and lets needletail detect its compression from the stream. - When any input is a stream, preprocessing feeds the digests from the batches it already hands the two writers, on a thread of its own: a record that leaves preprocessing unchanged is hashed from its batch, and only one that was changed or dropped is sent as read. A stream decodes serially, slower than hashing, so the thread keeps up. - Regular files keep their separate hashing pass, which overlaps the whole build: hashing GENCODE's 1.5 Gbases takes longer than the parallel-decoded preprocessing pass, so feeding it from that pass would either hold the pass back or queue the sequence in memory. Build time and peak memory on files are unchanged. - Both paths share one digest implementation, RefDigests; its printable-byte check now vectorizes. Indices built from GENCODE v50, Ensembl cDNA and a GRCh38 genome-decoy gentrome, as files and as FIFOs (the decoy list too), are file-for-file identical to develop's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016brRcptooQkxWr6TyWnwAE
ref_view found a reference's runs by binary search over all of them, and seed extension calls it once per seed: 5% of a genome-decoy run. Each reference's range of runs is now computed when the index loads (8 bytes per reference), so ref_view is two array reads. With this and the run-aware word match, a genome-decoy run (GRCh38 gentrome, GEUVADIS 10M pairs, 16 threads) takes 15.7 s against develop's 16.1 s, with 2.95 GB less peak memory and identical quant.sf. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016brRcptooQkxWr6TyWnwAE
The FIFO helper was inserted between that test and its doc comment, so the test lost it and the helper's own doc began with the test's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016brRcptooQkxWr6TyWnwAE
A user-facing download tool for one data source, never run against ENA itself, is a separate maintenance commitment from salmon reading streams. Streaming an ENA run needs only process substitution, which the docs can show. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016brRcptooQkxWr6TyWnwAE
…at likelihood Adds a section on what each method answers: bootstrap replicates resample the equivalence-class counts and re-run the optimizer; Gibbs samples come from the posterior given those counts. Each bootstrap replicate starts from the same uniform point, so where the likelihood is nearly flat across isoforms, replicates can understate an allocation the data barely constrain; the Gibbs sampler moves along it. Also adds a Gibbs check to the list, and notes that the cited report's two runs differed in platform and duplication rate and that its replicates were bootstraps. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016brRcptooQkxWr6TyWnwAE
salmon checked every uncompressed FASTQ for truncation in a pass of its own before mapping. On a pipe that pass took the first bytes, and the previous commit only skipped it for streams, leaving them unchecked: the FASTQ reader dropped a final record cut one or two lines in without an error, which is why the pass existed (#1162). paraseq-temp 0.6.3-pre.1 (through piscem-rs 0.12.3) refuses a truncated final record on the fill that reaches it, so paired inputs report the cut rather than a mismatch in record counts, and lets a reader name its input in format errors. salmon names each reader after its path, adding the option for a stream (`/dev/fd/62 (-2)`), and the preflight is removed. Plain FASTQ is read once: two 2.2 GB files at 16 threads quantify in 9.4 s instead of 10.4 s, with identical quant.sf. A FASTQ ending in a blank line, which the preflight rejected, now runs. Adds a "Reading from streams" section to the CLI reference. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016brRcptooQkxWr6TyWnwAE
…29-cb3772 docs: explain what inferential replicates do not measure
perf(map): hold reference sequences at two bits per base while mapping
…0-requant-b9d724 feat(io): read FASTQ and reference FASTA from streams
…rgo-patch-b0b64df564 chore(deps): bump the cargo-patch group with 8 updates
…ust-1.99-bookworm build(deps): bump rust from 1.98-bookworm to 1.99-bookworm
…site/smol-toml-1.9.1 chore(deps): bump smol-toml from 1.8.0 to 1.9.1 in /website
…site/develop/website-minor-af2242913b chore(deps): bump the website-minor group in /website with 2 updates
…r than k Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016brRcptooQkxWr6TyWnwAE
Quantifies four orphans (R1 or R2, forward or reverse, each with a mate that maps nowhere) in selective-alignment and sketch mode, and checks every record's read flag, reverse flag and SEQ. Before the fix, the forward R2 orphan was written with FLAG 153 (reverse) and a reverse-complemented SEQ. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016brRcptooQkxWr6TyWnwAE
…e-hits fix(map): do not reuse the previous fragment's hits for a mate shorter than k
…trand fix(quant): write a right orphan's SAM/BAM record on its own strand
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # crates/salmon-cli/tests/input_validation.rs
…-rerun fix(cli,quant): remove old results when a rerun starts and write quant.sf by rename
…s-em-code refactor(model,infer): delete unused bias and EM code and test the production path
fix(align): judge only the strand of a RAD hit under an outward library type
# Conflicts: # crates/salmon-cli/tests/input_validation.rs
…allel-n fix(cli): accept the documented --decoder parallel=N form
# Conflicts: # crates/salmon-align/tests/deterministic_bam.rs
fix(align): strip only /1 and /2 from read names when grouping mates
# Conflicts: # crates/salmon-cli/tests/input_validation.rs
…-order fix(quant): decide a deterministic single-end library type on every fragment
…hans-pairing fix(map): apply the pairing rules to a mate rescued by --recoverOrphans
…own-away fix(quant): do not collect bias models a pass will not use
# Conflicts: # crates/salmon-cli/tests/input_validation.rs
…decoys fix(quant): keep ambig_info.tsv row-aligned with quant.sf under --online
…ed-flags fix(cli): warn that the alignment error model flags do nothing under --rad
…gth-check fix(align): warn when a -t FASTA sequence has another length than its reference
# Conflicts: # crates/salmon-map/tests/collect_on_index.rs
fix(map): keep a sketch right orphan's score on its SAM record
…suffixes fix(core): use one list of compressed-file suffixes
…ersion-order fix(core): resolve --ignoreTxVersion collisions independently of hash order
…blame fix(infer): at the iteration cap, blame only the components still changing
# Conflicts: # crates/salmon-cli/tests/input_validation.rs
…rning fix(quant): do not report the bias seed EM as a convergence failure
…ounding fix(infer): sum a gathered step's L1 as a pushed step does
…ed-batch perf(quant): add to the shared processed count once per batch
Brings in Benjamin's second batch of audit fixes, #1289–#1308 (without #1301). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016brRcptooQkxWr6TyWnwAE
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016brRcptooQkxWr6TyWnwAE
The -k limit is now enforced, the decoy list must match the FASTA, -a refuses input whose mates are not adjacent and counts unmapped records, quantmerge refuses ambiguous names and damaged files, and salmon_map::extend_mem takes a RefView. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016brRcptooQkxWr6TyWnwAE
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release 2.11.0
This merges
developintomasterfor 2.11.0. The full notes are indocs/release-notes-2.11.0.md. The index format is unchanged.Contents:
quantandindexread FIFOs and process substitution (Benjamin Demaille, with maintainer edits).-a): orphan sides in RAD records (fix(rad): keep each orphan's own side and drop orphans beside a proper pair #1282, which also changes stranded reads-mode runs), single-end BAMs under-l A,SEQ *records under--errorModel,--hardFilter, outward library types.quantmergenames, BAMs not grouped by read name.-acounts unmapped records inpercent_mapped, and a rerun removes the oldquant.sffirst.--online --sketch --posBiasno longer panics.Results. Against 2.10.0 on x86 (muscle, GEUVADIS, GEUVADIS with the genome decoy, selective alignment and sketch, at 1, 8 and 16 threads),
quant.sfis identical in 31 of 38 runs. The other 7 are stranded muscle with selective alignment, which #1282 changes: 14 more fragments map, and 2,228 of 4.14M reads (0.05%) move between transcripts. Timing and memory tables are in the notes.Checks:
cargo test --release --workspace --locked: 435 passed. Clippy (-D warnings) and fmt are clean../scripts/bump_and_publish.sh 2.11.0 --dry-runpackages and verifies all ten crates at 2.11.0.After merging: run
./scripts/bump_and_publish.sh 2.11.0 --publishonmaster, set the GitHub release notes from the notes file, and open themaster→developsync PR.🤖 Generated with Claude Code
https://claude.ai/code/session_016brRcptooQkxWr6TyWnwAE