Skip to content

WebTransport: report the fate of outgoing datagrams via aggregate stats - #4091

Merged
jesup merged 1 commit into
mainfrom
users/jesup/report_datagrams
Oct 19, 2026
Merged

jesup merged 1 commit into
mainfrom
users/jesup/report_datagrams

Conversation

@jesup

@jesup jesup commented Oct 17, 2026 •

Copy link
Copy Markdown
Member

Re-land of #3991, which was merged into #4030's branch instead of main by mistake. Content is unchanged from the approved version.

Adds SessionStats::datagrams_sent_outgoing/datagrams_dropped_outgoing, backed by the per-session sent count from the previous commit, so that once a session's queue is empty, every datagram a caller handed in has been counted as sent, expired or dropped exactly once. Dropped covers byte-budget evictions, datagrams refused at enqueue as the lowest priority, datagrams dropped at packet-build time as too big for the path MTU, and datagrams still queued when the session closes. The Capsule fallback counts as sent on write.

close_session flushes the sent count and drops what is still queued before snapshotting the stats it returns, so the final values include the datagrams that call discards; a close or reset from the peer drops and counts the queue too. Both close paths expire stale datagrams first, so those count as expired rather than dropped. The per-tick sweep only visits the sessions the transport reports as having work.

Copilot AI balanced review requested due to automatic review settings October 17, 2026 13:24
@codecov

codecov Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.99%. Comparing base (d245301) to head (fb2f47c).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4091      +/-   ##
==========================================
- Coverage   97.13%   96.99%   -0.14%     
==========================================
  Files         113      118       +5     
  Lines       41524    41887     +363     
  Branches    41524    41887     +363     
==========================================
+ Hits        40334    40629     +295     
- Misses       1176     1234      +58     
- Partials       14       24      +10     
Flag Coverage Δ
freebsd 94.58% <100.00%> (-0.13%) ⬇️
linux 97.21% <100.00%> (+0.01%) ⬆️
macos 95.37% <100.00%> (+0.01%) ⬆️
windows 95.45% <100.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
neqo-common 99.31% <ø> (ø)
neqo-http3 95.41% <100.00%> (+0.03%) ⬆️
neqo-qpack 97.10% <ø> (ø)
neqo-transport 97.98% <ø> (+0.01%) ⬆️
neqo-udp 95.37% <ø> (ø)
mtu 89.13% <ø> (ø)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Idle server connections can leave sent and dropped datagram statistics stale because pending counters do not trigger HTTP/3 processing.

1 open issue
What changed in this PR

Adds aggregate outgoing WebTransport datagram statistics alongside expiry counts.

Changes:

  • Tracks sent and dropped datagrams per session.
  • Includes queued datagram outcomes in close-time statistics.
  • Adds accounting and pending-counter tests.
File Description
neqo-transport/​src/​quic_datagrams.rs Detects all pending outcome counters.
neqo-transport/​src/​datagram_queue.rs Adds pending-counter predicate.
neqo-transport/​src/​connection/​tests/​datagram.rs Tests pending sent counts.
neqo-transport/​src/​connection/​mod.rs Updates counter API documentation.
neqo-http3/​src/​webtransport.rs Reads statistics after closing.
neqo-http3/​src/​features/​extended_connect/​webtransport_session.rs Records sent and dropped totals.
neqo-http3/​src/​features/​extended_connect/​tests/​webtransport/​datagrams.rs Tests aggregate outcome accounting.
neqo-http3/​src/​features/​extended_connect/​stats.rs Exposes sent and dropped fields.
neqo-http3/​src/​features/​extended_connect/​session.rs Collects queue and capsule outcomes.
neqo-http3/​src/​connection.rs Drains counters and preserves closed-session access.
neqo-http3/​src/​connect_udp.rs Adapts to the close helper’s return value.

Review effort: Balanced

Comment thread neqo-http3/src/connection.rs
@codspeed

codspeed Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will regress 1 benchmark

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 3 improved benchmarks
❌ 1 regressed benchmark
✅ 97 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ Simulation decode 10 STREAM frames 3.8 µs 3.9 µs -2.74%
⚡ WallTime 1-conn/1-100mb-resp (aka. Download) 49.8 ms 47.7 ms +4.41%
⚡ WallTime quiche-neqo 44.5 ms 43.1 ms +3.08%
⚡ WallTime s2n-neqo 128.4 ms 125.4 ms +2.34%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing users/jesup/report_datagrams (fb2f47c) with main (d245301)

Open in CodSpeed

Base automatically changed from users/jesup/datagram_counting to main October 17, 2026 14:28
@jesup
jesup force-pushed the users/jesup/report_datagrams branch from 62589ab to 6325866 Compare October 17, 2026 17:07
@jesup

jesup commented Oct 17, 2026

Copy link
Copy Markdown
Member Author

This PR is part of a stack of 4 bookmarks:

  1. trunk()
  2. users/jesup/report_datagrams ← this PR
  3. users/jesup/expose_queue_capacity
  4. users/jesup/add_remote_max_streams
  5. users/jesup/anticipated_incoming_streams

Created with jj-stack

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No whole-change observations.

Comment thread neqo-http3/src/connection.rs
@jesup
jesup disabled auto-merge October 17, 2026 18:32
@jesup
jesup enabled auto-merge October 17, 2026 18:56
Comment thread neqo-http3/src/features/extended_connect/stats.rs Outdated
Comment thread neqo-transport/src/datagram_queue.rs Outdated
Adds SessionStats::datagrams_sent_outgoing/datagrams_dropped_outgoing,
backed by the per-session sent count from the previous commit, so that once
a session's queue is empty, every datagram a caller handed in has been
counted as sent, expired or dropped exactly once.  Dropped covers byte-budget
evictions, datagrams refused at enqueue as the lowest priority, datagrams
dropped at packet-build time as too big for the path MTU, and datagrams
still queued when the session closes.  The Capsule fallback counts as sent
on write.

close_session flushes the sent count and drops what is still queued before
snapshotting the stats it returns, so the final values include the
datagrams that call discards; a close or reset from the peer drops and
counts the queue too.  Both close paths expire stale datagrams first, so
those count as expired rather than dropped.  The per-tick sweep only visits
the sessions the transport reports as having work.
@jesup
jesup force-pushed the users/jesup/report_datagrams branch from 6325866 to fb2f47c Compare October 19, 2026 15:49

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No findings.

@github-actions

Copy link
Copy Markdown
Contributor

Failed Interop Tests

QUIC Interop Runner, client vs. server, differences relative to main at d245301.

neqo-pr as clientneqo-pr as server
neqo-pr vs. go-x-net: BP BA
neqo-pr vs. haproxy: BP BA
neqo-pr vs. kwik: 🚀LR S Z ⚠️3 U L1 C1 ⚠️C2 V2
neqo-pr vs. lsquic: L1 C1
neqo-pr vs. msquic: Z A L1 C1
neqo-pr vs. mvfst: A BA
neqo-pr vs. neqo: Z A
neqo-pr vs. nginx: BP BA
neqo-pr vs. ngtcp2: Z ⚠️L1 C1 CM
neqo-pr vs. picoquic: Z A
neqo-pr vs. quic-go: A
neqo-pr vs. quiche: BP BA
neqo-pr vs. s2n-quic: CM
neqo-pr vs. tquic: S BP BA
neqo-pr vs. xquic: S R Z A L1 C1
go-x-net vs. neqo-pr: CM
kwik vs. neqo-pr: BP BA CM
msquic vs. neqo-pr: ⚠️L1 CM
mvfst vs. neqo-pr: Z L1 C1 CM
neqo vs. neqo-pr: Z A
openssl vs. neqo-pr: LR M A CM
quic-go vs. neqo-pr: CM
quic-zig vs. neqo-pr: CM
quiche vs. neqo-pr: CM
quinn vs. neqo-pr: V2 CM
s2n-quic vs. neqo-pr: CM
tquic vs. neqo-pr: CM
xquic vs. neqo-pr: M CM
All results

Succeeded Interop Tests

QUIC Interop Runner, client vs. server

neqo-pr as client

neqo-pr as server

Unsupported Interop Tests

QUIC Interop Runner, client vs. server

neqo-pr as client

neqo-pr as server

@jesup
jesup added this pull request to the merge queue Oct 19, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Benchmark results

No significant performance differences relative to d245301.

All results
transfer/1-conn/1-100mb-req (aka. Upload)/mtu-1500
       time:   [51.107 ms 51.138 ms 51.170 ms]
       thrpt:  [1.9085 GiB/s 1.9097 GiB/s 1.9108 GiB/s]
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mild
transfer/1-conn/1-100mb-resp (aka. Download)/mtu-1500
       time:   [51.037 ms 51.073 ms 51.110 ms]
       thrpt:  [1.9107 GiB/s 1.9121 GiB/s 1.9134 GiB/s]
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mild
transfer/1-conn/1-1b-resp (aka. HPS)/mtu-1500
       time:   [2.9129 ms 2.9138 ms 2.9147 ms]
       thrpt:  [343.09   B/s 343.20   B/s 343.30   B/s]
Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) low mild
1 (1.00%) high mild
1 (1.00%) high severe
transfer/1-conn/10_000-parallel-1b-resp (aka. RPS)/mtu-1500
       time:   [66.594 ms 66.929 ms 67.266 ms]
       thrpt:  [148.66 Kelem/s 149.41 Kelem/s 150.16 Kelem/s]
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) low mild
1 (1.00%) high mild
streams-flow-controlled/walltime/1-streams/each-4194304-bytes
       time:   [10.709 ms 10.713 ms 10.716 ms]
       thrpt:  [373.26 MiB/s 373.39 MiB/s 373.52 MiB/s]
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) low mild
1 (1.00%) high mild
streams-flow-controlled/walltime/10-streams/each-1048576-bytes
       time:   [28.671 ms 28.684 ms 28.698 ms]
       thrpt:  [348.46 MiB/s 348.62 MiB/s 348.78 MiB/s]
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mild
streams/walltime/1-streams/each-1000-bytes
       time:   [131.10 µs 131.63 µs 132.30 µs]
       thrpt:  [7.2086 MiB/s 7.2452 MiB/s 7.2743 MiB/s]
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
streams/walltime/1000-streams/each-1-bytes
       time:   [3.9134 ms 3.9252 ms 3.9363 ms]
       thrpt:  [248.09 KiB/s 248.79 KiB/s 249.54 KiB/s]
streams/walltime/1000-streams/each-1000-bytes
       time:   [12.580 ms 12.590 ms 12.600 ms]
       thrpt:  [75.689 MiB/s 75.748 MiB/s 75.810 MiB/s]
Found 10 outliers among 100 measurements (10.00%)
1 (1.00%) low severe
5 (5.00%) low mild
4 (4.00%) high mild
transfer/walltime/pacing-false/same-seed
       time:   [2.0228 ms 2.0239 ms 2.0252 ms]
       thrpt:  [1.9289 GiB/s 1.9301 GiB/s 1.9311 GiB/s]
transfer/walltime/pacing-false/varying-seeds
       time:   [2.0038 ms 2.0049 ms 2.0062 ms]
       thrpt:  [1.9471 GiB/s 1.9484 GiB/s 1.9494 GiB/s]
transfer/walltime/pacing-true/same-seed
       time:   [2.0328 ms 2.0343 ms 2.0362 ms]
       thrpt:  [1.9184 GiB/s 1.9202 GiB/s 1.9216 GiB/s]
transfer/walltime/pacing-true/varying-seeds
       time:   [2.0170 ms 2.0186 ms 2.0207 ms]
       thrpt:  [1.9331 GiB/s 1.9351 GiB/s 1.9366 GiB/s]

Instructions per cycle

Criterion reported no significant timing changes.

All benchmarks
Benchmark IPC before IPC after ΔIPC
transfer/1-conn/1-100mb-req (aka. Upload)/mtu-1500 3.12 3.19 +2.2%
streams-flow-controlled/walltime/1-streams/each-4194304-bytes 3.05 2.98 -2.1%
streams/walltime/1000-streams/each-1-bytes 3.30 3.25 -1.5%
streams/walltime/1000-streams/each-1000-bytes 3.01 2.97 -1.3%
streams/walltime/1-streams/each-1000-bytes 2.51 2.52 +0.4%
transfer/walltime/pacing-true/same-seed 2.87 2.88 +0.4%
transfer/1-conn/10_000-parallel-1b-resp (aka. RPS)/mtu-1500 3.21 3.22 +0.3%
transfer/walltime/pacing-false/varying-seeds 2.86 2.87 +0.3%
transfer/walltime/pacing-false/same-seed 2.88 2.88 -0.2%
transfer/1-conn/1-1b-resp (aka. HPS)/mtu-1500 2.24 2.25 +0.2%
transfer/walltime/pacing-true/varying-seeds 2.86 2.87 +0.2%
transfer/1-conn/1-100mb-resp (aka. Download)/mtu-1500 3.22 3.22 +0.1%
streams-flow-controlled/walltime/10-streams/each-1048576-bytes 3.15 3.16 +0.0%
Profiles for profiler.firefox.com (64)

Download data for profiler.firefox.com or download performance comparison data.

@github-actions

Copy link
Copy Markdown
Contributor

Client/server transfer results

Performance differences relative to d245301.

Transfer of 33554432 bytes over loopback, min. 100 runs. All unit-less numbers are in milliseconds.

Client vs. server Mean±σ Min–Max Median±MAD MiB/s±σ ΔMedian
google-neqo-cubic 70.8 ± 0.4 69.8 – 71.7 70.9 ± 0.3 451.8 ± 2.4 💔 +1.1 (+1.5%)
neqo-neqo-cubic-nopacing 18.8 ± 0.2 18.2 – 19.5 18.8 ± 0.2 1704.3 ± 18.6 💚 -0.2 (-1.3%)
neqo-neqo-newreno 19.4 ± 0.1 19.1 – 19.8 19.4 ± 0.1 1647.8 ± 10.6 💚 -0.3 (-1.4%)
neqo-neqo-newreno-nopacing 18.8 ± 0.2 18.4 – 19.3 18.8 ± 0.2 1702.2 ± 19.5 💚 -0.2 (-1.1%)
quiche-neqo-cubic 31.5 ± 0.7 30.2 – 33.7 31.4 ± 0.7 1014.9 ± 23.9 💔 +0.3 (+1.0%)

Table above only shows statistically significant changes. See all results below.

All results

Transfer of 33554432 bytes over loopback, min. 100 runs. All unit-less numbers are in milliseconds.

Client vs. server Mean±σ Min–Max Median±MAD MiB/s±σ ΔMedian
google-google 136.6 ± 0.5 135.3 – 137.8 136.6 ± 0.5 234.3 ± 0.9
google-neqo-cubic 70.8 ± 0.4 69.8 – 71.7 70.9 ± 0.3 451.8 ± 2.4 💔 +1.1 (+1.5%)
neqo-google-cubic 243.5 ± 72.3 169.1 – 478.0 221.5 ± 34.4 131.4 ± 39.0 +6.8 (+3.2%)
neqo-neqo-cubic 20.0 ± 0.1 19.6 – 20.4 20.0 ± 0.1 1601.0 ± 11.6 -0.2 (-1.0%)
neqo-neqo-cubic-nopacing 18.8 ± 0.2 18.2 – 19.5 18.8 ± 0.2 1704.3 ± 18.6 💚 -0.2 (-1.3%)
neqo-neqo-newreno 19.4 ± 0.1 19.1 – 19.8 19.4 ± 0.1 1647.8 ± 10.6 💚 -0.3 (-1.4%)
neqo-neqo-newreno-nopacing 18.8 ± 0.2 18.4 – 19.3 18.8 ± 0.2 1702.2 ± 19.5 💚 -0.2 (-1.1%)
neqo-quiche-cubic 29.7 ± 0.3 29.0 – 30.3 29.7 ± 0.3 1076.9 ± 10.1 +0.1 (+0.2%)
neqo-s2n-cubic 37.5 ± 0.2 37.1 – 38.1 37.5 ± 0.2 852.3 ± 4.3 +0.0 (+0.0%)
quiche-neqo-cubic 31.5 ± 0.7 30.2 – 33.7 31.4 ± 0.7 1014.9 ± 23.9 💔 +0.3 (+1.0%)
quiche-quiche 35.5 ± 0.4 34.8 – 36.2 35.4 ± 0.4 902.0 ± 9.4
s2n-neqo-cubic 117.2 ± 0.5 115.7 – 118.7 117.2 ± 0.5 272.9 ± 1.1 +0.3 (+0.3%)
s2n-s2n ⚠️ 168.4 ± 30.7 137.7 – 331.9 160.7 ± 1.1 190.1 ± 34.6

Download data for profiler.firefox.com or download performance comparison data.

Merged via the queue into main with commit 031e2d0 Oct 19, 2026
308 of 312 checks passed
@jesup
jesup deleted the users/jesup/report_datagrams branch October 19, 2026 17:09
mxinden-bot pushed a commit to mxinden-bot/neqo that referenced this pull request Oct 20, 2026
User-visible changes since v0.33.0:

- Expose the outgoingMaxBufferedDatagrams and outgoingMaxAge setters as production API:
  mozilla#3990
- Expose the outgoing WebTransport datagram queue capacity per session:
  mozilla#4037
- Expose the per-datagram overhead in the datagram queue capacity:
  mozilla#4067
- Report the fate of outgoing WebTransport datagrams via aggregate stats:
  mozilla#4091
- Count datagrams actually sent per session:
  mozilla#4030
- Count a session's expired outgoing WebTransport datagrams:
  mozilla#3986
- Return NotAvailable instead of TooMuchData when the peer has no datagram path:
  mozilla#3989
- Put ABE (RFC 8511) behind a pref:
  mozilla#4090
- Add a log-payload feature:
  mozilla#4046
- Remove the deprecated ConnectionError alias:
  mozilla#4089
- Answer every PATH_CHALLENGE received on a path:
  mozilla#4101
- Honor a peer's minimum ACK delay above 50ms:
  mozilla#4068
- Use version-specific key update labels:
  mozilla#4080
- Compute QPACK MaxEntries from the peer's capacity:
  mozilla#4033
- Stop connection ID generation when the generator is exhausted:
  mozilla#4082
- Pad to the PMTUD probe size only when sending a probe:
  mozilla#3964
- Ignore ACKs of stale PMTUD probes:
  mozilla#4088
mxinden-bot pushed a commit to mxinden-bot/neqo that referenced this pull request Oct 20, 2026
User-visible changes since v0.33.0:

- Expose the outgoingMaxBufferedDatagrams and outgoingMaxAge setters as production API: [mozilla#3990](mozilla#3990)
- Expose the outgoing WebTransport datagram queue capacity per session: [mozilla#4037](mozilla#4037)
- Expose the per-datagram overhead in the datagram queue capacity: [mozilla#4067](mozilla#4067)
- Report the fate of outgoing WebTransport datagrams via aggregate stats: [mozilla#4091](mozilla#4091)
- Count datagrams actually sent per session: [mozilla#4030](mozilla#4030)
- Count a session's expired outgoing WebTransport datagrams: [mozilla#3986](mozilla#3986)
- Return NotAvailable instead of TooMuchData when the peer has no datagram path: [mozilla#3989](mozilla#3989)
- Put ABE (RFC 8511) behind a pref: [mozilla#4090](mozilla#4090)
- Add a log-payload feature: [mozilla#4046](mozilla#4046)
- Remove the deprecated ConnectionError alias: [mozilla#4089](mozilla#4089)
- Answer every PATH_CHALLENGE received on a path: [mozilla#4101](mozilla#4101)
- Honor a peer's minimum ACK delay above 50ms: [mozilla#4068](mozilla#4068)
- Use version-specific key update labels: [mozilla#4080](mozilla#4080)
- Compute QPACK MaxEntries from the peer's capacity: [mozilla#4033](mozilla#4033)
- Stop connection ID generation when the generator is exhausted: [mozilla#4082](mozilla#4082)
- Pad to the PMTUD probe size only when sending a probe: [mozilla#3964](mozilla#3964)
- Ignore ACKs of stale PMTUD probes: [mozilla#4088](mozilla#4088)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants