Skip to content

perf(endpoint): poll_transmit visits only the connections that may send - #14

Open
pylaterreur wants to merge 2 commits into
mp0rta:mainfrom
pylaterreur:ready-queue
Open

pylaterreur wants to merge 2 commits into
mp0rta:mainfrom
pylaterreur:ready-queue

Conversation

@pylaterreur

Copy link
Copy Markdown

Part of #12.

poll_transmit walked every connection from the round-robin cursor until one sent. With one busy connection among N idle ones, each packet cost about N connection visits.

The endpoint now keeps a BTreeSet of ready handles and walks it from the cursor, which keeps today's order exactly.

  • Every endpoint call that reaches a connection makes it ready: a packet for it, a path call, connection(h) (which hands out &mut Connection), and a handle_timeout pass after one of those.
  • A turn that sends nothing and leaves nothing for the next turn parks the connection. It stays parked until such a call, or until its timer fires: its deadline or a pacing release.
  • A turn that changes state without sending stays ready: one that installs a mapping the TCB holds back, trims the reinjection head, moves the repair state, or changes a subflow's TCP state.
  • apply_aborts, which also walked every connection on each handle_packet, looks only at the connections connection(h) handed out.

The full scan stays as a test-only oracle (Endpoint::full_scan). diff_tests runs randomized two-endpoint scenarios both ways and checks that they produce the same packets in the same order, and the same events, poll_timeout values, reads and call results.

  • The scenarios cover MPTCP and TCP, joins and path calls, loss, reordering, blackholes, stripped MPTCP options, payload corruption, and careless drivers: partial drains, late and spurious handle_timeout, short buffers.
  • cargo test runs 40 small scenarios, about 10 s in debug. DIFF_SEEDS=1..1000 cargo test --release -p mptcp-rs diff_tests -- --ignored runs larger ones.

Connections visited per packet sent, for a 300 KB transfer next to N idle MPTCP connections (scale_tests):

N 1 10 100 500 2000
main 3.96 21.8 200 992 3962
this PR 1.81 1.81 1.81 1.81 1.81

Instructions for a whole run (setup plus a 1 MB transfer, callgrind, calls in the driver contract's order): N=0 −0.2%, N=10 −31%, N=100 −78%, N=1000 −93%.

Testing:

  • cargo test, clippy, fmt, the 1.85 MSRV build, and the fuzz targets.
  • diff_tests on 40 small and 400 large scenarios.
  • A separate harness that runs main and this branch as two crates through 3000 more scenarios: identical packets, events and API results.
  • tests/interop on a Linux 7.0 VM: m1 and 23 server-role runs pass. These are base-a, m2-goodput, m2-lossy and m2-hetero, plus m2-remove and csum-fail1/2 with the kernel client, in both directions.

No public API change, no new dependency.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GwUvhmzgfkiZ6vo4uGyesN

poll_transmit walked every connection from the round-robin cursor until
one sent, calling update_recovery, select_control and the scheduler on
each, so with one busy connection among N idle ones every packet cost
about N visits (issue mp0rta#12).

The endpoint now keeps a BTreeSet of ready handles and walks it from the
cursor, which keeps today's order. Every endpoint call that reaches a
connection makes it ready: a packet for it, a path call, connection(h)
(which hands out &mut Connection), and a handle_timeout pass after one
of those. A turn that sends nothing and leaves nothing for the next turn
parks the connection until then, or until its wake time passes: its
deadline, or a pacing release (which poll_send_at leaves out for a
go-back's lone FIN), the times when the passing of time alone can give
it a segment to send. A turn that installs a held mapping, trims the
reinjection head, moves the repair state or changes a subflow's TCP
state leaves it ready. apply_aborts looks only at the connections the
application reached.

The full scan stays as a test-only oracle (Endpoint::full_scan), and
diff_tests runs randomized two-endpoint scenarios both ways and checks
that the packets, in order, the events, poll_timeout values, reads and
call results are the same. With 2000 idle MPTCP connections a bulk
transfer costs 1.8 visits per packet instead of 3962 (scale_tests). The
rogue-plan tests change the probe scheduler's plan from outside the
connection, so they mark it themselves (set_rogue).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GwUvhmzgfkiZ6vo4uGyesN

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants