random thought: I'm kinda surprised that Linux doesn't yet have an API where you can call sendmsg() with some flag that says "if you lose this data I can regenerate the exact same data and give it to you again". then the TCP stack could discard that data as soon as it is transmitted, or also discard untransmitted data if it wants to shrink the TX window because the peer has suddenly gotten way slower.
With some mechanism where, when the kernel notices that it needs to recover data it already discarded, the kernel can post a message to the socket's error queue saying "please give me that data again", and a sendmsg() API addition so that you can specify "this is a new copy of data I already gave you before". And userspace could use SOF_TIMESTAMPING_TX_ACK to figure out until when it needs to be ready to regenerate the data.

That might make it possible for servers sending mostly-static content to use larger transmit windows without having to worry about the memory usage of socket send buffers so much (since only HTTP headers and dynamic content would still consume send buffer space while the data is in flight), and if the TLS implementation was designed for use with this feature (storing mappings between TCP byteranges and corresponding plaintext file ranges, IVs, and MACs), that might even work with content served over HTTPS...

0

If you have a fediverse account, you can quote this note from your own instance. Search https://infosec.exchange/users/jann/statuses/114684577720913102 on your instance and quote it. (Note that quoting is not supported in Mastodon.)