Conversation
icing@chaos.social

Stefan Eissing

I re-implemented the rate limiting in , coming in the next release. A description.

https://eissing.org/icing/posts/curl-rate-limits/

1
2
0
toke

Toke Høiland-Jørgensen

Reply to @icing@chaos.social
@icing interesting! Do you do anything to limit the amount of buffering in the TCP stack when rate limiting (e.g., adjust the receive buffer size), and/or to pace out the traffic on send?
1
0
0
icing@chaos.social

Stefan Eissing

Reply to @toke

@toke No, we do not go into the adjusting TCP windows. Rate limits are enforced on the net data, not the overhead of TLS records or what else may be involved (like chunked encoding in HTTP/1.1).

Also, in HTTP/2 this would not be possible apart from edge cases. And HTTP/3 is another matter entirely.

1
0
0
toke

Toke Høiland-Jørgensen

Reply to @icing@chaos.social
@icing right, I do realise it would not be super portable across protocols (or OSes). So you're rate limiting the read()/write() socket calls and just letting the OS do its thing, essentially?
1
0
0
icing@chaos.social

Stefan Eissing

Reply to @toke

@toke On a HTTP/1.1 or FTP transfer, that is basically it, yes. With TLS it's the send/receive on the TLS API really as we do not meddle into how the TLS stack manages its records and it may try to read more from the socket.

0
0
1
About social.kernel.org

Terms of service

  1. Please do not use this service in violation of the Linux Kernel Code of Conduct. Doing so will result in your account suspension with the referral of the matter to the CoC committee.

  2. "Repeating"/"boosting" someone else's status on this platform will be treated as endorsement and will fall under rule #1.

  3. You are encouraged to use this platform to promote your work on the Linux Kernel, but there is no restriction on permitted topics (with the exception of anything covered by #1 above).

  4. There is no requirement to post in English, but it should be considered the primary language of communication on this platform.

Privacy notice

The admins of this service have access to all posted statuses. They aren't looking, but if it's something they shouldn't know about, then you should not post it on this platform.

Please see the Linux Foundation Privacy Policy, which applies to this platform as well.

Getting your own account

If you would like an account on this instance, please check that the following applies to you:

  • You are listed in MAINTAINERS or CREDITS

  • OR: You have a kernel.org account or email address

  • OR: You have a long and established history of involvement with the Linux Kernel

If the above is true and you agree with the Terms of Service and Privacy Notice listed above, please use these instructions to request an account: