Major performance problem with second request on same keep-alive connection

If I load the same URL through TS twice in a row to a server with keep-alives turned on I get really slow performance on the second request.
E.g. (loading 212M over loopback with uncachable content, but results are similar with cachable content):
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 212M 100 212M 0 0 140M 0 0:00:01 0:00:01 -::- 142M*
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 212M 100 212M 0 0 6397k 0 0:00:34 0:00:34 -::- 6400k*

If I turn off proxy.config.http.keep_alive_enabled_out the problem goes away, it can also be partially solved by making proxy.config.io.max_buffer_size really big (but it needs to be bigger for larger content, and that supports the comment below, and proves that it isn't the origin server's fault.)

When I turn on debug messages it looks like the IO loop for the second request only wakes up every 10ms (when an epoll_wait times out) and then it reads and writes, for the first request it goes much faster without those pauses. My theory is that the issue is related to the the fact that the outgoing connection fd gets added to the epoll fd for the thread handling the first request and the first user agent fd is added to the same epoll fd, and it stays on that epoll fd. But the new user agent request is on a different epoll fd which I assume means is tied to a different thread.