Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
Patch
Description
Under high load, requests can spend a long time waiting in RPC queue, probably longer than query timeout. However, current coprocessor timeout mechanism doesn't take RPC queue time into account. As a result, handling these requests not only waste server resources while doing nothing useful, but can also cause cascading failure when server crashes and client retries.
To recover from server overload ASAP, we should check query deadline at each stages of query processing, and avoid spending resources on query that will exceed their deadline.