Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
1.9.0
-
None
-
None
Description
The above piece of a Java flamegraph shows the main application thread apply}}ing operations to a {{KuduSession in AUTO_FLUSH_BACKGROUND mode. The doFlush call is meant to set up a callbacks that actually send the rows to Kudu and that are triggered by the fulfillment of a Deferred for the tablet locations. However, when the tablet locations are cached, this Deferred can be fulfilled basically instantly, so by the time that the apply}}ing thread calls {{addCallbacks, the deferred is complete, and as an optimization the async library executes the callbacks on the thread that is adding them to the fulfilled Deferred. This means the apply}}ing thread executes the code that sends rows to Kudu, which is the opposite of how {{AUTO_FLUSH_BACKGROUND is meant to work.
We lose out on the ability to have multiple batches in flight at once, and we serialize the application logic, apply, and actually sending rows to Kudu.