Details
-
Improvement
-
Status: Reopened
-
Not a Priority
-
Resolution: Unresolved
-
1.11.1
-
None
-
Flink 1.12 master on 26.08.2020
Description
Currently PojoSerializer.createInstance() uses reflection call to create a class instance. As this method is called on each stream element on deserialization, reflection overhead can become noticeable in serialization-bound cases when:
- Pojo class is small, so instantiation is noticeable.
- The job is not having heavy CPU-bound event processing.
See this flamegraph built for flink-benchmarks/SerializationFrameworkMiniBenchmarks.serializerPojo benchmark:
This Reflection.getCallerClass method consumes a lot of CPU, mostly doing a security check if we allowed to do this reflective call.
There is no true reason to perform this check on each deserializing event, so to speed things up we can just cache the constructor using MetaHandle, so this check will be performed only once. With this tiny fix, the getCallerClass is gone:
The benchmark result:
serializerPojo thrpt 100 487.706 ± 30.480 ops/ms serializerPojo thrpt 100 569.828 ± 8.815 ops/m
Which is +15% to throughput.
Attachments
Attachments
Issue Links
- causes
-
FLINK-19086 Performance regression 2020-08-27 in globalWindow benchmark
- Closed
- links to