Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-19052

Performance issue with PojoSerializer

    XMLWordPrintableJSON

Details

    Description

      Currently PojoSerializer.createInstance() uses reflection call to create a class instance. As this method is called on each stream element on deserialization, reflection overhead can become noticeable in serialization-bound cases when:

      1. Pojo class is small, so instantiation is noticeable.
      2. The job is not having heavy CPU-bound event processing.

      See this flamegraph built for flink-benchmarks/SerializationFrameworkMiniBenchmarks.serializerPojo benchmark:

      This Reflection.getCallerClass method consumes a lot of CPU, mostly doing a security check if we allowed to do this reflective call.

       

      There is no true reason to perform this check on each deserializing event, so to speed things up we can just cache the constructor using MetaHandle, so this check will be performed only once. With this tiny fix, the getCallerClass is gone:

       

       

      The benchmark result:

      serializerPojo thrpt 100 487.706 ± 30.480 ops/ms
      serializerPojo thrpt 100 569.828 ± 8.815 ops/m

      Which is +15% to throughput.

      Attachments

        1. image-2020-08-26-10-49-59-400.png
          41 kB
          Roman Grebennikov
        2. image-2020-08-26-10-46-19-800.png
          37 kB
          Roman Grebennikov

        Issue Links

          Activity

            People

              Unassigned Unassigned
              rgrebennikov Roman Grebennikov
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated: