Uploaded image for project: 'Apache Avro'
  1. Apache Avro
  2. AVRO-3438

NPE when serializing Avro GenericRecord with Kryo

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.11.0, 1.10.2
    • None
    • java
    • None

    Description

      We have an Apache Flink application, that processes Avro GenericRecords. Flink uses Kryo serialization. The serialization fails:

       
      com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
      Serialization trace:
      props (org.apache.avro.Schema$Field)
      fieldMap (org.apache.avro.Schema$RecordSchema)
      schema (org.apache.avro.generic.GenericData$Record)
      values (org.apache.avro.generic.GenericData$Record)
      values (org.apache.avro.generic.GenericData$Record)
      at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:82) ~[kryo-2.24.0.jar:?]
      at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:495) ~[kryo-2.24.0.jar:?]
      at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:599) ~[kryo-2.24.0.jar:?]
      at com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:95) ~[kryo-2.24.0.jar:?]
      at com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:21) ~[kryo-2.24.0.jar:?]
      at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:523) ~[kryo-2.24.0.jar:?]
      at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:61) ~[kryo-2.24.0.jar:?]
      at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:495) ~[kryo-2.24.0.jar:?]
      at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:523) ~[kryo-2.24.0.jar:?]
      at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:61) ~[kryo-2.24.0.jar:?]
      ...
      Caused by: java.lang.NullPointerException
      at org.apache.avro.JsonProperties$2$1$1.<init>(JsonProperties.java:175) ~[avro-1.10.2.jar:1.10.2]
      at org.apache.avro.JsonProperties$2$1.iterator(JsonProperties.java:174) ~[avro-1.10.2.jar:1.10.2]
      at com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:80) ~[kryo-2.24.0.jar:?]
      at com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:21) ~[kryo-2.24.0.jar:?]
      at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:523) ~[kryo-2.24.0.jar:?]
      at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:61) ~[kryo-2.24.0.jar:?]
      ...
       
      This problem occurs, because Kryo uses Unsafe to copy objects. When creating an object, Unsafe does not call the class constructor. Therefore the field props in the class JsonProperties is not properly initialized. The field props is implemented via anonymous inner classes. The NullPointerException occurs at JsonProperties.java:175, because the reference to the enclosing instance of AbstractSet is null.

      A possible fix is to avoid using anonymous classes, use normal classes instead.  A similar case: https://stackoverflow.com/questions/36902471/inner-class-reference-to-enclosing-class-instance-is-null

       

      Avro: 1.10.2

      Kryo: 2.24.0

      Flink: 1.12.4

      Attachments

        Activity

          People

            Unassigned Unassigned
            tashoyan Arseniy Tashoyan
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: