Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-7420

Move all Avro code to flink-avro

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • None
    • 1.4.0
    • Build System
    • None

    Description

      Problem

      Currently, the flink-avro project is a shell with some tests and mostly duplicate and dead code. The classes that use Avro are distributed quite wildly through the code base, and introduce multiple direct dependencies on Avro in a messy way.

      That way, we cannot create a proper fat Avro dependency in which we shade Jackson away.

      Also, we expose Avro as a direct and hard dependency on many Flink modules, while it should be a dependency that users that use Avro types selectively add.

      Suggested Changes

      We should move all Avro related classes to flink-avro, and give flink-avro a dependency on flink-core and flink-streaming-java.

      • AvroTypeInfo
      • AvroSerializer
      • AvroRowSerializationSchema
      • AvroRowDeserializationSchema

      To be able to move the the avro serialization code from flink-ore to flink-avro, we need to load the AvroTypeInformation reflectively, similar to how we load the WritableTypeInfo for Hadoop.

      Attachments

        Activity

          People

            aljoscha Aljoscha Krettek
            sewen Stephan Ewen
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: