Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-4700

JDBC driver cannot support TIMESTAMP data type

Details

    • Bug
    • Status: Resolved
    • P2
    • Resolution: Fixed
    • None
    • 2.6.0
    • dsl-sql
    • None

    Description

      Avatica allows column representation to be customized, so a timestamp can be stored as a variety of types. Joda ReadableInstant is none of these types: https://github.com/apache/calcite-avatica/blob/acb675de97b9b0743c09368820a770e2ceda05f8/core/src/main/java/org/apache/calcite/avatica/util/AbstractCursor.java#L162

      By default, it seems to be configured to store TIMESTAMP columns as long values. If you run the SQL shell and select a TIMESTAMP column, you get:

      ava.lang.ClassCastException: org.joda.time.Instant cannot be cast to java.lang.Number
              at org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.util.AbstractCursor$NumberAccessor.getNumber(AbstractCursor.java:726)
              at org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.util.AbstractCursor$TimestampFromNumberAccessor.getString(AbstractCursor.java:1026)
              at org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.avatica.AvaticaResultSet.getString(AvaticaResultSet.java:225)
              at sqlline.Rows$Row.<init>(Rows.java:183)
      

      So, essentially, Beam SQL Shell does not support timestamps.

      We may be able to:

      • override how the accessor for our existing storage is created
      • configure what the column representation is (this doesn't really help, since none of the choices are ours)
      • convert timestamps to longs in BeamEnumerableConverter; not sure how many conversions will be required here

      Attachments

        Activity

          People

            apilloud Andrew Pilloud
            kenn Kenneth Knowles
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 2h 50m
                2h 50m