Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-1324

Druid metadata query throws exception if there are non-standard aggregators

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Trivial
    • Resolution: Fixed
    • 1.9.0
    • 1.9.0
    • druid-adapter
    • None

    Description

      If the approximate histogram aggregator is used the druid metadata query fails as the aggregator json contains unexpected fields.

      Quick patch which just ignores the fields and the type: https://github.com/remerge/calcite/commit/6752c302eb95a294380891c6de8dca6349fe1ed6

      com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "resolution" (class org.apache.calcite.adapter.druid.DruidConnectionImpl$JsonAggregator), not marked as ignorable (3 known properties: "type", "name", "fieldName"])
       at [Source: sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@39bebb16; line: 208, column: 24] (through reference chain: java.util.ArrayList[0]->org.apache.calcite.adapter.druid.JsonSegmentMetadata["aggregators"]->java.util.LinkedHashMap["bid_price_hist_eur"]->org.apache.calcite.adapter.druid.JsonAggregator["resolution"])
      	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2203)
      	at com.google.common.cache.LocalCache.get(LocalCache.java:3937)
      	at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3941)
      	at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4824)
      	at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4830)
      	at com.google.common.cache.LocalCache$LocalLoadingCache.apply(LocalCache.java:4848)
      	at com.google.common.collect.Maps$AsMapView.get(Maps.java:793)
      	at org.apache.calcite.schema.impl.AbstractSchema.getTable(AbstractSchema.java:91)
      	at org.apache.calcite.jdbc.CachingCalciteSchema.getImplicitTable(CachingCalciteSchema.java:127)
      	at org.apache.calcite.jdbc.CalciteSchema.getTable(CalciteSchema.java:255)
      	at org.apache.calcite.jdbc.CalciteMetaImpl$12.apply(CalciteMetaImpl.java:417)
      	at org.apache.calcite.jdbc.CalciteMetaImpl$12.apply(CalciteMetaImpl.java:415)
      	at org.apache.calcite.linq4j.EnumerableDefaults$15$1.current(EnumerableDefaults.java:1893)
      	at org.apache.calcite.linq4j.Linq4j$CompositeEnumerable$1.current(Linq4j.java:498)
      	at org.apache.calcite.linq4j.EnumerableDefaults$33.moveNext(EnumerableDefaults.java:2805)
      	at org.apache.calcite.linq4j.EnumerableDefaults$17$1.moveNext(EnumerableDefaults.java:1970)
      	at org.apache.calcite.linq4j.EnumerableDefaults$33.moveNext(EnumerableDefaults.java:2804)
      	at org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.next(Linq4j.java:673)
      	at org.apache.calcite.avatica.util.IteratorCursor.next(IteratorCursor.java:46)
      	at org.apache.calcite.avatica.AvaticaResultSet.next(AvaticaResultSet.java:218)
      	at org.apache.calcite.jdbc.CalciteResultSet.next(CalciteResultSet.java:86)
      	at sqlline.IncrementalRows.hasNext(IncrementalRows.java:62)
      	at sqlline.TableOutputFormat.print(TableOutputFormat.java:33)
      	at sqlline.SqlLine.print(SqlLine.java:1652)
      	at sqlline.Commands.metadata(Commands.java:199)
      	at sqlline.Commands.tables(Commands.java:332)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
      	at sqlline.SqlLine.dispatch(SqlLine.java:803)
      	at sqlline.SqlLine.begin(SqlLine.java:681)
      	at sqlline.SqlLine.start(SqlLine.java:398)
      	at sqlline.SqlLine.main(SqlLine.java:292)
      Caused by: java.lang.RuntimeException: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "resolution" (class org.apache.calcite.adapter.druid.DruidConnectionImpl$JsonAggregator), not marked as ignorable (3 known properties: "type", "name", "fieldName"])
       at [Source: sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@39bebb16; line: 208, column: 24] (through reference chain: java.util.ArrayList[0]->org.apache.calcite.adapter.druid.JsonSegmentMetadata["aggregators"]->java.util.LinkedHashMap["bid_price_hist_eur"]->org.apache.calcite.adapter.druid.JsonAggregator["resolution"])
      	at com.google.common.base.Throwables.propagate(Throwables.java:160)
      	at org.apache.calcite.adapter.druid.DruidConnectionImpl.metadata(DruidConnectionImpl.java:375)
      	at org.apache.calcite.adapter.druid.DruidSchema$1.load(DruidSchema.java:72)
      	at org.apache.calcite.adapter.druid.DruidSchema$1.load(DruidSchema.java:68)
      	at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)
      	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319)
      	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282)
      	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197)
      	... 34 more
      Caused by: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "resolution" (class org.apache.calcite.adapter.druid.DruidConnectionImpl$JsonAggregator), not marked as ignorable (3 known properties: "type", "name", "fieldName"])
       at [Source: sun.net.www.protocol.http.HttpURLConnection$HttpInputStream@39bebb16; line: 208, column: 24] (through reference chain: java.util.ArrayList[0]->org.apache.calcite.adapter.druid.JsonSegmentMetadata["aggregators"]->java.util.LinkedHashMap["bid_price_hist_eur"]->org.apache.calcite.adapter.druid.JsonAggregator["resolution"])
      	at com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:51)
      	at com.fasterxml.jackson.databind.DeserializationContext.reportUnknownProperty(DeserializationContext.java:839)
      	at com.fasterxml.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:1045)
      	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1352)
      	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownVanilla(BeanDeserializerBase.java:1330)
      	at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:264)
      	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:125)
      	at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringMap(MapDeserializer.java:495)
      	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:341)
      	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:26)
      	at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:520)
      	at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:101)
      	at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:258)
      	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:125)
      	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:245)
      	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
      	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:25)
      	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3736)
      	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2810)
      	at org.apache.calcite.adapter.druid.DruidConnectionImpl.metadata(DruidConnectionImpl.java:360)
      	... 40 more
      

      Attachments

        Activity

          People

            julianhyde Julian Hyde
            martin.karlsch Martin Karlsch
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: