Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-3370

FLATTEN error with a where clause

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.0.0
    • 1.1.0
    • Functions - Drill
    • None

    Description

      I've got a JSON file which contains 150 JSON strings all like this:

      {"arr": [94]} {"arr": [39]} {"arr": [180]}

      I was trying to Flatten() the arrays and filter the values using such an SQL query:
      select flatten(arr) as a from dfs.`/data/test/jsonarray.150.json` where a > 100;
      However, it returned no result. Then I modified my expression like this:
      select a from (select flatten(arr) as a from dfs.`/data/test/jsonarray.150.json`) where a > 100;
      It then failed:
      Error: SYSTEM ERROR: org.apache.drill.exec.exception.SchemaChangeException: Failure while trying to materialize incoming schema. Errors:
      Error in expression at index 1. Error: Missing function implementation: [flatten(BIGINT-REPEATED)]. Full expression: --UNKNOWN EXPRESSION-..
      Fragment 0:0
      [Error Id: 1d71bf0e-48da-43f8-8b36-6a513120d7e0 on slave2:31010] (state=,code=0)
      After a lot of attempts, I finally got it work:
      select a from (select flatten(arr) as a from dfs.`/data/test/jsonarray.150.json` limit 10000000) where a > 100;
      See, I just added a "limit 10000000" in this query and I am wondering if this is a bug or what in Drill?
      Looking forward to your attention and help. Many thanks.

      Attachments

        1. jsonarray.150.json
          2 kB
          Joanlynn LIN
        2. DRILL-3370.patch
          4 kB
          Jason Altekruse

        Activity

          People

            jaltekruse Jason Altekruse
            joanlynn Joanlynn LIN
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: