Uploaded image for project: 'ORC'
  1. ORC
  2. ORC-744 LazyIO of non-filter columns
  3. ORC-980

Filter processing ignores the schema case-sensitivity flag

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 1.7.0, 1.8.0
    • 1.7.0, 1.8.0
    • Java
    • None

    Description

      The current SArg to Filter Conversion ignores the case-sensitivity setting during read and always perform case-sensitive search of fields even when asked to perform case-insensitive match using

      options()
      .searchArgument(sarg, new String[] color:#6a8759}"RIDX"{color)
      .allowSARGToFilter(true)
      .isSchemaEvolutionCaseAware(false)

      leading to the following exceptions that the field is not found in the schema

      java.lang.IllegalArgumentException: Field RIDX not found in struct<f1:bigint,f2:decimal(20,6),f3:bigint,f4:string,ridx:bigint>

      at org.apache.orc.impl.ParserUtils.findColumn(ParserUtils.java:399)
      at org.apache.orc.impl.ParserUtils.findColumn(ParserUtils.java:337)
      at org.apache.orc.impl.ParserUtils.findSubtype(ParserUtils.java:285)
      at org.apache.orc.TypeDescription.findSubtype(TypeDescription.java:820)
      at org.apache.orc.TypeDescription.findSubtype(TypeDescription.java:814)
      at org.apache.orc.impl.filter.leaf.LeafFilterFactory.createLeafVectorFilter(LeafFilterFactory.java:230)
      at org.apache.orc.impl.filter.FilterFactory.createSArgFilter(FilterFactory.java:123)
      at org.apache.orc.impl.filter.FilterFactory.createBatchFilter(FilterFactory.java:63)
      at org.apache.orc.impl.RecordReaderImpl.<init>(RecordReaderImpl.java:284)

      SArg to Filter conversion should respect the case-sensitivity option as supplied by the reader.

      Attachments

        Issue Links

          Activity

            People

              planka Pavan Lanka
              planka Pavan Lanka
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: