Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-17617

Remainder(%) expression.eval returns incorrect result

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.6.3, 2.0.1, 2.1.0
    • SQL

    Description

      Problem

      Remainder(%) expression returns incorrect result when using expression.eval to calculate the result. expression.eval is called in case like constant folding.

      scala> -5083676433652386516D  % 10
      res19: Double = -6.0
      
      // Wrong answer with eval!!!
      scala> Seq("-5083676433652386516D").toDF.select($"value" % 10).show
      |(value % 10)|
      +------------+
      |         0.0|
      +------------+
      
      // Triggers codegen, will  not do constant folding
      scala> sc.makeRDD(Seq("-5083676433652386516D")).toDF.select($"value" % 10).show
      +------------+
      |(value % 10)|
      +------------+
      |        -6.0|
      +------------+
      

      Behavior of postgres:

      seanzhong=# select -5083676433652386516.0  % 10;
       ?column? 
      ----------
           -6.0
      (1 row)
      

      Attachments

        Activity

          People

            clockfly Sean Zhong
            clockfly Sean Zhong
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: