Uploaded image for project: 'XMLBeans'
  1. XMLBeans
  2. XMLBEANS-410

xsd2inst incorrectly handling decimal precision

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • Version 2.4.1
    • Version 2.4.1
    • Tools
    • None
    • All OS, all hardware

    Description

      as an example, a xs:decimal (totalDig:20, fracDig:0) was creating 1000.00 whic of course, does not validate.

      The problem is in the org.apache.xmlbeans.impl.xsd2inst.SampleXmlUtil.java class.
      Root cause of the problem is that the programmer was calling the setScale() method on a BigDecimal, as though it modifies the existing BigDecimal object (i.e. BigDecimal bd = new BigDecimal("1000.00"); bd.setScale(1)) setScale does not modify the existing, but returns a new BigDecimal object. Therefore the correct call would be (BigDecimal bd = new BigDecimal("1000.00"); bd = bd.setScale(1)).
      The lines included below represent the repair of the formatDecimal() method which is the eronious function.

      // We have the number
      // Adjust the scale according to the totalDigits and fractionDigits
      int digits = 0;
      BigDecimal ONE = new BigDecimal(BigInteger.ONE);
      for (BigDecimal n = result; n.abs().compareTo(ONE) >= 0; digits++)
      n = n.movePointLeft(1);

      if (fractionDigits > 0)
      if (totalDigits >= 0)
      result = result.setScale(Math.max(fractionDigits, totalDigits - digits));
      else
      result = result.setScale(fractionDigits);
      else if (fractionDigits == 0)
      result = result.setScale(0);

      return result.toString();
      }

      I would recommend that the code tree be searched for other setScale method calls to see if others need to be fixed. I currently do not have write access to the subversion repository, so I am unable to check this fix in myself.

      I was unsure whether to code this as a minor or major problem. It appears that the difference between the two is the presence of a work-around, for which there is none in this case.

      Attachments

        Activity

          People

            Unassigned Unassigned
            mark.henning Mark D Henning
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 1h
                1h
                Remaining:
                Remaining Estimate - 1h
                1h
                Logged:
                Time Spent - Not Specified
                Not Specified