Uploaded image for project: 'Commons Numbers'
  1. Commons Numbers
  2. NUMBERS-174

Update Gamma functions using the Boost implementation

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Implemented
    • 1.0
    • 1.1
    • gamma
    • None

    Description

      The current regularised incomplete gamma functions for P and Q compute using a series representation for all input. This method is not robust to extreme arguments.

      The Gamma functions are used in Commons Statistics in the Gamma and Poisson distributions.

      Large values of the mean in the Poisson distribution have low
      precision for the CDF. The distribution also has a configurable
      threshold for convergence of the gamma function evaluation
      (STATISTICS-38). Ideally this implementation detail should be
      removed from the API.

      The Gamma distribution has a function switch based on a threshold to
      compute the PDF. This threshold results in incorrect values for
      certain parameters (STATISTICS-39).

      The function switch in the Gamma distribution is based on the
      documentation for the Boost gamma functions [1]. However it does not
      implement all the suggested optimisations detailed in the most recent
      Boost documentation. This includes avoiding using the converging
      series of the gamma function when the convergence is slow or unstable,
      i.e. addresses the need for a configurable convergence threshold in
      the Poisson distribution.

      One part of the evaluation of the incomplete gamma functions require
      accuracy in the leading term:

      x^a exp(-x) / gamma(a) = exp(a log(x) - x - loggamma(a)

      When x and a are large then using logs to compute this leads to
      cancellation. Use of logs to compute this term is done in the current
      implementation in numbers for RegularizedGamma P and Q. It is the
      source of low precision for the CDF for the Poisson distribution when
      the mean is large.

      I propose to port the Boost gamma functions to numbers. This will be a
      process similar to the recent port of the error functions to the same
      package in numbers. These functions improved both accuracy and
      speed over the existing implementation. Once the gamma functions are
      ported a comparison can be made between the existing and new
      implementations. 

      [1] https://www.boost.org/doc/libs/1_77_0/libs/math/doc/html/math_toolkit/sf_gamma/igamma.html

      Attachments

        Issue Links

          Activity

            People

              aherbert Alex Herbert
              aherbert Alex Herbert
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 10m
                  10m