Fix gamma CDF normalization and incomplete gamma fraction - #23
Open
gaoflow wants to merge 1 commit into
Open
Conversation
GammaDistribution#cdf was scaled by scale/Gamma(shape), so it returned values outside [0,1] (e.g. 3.0 for shape=2, scale=3) and saturated below 1.0 whenever the two did not cancel. incomplete_gamma is already regularized. gamma_fraction returned after its first loop iteration and never applied the Lentz update, leaving a 0.5-1.5% error on the x >= a+1 branch. The update now runs every iteration with the XMININ guards, mirroring beta_fraction, and the result is returned after the loop. Tests cover the [0,1] range for scale != Gamma(shape) and mpmath reference values on both incomplete-gamma branches.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
GammaDistribution#cdf returns values outside [0,1]: shape=2, scale=3 gives 3.0 at x=500, and shape=5, scale=10 saturates at 0.4167.
incomplete_gammais already the regularized CDF, so the extrascale / Gamma(shape)factors are spurious — they only cancel whenscale == Gamma(shape).The same path also mis-computes the upper tail:
gamma_fractionreturned after its first loop iteration and never applied the Lentz update, leaving ~0.5–1.5% error for x/scale >= shape+1. The update now runs every iteration with the standard XMININ guards, matching the structure ofbeta_fractionin the same file.Tests cover the [0,1] range and monotonicity for scale != Gamma(shape), plus cdf reference values from mpmath over both incomplete-gamma branches. Existing cdf deltas are tightened from 0.01, which previously absorbed the tail error.
Repro:
Rubystats::GammaDistribution.new(2.0, 3.0).cdf(500.0)→ 3.0 before, 1.0 after.