Gamma Distribution

Information about Gamma Distribution

Gamma
Probability density function Probability density plots of gamma distributions
Cumulative distribution function Cumulative distribution plots of gamma distributions
Parameters	$\text{[math]}$ shape (real) $\text{[math]}$ scale (real)
Support	$\text{[math]}$
Probability density function (pdf)	$\text{[math]}$
Cumulative distribution function (cdf)	$\text{[math]}$
Mean	$\text{[math]}$
Median	no simple closed form
Mode
Variance	$\text{[math]}$
Skewness	$\text{[math]}$
Excess kurtosis	$\text{[math]}$
Entropy	$\text{[math]}$ $\text{[math]}$
Moment-generating function (mgf)
Characteristic function	$\text{[math]}$

In probability theory and statistics, the gamma distribution is a two-parameter family of continuous probability distributions. It has a scale parameter θ and a shape parameter k. If k is an integer then the distribution represents the sum of k exponentially distributed random variables, each of which has mean θ.

Characterization

That a random variable X is gamma-distributed with scale θ and shape k is denoted

Probability density function

The probability density function of the gamma distribution can be expressed in terms of the gamma function:

$\text{[math]}$

(This parameterization is used in the infobox and the plots.)

Alternatively, the gamma distribution can be parameterized in terms of a shape parameter $\text{[math]}$ and an inverse scale parameter $\text{[math]}$ , called a rate parameter:

$\text{[math]}$

If $\text{[math]}$ is a positive integer, then

Both parameterizations are common because either can be more convenient depending on the situation.

Cumulative distribution function

The cumulative distribution function can be expressed in terms of the incomplete gamma function,

$\text{[math]}$

Properties

Summation

If X_i has a Γ(α_i, β) distribution for i = 1, 2, ..., N, then

$\text{[math]}$

provided all X_i are independent.

The gamma distribution exhibits infinite divisibility.

Scaling

For any t > 0 it holds that tX is distributed Γ(k, tθ), demonstrating that θ is a scale parameter.

Exponential family

The Gamma distribution is a two-parameter exponential family with natural parameters $\text{[math]}$ and $\text{[math]}$ , and natural statistics $\text{[math]}$ and $\text{[math]}$ .

Information entropy

The information entropy is given by

$\text{[math]}$

: $\text{[math]}$

where ψ(k) is the digamma function.

Kullback-Leibler divergence

The directed Kullback-Leibler divergence between Γ(α₀, β₀) ('true' distribution) and Γ(α, β) ('approximating' distribution) is given by

$\text{[math]}$

Laplace Transform

The Laplace transformation of the gamma distribution is

$\text{[math]}$

Parameter estimation

Maximum likelihood estimation

The likelihood function for N iid observations $\text{[math]}$ is

$\text{[math]}$

from which we calculate the log-likelihood function

$\text{[math]}$

Finding the maximum with respect to $\text{[math]}$ by taking the derivative and setting it equal to zero yields the maximum likelihood estimate of the θ parameter:

$\text{[math]}$

Substituting this into the log-likelihood function gives

$\text{[math]}$

Finding the maximum with respect to k by taking the derivative and setting it equal to zero yields

$\text{[math]}$

where

$\text{[math]}$

is the digamma function.

There is no closed-form solution for k. The function is numerically very well behaved, so if a numerical solution is desired, it can be found using, for example, Newton's method. An initial value of k can be found either using the method of moments, or using the approximation

$\text{[math]}$

If we let

$\text{[math]}$

then k is approximately

$\text{[math]}$

which is within 1.5% of the correct value. An explicit form for the Newton-Raphson update of this initial guess is given by Choi and Wette (1969) as the following expression:

$\text{[math]}$

where $\text{[math]}$ denotes the trigamma function (the derivative of the digamma function).

The digamma and trigamma functions can be difficult to calculate with high precision. However, approximations known to be good to several significant figures can be computed using the following approximation formulae:

$\text{[math]}$

and

$\text{[math]}$

For details, see Choi and Wette (1969).

Bayesian minimum mean-squared error

With known k and unknown $\text{[math]}$ , the posterior PDF for theta (using the standard scale-invariant prior for $\text{[math]}$ ) is

$\text{[math]}$

Denoting

$\text{[math]}$

Integration over θ can be carried out using a change of variables, revealing that 1/θ is gamma-distributed with parameters $\text{[math]}$ .

$\text{[math]}$

The moments can be computed by taking the ratio (m by m = 0)

$\text{[math]}$

which shows that the mean +/- standard deviation estimate of the posterior distribution for theta is

$\text{[math]}$ +/- $\text{[math]}$

Generating gamma-distributed random variables

Given the scaling property above, it is enough to generate gamma variables with β = 1 as we can later convert to any value of β with simple division.

Using the fact that a Γ(1, 1) distribution is the same as an Exp(1) distribution, and noting the method of generating exponential variables, we conclude that if U is uniformly distributed on (0, 1], then −ln(U) is distributed Γ(1, 1). Now, using the "α-addition" property of gamma distribution, we expand this result:

$\text{[math]}$

where U_k are all uniformly distributed on (0, 1] and independent.

All that is left now is to generate a variable distributed as Γ(δ, 1) for 0 < δ < 1 and apply the "α-addition" property once more. This is the most difficult part.

We provide an algorithm without proof. It is an instance of the acceptance-rejection method:

Let m be 1.
Generate , and — independent uniformly distributed on (0, 1] variables.
If , where $\text{[math]}$ , then go to step 4, else go to step 5.
Let . Go to step 6.
Let .
If $\text{[math]}$ , then increment m and go to step 2.
Assume $\text{[math]}$ to be the realization of $\text{[math]}$

Now, to summarize,

$\text{[math]}$

where [k] is the integral part of k, and ξ has been generated using the algorithm above with δ = {k} (the fractional part of k), U_k and V_l are distributed as explained above and are all independent.

The GNU Scientific Library (which has ports for Visual Studio) has robust routines for sampling many distributions including the Gamma distribution.

Related distributions

Specializations

If $\text{[math]}$ , then X has an exponential distribution with rate parameter λ.
If $\text{[math]}$ , then X is identical to χ²(ν), the chi-square distribution with ν degrees of freedom.
If $\text{[math]}$ is an integer, the gamma distribution is an Erlang distribution and is the probability distribution of the waiting time until the $\text{[math]}$ -th "arrival" in a one-dimensional Poisson process with intensity 1/θ.
If $\text{[math]}$ , then X has a Maxwell-Boltzmann distribution with parameter a.
$\text{[math]}$ , then $\text{[math]}$

Others

If X has a Γ(k, θ) distribution, then 1/X has an inverse-gamma distribution with parameters k and θ^-1.
If X and Y are independently distributed Γ(α, θ) and Γ(β, θ) respectively, then X / (X + Y) has a beta distribution with parameters α and β.
If X_i are independently distributed Γ(α_i,θ) respectively, then the vector (X₁ / S, ..., X_n / S), where S = X₁ + ... + X_n, follows a Dirichlet distribution with parameters α₁, ..., α_n.

References

R. V. Hogg and A. T. Craig. Introduction to Mathematical Statistics, 4th edition. New York: Macmillan, 1978. (See Section 3.3.)
Eric W. Weisstein, Gamma distribution at MathWorld.
Engineering Statistics Handbook
S. C. Choi and R. Wette. (1969) Maximum Likelihood Estimation of the Parameters of the Gamma Distribution and Their Bias, Technometrics, 11(4) 683-69

	Probability distributions [ • • [ edit] ]
Univariate	Multivariate
Discrete:	Benford • Bernoulli • binomial • Boltzmann • categorical • compound Poisson • discrete phase-type • degenerate • Gauss-Kuzmin • geometric • hypergeometric • logarithmic • negative binomial • parabolic fractal • Poisson • Rademacher • Skellam • uniform • Yule-Simon • zeta • Zipf • Zipf-Mandelbrot	Ewens • multinomial • multivariate Polya
Continuous:	Beta • Beta prime • Cauchy • chi-square • Dirac delta function • Coxian • Erlang • exponential • exponential power • F • fading • Fermi-Dirac • Fisher's z • Fisher-Tippett • Gamma • generalized extreme value • generalized hyperbolic • generalized inverse Gaussian • Half-Logistic • Hotelling's T-square • hyperbolic secant • hyper-exponential • hypoexponential • inverse chi-square (scaled inverse chi-square) • inverse Gaussian • inverse gamma (scaled inverse gamma) • Kumaraswamy • Landau • Laplace • Lvy • Lvy skew alpha-stable • logistic • log-normal • Maxwell-Boltzmann • Maxwell speed • Nakagami • normal (Gaussian) • normal-gamma • normal inverse Gaussian • Pareto • Pearson • phase-type • polar • raised cosine • Rayleigh • relativistic Breit-Wigner • Rice • shifted Gompertz • Student's t • triangular • truncated normal • type-1 Gumbel • type-2 Gumbel • uniform • Variance-Gamma • Voigt • von Mises • Weibull • Wigner semicircle • Wilks' lambda	Dirichlet • Generalized Dirichlet distribution . inverse-Wishart • Kent • matrix normal • multivariate normal • multivariate Student • von Mises-Fisher • Wigner quasi • Wishart
Miscellaneous:	bimodal • Cantor • conditional • equilibrium • exponential family • infinitely divisible • location-scale family • marginal • maximum entropy • posterior • prior • quasi • sampling • singular

Definition

..... Click the link for more information.

In mathematics, the real numbers may be described informally as numbers that can be given by an infinite decimal representation, such as 2.4871773339…. The real numbers include both rational numbers, such as 42 and −23/129, and irrational numbers, such as π and
..... Click the link for more information.

In probability theory and statistics, a scale parameter is a special kind of numerical parameter of a parametric family of probability distributions.

Definition

If a family of probability densities with parameter s is of the form

..... Click the link for more information.

In mathematics, a support of a function f from a set X to the real numbers R is a subset Y of X such that f (x) is zero for all x in X and outside Y.
..... Click the link for more information.

In mathematics, a probability density function (pdf) is a function that represents a probability distribution in terms of integrals.

Formally, a probability distribution has density f, if f
..... Click the link for more information.

In probability theory, the cumulative distribution function (CDF), also called probability distribution function or just distribution function,^[1] completely describes the probability distribution of a real-valued random variable X.
..... Click the link for more information.

expected value (or mathematical expectation, or mean) of a discrete random variable is the sum of the probability of each possible outcome of the experiment multiplied by the outcome value (or payoff).
..... Click the link for more information.

median is described as the number separating the higher half of a sample, a population, or a probability distribution, from the lower half. The median of a finite list of numbers can be found by arranging all the observations from lowest value to highest value and picking
..... Click the link for more information.

In statistics, mode means the most frequent value assumed by a random variable, or occurring in a sampling of a random variable. The term is applied both to probability distributions and to collections of experimental data.
..... Click the link for more information.

variance of a random variable (or somewhat more precisely, of a probability distribution) is one measure of statistical dispersion, averaging the squared distance of its possible values from the expected value.
..... Click the link for more information.

skewness is a measure of the asymmetry of the probability distribution of a real-valued random variable.

Introduction

Consider the distribution in the figure. The bars on the right side of the distribution taper differently than the bars on the left side.
..... Click the link for more information.

kurtosis (from the Greek word kurtos, meaning bulging) is a measure of the "peakedness" of the probability distribution of a real-valued random variable. Higher kurtosis means more of the variance is due to infrequent extreme deviations, as opposed to frequent
..... Click the link for more information.

Shannon entropy or information entropy is a measure of the uncertainty associated with a random variable.

Shannon entropy quantifies the information contained in a piece of data: it is the minimum average message length, in bits (if using base-2 logarithms), that must
..... Click the link for more information.

In probability theory and statistics, the moment-generating function of a random variable X is

wherever this expectation exists. The moment-generating function generates the moments of the probability distribution.
..... Click the link for more information.

In probability theory, the characteristic function of any random variable completely defines its probability distribution. On the real line it is given by the following formula, where X is any random variable with the distribution in question:

..... Click the link for more information.

Probability theory is the branch of mathematics concerned with analysis of random phenomena.^[1] The central objects of probability theory are random variables, stochastic processes, and events: mathematical abstractions of non-deterministic events or measured quantities
..... Click the link for more information.

Statistics is a mathematical science pertaining to the collection, analysis, interpretation or explanation, and presentation of data. It is applicable to a wide variety of academic disciplines, from the physical and social sciences to the humanities.
..... Click the link for more information.

probability distribution that assigns a probability to every subset (more precisely every measurable subset) of its state space in such a way that the probability axioms are satisfied.
..... Click the link for more information.

In probability theory and statistics, a scale parameter is a special kind of numerical parameter of a parametric family of probability distributions.

Definition

If a family of probability densities with parameter s is of the form

..... Click the link for more information.

In probability theory and statistics, a shape parameter is a special kind of numerical parameter of a parametric family of probability distributions.

Definition

..... Click the link for more information.

exponential distributions are a class of continuous probability distribution. They are often used to model the time between independent events that happen at a constant average rate.
..... Click the link for more information.

Gamma function (represented by the capitalized Greek letter Γ) is an extension of the factorial function to real and complex numbers. For a complex number z with positive real part it is defined by

..... Click the link for more information.

In mathematics, the gamma function is defined by a definite integral. The incomplete gamma function is defined as an integral function of the same integrand. There are two varieties of the incomplete gamma function: the upper incomplete gamma function
..... Click the link for more information.

In probability theory, to say that two events are independent, intuitively means that the occurrence of one event makes it neither more nor less probable that the other occurs.
..... Click the link for more information.

The concept of infinite divisibility arises in different ways in philosophy, physics, economics, order theory (a branch of mathematics), and probability theory (also a branch of mathematics).
..... Click the link for more information.

In probability theory and statistics, a scale parameter is a special kind of numerical parameter of a parametric family of probability distributions.

Definition

If a family of probability densities with parameter s is of the form

..... Click the link for more information.

In probability and statistics, an exponential family is any class of probability distributions having a certain form. This special form is chosen for mathematical convenience, on account of some useful algebraic properties; as well as for generality, as exponential families are in
..... Click the link for more information.

This article is copied from an article on Wikipedia.org - the free encyclopedia created and edited by online user community. The text was not checked or edited by anyone on our staff. Although the vast majority of the wikipedia encyclopedia articles provide accurate and timely information please do not assume the accuracy of any particular article. This article is distributed under the terms of GNU Free Documentation License.

Herod_Archelaus

iPedia Free Information Center

Gamma Distribution

Information about Gamma Distribution

Characterization

Probability density function

Cumulative distribution function

Properties

Summation

Scaling

Exponential family

Information entropy

Kullback-Leibler divergence

Laplace Transform

Parameter estimation

Maximum likelihood estimation

Bayesian minimum mean-squared error

Generating gamma-distributed random variables

Related distributions

Specializations

Others

References

See also

Definition

Definition

Introduction

Definition

Definition

Definition