Primitive Distributions¶

class
Distribution
(reparameterized=None)[source]¶ Bases:
object
Base class for parameterized probability distributions.
Distributions in Pyro are stochastic function objects with
.sample()
and.log_pdf()
methods. Pyro provides two versions of each stochastic function:(i) lowercase versions that take parameters:
x = dist.bernoulli(param) # Returns a sample of size size(param). p = dist.bernoulli.log_pdf(x, param) # Evaluates log probability of x.
and (ii) UpperCase distribution classes that can construct stochastic functions with fixed parameters:
d = dist.Bernoulli(param) x = d() # Samples a sample of size size(param). p = d.log_pdf(x) # Evaluates log probability of x.
Under the hood the lowercase versions are aliases for the UpperCase versions.
Note
Parameters and data should be of type torch.autograd.Variable and all methods return type torch.autograd.Variable unless otherwise noted.
Tensor Shapes:
Distributions provide a method
.shape()
for the tensor shape of samples:x = d.sample(*args, **kwargs) assert x.shape == d.shape(*args, **kwargs)
Pyro distinguishes two different roles for tensor shapes of samples:
 The leftmost dimension corresponds to iid batching, which can be
treated specially during inference via the
.batch_log_pdf()
method.  The rightmost dimensions correspond to event shape.
These shapes are related by the equation:
assert d.shape(*args, **kwargs) == (d.batch_shape(*args, **kwargs) + d.event_shape(*args, **kwargs))
There are exceptions, for instance, in the case of the Categorical distribution, without one hot encoding.
Distributions provide a vectorized
.batch_log_pdf()
method that evaluates the log probability density of each event in a batch independently, returning a tensor of shaped.batch_shape(x) + (1,)
:x = d.sample(*args, **kwargs) assert x.shape == d.shape(*args, **kwargs) log_p = d.batch_log_pdf(x, *args, **kwargs) assert log_p.shape == d.batch_shape(*args, **kwargs) + (1,)
Distributions may also support broadcasting of the
.log_pdf()
and.batch_log_pdf()
methods, which may each be evaluated with a sample tensor x that is larger than (but broadcastable from) the parameters. In this case,d.batch_shape(x)
will return the shape of the broadcasted batch shape using the data tensor x:x = d.sample() xx = torch.stack([x, x]) d.batch_log_pdf(xx).size() == d.batch_shape(xx) + (1,)) # returns True
Implementing New Distributions:
Derived classes must implement the following methods:
.sample()
,.batch_log_pdf()
,.batch_shape()
, and.event_shape()
. Discrete classes may also implement the.enumerate_support()
method to improve gradient estimates and set.enumerable = True
.Examples:
Take a look at the examples to see how they interact with inference algorithms.

analytic_mean
(*args, **kwargs)[source]¶ Analytic mean of the distribution, to be implemented by derived classes.
Note that this is optional, and currently only used for testing distributions.
Returns: Analytic mean. Return type: torch.autograd.Variable. Raises: NotImplementedError if mean cannot be analytically computed.

analytic_var
(*args, **kwargs)[source]¶ Analytic variance of the distribution, to be implemented by derived classes.
Note that this is optional, and currently only used for testing distributions.
Returns: Analytic variance. Return type: torch.autograd.Variable. Raises: NotImplementedError if variance cannot be analytically computed.

batch_log_pdf
(x, *args, **kwargs)[source]¶ Evaluates log probability densities for each of a batch of samples.
Parameters: x (torch.autograd.Variable) – A single value or a batch of values batched along axis 0. Returns: log probability densities as a onedimensional torch.autograd.Variable with same batch size as value and params. The shape of the result should be self.batch_size(). Return type: torch.autograd.Variable

batch_shape
(x=None, *args, **kwargs)[source]¶ The lefthand tensor shape of samples, used for batching.
Samples are of shape
d.shape(x) == d.batch_shape(x) + d.event_shape()
.Parameters: x – Data that is used to determine the batch shape. This is optional. If not specified, the distribution parameters are used to determine the shape of the batch that is returned from sample()
.Returns: Tensor shape used for batching. Return type: torch.Size Raises: ValueError if the parameters are not broadcastable to the data shape

enumerable
= False¶

enumerate_support
(*args, **kwargs)[source]¶ Returns a representation of the parametrized distribution’s support.
This is implemented only by discrete distributions.
Returns: An iterator over the distribution’s discrete support. Return type: iterator

event_dim
(*args, **kwargs)[source]¶ Returns: Number of dimensions of individual events. Return type: int

event_shape
(x=None, *args, **kwargs)[source]¶ The righthand tensor shape of samples, used for individual events. The event dimension(/s) is used to designate random variables that could potentially depend on each other, for instance in the case of Dirichlet or the categorical distribution, but could also simply be used for logical grouping, for example in the case of a normal distribution with a diagonal covariance matrix.
Samples are of shape d.shape(x) == d.batch_shape(x) + d.event_shape().
Returns: Tensor shape used for individual events. Return type: torch.Size

log_pdf
(x, *args, **kwargs)[source]¶ Evaluates total log probability density of a batch of samples.
Parameters: x (torch.autograd.Variable) – A value. Returns: total log probability density as a onedimensional torch.autograd.Variable of size 1. Return type: torch.autograd.Variable

reparameterized
= False¶

sample
(*args, **kwargs)[source]¶ Samples a random value.
For tensor distributions, the returned Variable should have the same .size() as the parameters, unless otherwise noted.
Returns: A random value or batch of random values (if parameters are batched). The shape of the result should be self.size(). Return type: torch.autograd.Variable
 The leftmost dimension corresponds to iid batching, which can be
treated specially during inference via the
Bernoulli¶

class
Bernoulli
(ps=None, logits=None, batch_size=None, log_pdf_mask=None, *args, **kwargs)[source]¶ Bases:
pyro.distributions.distribution.Distribution
Bernoulli distribution.
Distribution over a vector of independent Bernoulli variables. Each element of the vector takes on a value in {0, 1}.
This is often used in conjunction with torch.nn.Sigmoid to ensure the ps parameters are in the interval [0, 1].
Parameters:  ps (torch.autograd.Variable) – Probabilities. Should lie in the interval [0,1].
 logits – Log odds, i.e. \(\log(\frac{p}{1  p})\). Either ps or logits should be specified, but not both.
 batch_size – The number of elements in the batch used to generate a sample. The batch dimension will be the leftmost dimension in the generated sample.
 log_pdf_mask – Tensor that is applied to the batch log pdf values as a multiplier. The most common use case is supplying a boolean tensor mask to mask out certain batch sites in the log pdf computation.

analytic_mean
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.analytic_mean()
.

analytic_var
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.analytic_var()
.

batch_log_pdf
(x)[source]¶ Ref:
pyro.distributions.distribution.Distribution.batch_log_pdf()

batch_shape
(x=None)[source]¶ Ref:
pyro.distributions.distribution.Distribution.batch_shape()
.

enumerable
= True¶

enumerate_support
()[source]¶ Returns the Bernoulli distribution’s support, as a tensor along the first dimension.
Note that this returns support values of all the batched RVs in lockstep, rather than the full cartesian product. To iterate over the cartesian product, you must construct univariate Bernoullis and use itertools.product() over all univariate variables (may be expensive).
Returns: torch variable enumerating the support of the Bernoulli distribution. Each item in the return value, when enumerated along the first dimensions, yields a value from the distribution’s support which has the same dimension as would be returned by sample. Return type: torch.autograd.Variable.

event_shape
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.event_shape()
.
Beta¶

class
Beta
(alpha, beta, batch_size=None, *args, **kwargs)[source]¶ Bases:
pyro.distributions.distribution.Distribution
Univariate beta distribution parameterized by alpha and beta.
This is often used in conjunction with torch.nn.Softplus to ensure alpha and beta parameters are positive.
Parameters:  alpha (torch.autograd.Variable) – Lower shape parameter. Should be positive.
 beta (torch.autograd.Variable) – Upper shape parameter. Should be positive.

analytic_mean
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.analytic_mean()

analytic_var
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.analytic_var()

batch_log_pdf
(x)[source]¶ Ref:
pyro.distributions.distribution.Distribution.batch_log_pdf()

batch_shape
(x=None)[source]¶ Ref:
pyro.distributions.distribution.Distribution.batch_shape()

event_shape
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.event_shape()
.
Categorical¶

class
Categorical
(ps=None, vs=None, logits=None, one_hot=True, batch_size=None, log_pdf_mask=None, *args, **kwargs)[source]¶ Bases:
pyro.distributions.distribution.Distribution
Categorical (discrete) distribution.
Discrete distribution over elements of vs with \(P(vs[i]) \propto ps[i]\). If
one_hot=True
,.sample()
returns a onehot vector; otherwise.sample()
returns the category index.Parameters:  ps (torch.autograd.Variable) – Probabilities. These should be nonnegative and normalized along the rightmost axis.
 logits (torch.autograd.Variable) – Log probability values. When exponentiated, these should sum to 1 along the last axis. Either ps or logits should be specified but not both.
 vs (list or numpy.ndarray or torch.autograd.Variable) – Optional list of values in the support.
 one_hot – Whether
sample()
returns a one_hot sample. Defaults to False if vs is specified, or True if vs is not specified.  batch_size (int) – Optional number of elements in the batch used to generate a sample. The batch dimension will be the leftmost dimension in the generated sample.

batch_log_pdf
(x)[source]¶ Evaluates log probability densities for one or a batch of samples and parameters. The last dimension for ps encodes the event probabilities, and the remaining dimensions are considered batch dimensions.
ps and vs are first broadcasted to the size of the data x. The data tensor is used to to create a mask over vs where a 1 in the mask indicates that the corresponding value in vs was selected. Since, ps and vs have the same size, this mask when applied over ps gives the probabilities of the selected events. The method returns the logarithm of these probabilities.
Returns: tensor with log probabilities for each of the batches. Return type: torch.autograd.Variable

batch_shape
(x=None)[source]¶ Ref:
pyro.distributions.distribution.Distribution.batch_shape()

enumerable
= True¶

enumerate_support
()[source]¶ Returns the categorical distribution’s support, as a tensor along the first dimension.
Note that this returns support values of all the batched RVs in lockstep, rather than the full cartesian product. To iterate over the cartesian product, you must construct univariate Categoricals and use itertools.product() over all univariate variables (but this is very expensive).
Parameters:  ps (torch.autograd.Variable) – Tensor where the last dimension denotes the event probabilities, p_k, which must sum to 1. The remaining dimensions are considered batch dimensions.
 vs (list or numpy.ndarray or torch.autograd.Variable) – Optional parameter, enumerating the items in the support. This could either
have a numeric or string type. This should have the same dimension as
ps
.  one_hot (boolean) – Denotes whether one hot encoding is enabled. This is True by default. When set to false, and no explicit vs is provided, the last dimension gives the onehot encoded value from the support.
Returns: Torch variable or numpy array enumerating the support of the categorical distribution. Each item in the return value, when enumerated along the first dimensions, yields a value from the distribution’s support which has the same dimension as would be returned by sample. If
one_hot=True
, the last dimension is used for the onehot encoding.Return type: torch.autograd.Variable or numpy.ndarray.

event_shape
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.event_shape()

sample
()[source]¶ Returns a sample which has the same shape as ps (or vs), except that if
one_hot=True
(and no vs is specified), the last dimension will have the same size as the number of events. The type of the sample is numpy.ndarray if vs is a list or a numpy array, else a tensor is returned.Returns: sample from the Categorical distribution Return type: numpy.ndarray or torch.LongTensor
Cauchy¶

class
Cauchy
(mu, gamma, batch_size=None, *args, **kwargs)[source]¶ Bases:
pyro.distributions.distribution.Distribution
Cauchy (a.k.a. Lorentz) distribution.
This is a continuous distribution which is roughly the ratio of two Gaussians if the second Gaussian is zero mean. The distribution is over tensors that have the same shape as the parameters mu`and `gamma, which in turn must have the same shape as each other.
This is often used in conjunction with torch.nn.Softplus to ensure the gamma parameter is positive.
Parameters:  mu (torch.autograd.Variable) – Location parameter.
 gamma (torch.autograd.Variable) – Scale parameter. Should be positive.

analytic_mean
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.analytic_mean()

analytic_var
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.analytic_var()

batch_log_pdf
(x)[source]¶ Ref:
pyro.distributions.distribution.Distribution.batch_log_pdf()

batch_shape
(x=None)[source]¶ Ref:
pyro.distributions.distribution.Distribution.batch_shape()

event_shape
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.event_shape()
Delta¶

class
Delta
(v, batch_size=None, *args, **kwargs)[source]¶ Bases:
pyro.distributions.distribution.Distribution
Degenerate discrete distribution (a single point).
Discrete distribution that assigns probability one to the single element in its support. Delta distribution parameterized by a random choice should not be used with MCMC based inference, as doing so produces incorrect results.
Parameters: v (torch.autograd.Variable) – The single support element. 
batch_log_pdf
(x)[source]¶ Ref:
pyro.distributions.distribution.Distribution.batch_log_pdf()

batch_shape
(x=None)[source]¶ Ref:
pyro.distributions.distribution.Distribution.batch_shape()

enumerable
= True¶

enumerate_support
(v=None)[source]¶ Returns the delta distribution’s support, as a tensor along the first dimension.
Parameters: v – torch variable where each element of the tensor represents the point at which the delta distribution is concentrated. Returns: torch variable enumerating the support of the delta distribution. Return type: torch.autograd.Variable.

event_shape
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.event_shape()

Normal¶

class
Normal
(mu, sigma, batch_size=None, log_pdf_mask=None, *args, **kwargs)[source]¶ Bases:
pyro.distributions.distribution.Distribution
Univariate normal (Gaussian) distribution.
A distribution over tensors in which each element is independent and Gaussian distributed, with its own mean and standard deviation. The distribution is over tensors that have the same shape as the parameters mu and sigma, which in turn must have the same shape as each other.
This is often used in conjunction with torch.nn.Softplus to ensure the sigma parameters are positive.
Parameters:  mu (torch.autograd.Variable) – Means.
 sigma (torch.autograd.Variable) – Standard deviations. Should be positive and the same shape as mu.

analytic_mean
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.analytic_mean()

analytic_var
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.analytic_var()

batch_log_pdf
(x)[source]¶ Diagonal Normal loglikelihood
Ref:
pyro.distributions.distribution.Distribution.batch_log_pdf()

batch_shape
(x=None)[source]¶ Ref:
pyro.distributions.distribution.Distribution.batch_shape()

event_shape
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.event_shape()

reparameterized
= True¶
Exponential¶

class
Exponential
(lam, batch_size=None, *args, **kwargs)[source]¶ Bases:
pyro.distributions.distribution.Distribution
Exponential parameterized by scale lambda.
This is often used in conjunction with torch.nn.Softplus to ensure the lam parameter is positive.
Parameters: lam (torch.autograd.Variable) – Scale parameter (a.k.a. lambda). Should be positive. 
analytic_mean
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.analytic_mean()

analytic_var
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.analytic_var()

batch_log_pdf
(x)[source]¶ Ref:
pyro.distributions.distribution.Distribution.batch_log_pdf()

batch_shape
(x=None)[source]¶ Ref:
pyro.distributions.distribution.Distribution.batch_shape()

event_shape
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.event_shape()

reparameterized
= True¶

Gamma¶

class
Gamma
(alpha, beta, batch_size=None, *args, **kwargs)[source]¶ Bases:
pyro.distributions.distribution.Distribution
Gamma distribution parameterized by alpha and beta.
This is often used in conjunction with torch.nn.Softplus to ensure alpha and beta parameters are positive.
Parameters:  alpha (torch.autograd.Variable) – Shape parameter. Should be positive.
 beta (torch.autograd.Variable) – Shape parameter. Should be positive. Shouldb be the same shape as alpha.

analytic_mean
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.analytic_mean()

analytic_var
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.analytic_var()

batch_log_pdf
(x)[source]¶ Ref:
pyro.distributions.distribution.Distribution.batch_log_pdf()

batch_shape
(x=None)[source]¶ Ref:
pyro.distributions.distribution.Distribution.batch_shape()

event_shape
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.event_shape()
HalfCauchy¶

class
HalfCauchy
(mu, gamma, batch_size=None, *args, **kwargs)[source]¶ Bases:
pyro.distributions.distribution.Distribution
HalfCauchy distribution.
This is a continuous distribution with lowerbounded domain (x > mu). See also the Cauchy distribution.
This is often used in conjunction with torch.nn.Softplus to ensure the gamma parameter is positive.
Parameters:  mu – mean (tensor)
 gamma – scale (tensor (0, Infinity))

batch_log_pdf
(x)[source]¶ Ref:
pyro.distributions.distribution.Distribution.batch_log_pdf()

batch_shape
(x=None)[source]¶ Ref:
pyro.distributions.distribution.Distribution.batch_shape()

event_shape
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.event_shape()
LogNormal¶

class
LogNormal
(mu, sigma, batch_size=None, *args, **kwargs)[source]¶ Bases:
pyro.distributions.distribution.Distribution
Lognormal distribution.
A distribution over probability vectors obtained by exptransforming a random variable drawn from
Normal({mu: mu, sigma: sigma})
.This is often used in conjunction with torch.nn.Softplus to ensure the sigma parameters are positive.
Parameters:  mu (torch.autograd.Variable) – log mean parameter.
 sigma (torch.autograd.Variable) – log standard deviations. Should be positive.

analytic_mean
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.analytic_mean()

analytic_var
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.analytic_var()

batch_log_pdf
(x)[source]¶ Ref:
pyro.distributions.distribution.Distribution.batch_log_pdf()

batch_shape
(x=None)[source]¶ Ref:
pyro.distributions.distribution.Distribution.batch_shape()

event_shape
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.event_shape()

reparameterized
= True¶

sample
()[source]¶ Reparameterized lognormal sampler. Ref:
pyro.distributions.distribution.Distribution.sample()
Multinomial¶

class
Multinomial
(ps, n, batch_size=None, *args, **kwargs)[source]¶ Bases:
pyro.distributions.distribution.Distribution
Multinomial distribution.
Distribution over counts for n independent Categorical(ps) trials.
This is often used in conjunction with torch.nn.Softmax to ensure probabilites ps are normalized.
Parameters:  ps (torch.autograd.Variable) – Probabilities (real). Should be positive and should normalized over the rightmost axis.
 n (int) – Number of trials. Should be positive.

analytic_mean
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.analytic_mean()

analytic_var
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.analytic_var()

batch_log_pdf
(x)[source]¶ Ref:
pyro.distributions.distribution.Distribution.batch_log_pdf()

batch_shape
(x=None)[source]¶ Ref:
pyro.distributions.distribution.Distribution.batch_shape()

event_shape
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.event_shape()
Poisson¶

class
Poisson
(lam, batch_size=None, *args, **kwargs)[source]¶ Bases:
pyro.distributions.distribution.Distribution
Poisson distribution over integers parameterized by scale lambda.
This is often used in conjunction with torch.nn.Softplus to ensure the lam parameter is positive.
Parameters: lam (torch.autograd.Variable) – Mean parameter (a.k.a. lambda). Should be positive. 
analytic_mean
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.analytic_mean()

analytic_var
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.analytic_var()

batch_log_pdf
(x)[source]¶ Ref:
pyro.distributions.distribution.Distribution.batch_log_pdf()

batch_shape
(x=None)[source]¶ Ref:
pyro.distributions.distribution.Distribution.batch_shape()

event_shape
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.event_shape()

Uniform¶

class
Uniform
(a, b, batch_size=None, *args, **kwargs)[source]¶ Bases:
pyro.distributions.distribution.Distribution
Uniform distribution over the continuous interval [a, b].
Parameters:  a (torch.autograd.Variable) – lower bound (real).
 b (torch.autograd.Variable) – upper bound (real). Should be greater than a.

analytic_mean
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.analytic_mean()

analytic_var
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.analytic_var()

batch_log_pdf
(x)[source]¶ Ref:
pyro.distributions.distribution.Distribution.batch_log_pdf()

batch_shape
(x=None)[source]¶ Ref:
pyro.distributions.distribution.Distribution.batch_shape()

event_shape
()[source]¶ Ref:
pyro.distributions.distribution.Distribution.event_shape()

reparameterized
= False¶