Transformed Distribution¶

TransformedDistribution¶

class TransformedDistribution(base_distribution, bijectors, *args, **kwargs)[source]

Transforms the base distribution by applying a sequence of Bijectors to it. This results in a scorable distribution (i.e. it has a log_pdf() method).

Parameters: base_distribution (pyro.distribution.Distribution) – a (continuous) base distribution; samples from this distribution are passed through the sequence of Bijectors to yield a sample from the TransformedDistribution bijectors – either a single Bijector or a sequence of Bijectors wrapped in a nn.ModuleList the transformed distribution
batch_log_pdf(y, *args, **kwargs)[source]
batch_shape(x=None, *args, **kwargs)[source]
event_shape(*args, **kwargs)[source]
log_pdf(y, *args, **kwargs)[source]
Parameters: y (torch.autograd.Variable) – a value sampled from the transformed distribution the score (the log pdf) of y torch.autograd.Variable

Scores the sample by inverting the bijector(s) and computing the score using the score of the base distribution and the log det jacobian

sample(*args, **kwargs)[source]
Returns: a sample y torch.autograd.Variable

Sample from base distribution and pass through bijector(s)

Bijector¶

class Bijector(*args, **kwargs)[source]

Bases: torch.nn.modules.module.Module

Abstract class Bijector. Bijector are bijective transformations with computable log det jacobians. They are meant for use in TransformedDistribution.

inverse(*args, **kwargs)[source]

Virtual inverse method

Inverts the bijection y => x.

log_det_jacobian(*args, **kwargs)[source]

Virtual logdet jacobian method.

Computes the log det jacobian |dy/dx|

InverseAutoRegressiveFlow¶

class InverseAutoregressiveFlow(input_dim, hidden_dim, sigmoid_bias=2.0, permutation=None)[source]

An implementation of an Inverse Autoregressive Flow. Together with the TransformedDistribution this provides a way to create richer variational approximations.

Example usage:

>>> base_dist = Normal(...)
>>> iaf = InverseAutoregressiveFlow(...)
>>> pyro.module("my_iaf", iaf)
>>> iaf_dist = TransformedDistribution(base_dist, iaf)


Note that this implementation is only meant to be used in settings where the inverse of the Bijector is never explicitly computed (rather the result is cached from the forward call). In the context of variational inference, this means that the InverseAutoregressiveFlow should only be used in the guide, i.e. in the variational distribution. In other contexts the inverse could in principle be computed but this would be a (potentially) costly computation that scales with the dimension of the input (and in any case support for this is not included in this implementation).

Parameters: input_dim (int) – dimension of input hidden_dim (int) – hidden dimension (number of hidden units) sigmoid_bias (float) – bias on the hidden units fed into the sigmoid; default=2.0 permutation (bool) – whether the order of the inputs should be permuted (by default the conditional dependence structure of the autoregression follows the sequential order)

References:

1. Improving Variational Inference with Inverse Autoregressive Flow [arXiv:1606.04934] Diederik P. Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, Ilya Sutskever, Max Welling

2. Variational Inference with Normalizing Flows [arXiv:1505.05770] Danilo Jimenez Rezende, Shakir Mohamed

3. MADE: Masked Autoencoder for Distribution Estimation [arXiv:1502.03509] Mathieu Germain, Karol Gregor, Iain Murray, Hugo Larochelle

get_arn()[source]
Return type: pyro.nn.AutoRegressiveNN

Return the AutoRegressiveNN associated with the InverseAutoregressiveFlow

inverse(y, *args, **kwargs)[source]
Parameters: y (torch.autograd.Variable) – the output of the bijection

Inverts y => x. As noted above, this implementation is incapable of inverting arbitrary values y; rather it assumes y is the result of a previously computed application of the bijector to some x (which was cached on the forward call)

log_det_jacobian(y, *args, **kwargs)[source]

Calculates the determinant of the log jacobian