Automatic Guide Generation

The pyro.contrib.autoguide module provides algorithms to automatically generate guides from simple models, for use in SVI. For example to generate a mean field Gaussian guide:

def model():
    ...

guide = AutoDiagonalNormal(model)  # a mean field guide
svi = SVI(model, guide, Adam({'lr': 1e-3}), Trace_ELBO())

Automatic guides can also be combined using pyro.poutine.block() and AutoGuideList.

AutoGuide

class AutoGuide(model, prefix='auto')[source]

Bases: object

Base class for automatic guides.

Derived classes must implement the __call__() method.

Auto guides can be used individually or combined in an AutoGuideList object.

Parameters:
  • model (callable) – a pyro model
  • prefix (str) – a prefix that will be prefixed to all param internal sites
__call__(*args, **kwargs)[source]

A guide with the same *args, **kwargs as the base model.

Returns:A dict mapping sample site name to sampled value.
Return type:dict
sample_latent(*args, **kwargs)[source]

Samples an encoded latent given the same *args, **kwargs as the base model.

AutoGuideList

class AutoGuideList(model, prefix='auto')[source]

Bases: pyro.contrib.autoguide.AutoGuide

Container class to combine multiple automatic guides.

Example usage:

guide = AutoGuideList(my_model)
guide.add(AutoDiagonalNormal(poutine.block(model, hide=["assignment"])))
guide.add(AutoDiscreteParallel(poutine.block(model, expose=["assignment"])))
svi = SVI(model, guide, optim, Trace_ELBO())
Parameters:
  • model (callable) – a Pyro model
  • prefix (str) – a prefix that will be prefixed to all param internal sites
__call__(*args, **kwargs)[source]

A composite guide with the same *args, **kwargs as the base model.

Returns:A dict mapping sample site name to sampled value.
Return type:dict
add(part)[source]

Add an automatic guide for part of the model. The guide should have been created by blocking the model to restrict to a subset of sample sites. No two parts should operate on any one sample site.

Parameters:part (AutoGuide) – a partial guide to add

AutoDelta

class AutoDelta(model, prefix='auto')[source]

Bases: pyro.contrib.autoguide.AutoGuide

This implementation of AutoGuide uses Delta distributions to construct a MAP guide over the entire latent space. The guide does not depend on the model’s *args, **kwargs.

Usage:

guide = AutoDelta(model)
svi = SVI(model, guide, ...)

By default latent variables are randomly initialized by the model. To change this default behavior the user should call pyro.param() before beginning inference, with "auto_" prefixed to the targetd sample site names e.g. for sample sites named “level” and “concentration”, initialize via:

pyro.param("auto_level", torch.tensor([-1., 0., 1.]))
pyro.param("auto_concentration", torch.ones(k),
           constraint=constraints.positive)
__call__(*args, **kwargs)[source]

An automatic guide with the same *args, **kwargs as the base model.

Returns:A dict mapping sample site name to sampled value.
Return type:dict
median(*args, **kwargs)[source]

Returns the posterior median value of each latent variable.

Returns:A dict mapping sample site name to median tensor.
Return type:dict

AutoContinuous

class AutoContinuous(model, prefix='auto')[source]

Bases: pyro.contrib.autoguide.AutoGuide

Base class for implementations of continuous-valued Automatic Differentiation Variational Inference [1].

Each derived class implements its own sample_latent() method.

Assumes model structure and latent dimension are fixed, and all latent variables are continuous.

Parameters:model (callable) – a Pyro model

Reference:

[1] ‘Automatic Differentiation Variational Inference’,
Alp Kucukelbir, Dustin Tran, Rajesh Ranganath, Andrew Gelman, David M. Blei
__call__(*args, **kwargs)[source]

An automatic guide with the same *args, **kwargs as the base model.

Returns:A dict mapping sample site name to sampled value.
Return type:dict
median(*args, **kwargs)[source]

Returns the posterior median value of each latent variable.

Returns:A dict mapping sample site name to median tensor.
Return type:dict
quantiles(quantiles, *args, **kwargs)[source]

Returns posterior quantiles each latent variable. Example:

print(guide.quantiles([0.05, 0.5, 0.95]))
Parameters:quantiles (torch.Tensor or list) – A list of requested quantiles between 0 and 1.
Returns:A dict mapping sample site name to a list of quantile values.
Return type:dict
sample_latent(*args, **kwargs)[source]

Samples an encoded latent given the same *args, **kwargs as the base model.

AutoMultivariateNormal

class AutoMultivariateNormal(model, prefix='auto')[source]

Bases: pyro.contrib.autoguide.AutoContinuous

This implementation of AutoContinuous uses a Cholesky factorization of a Multivariate Normal distribution to construct a guide over the entire latent space. The guide does not depend on the model’s *args, **kwargs.

Usage:

guide = AutoMultivariateNormal(model)
svi = SVI(model, guide, ...)

By default the mean vector is initialized to zero and the Cholesky factor is initialized to the identity. To change this default behavior the user should call pyro.param() before beginning inference, e.g.:

latent_dim = 10
pyro.param("auto_loc", torch.randn(latent_dim))
pyro.param("auto_scale_tril", torch.tril(torch.rand(latent_dim)),
           constraint=constraints.lower_cholesky)
sample_latent(*args, **kwargs)[source]

Samples the (single) multivariate normal latent used in the auto guide.

AutoDiagonalNormal

class AutoDiagonalNormal(model, prefix='auto')[source]

Bases: pyro.contrib.autoguide.AutoContinuous

This implementation of AutoContinuous uses a Normal distribution with a diagonal covariance matrix to construct a guide over the entire latent space. The guide does not depend on the model’s *args, **kwargs.

Usage:

guide = AutoDiagonalNormal(model)
svi = SVI(model, guide, ...)

By default the mean vector is initialized to zero and the scale is initialized to the identity. To change this default behavior the user should call pyro.param() before beginning inference, e.g.:

latent_dim = 10
pyro.param("auto_loc", torch.randn(latent_dim))
pyro.param("auto_scale", torch.ones(latent_dim),
           constraint=constraints.positive)
sample_latent(*args, **kwargs)[source]

Samples the (single) diagnoal normal latent used in the auto guide.

AutoLowRankMultivariateNormal

class AutoLowRankMultivariateNormal(model, prefix='auto', rank=1)[source]

Bases: pyro.contrib.autoguide.AutoContinuous

This implementation of AutoContinuous uses a low rank plus diagonal Multivariate Normal distribution to construct a guide over the entire latent space. The guide does not depend on the model’s *args, **kwargs.

Usage:

guide = AutoLowRankMultivariateNormal(model, rank=10)
svi = SVI(model, guide, ...)

By default the D_term is initialized to 1/2 and the W_term is intialized randomly such that W_term.t().matmul(W_term) is half the identity matrix. To change this default behavior the user should call pyro.param() before beginning inference, e.g.:

latent_dim = 10
pyro.param("auto_loc", torch.randn(latent_dim))
pyro.param("auto_W_term", torch.randn(latent_dim)))
pyro.param("auto_D_term", torch.randn(latent_dim).exp()),
           constraint=constraints.positive)
Parameters:
  • model (callable) – a generative model
  • rank (int) – the rank of the low-rank part of the covariance matrix
  • prefix (str) – a prefix that will be prefixed to all param internal sites
sample_latent(*args, **kwargs)[source]

Samples the (single) multivariate normal latent used in the auto guide.

AutoDiscreteParallel

class AutoDiscreteParallel(model, prefix='auto')[source]

Bases: pyro.contrib.autoguide.AutoGuide

A discrete mean-field guide that learns a latent discrete distribution for each discrete site in the model.

__call__(*args, **kwargs)[source]

An automatic guide with the same *args, **kwargs as the base model.

Returns:A dict mapping sample site name to sampled value.
Return type:dict