Optimization

The module pyro.optim provides support for optimization in Pyro. In particular it provides PyroOptim, which is used to wrap PyTorch optimizers and manage optimizers for dynamically generated parameters (see the tutorial SVI Part I for a discussion). Any custom optimization algorithms are also to be found here.

Pyro Optimizers

AdagradRMSProp(optim_args)[source]

A wrapper for an optimizer that is a mash-up of Adagrad and RMSprop.

ClippedAdam(optim_args)[source]

A wrapper for a modification of the Adam optimization algorithm that supports gradient clipping.

class PyroOptim(optim_constructor, optim_args)[source]

Bases: object

A wrapper for torch.optim.Optimizer objects that helps with managing dynamically generated parameters.

Parameters:
  • optim_constructor – a torch.optim.Optimizer
  • optim_args – a dictionary of learning arguments for the optimizer or a callable that returns such dictionaries
get_state()[source]

Get state associated with all the optimizers in the form of a dictionary with key-value pairs (parameter name, optim state dicts)

load(filename)[source]
Parameters:filename – file name to load from

Load optimizer state from disk

save(filename)[source]
Parameters:filename – file name to save to

Save optimizer state to disk

set_state(state_dict)[source]

Set the state associated with all the optimizers using the state obtained from a previous call to get_state()

Pytorch Optimizers

LBFGS(optim_args)

Wraps torch.optim.LBFGS with PyroOptim.

Adamax(optim_args)

Wraps torch.optim.Adamax with PyroOptim.

Adagrad(optim_args)

Wraps torch.optim.Adagrad with PyroOptim.

SGD(optim_args)

Wraps torch.optim.SGD with PyroOptim.

Adam(optim_args)

Wraps torch.optim.Adam with PyroOptim.

Rprop(optim_args)

Wraps torch.optim.Rprop with PyroOptim.

ASGD(optim_args)

Wraps torch.optim.ASGD with PyroOptim.

RMSprop(optim_args)

Wraps torch.optim.RMSprop with PyroOptim.

SparseAdam(optim_args)

Wraps torch.optim.SparseAdam with PyroOptim.

Adadelta(optim_args)

Wraps torch.optim.Adadelta with PyroOptim.