vae.VAE

class vae.VAE(n_inputs, n_latent, n_encoder=[], n_decoder=[], visible_type='binary', dropout_rate=None, nonlinearity=<function relu>, weight_normalization=False, importance_weighting=False, min_divergence=0.0, **kwargs)

Variational Autoencoder

Parameters:
n_inputs : int

Number of input (visible) variables.

n_latent : int

Number of latent variables.

n_encoder : list of int, optional (default=[])

Number of hidden units for the encoders.

n_decoder : list of int, optional (default=[])

Number of hidden units for the decoders.

visible_type : str, optional (default=’binary’):

Type of the visible variables. Can be ‘binary’ or ‘real’.

dropout_rate : float or None, optional (default=None)

Dropout rate, if any, to apply to the input.

nonlinearity : optional (default=tf.nn.relu)

Nonlinearity to use for hidden units.

weight_normalization : bool, optional (default=False)

Whether to use weight normalization.

importance_weighting : bool, optional (default=False)

Whether to use importance weighting for training the model.

min_divergence : float, optional (default=0.0)

Minimum KL divergence per latent dimension. Default=0.0.

optimizer : str, optional (default=’Adam’)

Optimizer to use for training the model. See tf.contrib.layers.OPTIMIZER_CLS_NAMES for available options.

learning_rate : float, optional (default=0.001)

Learning rate for training the model.

learning_rate_decay_fn : optional (default=None)

Function for decaying the learning rate. Takes learning_rate and global_step, and return the decayed learning rate.

clip_gradients : float or None, optional (default=None)

If provided, global clipping is applied to prevent the gradient norms from exceeding the provided value.

model_dir : str or None, optional (default=None)

Path to the model directory. Defaults to the current working directory.

debug : bool, optional (default=False):

Whether to open the TensorFlow session in debug mode.

Attributes:
model_dir

str: Directory for saving and restoring the model

Methods

collect_stats(data, tensors[, enable_summaries]) Collect statistics on the provided data
evaluate(data[, tensors, combine_func, …]) Evaluate the model on the provided data.
fit(train_data[, validation_data, epochs, …]) Train the model using the provided training data
collect_stats(data, tensors, enable_summaries=False, **feed_dict)

Collect statistics on the provided data

Parameters:
data

Input data. Can be a NumPy array, a SciPy sparse matrix, or a Dataset object.

tensors: list of str

List containing names of statistics to collect.

enable_summaries : bool, optional (default=False)

Whether to write summaries to disk.

**feed_dict

Additional keyword arguments to append to the feed dictionary used for evaluation.

Returns:
results : list

The collected statistics.

evaluate(data, tensors=None, combine_func=<function _concat>, enable_summaries=False, batch_size=100, n_samples=1, training=False)

Evaluate the model on the provided data.

Any of the following can be provided for the tensors argument:

  • z_mean : Mean of latent variables.
  • z_sd : Standard deviation of latent variables.
  • x_mean : Mean of visible variables (reconstructions).
  • x_sd : Standard deviation of real-valued visible variables (reconstructions).
  • x_mean_combined : Mean of visible variables (reconstructions), averaged over drawn samples.
  • x_sd_combined : Standard deviation of real-valued visible variables (reconstructions), averaged over drawn samples.
  • log_p_x_given_z: log[p(x|z)].
  • log_p_x_given_z_combined: log[p(x|z)], combined (using ops.reduce_logmeanexp()) over drawn samples.
  • log_p_x: log[p(x|z)].
  • log_p_x_combined: log[p(x)], combined (using ops.reduce_logmeanexp()) over drawn samples.
  • NLL: Negative log-likelihood, i.e., mean(log_p_x_combined).
  • Reconstruction Error: Reconstruction error, i.e., mean(log_p_x_given_z_combined).
Parameters:
data

Input data. Can be a NumPy array, a SciPy sparse matrix, or a Dataset object.

tensors : list of str or None, optional (default=None)

List of tensor names to collect. Defaults to all available tensors.

combine_func : optional

Function to use to combine collected data from all batches. If set to None, each batch is returned separately. Defaults to concatenation along the first axis.

enable_summaries : bool, optional (default=False)

Whether to write summaries to disk.

batch_size : int, optional (default=100)

Size of mini-batches.

n_samples : int, optional (default=1)

Number of samples for the Monte Carlo estimator, i.e., number of samples to draw from the encoder.

training : bool, optional (default=False)

Whether to evaluate in training mode. Ignored if not using dropout.

fit(train_data, validation_data=None, epochs=1, shuffle=True, restore=False, summary_steps=None, init_feed_dict={}, validation_feed_dict={}, batch_size=100, n_samples=1)

Train the model using the provided training data

Parameters:
train_data

Training data. Can be a NumPy array, a SciPy sparse matrix, or a Dataset object.

validation_data : optional (default=None)

Validation data. Can be a NumPy array, a SciPy sparse matrix, or a Dataset object.

epochs : int (default=1)

Number of training epochs.

shuffle : bool, optional (default=True)

Whether to shuffle training data before each epoch. Ignored if training data is a Dataset object.

restore : bool, optional (default=False)

Whether to restore the model from a previous checkpoint.

summary_steps : int or None, optional (default=None)

Number of steps between writing summaries, or None for disabling summaries.

init_feed_dict : dict, optional (default={})

Feed dictionary to append for initialization.

validation_feed_dict : dict, optional (default={})

Feed dictionary to append for validation.

batch_size : int, optional (default=100)

Size of mini-batches.

n_samples : int, optional (default=1)

Number of samples for the Monte Carlo estimator, i.e., number of samples to draw from the encoder.

Returns:
self
model_dir

str: Directory for saving and restoring the model