Inference¶

mellon.inference.calculate_batch_elbo(logprob, rng, params, num_samples)View on GitHub ¶

Calculates the average ELBO over a batch of random samples.

Parameters:

logprob – log probability of the sample
rng – random number generator key
params – parameters of the Gaussian distribution
num_samples – number of samples in the batch

Returns:

batch ELBO

mellon.inference.calculate_elbo(logprob, rng, mean, log_std)View on GitHub ¶

Calculates the single-sample Monte Carlo estimate of the variational lower bound (ELBO).

Parameters:

logprob – log probability of the sample
rng – random number generator
mean – mean of the Gaussian distribution
log_std – logarithm of standard deviation of the Gaussian distribution

Returns:

ELBO estimate

mellon.inference.calculate_gaussian_logpdf(x, mean, log_std)View on GitHub ¶

Calculates the log probability density function of a multivariate Gaussian with diagonal covariance.

Parameters:

x – value to evaluate the Gaussian on
mean – mean of the Gaussian distribution
log_std – logarithm of standard deviation of the Gaussian distribution

Returns:

log pdf of the Gaussian distribution

mellon.inference.compute_conditional(x, landmarks, pre_transformation, pre_transformation_std, y, mu, cov_func, L, Lp=None, sigma=0, jitter=1e-06, y_is_mean=False, with_uncertainty=False)View on GitHub ¶

Builds the mean function of the Gaussian process, conditioned on the function values (e.g., log-density) on x. Returns an instance of mellon.Predictor that acts as a function, defined on the whole domain of x.

Parameters:

x (array-like) – The training instances.
landmarks (array-like or None) – The landmark points for fast sparse computation. Landmarks can be None if not using landmark points.
pre_transformation (array-like or None) – The pre-transformed latent function representation.
pre_transformation_std (array-like or None) – Standard deviation of the parameters, e.g., as inferred by ADVI.
y (array-like) – The function values at each point in x.
mu (float) – The original Gaussian process mean \(\mu\).
cov_func (function) – The Gaussian process covariance function.
L (array-like) – The matrix \(L\) used to transform the latent function representation to thr Gaussin Process mean. Typically \(L L^\top \approx K\), where \(K\) is the covariance matrix of the Gaussian Process.
Lp (array-like, optional) – A matrix such that \(L_p L_p^\top = \Sigma_p\), where \(\Sigma_p\) is the covariance matrix of the Gaussian Process on the inducing points.
sigma (float, optional) – White noise variance, by default 0.
jitter (float, optional) – A small amount to add to the diagonal for stability, by default 1e-6.
y_is_mean (bool) – Wether to consider y the GP mean or a noise measurment subject to sigma or y_cov_factor. Has no effect if L is passed. Defaults to False.
with_uncertainty (bool) – Wether to compute covariance functions and predictive uncertainty. Defaults to False.

Returns:

The conditioned Gaussian process mean function.

Return type:

mellon.Predictor

mellon.inference.compute_conditional_explog(x, landmarks, pre_transformation, pre_transformation_std, y, mu, cov_func, L, Lp, sigma=0, jitter=1e-06, y_is_mean=False, with_uncertainty=False)View on GitHub ¶

Builds the exp-mean function of the Gaussian process, conditioned on the function log-values (e.g., dimensionality) on x. Returns a function that is defined on the whole domain of x.

Parameters:

x (array-like) – The training instances.
landmarks (array-like or None) – The landmark points for fast sparse computation. Landmarks can be None if not using landmark points.
pre_transformation (array-like or None) – The pre-transformed latent function representation.
pre_transformation_std (array-like or None) – Standard deviation of the parameters, e.g., as inferred by ADVI.
y (array-like) – The function values at each point in x.
mu (float) – The original Gaussian process mean \(\mu\).
cov_func (function) – The Gaussian process covariance function.
L (array-like) – The matrix \(L\) used to transform the latent function representation to thr Gaussin Process mean. Typically \(L L^\top \approx K\), where \(K\) is the covariance matrix of the Gaussian Process.
Lp (array-like, optional) – A matrix such that \(L_p L_p^\top = \Sigma_p\), where \(\Sigma_p\) is the covariance matrix of the Gaussian Process on the inducing points.
sigma (float, optional) – White noise variance, by default 0.
jitter (float, optional) – A small amount to add to the diagonal for stability, by default 1e-6.
y_is_mean (bool) – Wether to consider y the GP mean or a noise measurment subject to sigma or y_cov_factor. Has no effect if L is passed. Defaults to False.
with_uncertainty (bool) – Wether to compute covariance functions and predictive uncertainty. Defaults to False.

Returns:

The conditioned Gaussian process mean function.

Return type:

mellon.Predictor

mellon.inference.compute_conditional_times(x, landmarks, pre_transformation, pre_transformation_std, y, mu, cov_func, L, Lp, sigma=0, jitter=1e-06, y_is_mean=False, with_uncertainty=False)View on GitHub ¶

Builds the mean function of the Gaussian process, conditioned on the function values (e.g., log-density) on x, taking into account the associated times of each sample. Returns an instance of mellon.Predictor that acts as a function, defined on the whole domain of x, and can be evaluated at any given time.

Parameters:

x (array-like) – The training instances.
landmarks (array-like or None) – The landmark points for fast sparse computation. Landmarks can be None if not using landmark points.
pre_transformation (array-like or None) – The pre-transformed latent function representation.
pre_transformation_std (array-like or None) – Standard deviation of the parameters, e.g., as inferred by ADVI.
y (array-like) – The function values at each point in x.
mu (float) – The original Gaussian process mean \(\mu\).
cov_func (function) – The Gaussian process covariance function.
L (array-like) – A matrix such that \(L L^\top \approx K\), where \(K\) is the covariance matrix of the Gaussian Process.
Lp (array-like, optional) – A matrix such that \(L_p L_p^\top = \Sigma_p\), where \(\Sigma_p\) is the covariance matrix of the Gaussian Process on the inducing points.
sigma (float, optional) – White noise variance, by default 0.
jitter (float, optional) – A small amount to add to the diagonal for stability, by default 1e-6.
y_is_mean (bool) – Wether to consider y the GP mean or a noise measurment subject to sigma or y_cov_factor. Has no effect if L is passed. Defaults to False.
with_uncertainty (bool) – Wether to compute covariance functions and predictive uncertainty. Defaults to False.

Returns:

The conditioned Gaussian process mean function that also accepts a ‘times’ argument to account for time-sensitive inferences.

Return type:

mellon.Predictor

mellon.inference.compute_dimensionality_loss_func(distances, transform, k)View on GitHub ¶

Computes the Bayesian loss function -(prior(\(z\)) + likelihood(transform(\(z\)))) for dimensionality inference.

Parameters:

distances (array-like) – The observed k nearest neighbor distances.
transform (function) – Maps \(z \sim \text{Normal}(0, I) \rightarrow \log(f) \sim \text{Normal}(\mu, K')\), where \(I\) is the identity matrix and \(K \approx K' = L L^\top\), where \(K\) is the covariance matrix.
k (int) – dimension of transform input

Returns:

loss_func - The Bayesian loss function

Return type:

function, function

mellon.inference.compute_dimensionality_transform(mu_dim, mu_dens, L)View on GitHub ¶

Computes a function transform that maps \(z \sim \text{Normal}(0, I) \rightarrow \log(f) \sim \text{Normal}(\mu, K')\), where \(I\) is the identity matrix and \(K \approx K' = L L^\top\), where \(K\) is the covariance matrix.

Parameters:

mu (float) – The Gaussian process mean \(\mu\).
L (array-like) – A matrix such that \(L L^\top \approx K\), where \(K\) is the covariance matrix.

Returns:

transform - The transform function \(z \rightarrow f\).

mellon.inference.compute_log_density_x(pre_transformation, transform)View on GitHub ¶

Computes the log density at the training points.

Parameters:

pre_transformation (array-like) – \(z \sim \text{Normal}(0, I)\)
transform (function) – A function \(z \sim \text{Normal}(0, I) \rightarrow f \sim \text{Normal}(\mu, K')\), where \(I\) is the identity matrix and \(K \approx K' = L L^\top\), where \(K\) is the covariance matrix.

Returns:

log_density_x - The log density at the training points.

mellon.inference.compute_loss_func(nn_distances, d, transform, k)View on GitHub ¶

Computes the Bayesian loss function -(prior(\(z\)) + likelihood(transform(\(z\)))).

Parameters:

nn_distances (array-like) – The observed nearest neighbor distances.
d (int) – The dimensionality of the data.
transform (function) – Maps \(z \sim \text{Normal}(0, I) \rightarrow f \sim \text{Normal}(\mu, K')\), where \(I\) is the identity matrix and \(K \approx K' = L L^\top\), where \(K\) is the covariance matrix.
k (int) – dimension of transform input

Returns:

loss_func - The Bayesian loss function

Return type:

function, function

mellon.inference.compute_parameter_cov_factor(pre_transformation_std, L)View on GitHub ¶

Computes \(\Sigma_L\) the left factor of the covariance matrix of log_density_x the mean function of the Gaussian Process on the training data points. The uncertainty of the mean function comes from the uncertainty of the inferred model parameters quantified by pre_transformation_std.

Parameters:

pre_transformation_std (array-like) – Standard deviation of the parameters, e.g., as inferred by ADVI.
L (array-like) – A matrix such that \(L L^\top \approx K\), where \(K\) is the covariance matrix of the Gaussian Process.

Returns:

sigma_L - The left factor of the covariance matrix of the transformed parameters.

mellon.inference.compute_transform(mu, L)View on GitHub ¶

Computes a function transform that maps \(z \sim \text{Normal}(0, I) \rightarrow f \sim \text{Normal}(\mu, K')\), where \(I\) is the identity matrix and \(K \approx K' = L L^\top\), where \(K\) is the covariance matrix.

Parameters:

mu (float) – The Gaussian process mean:math:mu.
L (array-like) – A matrix such that \(L L^\top \approx K\), where \(K\) is the covariance matrix.

Returns:

transform - The transform function \(z \rightarrow f\).

mellon.inference.generate_gaussian_sample(rng, mean, log_std)View on GitHub ¶

Generates a single sample from a multivariate Gaussian with diagonal covariance.

Parameters:

rng – random number generator
mean – mean of the Gaussian distribution
log_std – logarithm of standard deviation of the Gaussian distribution

Returns:

sample from the Gaussian distribution

mellon.inference.minimize_adam(loss_func, initial_value, n_iter=100, init_learn_rate=0.1, jit=False)View on GitHub ¶

Minimizes function with a starting guess of initial_value using adam and exponentially decaying learning rate.

Parameters:

loss_func (function) – The loss function to minimize.
initial_value (array-like) – The initial guess.
n_iter (integer) – The number of optimization iterations. Defaults to 100.
init_learn_rate (float) – The initial learn rate. Defaults to 1.

Returns:

Results - A named tuple containing pre_transformation, opt_state, losses: The optimized parameters, final state of the optimizer, and history of loss values,

Return type:

array-like, array-like, Object

mellon.inference.minimize_lbfgsb(loss_func, initial_value, jit=False)View on GitHub ¶

Minimizes function with a starting guess of initial_value.

Parameters:

loss_func (function) – Loss function to minimize.
initial_value (array-like) – Initial guess.

Returns:

Results - A named tuple containing pre_transformation, opt_state, loss: The optimized parameters, final state of the optimizer, and the final loss value,

Return type:

array-like, array-like, Object

mellon.inference.run_advi(loss_func, initial_parameters, n_iter=100, init_learn_rate=0.1, nsamples=40, jit=False)View on GitHub ¶

Performs automatic differentiation variational inference (ADVI) to fit a Gaussian approximation to an intractable, unnormalized density.

Parameters:

loss_func – function to calculate the loss
initial_parameters – initial parameters for the optimization
n_iter – number of iterations for the optimization (default: DEFAULT_N_ITER)
init_learn_rate (float) – The initial learn rate. Defaults to 1.

Returns:

parameters, standard deviations, and loss after the optimization

Returns:

Results - A named tuple containing pre_transformation, pre_transformation_std, losses: The optimized parameters, the optimized standard deviations, and a history of ELBO values.

Return type:

array-like, array-like, Object