functions.py

---

pycmtensor.functions

PyCMTensor functions module

relu(x, alpha=0.0)

Compute the element-wise rectified linear activation function.

Source from Theano 0.7.1

Parameters:

Name	Type	Description	Default
x	TensorVariable	The input symbolic tensor.	required
alpha	float or TensorSharedVariable	The slope for negative input. A value between 0 and 1. Default is 0.	0.0

Returns:

Type	Description
TensorVariable	The element-wise rectified linear activation function applied to x.

neg_relu(x, alpha=0.0)

negative variant of relu

exp_mov_average(batch_avg, moving_avg, alpha=0.1)

Calculates the exponential moving average (EMA) of a new minibatch

Parameters:

Name	Type	Description	Default
batch_avg	array - like	The mean batch value.	required
moving_avg	array - like	The accumulated mean.	required
alpha	float	The ratio of moving average to batch average.	0.1

Returns:

Type	Description
TensorVariable	The new moving average

Note

The moving average will decay by the difference between the existing value and the new value multiplied by the moving average factor. A higher alpha value results in faster changing moving average.

Formula:

\[ x_{EMA} = \alpha * x_t + x_{EMA} * (1-\alpha) \]

logit(utility, avail=None)

Computes the Logit function, with availability conditions.

Parameters:

Name	Type	Description	Default
utility	Union[list, tuple, TensorVariable]	utility equations	required
avail	Union[list, tuple, TensorVariable]	availability conditions, if no availability conditions are provided, defaults to 1 for all availabilities.	None

Returns:

Type	Description
TensorVariable	A NxM matrix of probabilities.

Note

The 0-th dimension is the numbering of alternatives, the N-th dimension is the size of the input (# rows).

log_likelihood(prob, y, index=None)

Symbolic representation of the log likelihood cost function.

Parameters:

Name	Type	Description	Default
prob	TensorVariable	choice probabilites tensor	required
y	TensorVariable	choice variable tensor	required
index	TensorVariable	index tensor, if None, dynamically get the same of the y tensor	None

Returns:

Type	Description
TensorVariable	a symbolic tensor of the log likelihood

Note

The 0-th dimension is the numbering of alternatives, the N-th dimension is the size of the input (# rows).

rmse(y_hat, y)

Computes the root mean squared error (RMSE) between pairs of observations

Parameters:

Name	Type	Description	Default
y_hat	TensorVariable	model estimated values	required
y	TensorVariable	ground truth values	required

Returns:

Type	Description
TensorVariable	symbolic scalar representation of the rmse

Note

Tensor is flattened to a dim=1 vector if the input tensor is dim=2.

mae(y_hat, y)

Computes the mean absolute error (MAE) between pairs of observations

Parameters:

Name	Type	Description	Default
y_hat	TensorVariable	model estimated values	required
y	TensorVariable	ground truth values	required

Returns:

Type	Description
TensorVariable	symbolic scalar representation of the mean absolute error

Note

Tensor is flattened to a dim=1 vector if the input tensor is dim=2.

kl_divergence(p, q)

Computes the KL divergence loss between discrete distributions p and q.

Parameters:

Name	Type	Description	Default
p	TensorVariable	model output probabilities	required
q	TensorVariable	ground truth probabilities	required

Returns:

Type	Description
TensorVariable	a symbolic representation of the KL loss

Note

Formula:

\[ L = \begin{cases} \sum_{i=1}^N (p_i * log(p_i/q_i)) & p>0\\ 0 & p<=0 \end{cases} \]

kl_multivar_norm(m0, v0, m1, v1, epsilon=1e-06)

Computes the KL divergence loss between two multivariate normal distributions.

Parameters:

Name	Type	Description	Default
m0	TensorVariable	mean vector of the first Normal m.v. distribution \(N_0\)	required
v0	TensorVariable	(co-)variance matrix of the first Normal m.v. distribution \(N_0\)	required
m1	TensorVariable	mean vector of the second Normal m.v. distribution \(N_1\)	required
v1	TensorVariable	(co-)variance of the second Normal m.v. distribution \(N_1\)	required
epsilon	float	small value to prevent divide-by-zero error	1e-06

Note

k = dimension of the distribution.

Formula:

\[ D_{KL}(N_0||N_1) = 0.5 * \Big(\ln\big(\frac{|v_1|}{|v_0|}\big) + trace(v_1^{-1} v_0) + (m_1-m_0)^T v_1^{-1} (m_1-m_0) - k\Big) \]

In variational inference, the kl divergence is the relative entropy between a diagonal multivariate Normal and a standard Normal distribution, \(N(0, 1)\), therefore, for VI, m1=1, v1=1

For two univariate distributions, dimensions of m0,m1,v0,v1 = 0

errors(prob, y)

Symbolic representation of the discrete prediction as a percentage error.

Parameters:

Name	Type	Description	Default
prob	TensorVariable	choice probabilites tensor	required
y	TensorVariable	choice variable tensor	required

Returns:

Type	Description
TensorVariable	the mean prediction error over y

second_order_derivative(cost, params)

Symbolic representation of the 2^nd order Hessian matrix given cost.

Parameters:

Name	Type	Description	Default
cost	TensorVariable	function to compute the gradients over	required
params	List[Beta]	params to compute the gradients over	required

Returns:

Type	Description
TensorVariable	the Hessian matrix of the cost function wrt to the params

Note

Parameters with status=1 are ignored.

first_order_derivative(cost, params)

Symbolic representation of the 1^st order gradient vector given the cost.

Parameters:

Name	Type	Description	Default
cost	TensorVariable	function to compute the gradients over	required
params	List[Beta]	params to compute the gradients over	required

Returns:

Type	Description
TensorVariable	the gradient vector of the cost function wrt to the params

Note

Parameters with status=1 are ignored.