theanets.losses.CrossEntropy¶
-
class
theanets.losses.
CrossEntropy
(target, weight=1.0, weighted=False, output_name='out')[source]¶ Cross-entropy (XE) loss function for classifiers.
Parameters: - target : int
Number of dimensions required to store the target values for computing the loss.
- weight : float, optional
The importance of this loss for the model being trained. Defaults to 1.
- weighted : bool, optional
If True, a floating-point array of weights with the same dimensions as
out_dim
will be required to compute the “weighted” loss. Defaults to False.- output_name : str, optional
Name of the network output to tap for computing the loss. Defaults to ‘out:out’, the name of the default output of the last layer in a linear network.
Notes
The cross-entropy between a “true” distribution over discrete classes \(p(t)\) and a “model” distribution over predicted classes \(q(x)\) is the expected number of bits needed to store the model distribution, under the expectation of the true distribution. Mathematically, this loss computes:
\[\mathcal{L}(x, t) = - \sum_{k=1}^K p(t=k) \log q(x=k)\]The loss value is similar to the KL divergence between \(p\) and \(q\), but it is specifically aimed at classification models. When using this loss, targets are assumed to be integers in the half-open interval \([0, k)\); internally, the loss is computed by first taking the log of the model distribution and then summing up only the entries in the resulting array corresponding to the true class.
Attributes: - weight : float, optional
The importance of this loss for the model being trained.
- output_name : str
Name of the network output to tap for computing the loss.
-
__init__
(target, weight=1.0, weighted=False, output_name='out')[source]¶ x.__init__(…) initializes x; see help(type(x)) for signature
Methods
__init__
(target[, weight, weighted, output_name])x.__init__(…) initializes x; see help(type(x)) for signature accuracy
(outputs)Build a Theano expression for computing the accuracy of graph output. log
()Log some diagnostic info about this loss. Attributes
variables
A list of Theano variables used in this loss. -
accuracy
(outputs)[source]¶ Build a Theano expression for computing the accuracy of graph output.
Parameters: - outputs : dict of Theano expressions
A dictionary mapping network output names to Theano expressions representing the outputs of a computation graph.
Returns: - acc : Theano expression
A Theano expression representing the accuracy of the output compared to the target data.