theanets.layers.recurrent.RNN¶
-
class
theanets.layers.recurrent.
RNN
(size, inputs, name=None, activation='relu', **kwargs)¶ Standard recurrent network layer.
Notes
There are many different styles of recurrent network layers, but the one implemented here is known as an Elman layer or an SRN (Simple Recurrent Network) – the output from the layer at the previous time step is incorporated into the input of the layer at the current time step.
\[h_t = \sigma(x_t W_{xh} + h_{t-1} W_{hh} + b)\]Here, \(\sigma(\cdot)\) is the activation function of the layer, and the subscript represents the time step of the data being processed. The state of the hidden layer at time \(t\) depends on the input at time \(t\) and the state of the hidden layer at time \(t-1\).
Parameters
b
— biasxh
— matrix connecting inputs to hiddenshh
— matrix connecting hiddens to hiddens
Outputs
out
— the post-activation state of the layerpre
— the pre-activation state of the layer
-
__init__
(size, inputs, name=None, activation='relu', **kwargs)¶
Methods
__init__
(size, inputs[, name, activation])add_bias
(name, size[, mean, std])Helper method to create a new bias vector. add_weights
(name, nin, nout[, mean, std, ...])Helper method to create a new weight matrix. connect
(inputs)Create Theano variables representing the outputs of this layer. find
(key)Get a shared variable for a parameter by name. initial_state
(name, batch_size)Return an array of suitable for representing initial state. log
()Log some information about this layer. output_name
([name])Return a fully-scoped name for the given layer output. setup
()Set up the parameters and initial values for this layer. to_spec
()Create a specification dictionary for this layer. transform
(inputs)Transform the inputs for this layer into an output for the layer. Attributes
input_size
For networks with one input, get the input size. num_params
Total number of learnable parameters in this layer. params
A list of all parameters in this layer. -
setup
()¶ Set up the parameters and initial values for this layer.
-
transform
(inputs)¶ Transform the inputs for this layer into an output for the layer.
Parameters: inputs : dict of theano expressions
Symbolic inputs to this layer, given as a dictionary mapping string names to Theano expressions. See
base.Layer.connect()
.Returns: outputs : dict of theano expressions
A map from string output names to Theano expressions for the outputs from this layer. This layer type generates a “pre” output that gives the unit activity before applying the layer’s activation function, and an “out” output that gives the post-activation output.
updates : list of update pairs
A sequence of updates to apply inside a theano function.