misc

Convenient classes (and functions) that do not fit any other category.

identity(tensor, **_)[source]

Simply pass through the argument and ignore keyword arguments.

This is a placeholder for instances where a default function is required.

Parameters:: tensor (Tensor) – Any argument (typically a tensor) to be passed straight through.
Returns:: The tensor passed in as argument.
Return type:: Tensor

class Identity(*_, **__)[source]

Bases: Block

PyTorch module that passes a tensor right through, doing nothing.

This is a placeholder for instances where a default module is required. Providing any number of (keyword) arguments on instantiation is permitted, but they will be ignored.

forward(tensor, **_)[source]

Simply pass through the argument and ignore keyword arguments.

Parameters:: tensor (Tensor) – Any argument (typically a tensor) to be passed straight through.
Returns:: The tensor passed in as argument.
Return type:: Tensor

new(*_, **__)[source]

Return a fresh, new instance.

Providing any number of (keyword) arguments is permitted, but they will be ignored.

Returns:: A fresh, new instance of itself.
Return type:: Identity

reset_parameters()[source]: Does nothing because there are no internal parameters to reset.

class Finalizer(mod_dim, *activations, **kwargs)[source]

Bases: Resettable

Extract one or more numbers from the final layer of a neural network.

Instances of this class serve as a convenient final layer in any neural network, no matter if it is for regression, for classification, for multiple targets, or if you predict the parameters of a probability distribution. The last activations of your network are going to be passed through as many linear layers as you need outputs, each passed through their own (and potentially different) non-linearity to give you the desired number of outputs and the desired value range for each output.

Parameters:

mod_dim (int) – The size of the last dimension of the input tensor, essentially the “width” of the neural network before it is to be collapsed to the final output.
*activations (Module) – Specify as many activations (instances of PyTorch Module) as you want outputs (e.g., Sigmoid() for binary classification, Softplus() for strictly positive regression targets, etc.). For unbounded regression targets, where you want no activation function at all, use Identity().
**kwargs – Keyword arguments are passed on to all linear layers.

References

forward(inp)[source]

Forward pass for generating mean and matching standard deviation.

Parameters:: inp (Tensor) – The activations after the last hidden layer in your network. The size of the last dimension is expected to be mod_dim.
Returns:: A tensor with the predicted mean values and a tensor with the predicted standard deviations that are guaranteed to be greater or equal to the square root of the mean. Both have the same dimension and size, viz., that of the inp with the size of the last dimension shrunk to 1.
Return type:: tuple

new(mod_dim=None, beta=None, threshold=None, **kwargs)[source]

Return a fresh instance with the same or updated parameters.

Parameters:

mod_dim (int, optional) – The size of the last dimension of the input tensor. Overwrites the mod_dim of the current instance if given. Defaults to None.
beta (float, optional) – Scaling parameter ot the Softplus activation function. Overwrites the beta of the current instance if given. Defaults to None.
threshold (float, optional) – The Softplus activation is approximated as a linear function for values greater than this. Overwrites the threshold of the current instance if given. Defaults to None.
**kwargs – Additional keyword arguments are merged into the keyword arguments of the current instance and are then passed through to the linear layers together.

Returns:

A fresh, new instance of itself.

Return type:

NegativeBinomialFinalizer

reset_parameters()[source]: Re-initialize all internal parameters.

class Compile(inplace=True, model=None, **kwargs)[source]

Bases: object

Partial of the compile top-level function or Module method.

Parameters:

inplace (bool, optional) – Whether to compile the model in place (by calling its compile method) or create a new, compiled instance. Defaults to True
model (Module, optional) – For convenience, the model to compile can already be given at instantiation. However, Nothing will happen until instances are called.
**kwargs – Additional keyword arguments are forwarded to the compile function or method call. See the Documentation for details.

__call__(model=None, **kwargs)[source]

Compile a Module with the given options.

Parameters:

model (Module, optional) – If no model was given on instantiation, one must be given here. Otherwise, there would be nothing to compile. If a model was given on instantiation and one is given here, the latter replaces the former. Defaults to None.
**kwargs – Additional keyword arguments are merged into those given at instantiation and then forward to the compile function or method call. See the Documentation for details.

Returns:

The compiled module.

Return type:

Module

Raises:

CompileError – If no model was given, neither at instantiation, nor when calling instances.

class Cat(dim=0)[source]

Bases: ArgRepr

Simple partial of PyTorch’s top-level cat function.

Parameters:: dim (int, optional) – The dimension along which to concatenate the tensors. Defaults to 0.

__call__(tensors)[source]

Concatenate the given tensors along one of their dimensions.

Parameters:: tensors (tuple or list of tensors) – Tensors to concatenate along an existing dimension.
Returns:: The concatenated tensors.
Return type:: Tensor

class LazyCatDim0(tensors)[source]

Bases: object

Lazily concatenate a sequence of tensors along their first dimension.

Concatenating a large number of even small tensors (or a small number of large tensors) causes a memory spike because, temporarily, two copies of all tensors are needed. Sometimes, this simply cannot be avoided. However, when only a small part of the full concatenation of all tensors is needed at any given time, e.g., when chopping off micro-batches of training data to feed to a model, the present class provides an alternative: Constituent tensors are kept as is and concatenation is only performed when slices or element(s) along the first dimension are requested. These are selected from the constituents first and only then concatenated (and, thus, copied). Slicing and element selection of further dimensions is delayed until selection and concatenation along the first dimension is completed.

Parameters:

tensors (iterable) – The iterable of tensors to cache, all of which must have the same number of dimensions and the exact same sizes in all dimensions but the first.

Raises:

ShapeError – If any tensor is a scalar, that is, has zero dimensions, or if the shape after the first dimension is not the same across all tensors. Also, if there are no tensors to wrap.
DeviceError – If tensors are spread over multiple devices.
DTypeError – If tensors have multiple dtypes.

property device: The common device of all cached tensors.

property dtype: The common dtype of all cached tensors.

property lookup: A lazily computed and cached lookup table for indices.

property shape: The shape of the full concatenation of the wrapped tensors.

size(dim=None)[source]

The size of the full concatenation of the wrapped tensors.

Parameters:

dim (int, optional) – The dimension for which to return the size. Defaults to None. If not given the shape of the full concatenation of all cached tensors is returned.

Returns:

int – If the size along a certain dimension was requested.
Size – A PyTorch Size object specifying the shape of the full concatenation of all cached tensors.

to(*args)[source]

Returns a new instance of self, wrapping transformed tensors.

See the PyTorch documentation for possible call signatures, but keep in mind that all listed operations will create a copy of the wrapped tensors after all.