Split Type-A Activation Functions

Type-A activation functions consist of two real-valued functions, \(G_\mathbb{R}(\cdot)\) and \(G_\mathbb{I}(\cdot)\), which are applied to the real and imaginary parts of the input tensor, respectively, as

\[G(\mathbf{z}) = G_\mathbb{R}(\mathbf{x}) + j G_\mathbb{I}(\mathbf{y}),\]

where \(\mathbf{z} = \mathbf{x} + j\mathbf{y}\).

In most cases, \(G_\mathbb{R}(\cdot) = G_\mathbb{I}(\cdot)\); however, \(G_\mathbb{R}(\cdot)\) and \(G_\mathbb{I}(\cdot)\) can also be distinct functions. A generalized Type-A split activation function is defined in GeneralizedSplitActivation, which accepts two real-valued torch.nn.Module objects for \(G_\mathbb{R}(\cdot)\) and \(G_\mathbb{I}(\cdot)\), respectively.

class complextorch.nn.modules.activation.split_type_A.CSigmoid

Alias for the CVSplitSigmoid

Implements the operation:

\[G(\mathbf{z}) = \text{sigmoid}(\mathbf{x}) + j \text{sigmoid}(\mathbf{y})\]
class complextorch.nn.modules.activation.split_type_A.CTanh

Alias for the CVSplitTanh

Implements the operation:

\[G(\mathbf{z}) = \tanh(\mathbf{x}) + j \tanh(\mathbf{y})\]

Based on work from the following paper:

A Hirose, S Yoshida. Generalization characteristics of complex-valued feedforward neural networks in relation to signal coherence.

class complextorch.nn.modules.activation.split_type_A.CVSplitAbs

Split Absolute Value Activation Function.

Implements the operation:

\[G(\mathbf{z}) = |\mathbf{x}| + j |\mathbf{y}|\]

Based on work from the following paper:

A Marseet, F Sahin. Application of complex-valued convolutional neural network for next generation wireless networks.

forward(input: CVTensor) → CVTensor

Computes the Type-A split abs() activation function.

Parameters:

input (CVTensor) – input tensor

Returns:

\(|\mathbf{x}| + j |\mathbf{y}|\)

Return type:

CVTensor

class complextorch.nn.modules.activation.split_type_A.CVSplitSigmoid

Split Complex-Valued Sigmoid

Implements the operation:

\[G(\mathbf{z}) = \text{sigmoid}(\mathbf{x}) + j \text{sigmoid}(\mathbf{y})\]
class complextorch.nn.modules.activation.split_type_A.CVSplitTanh

Split Complex-Valued Hyperbolic Tangent

Implements the operation:

\[G(\mathbf{z}) = \tanh(\mathbf{x}) + j \tanh(\mathbf{y})\]

Based on work from the following paper:

A Hirose, S Yoshida. Generalization characteristics of complex-valued feedforward neural networks in relation to signal coherence.

class complextorch.nn.modules.activation.split_type_A.GeneralizedSplitActivation(activation_r: Module, activation_i: Module)

Generalized Split Type-A Activation Function

Operates on the real and imaginary parts separately.

Implements the operation:

\[G(\mathbf{z}) = G_\mathbb{R}(\mathbf{x}) + j G_\mathbb{I}(\mathbf{y}),\]

where \(\mathbf{z} = \mathbf{x} + j\mathbf{y}\).

Type-A nomenclature is defined in the following paper:

J. A. Barrachina, C. Ren, G. Vieillard, C. Morisseau, and J.-P. Ovarlez. Theory and Implementation of Complex-Valued Neural Networks.

forward(input: CVTensor) → CVTensor

Computes the generalized Type-A split activation function.

Parameters:

input (CVTensor) – input tensor

Returns:

\(\texttt{activation_r}(input.real) + j\texttt{activation_i}(input.imag)\)

Return type:

CVTensor