Complex-Valued Rectified Linear Units

The Rectified Linear Unit (ReLU) is the most common activation function in modern data-driven algorithms. Hence, it is oft-extended to the complex domain. However, whereas its nonlinearity lends itself naturally to the real domain, its application to the complex domain, specifically its activation in different quadrants of the complex plane, has led to further investigation.

These variants of the complex-valued ReLU are all Type-A split activation functions, meaning they apply a function separately to the real and imaginary parts of the input tensor, as detailed in Split Type-A.

class complextorch.nn.modules.activation.complex_relu.CPReLU

Split Complex-Valued Parametric Rectified Linear Unit

Split Type-A extension of the Parametric ReLU for complex-valued tensors.

Implements the operation:

\[G(\mathbf{z}) = \texttt{PReLU}(\mathbf{x}) + j \texttt{PReLU}(\mathbf{y})\]

Based on work from the following paper:

H. Jing, S. Li, K. Miao, S. Wang, X. Cui, G. Zhao and H. Sun. Enhanced Millimeter-Wave 3-D Imaging via Complex-Valued Fully Convolutional Neural Network.

class complextorch.nn.modules.activation.complex_relu.CReLU(inplace: bool = True)

Split Complex-Valued Rectified Linear Unit

Implements the operation:

\[G(\mathbf{z}) = \texttt{ReLU}(\mathbf{x}) + j \texttt{ReLU}(\mathbf{y})\]

Alias for CVSplitReLU. The nomenclature CReLU is used only in certain literature to denote the split complex-valued rectified linear unit.

class complextorch.nn.modules.activation.complex_relu.CVSplitReLU(inplace: bool = True)

Split Complex-Valued Rectified Linear Unit

Implements the operation:

\[G(\mathbf{z}) = \texttt{ReLU}(\mathbf{x}) + j \texttt{ReLU}(\mathbf{y})\]

Based on work from the following paper:

Jingkun Gao, Bin Deng, Yuliang Qin, Hongqiang Wang and Xiang Li. Enhanced Radar Imaging Using a Complex-valued Convolutional Neural Network.