Convolution Layers
- class complextorch.nn.modules.conv.CVConv1d(in_channels: int, out_channels: int, kernel_size: int | Tuple[int], stride: int | Tuple[int] = 1, padding: str | int | Tuple[int] = 0, dilation: int | Tuple[int] = 1, groups: int = 1, bias: bool = True, padding_mode: str = 'zeros', device=None, dtype=None)
1-D Complex-Valued Convolution
Based on the PyTorch torch.nn.Conv1d implementation.
Employs Gauss’ multiplication trick to reduce number of computations by 25% compare with the naive implementation.
The most common implementation of complex-valued convolution entails the following computation:
\[G(\mathbf{z}) = \text{conv}(\mathbf{z}_\mathbb{R}, \mathbf{W}_\mathbb{R}, \mathbf{b}_\mathbb{R})) - \text{conv}(\mathbf{z}_\mathbb{I}, \mathbf{W}_\mathbb{I}, \mathbf{b}_\mathbb{I})) + j(\text{conv}(\mathbf{z}_\mathbb{R}, \mathbf{W}_\mathbb{I}, \mathbf{b}_\mathbb{I})) + \text{conv}(\mathbf{z}_\mathbb{I}, \mathbf{W}_\mathbb{R}, \mathbf{b}_\mathbb{R})))\]where \(\mathbf{W}\) and \(\mathbf{b}\) are the complex-valued weight and bias tensors, respectively, and \(\text{conv}(\cdot)\) is the conovlution operator.
By comparison, using Gauss’ trick, the complex-vauled convolution can be implemented as:
\[ \begin{align}\begin{aligned}t1 =& \text{conv}(\mathbf{z}_\mathbb{R}, \mathbf{W}_\mathbb{R}, \mathbf{b}_\mathbb{R}))\\t2 =& \text{conv}(\mathbf{z}_\mathbb{I}, \mathbf{W}_\mathbb{I}, \mathbf{b}_\mathbb{I}))\\t3 =& \text{conv}(\mathbf{z}_\mathbb{R} + \mathbf{z}_\mathbb{I}, \mathbf{W}_\mathbb{R} + \mathbf{W}_\mathbb{I}, \mathbf{b}_\mathbb{R} + \mathbf{b}_\mathbb{I}))\\G(\mathbf{z}) =& t1 - t2 + j(t3 - t2 - t1)\end{aligned}\end{align} \]requiring only 3 convolution operations.
- class complextorch.nn.modules.conv.CVConv2d(in_channels: int, out_channels: int, kernel_size: int | Tuple[int, int], stride: int | Tuple[int, int] = 1, padding: str | int | Tuple[int, int] = 0, dilation: int | Tuple[int, int] = 1, groups: int = 1, bias: bool = True, padding_mode: str = 'zeros', device=None, dtype=None)
2-D Complex-Valued Convolution
Based on the PyTorch torch.nn.Conv2d implementation.
Employs Gauss’ multiplication trick to reduce number of computations by 25% compare with the naive implementation.
The most common implementation of complex-valued convolution entails the following computation:
\[G(\mathbf{z}) = \text{conv}(\mathbf{z}_\mathbb{R}, \mathbf{W}_\mathbb{R}, \mathbf{b}_\mathbb{R})) - \text{conv}(\mathbf{z}_\mathbb{I}, \mathbf{W}_\mathbb{I}, \mathbf{b}_\mathbb{I})) + j(\text{conv}(\mathbf{z}_\mathbb{R}, \mathbf{W}_\mathbb{I}, \mathbf{b}_\mathbb{I})) + \text{conv}(\mathbf{z}_\mathbb{I}, \mathbf{W}_\mathbb{R}, \mathbf{b}_\mathbb{R})))\]where \(\mathbf{W}\) and \(\mathbf{b}\) are the complex-valued weight and bias tensors, respectively, and \(\text{conv}(\cdot)\) is the conovlution operator.
By comparison, using Gauss’ trick, the complex-vauled convolution can be implemented as:
\[ \begin{align}\begin{aligned}t1 =& \text{conv}(\mathbf{z}_\mathbb{R}, \mathbf{W}_\mathbb{R}, \mathbf{b}_\mathbb{R}))\\t2 =& \text{conv}(\mathbf{z}_\mathbb{I}, \mathbf{W}_\mathbb{I}, \mathbf{b}_\mathbb{I}))\\t3 =& \text{conv}(\mathbf{z}_\mathbb{R} + \mathbf{z}_\mathbb{I}, \mathbf{W}_\mathbb{R} + \mathbf{W}_\mathbb{I}, \mathbf{b}_\mathbb{R} + \mathbf{b}_\mathbb{I}))\\G(\mathbf{z}) =& t1 - t2 + j(t3 - t2 - t1)\end{aligned}\end{align} \]requiring only 3 convolution operations.
- class complextorch.nn.modules.conv.CVConv3d(in_channels: int, out_channels: int, kernel_size: int | Tuple[int, int, int], stride: int | Tuple[int, int, int] = 1, padding: str | int | Tuple[int, int, int] = 0, dilation: int | Tuple[int, int, int] = 1, groups: int = 1, bias: bool = True, padding_mode: str = 'zeros', device=None, dtype=None)
3-D Complex-Valued Convolution
Based on the PyTorch torch.nn.Conv3d implementation.
Employs Gauss’ multiplication trick to reduce number of computations by 25% compare with the naive implementation.
The most common implementation of complex-valued convolution entails the following computation:
\[G(\mathbf{z}) = \text{conv}(\mathbf{z}_\mathbb{R}, \mathbf{W}_\mathbb{R}, \mathbf{b}_\mathbb{R})) - \text{conv}(\mathbf{z}_\mathbb{I}, \mathbf{W}_\mathbb{I}, \mathbf{b}_\mathbb{I})) + j(\text{conv}(\mathbf{z}_\mathbb{R}, \mathbf{W}_\mathbb{I}, \mathbf{b}_\mathbb{I})) + \text{conv}(\mathbf{z}_\mathbb{I}, \mathbf{W}_\mathbb{R}, \mathbf{b}_\mathbb{R})))\]where \(\mathbf{W}\) and \(\mathbf{b}\) are the complex-valued weight and bias tensors, respectively, and \(\text{conv}(\cdot)\) is the conovlution operator.
By comparison, using Gauss’ trick, the complex-vauled convolution can be implemented as:
\[ \begin{align}\begin{aligned}t1 =& \text{conv}(\mathbf{z}_\mathbb{R}, \mathbf{W}_\mathbb{R}, \mathbf{b}_\mathbb{R}))\\t2 =& \text{conv}(\mathbf{z}_\mathbb{I}, \mathbf{W}_\mathbb{I}, \mathbf{b}_\mathbb{I}))\\t3 =& \text{conv}(\mathbf{z}_\mathbb{R} + \mathbf{z}_\mathbb{I}, \mathbf{W}_\mathbb{R} + \mathbf{W}_\mathbb{I}, \mathbf{b}_\mathbb{R} + \mathbf{b}_\mathbb{I}))\\G(\mathbf{z}) =& t1 - t2 + j(t3 - t2 - t1)\end{aligned}\end{align} \]requiring only 3 convolution operations.
- class complextorch.nn.modules.conv.CVConvTranpose1d(in_channels: int, out_channels: int, kernel_size: int | Tuple[int], stride: int | Tuple[int] = 1, padding: int | Tuple[int] = 0, output_padding: int | Tuple[int] = 0, groups: int = 1, bias: bool = True, dilation: int | Tuple[int] = 1, padding_mode: str = 'zeros', device=None, dtype=None)
1-D Complex-Valued Transposed Convolution
Based on the PyTorch torch.nn.ConvTranspose1d implementation.
Employs Gauss’ multiplication trick to reduce number of computations by 25% compare with the naive implementation.
The most common implementation of complex-valued convolution entails the following computation:
\[G(\mathbf{z}) = convT(\mathbf{z}_\mathbb{R}, \mathbf{W}_\mathbb{R}, \mathbf{b}_\mathbb{R})) - convT(\mathbf{z}_\mathbb{I}, \mathbf{W}_\mathbb{I}, \mathbf{b}_\mathbb{I})) + j(convT(\mathbf{z}_\mathbb{R}, \mathbf{W}_\mathbb{I}, \mathbf{b}_\mathbb{I})) + convT(\mathbf{z}_\mathbb{I}, \mathbf{W}_\mathbb{R}, \mathbf{b}_\mathbb{R})))\]where \(\mathbf{W}\) and \(\mathbf{b}\) are the complex-valued weight and bias tensors, respectively, and \(convT(\cdot)\) is the transposed conovlution operator.
By comparison, using Gauss’ trick, the complex-vauled convolution can be implemented as:
\[ \begin{align}\begin{aligned}t1 =& convT(\mathbf{z}_\mathbb{R}, \mathbf{W}_\mathbb{R}, \mathbf{b}_\mathbb{R}))\\t2 =& convT(\mathbf{z}_\mathbb{I}, \mathbf{W}_\mathbb{I}, \mathbf{b}_\mathbb{I}))\\t3 =& convT(\mathbf{z}_\mathbb{R} + \mathbf{z}_\mathbb{I}, \mathbf{W}_\mathbb{R} + \mathbf{W}_\mathbb{I}, \mathbf{b}_\mathbb{R} + \mathbf{b}_\mathbb{I}))\\G(\mathbf{z}) =& t1 - t2 + j(t3 - t2 - t1)\end{aligned}\end{align} \]requiring only 3 transposed convolution operations.
- class complextorch.nn.modules.conv.CVConvTranpose2d(in_channels: int, out_channels: int, kernel_size: int | Tuple[int], stride: int | Tuple[int] = 1, padding: int | Tuple[int] = 0, output_padding: int | Tuple[int] = 0, groups: int = 1, bias: bool = True, dilation: int | Tuple[int] = 1, padding_mode: str = 'zeros', device=None, dtype=None)
2-D Complex-Valued Transposed Convolution
Based on the PyTorch torch.nn.ConvTranspose2d implementation.
Employs Gauss’ multiplication trick to reduce number of computations by 25% compare with the naive implementation.
The most common implementation of complex-valued convolution entails the following computation:
\[G(\mathbf{z}) = convT(\mathbf{z}_\mathbb{R}, \mathbf{W}_\mathbb{R}, \mathbf{b}_\mathbb{R})) - convT(\mathbf{z}_\mathbb{I}, \mathbf{W}_\mathbb{I}, \mathbf{b}_\mathbb{I})) + j(convT(\mathbf{z}_\mathbb{R}, \mathbf{W}_\mathbb{I}, \mathbf{b}_\mathbb{I})) + convT(\mathbf{z}_\mathbb{I}, \mathbf{W}_\mathbb{R}, \mathbf{b}_\mathbb{R})))\]where \(\mathbf{W}\) and \(\mathbf{b}\) are the complex-valued weight and bias tensors, respectively, and \(convT(\cdot)\) is the transposed conovlution operator.
By comparison, using Gauss’ trick, the complex-vauled convolution can be implemented as:
\[ \begin{align}\begin{aligned}t1 =& convT(\mathbf{z}_\mathbb{R}, \mathbf{W}_\mathbb{R}, \mathbf{b}_\mathbb{R}))\\t2 =& convT(\mathbf{z}_\mathbb{I}, \mathbf{W}_\mathbb{I}, \mathbf{b}_\mathbb{I}))\\t3 =& convT(\mathbf{z}_\mathbb{R} + \mathbf{z}_\mathbb{I}, \mathbf{W}_\mathbb{R} + \mathbf{W}_\mathbb{I}, \mathbf{b}_\mathbb{R} + \mathbf{b}_\mathbb{I}))\\G(\mathbf{z}) =& t1 - t2 + j(t3 - t2 - t1)\end{aligned}\end{align} \]requiring only 3 transposed convolution operations.
- class complextorch.nn.modules.conv.CVConvTranpose3d(in_channels: int, out_channels: int, kernel_size: int | Tuple[int], stride: int | Tuple[int] = 1, padding: int | Tuple[int] = 0, output_padding: int | Tuple[int] = 0, groups: int = 1, bias: bool = True, dilation: int | Tuple[int] = 1, padding_mode: str = 'zeros', device=None, dtype=None)
3-D Complex-Valued Transposed Convolution
Based on the PyTorch torch.nn.ConvTranspose3d implementation.
Employs Gauss’ multiplication trick to reduce number of computations by 25% compare with the naive implementation.
The most common implementation of complex-valued convolution entails the following computation:
\[G(\mathbf{z}) = convT(\mathbf{z}_\mathbb{R}, \mathbf{W}_\mathbb{R}, \mathbf{b}_\mathbb{R})) - convT(\mathbf{z}_\mathbb{I}, \mathbf{W}_\mathbb{I}, \mathbf{b}_\mathbb{I})) + j(convT(\mathbf{z}_\mathbb{R}, \mathbf{W}_\mathbb{I}, \mathbf{b}_\mathbb{I})) + convT(\mathbf{z}_\mathbb{I}, \mathbf{W}_\mathbb{R}, \mathbf{b}_\mathbb{R})))\]where \(\mathbf{W}\) and \(\mathbf{b}\) are the complex-valued weight and bias tensors, respectively, and \(convT(\cdot)\) is the transposed conovlution operator.
By comparison, using Gauss’ trick, the complex-vauled convolution can be implemented as:
\[ \begin{align}\begin{aligned}t1 =& convT(\mathbf{z}_\mathbb{R}, \mathbf{W}_\mathbb{R}, \mathbf{b}_\mathbb{R}))\\t2 =& convT(\mathbf{z}_\mathbb{I}, \mathbf{W}_\mathbb{I}, \mathbf{b}_\mathbb{I}))\\t3 =& convT(\mathbf{z}_\mathbb{R} + \mathbf{z}_\mathbb{I}, \mathbf{W}_\mathbb{R} + \mathbf{W}_\mathbb{I}, \mathbf{b}_\mathbb{R} + \mathbf{b}_\mathbb{I}))\\G(\mathbf{z}) =& t1 - t2 + j(t3 - t2 - t1)\end{aligned}\end{align} \]requiring only 3 transposed convolution operations.
- class complextorch.nn.modules.conv.SlowCVConv1d(in_channels: int, out_channels: int, kernel_size: int, stride: int = 1, padding: int = 0, dilation: int = 1, groups: int = 1, bias: bool = False)
Slow Complex-Valued 1-D Convolution
Implemented using torch.nn.Conv1d and complex-valued tensors.
Slower than using CVTensor. PyTorch must have some additional overhead that makes this method significantly slower than using CVTensors and the other CVConv layers