建设网站软件,手机优化网站建设,曲周专业做网站,淄博网站建设优化seo文章目录 稀疏矩阵的格式coocsrcsc Construction of Sparse COO tensorsConstruction of CSR tensorsLinear Algebra operations#xff08;稀疏与稠密之间混合运算#xff09;Tensor methods and sparse#xff08;与稀疏有关的tensor成员函数#xff09;coo张量可用的ten… 文章目录 稀疏矩阵的格式coocsrcsc Construction of Sparse COO tensorsConstruction of CSR tensorsLinear Algebra operations稀疏与稠密之间混合运算Tensor methods and sparse与稀疏有关的tensor成员函数coo张量可用的tensor成员函数经实测csr也有一些可以用比如dim() Torch functions specific to sparse Tensors与稀疏有关的torch函数支持稀疏张量的常规torch函数支持稀疏张量的一元函数 稀疏矩阵的格式
目前torch.sparse和scipy.sparse模块比较支持的主流的稀疏矩阵格式有coo格式、csr格式和csc格式这三种格式中可供使用的API也最多。
coo
将矩阵中非零元素的坐标和值分开存储在3个数组中3个数组长度必须相同表示有n个非零元素。 csr
分 Index Pointers、Indices、Data3个数组存储。
Index Pointers第 i个元素记录这个矩阵的第 i行的第1个非零值在 Data数组的起始位置第 i1个元素记录这个矩阵的第 i行的最后一个非零值在 Data数组的终止位置不包含右边界。因此这个矩阵的行数等于 len(Index Pointers)-1第 i行非零值的个数等于 Index Pointers[i1]-Index Pointers[i]。Indices第 i个元素记录这个矩阵的第 i个非零值的列坐标。Data第 i个元素记录这个矩阵的第 i个非零值的具体数值排列顺序严格按照行优先列次先。 csc
与csr唯一的不同在于列优先其他规则一模一样。 Construction of Sparse COO tensors
常规构建 i [[0, 1, 1],[2, 0, 2]]v [3, 4, 5]s torch.sparse_coo_tensor(i, v, (2, 3))s
tensor(indicestensor([[0, 1, 1],[2, 0, 2]]),valuestensor([3, 4, 5]),size(2, 3), nnz3, layouttorch.sparse_coo)s.to_dense()
tensor([[0, 0, 3],[4, 0, 5]])torch中稀疏矩阵的存储方式记录在 tensor.layout中可以通过检查 torch.layout torch.sparse_coo来判断是否是coo张量。此外稠密张量的 layout等于 strided。
稠密混合的coo张量 i [[0, 1, 1],[2, 0, 2]]v [[3, 4], [5, 6], [7, 8]]s torch.sparse_coo_tensor(i, v, (2, 3, 2))s
tensor(indicestensor([[0, 1, 1],[2, 0, 2]]),valuestensor([[3, 4],[5, 6],[7, 8]]),size(2, 3, 2), nnz3, layouttorch.sparse_coo)此方案与常规的coo构建方式不同values中每个元素可以是一个向量表示对应坐标的稠密张量因此创建出的coo张量也多出了一个维度。
带有重复坐标的coo张量 i [[1, 1]]v [3, 4]storch.sparse_coo_tensor(i, v, (3,))s
tensor(indicestensor([[1, 1]]),valuestensor( [3, 4]),size(3,), nnz2, layouttorch.sparse_coo)s.to_dense()
tensor([0, 7, 0])如果输入的坐标有重复则创建出的coo张量会自动把坐标重复的元素值相加。此外可以通过成员函数 .coalesce()把重复坐标的元素值相加将这个coo转换成一个不重复的张量也可以通过 .is_coalesced()检查这个coo是否存在重复的坐标。
Construction of CSR tensors
按照 Index Pointers、Indices、Data三个数组的定义构建即可。 crow_indices torch.tensor([0, 2, 4])col_indices torch.tensor([0, 1, 0, 1])values torch.tensor([1, 2, 3, 4])csr torch.sparse_csr_tensor(crow_indices, col_indices, values, dtypetorch.float64)csr
tensor(crow_indicestensor([0, 2, 4]),col_indicestensor([0, 1, 0, 1]),valuestensor([1., 2., 3., 4.]), size(2, 2), nnz4,dtypetorch.float64)csr.to_dense()
tensor([[1., 2.],[3., 4.]], dtypetorch.float64)Linear Algebra operations稀疏与稠密之间混合运算
M表示2-D张量V表示1-D张量f表示标量*表示逐元素乘法表示矩阵乘法。M[SparseSemiStructured]表示一种半结构化的稀疏矩阵此处不再展开可以自行去torch官网察看。
PyTorch operationSparse gradLayout signaturetorch.mv()noM[sparse_coo] V[strided] - V[strided]torch.mv()noM[sparse_csr] V[strided] - V[strided]torch.matmul()noM[sparse_coo] M[strided] - M[strided]torch.matmul()noM[sparse_csr] M[strided] - M[strided]torch.matmul()noM[SparseSemiStructured] M[strided] - M[strided]torch.matmul()noM[strided] M[SparseSemiStructured] - M[strided]torch.mm()noM[strided] M[SparseSemiStructured] - M[strided]torch.mm()noM[sparse_coo] M[strided] - M[strided]torch.mm()noM[SparseSemiStructured] M[strided] - M[strided]torch.sparse.mm()yesM[sparse_coo] M[strided] - M[strided]torch.smm()noM[sparse_coo] M[strided] - M[sparse_coo]torch.hspmm()noM[sparse_coo] M[strided] - M[hybrid sparse_coo]torch.bmm()noT[sparse_coo] T[strided] - T[strided]torch.addmm()nof * M[strided] f * (M[sparse_coo] M[strided]) - M[strided]torch.addmm()nof * M[strided] f * (M[SparseSemiStructured] M[strided]) - M[strided]torch.addmm()nof * M[strided] f * (M[strided] M[SparseSemiStructured]) - M[strided]torch.sparse.addmm()yesf * M[strided] f * (M[sparse_coo] M[strided]) - M[strided]torch.sspaddmm()nof * M[sparse_coo] f * (M[sparse_coo] M[strided]) - M[sparse_coo]torch.lobpcg()noGENEIG(M[sparse_coo]) - M[strided], M[strided]torch.pca_lowrank()yesPCA(M[sparse_coo]) - M[strided], M[strided], M[strided]torch.svd_lowrank()yesSVD(M[sparse_coo]) - M[strided], M[strided], M[strided]
以上API中如果 Layout signature中提供了 或者 *操作符就不需要记住API直接通过操作符即可隐式调用对应的API。如 a torch.tensor([[0, 0, 1, 0], [1, 2, 0, 0], [0, 0, 0, 0]], dtypetorch.float64)sp a.to_sparse_csr()vec torch.randn(4, 1, dtypetorch.float64)sp.matmul(vec)
tensor([[ 0.4788],[-3.2338],[ 0.0000]], dtypetorch.float64)sp vec
tensor([[ 0.4788],[-3.2338],[ 0.0000]], dtypetorch.float64)需要注意的是使用操作符在稀疏张量和稠密张量之间乘法运算时返回的都是稠密张量。如果想要返回稀疏张量需要显式使用torch.smm()。
torch同样支持稀疏与稀疏之间的运算但要求输入的稀疏张量必须具有相同的稀疏结构否则会报错返回的稀疏张量的稀疏结构也与输入相同。
乘法运算 a torch.tensor([[0, 0, 1, 0], [1, 2, 0, 0], [0, 1, 0, 0], [1, 0, 0, 0]], dtypetorch.float64)b torch.tensor([[0, 0, 2, 0], [3, 1, 0, 0], [0, 0, 4, 0], [1, 0, 0, 1]], dtypetorch.float64)sp1 a.to_sparse_coo()sp2 b.to_sparse_coo()sp1 sp2
tensor(indicestensor([[0, 1, 1, 1, 2, 2, 3],[2, 0, 1, 2, 0, 1, 2]]),valuestensor([4., 6., 2., 2., 3., 1., 2.]),size(4, 4), nnz7, dtypetorch.float64, layouttorch.sparse_coo)加法运算 a torch.tensor([[0, 0, 1, 0], [1, 2, 0, 0], [0, 1, 0, 0], [1, 0, 0, 0]], dtypetorch.float64)b torch.tensor([[0, 0, 2, 0], [3, 1, 0, 0], [0, 0, 4, 0], [1, 0, 0, 1]], dtypetorch.float64)sp1 a.to_sparse_coo()sp2 b.to_sparse_coo()sp3 b.to_sparse_csr()sp1 sp2
tensor(indicestensor([[0, 1, 1, 2, 2, 3, 3],[2, 0, 1, 1, 2, 0, 3]]),valuestensor([3., 4., 3., 1., 4., 2., 1.]),size(4, 4), nnz7, dtypetorch.float64, layouttorch.sparse_coo)sp1 sp3
UserWarning: Sparse CSR tensor support is in beta state. If you miss a functionality in the sparse tensor support, please submit a feature request to https://github.com/pytorch/pytorch/issues. (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\SparseCsrTensorImpl.cpp:55.)sp3 b.to_sparse_csr()
Traceback (most recent call last):File C:\Users\Xu Han\Desktop\pycharm-projects\MD_notes\main.py, line 18, in moduleprint(sp1 sp3)
RuntimeError: memory format option is only supported by strided tensorsTensor methods and sparse与稀疏有关的tensor成员函数
PyTorch operationreturnTensor.is_sparseIsTrue if the Tensor uses sparse COO storage layout, False otherwise.Tensor.is_sparse_csrIsTrue if the Tensor uses sparse CSR storage layout, False otherwise.Tensor.dense_dimReturn the number of dense dimensions in a sparse tensorself.Tensor.sparse_dimReturn the number of sparse dimensions in a sparse tensorself.
这里打断一下表格讲解一下dense_dim和sparse_dim的含义。上文中我们曾构建过稠密混合的coo张量如下 i [[0, 1, 1],[2, 0, 2]]v [[3, 4], [5, 6], [7, 8]]s torch.sparse_coo_tensor(i, v, (2, 3, 2))s
tensor(indicestensor([[0, 1, 1],[2, 0, 2]]),valuestensor([[3, 4],[5, 6],[7, 8]]),size(2, 3, 2), nnz3, layouttorch.sparse_coo)那么对于这个tensor它的dense_dim为1sparse_dim为2。
此外在进行稀疏与稀疏之间的数学运算时一定要保证稀疏张量的sparse_dim等于2.
继续表格。
PyTorch operationreturnTensor.sparse_maskReturns a new sparse tensor with values from a strided tensorself filtered by the indices of the sparse tensor mask.Tensor.to_sparseReturns a sparse copy of the tensor.Tensor.to_sparse_cooConvert a tensor to coordinate format.Tensor.to_sparse_csrConvert a tensor to compressed row storage format (CSR).Tensor.to_sparse_cscConvert a tensor to compressed column storage (CSC) format.Tensor.to_sparse_bsrConvert a tensor to a block sparse row (BSR) storage format of given blocksize.Tensor.to_sparse_bscConvert a tensor to a block sparse column (BSC) storage format of given blocksize.Tensor.to_denseCreates a strided copy ofself if self is not a strided tensor, otherwise returns self.Tensor.valuesReturn the values tensor of a sparse COO tensor.
以下是仅限coo张量的成员
PyTorch operationreturnTensor.coalesceReturns a coalesced copy ofself if self is an uncoalesced tensor.Tensor.sparse_resize_Resizesself sparse tensor to the desired size and the number of sparse and dense dimensions.Tensor.sparse_resize_and_clear_Removes all specified elements from a sparse tensorself and resizes self to the desired size and the number of sparse and dense dimensions.Tensor.is_coalescedReturnsTrue if self is a sparse COO tensor that is coalesced, False otherwise.Tensor.indicesReturn the indices tensor of a sparse COO tensor.
以下是仅限csr和bsr张量的成员
PyTorch operationreturnTensor.crow_indicesReturns the tensor containing the compressed row indices of theself tensor when self is a sparse CSR tensor of layout sparse_csr.Tensor.col_indicesReturns the tensor containing the column indices of theself tensor when self is a sparse CSR tensor of layout sparse_csr.
以下是仅限csc和bsc张量的成员
PyTorch operationreturnTensor.row_indices…Tensor.ccol_indices…
coo张量可用的tensor成员函数经实测csr也有一些可以用比如dim()
add() add_() addmm() addmm_() any() asin() asin_() arcsin() arcsin_() bmm() clone() deg2rad() deg2rad_() detach() detach_() dim() div() div_() floor_divide() floor_divide_() get_device() index_select() isnan() log1p() log1p_() mm() mul() mul_() mv() narrow_copy() neg() neg_() negative() negative_() numel() rad2deg() rad2deg_() resize_as_() size() pow() sqrt() square() smm() sspaddmm() sub() sub_() t() t_() transpose() transpose_() zero_()
Torch functions specific to sparse Tensors与稀疏有关的torch函数
PyTorch operationreturnsparse_coo_tensorConstructs a sparse tensor in COO(rdinate) format with specified values at the givenindices.sparse_csr_tensorConstructs a sparse tensor in CSR (Compressed Sparse Row) with specified values at the givencrow_indices and col_indices.sparse_csc_tensorConstructs a sparse tensor in CSC (Compressed Sparse Column) with specified values at the givenccol_indices and row_indices.sparse_bsr_tensorConstructs a sparse tensor in BSR (Block Compressed Sparse Row)) with specified 2-dimensional blocks at the givencrow_indices and col_indices.sparse_bsc_tensorConstructs a sparse tensor in BSC (Block Compressed Sparse Column)) with specified 2-dimensional blocks at the givenccol_indices and row_indices.sparse_compressed_tensorConstructs a sparse tensor in Compressed Sparse format - CSR, CSC, BSR, or BSC - with specified values at the givencompressed_indices and plain_indices.sparse.sumReturn the sum of each row of the given sparse tensor.sparse.addmmThis function does exact same thing as torch.addmm() in the forward, except that it supports backward for sparse COO matrixmat1.sparse.sampled_addmmPerforms a matrix multiplication of the dense matricesmat1 and mat2 at the locations specified by the sparsity pattern of input.sparse.mmPerforms a matrix multiplication of the sparse matrixmat1sspaddmmMatrix multiplies a sparse tensormat1 with a dense tensor mat2, then adds the sparse tensor input to the result.hspmmPerforms a matrix multiplication of a sparse COO matrixmat1 and a strided matrix mat2.smmPerforms a matrix multiplication of the sparse matrixinput with the dense matrix mat.sparse.softmaxApplies a softmax function.sparse.log_softmaxApplies a softmax function followed by logarithm.sparse.spdiagsCreates a sparse 2D tensor by placing the values from rows ofdiagonals along specified diagonals of the output
支持稀疏张量的常规torch函数
cat() dstack() empty() empty_like() hstack() index_select() is_complex() is_floating_point() is_nonzero() is_same_size() is_signed() is_tensor() lobpcg() mm() native_norm() pca_lowrank() select() stack() svd_lowrank() unsqueeze() vstack() zeros() zeros_like()
支持稀疏张量的一元函数
The following operators currently support sparse COO/CSR/CSC/BSR tensor inputs.
abs() asin() asinh() atan() atanh() ceil() conj_physical() floor() log1p() neg() round() sin() sinh() sign() sgn() signbit() tan() tanh() trunc() expm1() sqrt() angle() isinf() isposinf() isneginf() isnan() erf() erfinv()
