column permutation of sparse matrix

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

column permutation of sparse matrix

khai
Hi,

I'm very new to working with sparse matrices and would like to know how I can column permute a sparse matrix. Here is a small example:

> M1 <- spMatrix(nrow=5,ncol=6,i=sample(5,15,replace=TRUE),j=sample(6,15,replace=TRUE),x=round_any(rnorm(15,2),0.001))
> M1
5 x 6 sparse Matrix of class "dgTMatrix"

[1,] 2.983      .     1.656    5.003    .        .
[2,]    .            .     2.990        .         .        .
[3,]    .     0.592   5.349    1.115    .        .
[4,] 1.836      .     2.804        .         .        .
[5,]    .     6.961       .             .         .     1.077

I know I can permute entire columns this way

> M1[,sample(6,6)]
5 x 6 sparse Matrix of class "dgTMatrix"

[1,] 5.003        .           .           .       1.656    2.983
[2,]     .             .           .           .       2.990       .
[3,] 1.115    0.592      .           .       5.349       .
[4,]     .             .           .          .       2.804     1.836
[5,]     .         6.961   1.077     .          .            .

But I would like the new sparse matrix to look like this...where only the nonzero elements are permuted.

[1,] 1.656      .     5.003    2.983    .        .
[2,]    .            .     2.990        .         .        .
[3,]    .    5.349    1.115    0.592    .        .
[4,] 2.804      .     1.836        .         .        .
[5,]    .     1.077       .             .         .     6.961

Thanks in advance for any advice!
Reply | Threaded
Open this post in threaded view
|

Re: column permutation of sparse matrix

Adams, Jean
khai wrote on 12/19/2011 11:26:55 PM:

> Hi,
>
> I'm very new to working with sparse matrices and would like to know how
I

> can column permute a sparse matrix. Here is a small example:
>
> > M1 <-
> > spMatrix(nrow=5,ncol=6,i=sample(5,15,replace=TRUE),j=sample(6,
> 15,replace=TRUE),x=round_any(rnorm(15,2),0.001))
> > M1
> 5 x 6 sparse Matrix of class "dgTMatrix"
>
> [1,] 2.983      .     1.656    5.003    .        .
> [2,]    .            .     2.990        .         .        .
> [3,]    .     0.592   5.349    1.115    .        .
> [4,] 1.836      .     2.804        .         .        .
> [5,]    .     6.961       .             .         .     1.077
>
> I know I can permute entire columns this way
>
> > M1[,sample(6,6)]
> 5 x 6 sparse Matrix of class "dgTMatrix"
>
> [1,] 5.003        .           .           .       1.656    2.983
> [2,]     .             .           .           .       2.990       .
> [3,] 1.115    0.592      .           .       5.349       .
> [4,]     .             .           .          .       2.804     1.836
> [5,]     .         6.961   1.077     .          .            .
>
> But I would like the new sparse matrix to look like this...where only
the
> nonzero elements are permuted.
>
> [1,] 1.656      .     5.003    2.983    .        .
> [2,]    .            .     2.990        .         .        .
> [3,]    .    5.349    1.115    0.592    .        .
> [4,] 2.804      .     1.836        .         .        .
> [5,]    .     1.077       .             .         .     6.961
>
> Thanks in advance for any advice!


I don't have experience with sparse matrices, but I was able to get this
to work by converting the sparse matrix to a "base" matrix and back again.


library(Matrix)

nonzero.cols <- !apply(M1==0, 2, all)
M2 <- as.matrix(M1)
reord <- sample(seq(dim(M1)[2])[nonzero.cols])
M2[, nonzero.cols] <- as.matrix(M1[, reord])
Matrix(M2, sparse=TRUE)


Jean
        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: column permutation of sparse matrix

Douglas Bates-2
On Tue, Dec 20, 2011 at 8:20 AM, Jean V Adams <[hidden email]> wrote:

Hi Jean,

> khai wrote on 12/19/2011 11:26:55 PM:
>
>> Hi,
>>
>> I'm very new to working with sparse matrices and would like to know how
> I
>> can column permute a sparse matrix. Here is a small example:
>>
>> > M1 <-
>> > spMatrix(nrow=5,ncol=6,i=sample(5,15,replace=TRUE),j=sample(6,
>> 15,replace=TRUE),x=round_any(rnorm(15,2),0.001))
>> > M1
>> 5 x 6 sparse Matrix of class "dgTMatrix"
>>
>> [1,] 2.983      .     1.656    5.003    .        .
>> [2,]    .            .     2.990        .         .        .
>> [3,]    .     0.592   5.349    1.115    .        .
>> [4,] 1.836      .     2.804        .         .        .
>> [5,]    .     6.961       .             .         .     1.077
>>
>> I know I can permute entire columns this way
>>
>> > M1[,sample(6,6)]
>> 5 x 6 sparse Matrix of class "dgTMatrix"
>>
>> [1,] 5.003        .           .           .       1.656    2.983
>> [2,]     .             .           .           .       2.990       .
>> [3,] 1.115    0.592      .           .       5.349       .
>> [4,]     .             .           .          .       2.804     1.836
>> [5,]     .         6.961   1.077     .          .            .
>>
>> But I would like the new sparse matrix to look like this...where only
> the
>> nonzero elements are permuted.
>>
>> [1,] 1.656      .     5.003    2.983    .        .
>> [2,]    .            .     2.990        .         .        .
>> [3,]    .    5.349    1.115    0.592    .        .
>> [4,] 2.804      .     1.836        .         .        .
>> [5,]    .     1.077       .             .         .     6.961
>>
>> Thanks in advance for any advice!

A peculiar request but you can do that by permuting the 'x' slot in
your original matrix.

> set.seed(1)
> (M1 <- spMatrix(nrow=5,ncol=6,i=sample(5,15,replace=TRUE),j=sample(6, 15,replace=TRUE),x=round(rnorm(15,2),3)))
5 x 6 sparse Matrix of class "dgTMatrix"

[1,] .     .     2.738  .     3.595 .
[2,] .     .     .      .     .     .
[3,] 3.879 .     4.289 -0.215 1.374 .
[4,] .     4.966 1.180  1.379 .     2.487
[5,] 3.512 .     .      .     2.330 .
> (nnz <- nnzero(M1))
[1] 15
> M2 <- M1
> M2@x <- M2@x[sample(nnz, nnz)]
> M2
5 x 6 sparse Matrix of class "dgTMatrix"

[1,] .     .     2.487 .      3.595 .
[2,] .     .     .     .      .     .
[3,] 3.709 .     2.875 3.512 -0.215 .
[4,] .     3.764 1.164 2.576  .     3.125
[5,] 2.184 .     .     .      2.738 .

>
> I don't have experience with sparse matrices, but I was able to get this
> to work by converting the sparse matrix to a "base" matrix and back again.
>
>
> library(Matrix)
>
> nonzero.cols <- !apply(M1==0, 2, all)
> M2 <- as.matrix(M1)
> reord <- sample(seq(dim(M1)[2])[nonzero.cols])
> M2[, nonzero.cols] <- as.matrix(M1[, reord])
> Matrix(M2, sparse=TRUE)

This solution has the disadvantage of converting the sparse matrix to
a dense matrix, which may end up producing a much larger object.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: column permutation of sparse matrix

khai
Thanks very much for responses. The 2 suggestions each do a part of what I want. Jean's proposal permutes nonzero elements within each row but shifts the location of the nonzero element around in the j slot of each i row.  And Douglas's idea keeps the second part but permutes the entire x slot values. I'll try to see how I can combine these 2 ideas but it's not immediately obvious to me. =}

-khai
Reply | Threaded
Open this post in threaded view
|

Re: column permutation of sparse matrix

Adams, Jean
khai wrote on 12/20/2011 01:24:10 PM:

> Thanks very much for responses. The 2 suggestions each do a part of what
I
> want. Jean's proposal permutes nonzero elements within each row but
shifts
> the location of the nonzero element around in the j slot of each i row.
And
> Douglas's idea keeps the second part but permutes the entire x slot
values.
> I'll try to see how I can combine these 2 ideas but it's not immediately
> obvious to me. =}
>
> -khai


I'm not sure what you mean by shifting "the location of the nonzero
element around in the j slot of each i row."

My code shifts the nonzero columns around at random, but keeps the zero
column in its original location.  I thought that this is what you wanted
to do.

In the example below, the zero column starts as column 5 and ends as
column 5, and the nonzero columns are reordered.  Columns 1, 2, 3, 4, 5, 6
end up as 2, 1, 6, 3, 5, 4.

Jean


> M1
5 x 6 sparse Matrix of class "dgTMatrix"
 
[1,] .     6.569 5.592 .    . .
[2,] 1.163 2.688 .     1.85 . 3.217
[3,] 2.527 3.375 .     .    . 6.481
[4,] .     .     .     .    . 1.483
[5,] 1.004 .     .     .    . .
> nonzero.cols <- !apply(M1==0, 2, all)
> M2 <- as.matrix(M1)
> reord <- sample(seq(dim(M1)[2])[nonzero.cols])
> M2[, nonzero.cols] <- as.matrix(M1[, reord])
> Matrix(M2, sparse=TRUE)
5 x 6 sparse Matrix of class "dgCMatrix"
 
[1,] 6.569 .     .     5.592 . .
[2,] 2.688 1.163 3.217 .     . 1.85
[3,] 3.375 2.527 6.481 .     . .
[4,] .     .     1.483 .     . .
[5,] .     1.004 .     .     . .

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: column permutation of sparse matrix

cberry
In reply to this post by Douglas Bates-2
Douglas Bates <[hidden email]> writes:

> On Tue, Dec 20, 2011 at 8:20 AM, Jean V Adams <[hidden email]> wrote:
>
> Hi Jean,
>
>> khai wrote on 12/19/2011 11:26:55 PM:
>>
>>> Hi,
>>>
>>> I'm very new to working with sparse matrices and would like to know how
>> I
>>> can column permute a sparse matrix. Here is a small example:
>>>
>>> > M1 <-
>>> > spMatrix(nrow=5,ncol=6,i=sample(5,15,replace=TRUE),j=sample(6,
>>> 15,replace=TRUE),x=round_any(rnorm(15,2),0.001))
>>> > M1
>>> 5 x 6 sparse Matrix of class "dgTMatrix"
>>>
>>> [1,] 2.983      .     1.656    5.003    .        .
>>> [2,]    .            .     2.990        .         .        .
>>> [3,]    .     0.592   5.349    1.115    .        .
>>> [4,] 1.836      .     2.804        .         .        .
>>> [5,]    .     6.961       .             .         .     1.077
>>>
>>> I know I can permute entire columns this way
>>>
>>> > M1[,sample(6,6)]
>>> 5 x 6 sparse Matrix of class "dgTMatrix"
>>>
>>> [1,] 5.003        .           .           .       1.656    2.983
>>> [2,]     .             .           .           .       2.990       .
>>> [3,] 1.115    0.592      .           .       5.349       .
>>> [4,]     .             .           .          .       2.804     1.836
>>> [5,]     .         6.961   1.077     .          .            .
>>>
>>> But I would like the new sparse matrix to look like this...where only
>> the
>>> nonzero elements are permuted.
>>>
>>> [1,] 1.656      .     5.003    2.983    .        .
>>> [2,]    .            .     2.990        .         .        .
>>> [3,]    .    5.349    1.115    0.592    .        .
>>> [4,] 2.804      .     1.836        .         .        .
>>> [5,]    .     1.077       .             .         .     6.961
>>>
>>> Thanks in advance for any advice!
>
> A peculiar request but you can do that by permuting the 'x' slot in
> your original matrix.
>
>> set.seed(1)
>> (M1 <- spMatrix(nrow=5,ncol=6,i=sample(5,15,replace=TRUE),j=sample(6, 15,replace=TRUE),x=round(rnorm(15,2),3)))
> 5 x 6 sparse Matrix of class "dgTMatrix"
>
> [1,] .     .     2.738  .     3.595 .
> [2,] .     .     .      .     .     .
> [3,] 3.879 .     4.289 -0.215 1.374 .
> [4,] .     4.966 1.180  1.379 .     2.487
> [5,] 3.512 .     .      .     2.330 .
>> (nnz <- nnzero(M1))
> [1] 15
>> M2 <- M1
>> M2@x <- M2@x[sample(nnz, nnz)]
>> M2
> 5 x 6 sparse Matrix of class "dgTMatrix"
>
> [1,] .     .     2.487 .      3.595 .
> [2,] .     .     .     .      .     .
> [3,] 3.709 .     2.875 3.512 -0.215 .
> [4,] .     3.764 1.164 2.576  .     3.125
> [5,] 2.184 .     .     .      2.738 .
>

I do not think this is what was wanted.

M1 and M2 are matrices with 12 non-zero elements, not 15 as nnzero() reports.

The dgTMatrix representation effectively treats the slots of M1 like this:

  res <- tapply(M1@x, list(M1@i,M1@j), sum)
  res[is.na(res)] <- 0
  res

So permuting 'x' will return a different dgTMatrix with the same number
of non-zero elements in place of the existing non-zero elements, but not
necessarily having the same values when the (i,j) pairs are not
unique. (e.g. 3.709 is seen in M2, but not M1)

I think the OP was asking for permutations of the non-zero values in
'res', rather than permutations of M1@x.

HTH,

Chuck

>>
>> I don't have experience with sparse matrices, but I was able to get this
>> to work by converting the sparse matrix to a "base" matrix and back again.
>>
>>
>> library(Matrix)
>>
>> nonzero.cols <- !apply(M1==0, 2, all)
>> M2 <- as.matrix(M1)
>> reord <- sample(seq(dim(M1)[2])[nonzero.cols])
>> M2[, nonzero.cols] <- as.matrix(M1[, reord])
>> Matrix(M2, sparse=TRUE)
>
> This solution has the disadvantage of converting the sparse matrix to
> a dense matrix, which may end up producing a much larger object.
>

--
Charles C. Berry                            Dept of Family/Preventive Medicine
cberry at ucsd edu    UC San Diego
http://famprevmed.ucsd.edu/faculty/cberry/  La Jolla, San Diego 92093-0901

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: column permutation of sparse matrix

khai
This post was updated on .
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

Re: column permutation of sparse matrix

khai
In reply to this post by cberry
Sorry. I erred somewhere in making the original M1 matrix using sampling call in the spMatrix function. I still don't really understand why the matrix values do not match exactly with M1@x.  If I regenerate M1 this way...

> i=c(1:5,5:1)
> j=c(1:5,2:6)
> x=rnorm(10,2)
> M1 <- spMatrix(nrow=5,ncol=6,i=i,j=j,x=x)
> M1
5 x 6 sparse Matrix of class "dgTMatrix"

[1,] 1.876193 .        .        .        .        2.040574
[2,] .        1.032214 .        .        1.786480 .
[3,] .        .        1.716743 3.711979 .        .
[4,] .        .        2.534000 3.302921 .        .
[5,] .        1.084763 .        .        1.511992 .



Then the str(M1) makes sense and all the methods described so far appears to work as expected.

> M1@x
 [1] 1.876193 1.032214 1.716743 3.302921 1.511992 1.084763 2.534000 3.711979
 [9] 1.786480 2.040574


The solution is still not apparent to me though without doing a lapply.

spVecs <- lapply(1:nrow(M1), function(a)
        {
                nnz <- nnzero(M1[a,])
                tmp <- as(M1[a,],'sparseVector')
                tmp@x <- tmp@x[sample(nnz,nnz)]
                return(tmp)
        })

To get back a sparse matrix...I can do this.

        x <- do.call(c,lapply(spVecs,function(v) v@x))
        j= do.call(c,lapply(spVecs,function(v) v@i))
        i = do.call(c,lapply(1:length(spVecs), function(n) rep(n,length(spVecs[[n]]@i))))
        permM1 <- sparseMatrix(i=i,j=j,x=x)

> permM1
5 x 6 sparse Matrix of class "dgCMatrix"

[1,] 2.040574 .        .        .        .        1.876193
[2,] .        1.786480 .        .        1.032214 .
[3,] .        .        1.716743 3.711979 .        .
[4,] .        .        2.534000 3.302921 .        .
[5,] .        1.511992 .        .        1.084763 .

In this case, rows 3 and 4 remained unchanged.  But now this takes a really long time for a large sparse matrix.

Thanks for any advice.