Non-positive definite cross-covariance matrices

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Non-positive definite cross-covariance matrices

Jeff Bassett
I am creating covariance matrices from sets of points, and I am having
frequent problems where I create matrices that are non-positive
definite.  I've started using the corpcor package, which was
specifically designed to address these types of problems.  It has
solved many of my problems, but I still have one left.

One of the matrices I need to calculate is a cross-covariance matrix.
In other words, I need to calculate cov(A, B), where A and B are each
a matrix defining a set of points.  The corpcor package does not seem
to be able to perform this operation.

Can anyone suggest a way to create cross-covariance matrices that are
guaranteed (or at least likely) to be positive definite, either using
corpcor or another package?

I'm using R 2.8.1 and corpcor 1.5.2 on Mac OS X 10.5.8.

- Jeff

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Non-positive definite cross-covariance matrices

Giovanni Petris
What made you think that a cross-covariance matrix should be positive
definite? Id does not even need to be a square matrix, or symmetric.

Giovanni Petris

On Mon, 2010-11-15 at 12:58 -0500, Jeff Bassett wrote:

> I am creating covariance matrices from sets of points, and I am having
> frequent problems where I create matrices that are non-positive
> definite.  I've started using the corpcor package, which was
> specifically designed to address these types of problems.  It has
> solved many of my problems, but I still have one left.
>
> One of the matrices I need to calculate is a cross-covariance matrix.
> In other words, I need to calculate cov(A, B), where A and B are each
> a matrix defining a set of points.  The corpcor package does not seem
> to be able to perform this operation.
>
> Can anyone suggest a way to create cross-covariance matrices that are
> guaranteed (or at least likely) to be positive definite, either using
> corpcor or another package?
>
> I'm using R 2.8.1 and corpcor 1.5.2 on Mac OS X 10.5.8.
>
> - Jeff
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Non-positive definite cross-covariance matrices

Jeff Bassett
Giovanni,

Both matrices describing the points (A and B in my example) are the
same size, so the resulting matrix will always be square.  Also, the
equation I'm using is essentially the following identity:

Var(A + B) = Var(A) + Var(B) + Cov(A, B) + Cov(B, A)

All the covariance matrices that result from the Var() terms should be
positive definite, and while it seems possible that either of those
resulting from the Cov() terms may not be, the sum of the two should.
Do you agree?

Of course, the above identity only holds if the data is normally
distributed.  The Mardia test for multivariate normality in fact shows
that my data is not.  This may ultimately be my problem.  So maybe I
should be asking if you could point me toward some packages that can
transform my data so that it is normally distributed.

- Jeff


On Mon, Nov 15, 2010 at 3:56 PM, Giovanni Petris <[hidden email]> wrote:

> What made you think that a cross-covariance matrix should be positive
> definite? Id does not even need to be a square matrix, or symmetric.
>
> Giovanni Petris
>
> On Mon, 2010-11-15 at 12:58 -0500, Jeff Bassett wrote:
>> I am creating covariance matrices from sets of points, and I am having
>> frequent problems where I create matrices that are non-positive
>> definite.  I've started using the corpcor package, which was
>> specifically designed to address these types of problems.  It has
>> solved many of my problems, but I still have one left.
>>
>> One of the matrices I need to calculate is a cross-covariance matrix.
>> In other words, I need to calculate cov(A, B), where A and B are each
>> a matrix defining a set of points.  The corpcor package does not seem
>> to be able to perform this operation.
>>
>> Can anyone suggest a way to create cross-covariance matrices that are
>> guaranteed (or at least likely) to be positive definite, either using
>> corpcor or another package?
>>
>> I'm using R 2.8.1 and corpcor 1.5.2 on Mac OS X 10.5.8.
>>
>> - Jeff
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
>
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Non-positive definite cross-covariance matrices

plangfelder
On Tue, Nov 16, 2010 at 9:40 AM, Jeff Bassett <[hidden email]> wrote:

> Giovanni,
>
> Both matrices describing the points (A and B in my example) are the
> same size, so the resulting matrix will always be square.  Also, the
> equation I'm using is essentially the following identity:
>
> Var(A + B) = Var(A) + Var(B) + Cov(A, B) + Cov(B, A)
>
> All the covariance matrices that result from the Var() terms should be
> positive definite, and while it seems possible that either of those
> resulting from the Cov() terms may not be, the sum of the two should.
> Do you agree?
>
> Of course, the above identity only holds if the data is normally
> distributed.

Hi Jeff,

as far as I can see, the identity above is an identity and holds
irrespective of how your data is distributed (just write out the
difinition of var and cov and you will see that the equation is always
true).

It is easy to come up with examples where Cov(A, B) + Cov(B, A) is not
positive definite. As an extreme example, consider a matrix A (say 10
columns, 100 rows) such that the off-diagonal covariances are all zero
and the columns are standardized, so cov(A) = diag(1, 1, 1, ...). Then
take B = -A, so cov(A, B) = cov(B, A) = diag(-1, -1, -1, ...).
Obviously, cov(A, B) + cov(B, A) is not positively definite, in fact
it is negative definite.

If the matrices A and B are completely independent, it is not very
likely that cov(A,B) + cov(B,A) will be positive definite. For
example, if A and B had just one column, there's a 50-50 chance that
cov(A, B) is negative (single number). When you have more than one
column, the chance is even less than 50-50 because all eigenvalues
would have to be positive.

You may be able to generate matrices whose cov(A, B) is positive
definite by starting with a matrix A, then generating a random matrix
B0, subtracting from the columns of B0 the projections of columns of
B0 into the columns of A, then adding a small multiple of A to get B.
But this may not be what you want or need.

Alternatively (and more likely), something in your approach may need
re-thinking.

HTH,

Peter


> - Jeff
>
>
> On Mon, Nov 15, 2010 at 3:56 PM, Giovanni Petris <[hidden email]> wrote:
>> What made you think that a cross-covariance matrix should be positive
>> definite? Id does not even need to be a square matrix, or symmetric.
>>
>> Giovanni Petris
>>
>> On Mon, 2010-11-15 at 12:58 -0500, Jeff Bassett wrote:
>>> I am creating covariance matrices from sets of points, and I am having
>>> frequent problems where I create matrices that are non-positive
>>> definite.  I've started using the corpcor package, which was
>>> specifically designed to address these types of problems.  It has
>>> solved many of my problems, but I still have one left.
>>>
>>> One of the matrices I need to calculate is a cross-covariance matrix.
>>> In other words, I need to calculate cov(A, B), where A and B are each
>>> a matrix defining a set of points.  The corpcor package does not seem
>>> to be able to perform this operation.
>>>
>>> Can anyone suggest a way to create cross-covariance matrices that are
>>> guaranteed (or at least likely) to be positive definite, either using
>>> corpcor or another package?
>>>
>>> I'm using R 2.8.1 and corpcor 1.5.2 on Mac OS X 10.5.8.
>>>
>>> - Jeff
>>>
>>> ______________________________________________
>>> [hidden email] mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Non-positive definite cross-covariance matrices

Jeff Bassett
On Tue, Nov 16, 2010 at 1:49 PM, Peter Langfelder
<[hidden email]> wrote:
>
> It is easy to come up with examples where Cov(A, B) + Cov(B, A) is not
> positive definite. As an extreme example, consider a matrix A (say 10
> columns, 100 rows) such that the off-diagonal covariances are all zero
> and the columns are standardized, so cov(A) = diag(1, 1, 1, ...). Then
> take B = -A, so cov(A, B) = cov(B, A) = diag(-1, -1, -1, ...).
> Obviously, cov(A, B) + cov(B, A) is not positively definite, in fact
> it is negative definite.

Peter,

I see your point.  As it turns out though, what I'm trying to
calculate is heritability using a slightly modified version of an
equation from multivariate quantitative genetics.  Theoretically I
suppose a heritability matrix could be non-positive definite, but in
practice it almost never is, at least from what I understand.

I think I've found a solution to my problem though.  The equation I
showed before can be rearranged so that the cross-covariance terms are
described in terms of the Var() terms.

Cov(A, B) + Cov(B, A) = Var(A) + Var(B) - Var(A + B)

Since the corpcor package can calculate positive definite versions of
all the Var() terms, I can then calculate the sum of the
cross-covariance terms from those.  I've done some preliminary tests,
and it seems to be working quite well.

Thanks for all the help,
- Jeff

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Non-positive definite cross-covariance matrices

plangfelder
> Peter,
>
> I see your point.  As it turns out though, what I'm trying to
> calculate is heritability using a slightly modified version of an
> equation from multivariate quantitative genetics.  Theoretically I
> suppose a heritability matrix could be non-positive definite, but in
> practice it almost never is, at least from what I understand.
>
> I think I've found a solution to my problem though.  The equation I
> showed before can be rearranged so that the cross-covariance terms are
> described in terms of the Var() terms.
>
> Cov(A, B) + Cov(B, A) = Var(A) + Var(B) - Var(A + B)
>
> Since the corpcor package can calculate positive definite versions of
> all the Var() terms, I can then calculate the sum of the
> cross-covariance terms from those.  I've done some preliminary tests,
> and it seems to be working quite well.

Hi Jeff,

well, if it works for you, then use it (note that the right hand side
of the equation should have reversed sign), although I have to say
it's not clear to me how the rearrangement helps.  Even if all three
variance matrices are positive definite, their sum need not be.

Peter

Peter

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Non-positive definite cross-covariance matrices

Mike Marchywka








----------------------------------------

> Date: Tue, 16 Nov 2010 17:39:57 -0800
> From: [hidden email]
> To: [hidden email]
> CC: [hidden email]
> Subject: Re: [R] Non-positive definite cross-covariance matrices
>
> > Peter,
> >
> > I see your point.  As it turns out though, what I'm trying to
> > calculate is heritability using a slightly modified version of an
> > equation from multivariate quantitative genetics.  Theoretically I
> > suppose a heritability matrix could be non-positive definite, but in
> > practice it almost never is, at least from what I understand.
> >
> > I think I've found a solution to my problem though.  The equation I
> > showed before can be rearranged so that the cross-covariance terms are
> > described in terms of the Var() terms.
> >
> > Cov(A, B) + Cov(B, A) = Var(A) + Var(B) - Var(A + B)
> >
> > Since the corpcor package can calculate positive definite versions of
> > all the Var() terms, I can then calculate the sum of the
> > cross-covariance terms from those.  I've done some preliminary tests,
> > and it seems to be working quite well.
>
> Hi Jeff,
>
> well, if it works for you, then use it (note that the right hand side
> of the equation should have reversed sign), although I have to say
> it's not clear to me how the rearrangement helps. Even if all three
> variance matrices are positive definite, their sum need not be.
>
Do you have a link to a paper on what you are trying to do?
It is quite common in many related expressions
to have quadratic terms and then get cross terms that don't act the same way.
The elements of A and B are ordered numbers right rather than codes
for bases?

Are these the formula you are talking about?
With scalars order not matter

http://mathworld.wolfram.com/Covariance.html

usually you expect matricies to be about the same and often
are for many special cases ( usually things like AB=BA etc) .
In this case, if you
look at the defintions, it seems the cross terms differ just
in being transpose of each other,

http://en.wikipedia.org/wiki/Covariance_matrix


Positive-sort-of-definite would seem to imply something
about the data. I guess one question might be that
given what you think this thing measures and given
what you think your data is, should it have some
properties like being positive? What does positive
mean in the system under consideration?


> Peter
>
> Peter
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
     
______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.