Re: fdHess function

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: fdHess function

Douglas Bates-2
Your question is better addressed to the [hidden email] mailing list,
which I am copying on this reply.

You are confusing a statistical concept, the Fisher Information matrix,
with a numerical concept, the Hessian matrix of a scalar function of a
vector argument.

The Fisher information matrix is the Hessian matrix of a particular
function at its optimum and I have forgotten whether that function is the
log-likelihood or negative twice the log-likelihood or ...  Rather than get
it wrong I am sending a copy of this reply to the list where many of the
readers will be able to answer you more reliably than I can.


On Tue, Jan 22, 2013 at 1:22 PM, Marcos Coque Jr <[hidden email]>wrote:

> Dear Bates,
>
> I am using the fdHess function for R language.
> And I have a question.
>
> What is the relationship with the Hessian and Fisher Information in your
> function?
> Because I think that Fisher Information=-Hessian, but I found the oposite
> in your function.
> Maybe I be something wrong...
>
> Thanks,
>
> Marcos
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: fdHess function

mark leeds
Hi Doug: I was just looking at this coincidentally. When X is a vector, the
Fisher Information I_{theta} = the negative expectation of the second
derivatives of the log likelihood. So it's a matrix.  In other words,
I_theta = E(partial^2 /partial theta^2(log(X,theta).) where X is a vector.

But, even though the the Fisher Information has a seemingly nice formula, (
and this is where my confusion arose when I was dealing with this and why
I'm looking at it right
now. I have  short document that I wrote to myself  explaining it so if
anyone wants it, email me individually. It's nothing earth shattering !!!!!
) in many cases taking the that expectation is not easy so the  Fischer
Information is approximated by its empirical counterpart which is obtained
by summing each of the elements in the matrix given the n observations and
then dividing each of the elements in the matrix by n.













On Tue, Jan 22, 2013 at 3:27 PM, Douglas Bates <[hidden email]> wrote:

> Your question is better addressed to the [hidden email] mailing
> list,
> which I am copying on this reply.
>
> You are confusing a statistical concept, the Fisher Information matrix,
> with a numerical concept, the Hessian matrix of a scalar function of a
> vector argument.
>
> The Fisher information matrix is the Hessian matrix of a particular
> function at its optimum and I have forgotten whether that function is the
> log-likelihood or negative twice the log-likelihood or ...  Rather than get
> it wrong I am sending a copy of this reply to the list where many of the
> readers will be able to answer you more reliably than I can.
>
>
> On Tue, Jan 22, 2013 at 1:22 PM, Marcos Coque Jr <[hidden email]
> >wrote:
>
> > Dear Bates,
> >
> > I am using the fdHess function for R language.
> > And I have a question.
> >
> > What is the relationship with the Hessian and Fisher Information in your
> > function?
> > Because I think that Fisher Information=-Hessian, but I found the oposite
> > in your function.
> > Maybe I be something wrong...
> >
> > Thanks,
> >
> > Marcos
> >
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: fdHess function

mark leeds
I neglected to mention that, once you get either I_theta or some empirical
estimate
of it, you then invert it to get an estimate of the asymptotic covariance
matrix of the
MLE.


On Tue, Jan 22, 2013 at 3:48 PM, Mark Leeds <[hidden email]> wrote:

> Hi Doug: I was just looking at this coincidentally. When X is a vector,
> the Fisher Information I_{theta} = the negative expectation of the second
> derivatives of the log likelihood. So it's a matrix.  In other words,
> I_theta = E(partial^2 /partial theta^2(log(X,theta).) where X is a vector.
>
> But, even though the the Fisher Information has a seemingly nice formula,
> ( and this is where my confusion arose when I was dealing with this and why
> I'm looking at it right
> now. I have  short document that I wrote to myself  explaining it so if
> anyone wants it, email me individually. It's nothing earth shattering !!!!!
> ) in many cases taking the that expectation is not easy so the  Fischer
> Information is approximated by its empirical counterpart which is obtained
> by summing each of the elements in the matrix given the n observations and
> then dividing each of the elements in the matrix by n.
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Tue, Jan 22, 2013 at 3:27 PM, Douglas Bates <[hidden email]>wrote:
>
>> Your question is better addressed to the [hidden email] mailing
>> list,
>> which I am copying on this reply.
>>
>> You are confusing a statistical concept, the Fisher Information matrix,
>> with a numerical concept, the Hessian matrix of a scalar function of a
>> vector argument.
>>
>> The Fisher information matrix is the Hessian matrix of a particular
>> function at its optimum and I have forgotten whether that function is the
>> log-likelihood or negative twice the log-likelihood or ...  Rather than
>> get
>> it wrong I am sending a copy of this reply to the list where many of the
>> readers will be able to answer you more reliably than I can.
>>
>>
>> On Tue, Jan 22, 2013 at 1:22 PM, Marcos Coque Jr <[hidden email]
>> >wrote:
>>
>> > Dear Bates,
>> >
>> > I am using the fdHess function for R language.
>> > And I have a question.
>> >
>> > What is the relationship with the Hessian and Fisher Information in your
>> > function?
>> > Because I think that Fisher Information=-Hessian, but I found the
>> oposite
>> > in your function.
>> > Maybe I be something wrong...
>> >
>> > Thanks,
>> >
>> > Marcos
>> >
>>
>>         [[alternative HTML version deleted]]
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.