fitted from plm

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

fitted from plm

Max Brown-2
Hi,

I am estimating a (fixed-effects) model with plm, for which I would like
to get the fitted values. If I call fitted() on my estimated model, it
returns NULL.

How do I get the fitted values out of the plm object?

Thanks.

Max

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: fitted from plm

Achim Zeileis-4
On Wed, 20 Oct 2010, [hidden email] wrote:

> Hi,
>
> I am estimating a (fixed-effects) model with plm, for which I would like
> to get the fitted values. If I call fitted() on my estimated model, it
> returns NULL.
>
> How do I get the fitted values out of the plm object?

I think there is currently no method for that. In many cases, you can
compute them fairly easily from the residuals via

   y - residuals(...)

And if the model frame is retained you find y in the $model element.
Hence, a dirty hack that should work is

   fitted.panelmodel <- function(object, ...)
     object$model[[1]] - object$residuals

However, beware of models where the number of observations might be
reduced due to lags and differencing.

And of course this covers just in-sample predictions. The out-of-sample
case would have to be done by hand.

Yves and Giovanni (cc) might be able to provide more/better/correct
information...

Best,
Z

> Thanks.
>
> Max
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: fitted from plm

Millo Giovanni
Hello.
Correct, IMHO: Achim's suggestion is the best way to get in-sample fitted values, w.r.t. e.g. direct calculation as yhat=X%*%coef, as it implicitly takes care of fixed effects etc..
A method for out-of-sample use is not implemented yet. It will eventually be, but there is no "schedule" for now. There are peculiar design issues: one for all, double dimensionality (prediction on the N dimension is meaningless for FE models etc. etc.).

Best,
Giovanni

-----Original Message-----
From: Achim Zeileis [mailto:[hidden email]]
Sent: Wed 20/10/2010 17.16
To: [hidden email]
Cc: [hidden email]; [hidden email]; Millo Giovanni
Subject: Re: [R] fitted from plm
 
On Wed, 20 Oct 2010, [hidden email] wrote:

> Hi,
>
> I am estimating a (fixed-effects) model with plm, for which I would like
> to get the fitted values. If I call fitted() on my estimated model, it
> returns NULL.
>
> How do I get the fitted values out of the plm object?

I think there is currently no method for that. In many cases, you can
compute them fairly easily from the residuals via

   y - residuals(...)

And if the model frame is retained you find y in the $model element.
Hence, a dirty hack that should work is

   fitted.panelmodel <- function(object, ...)
     object$model[[1]] - object$residuals

However, beware of models where the number of observations might be
reduced due to lags and differencing.

And of course this covers just in-sample predictions. The out-of-sample
case would have to be done by hand.

Yves and Giovanni (cc) might be able to provide more/better/correct
information...

Best,
Z

> Thanks.
>
> Max
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>


 
Ai sensi del D.Lgs. 196/2003 si precisa che le informazi...{{dropped:15}}

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: fitted from plm

Max Brown-2

Thanks Giovanni and Achim.

Just to confirm - the order of observations in residuals(mymodel) is
exactly the same as in the dataframe that I pass to plm, so that I can
extract the fitted values for an individual by calculating

yhat = y - residuals(mymodel)

as you describe, and then pick out individuals by comparing my yhat
vector with my y vector in the dataframe, right? (The residuals aren't
stacked in a funny way?)

Thanks.

Max

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

R: fitted from plm

Millo Giovanni
Hi Max. You're welcome.
Re your question below, the input order is not always preserved: observations are always reordered by (individual, time) internally, so that the output you get is ordered accordingly. Most of the time, this is also the order you'd input them, but this is not necessarily so.

Therefore if you do as Achim said, taking both y and residuals from the 'panelmodel' object, you are safe; while if you take residuals from there, y form the original data then you are exposed to the risk that the original data be ordered differently.

Here a quick illustration on the venerable Grunfeld data (some output suppressed, some comments added):

##################################################
> library(plm)
> data(Grunfeld)
> head(Grunfeld) # original data: ordered as stacked time series
  firm year   inv  value capital
1    1 1935 317.6 3078.5     2.8
2    1 1936 391.8 4661.7    52.6
3    1 1937 410.6 5387.1   156.9
4    1 1938 257.7 2792.2   209.2
5    1 1939 330.8 4313.2   203.4
6    1 1940 461.2 4643.9   207.2
>
> fm<-inv~value+capital
> remod<-plm(fm, Grunfeld, model="random")
> head(residuals(remod))
         1          2          3          4          5          6
  44.97404  -69.97550 -162.94693  -47.09013 -139.18021  -46.25567
> head(remod$model[[1]]) # this is 'y' (='inv')
    1     2     3     4     5     6
317.6 391.8 410.6 257.7 330.8 461.2
> head(remod$model[[2]]) # this is 'X[,1]' (='value')
     1      2      3      4      5      6
3078.5 4661.7 5387.1 2792.2 4313.2 4643.9
>
> ## reorder as stacked cross-sections
> Grun2<-Grunfeld[order(Grunfeld$year, Grunfeld$firm),]
>
> head(Grun2)
    firm year    inv  value capital
1      1 1935 317.60 3078.5     2.8
21     2 1935 209.90 1362.4    53.8
41     3 1935  33.10 1170.6    97.8
61     4 1935  40.29  417.5    10.5
81     5 1935  39.68  157.7   183.2
101    6 1935  20.36  197.0     6.5
>
> remod<-plm(fm, Grun2, model="random")
> head(residuals(remod))
         1          2          3          4          5          6
  44.97404  -69.97550 -162.94693  -47.09013 -139.18021  -46.25567
> head(remod$model[[1]])
    1     2     3     4     5     6
317.6 391.8 410.6 257.7 330.8 461.2
>
> ## output maintains standard order!
>
> ## reorder as completely random
> Grun3<-Grunfeld[order(runif(dim(Grunfeld)[[1]])),]
>
> head(Grun3)
    firm year    inv   value capital
27     2 1941 472.80 2380.50  261.40
2      1 1936 391.80 4661.70   52.60
184   10 1938   1.99   58.72    4.56
106    6 1940  28.54  298.00   52.50
79     4 1953 174.93 1001.50  346.10
194   10 1948   5.66   68.00    5.38
>
> remod<-plm(fm, Grun3, model="random")
> head(residuals(remod))
         1          2          3          4          5          6
  44.97404  -69.97550 -162.94693  -47.09013 -139.18021  -46.25567
> head(remod$model[[1]])
    1     2     3     4     5     6
317.6 391.8 410.6 257.7 330.8 461.2
>
> ## same as above
#############################################

Take heed, of course, of the 'index' argument to the model (you don't need it only if data have individual and time indices in the first two cols., respectively).
HTH,
Giovanni

PS if you'd prefer us to stack residuals "in a funny way", just ask ;^)

-----Messaggio originale-----
Da: [hidden email] [mailto:[hidden email]]
Inviato: giovedì 21 ottobre 2010 16:22
A: Millo Giovanni
Cc: Achim Zeileis; [hidden email]
Oggetto: Re: fitted from plm


Thanks Giovanni and Achim.

Just to confirm - the order of observations in residuals(mymodel) is exactly the same as in the dataframe that I pass to plm, so that I can extract the fitted values for an individual by calculating

yhat = y - residuals(mymodel)

as you describe, and then pick out individuals by comparing my yhat vector with my y vector in the dataframe, right? (The residuals aren't stacked in a funny way?)

Thanks.

Max



Ai sensi del D.Lgs. 196/2003 si precisa che le informazi...{{dropped:13}}

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: fitted from plm

Max Brown-2
Wow, thanks for the very detailed reply!

Max

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.