stepwise regression

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

stepwise regression

Jinsong Zhao
Dear all,

I have encountered a problem when perform stepwise regression.
The dataset have more 9 independent variables, but 7 observation.

In R, before performing stepwise, a lm object should be given.
fm <- lm(y ~ X1 + X2 + X3 + X11 + X22 + X33 + X12 + X13 + X23)

However, summary(fm) will give:

Residual standard error: NaN on 0 degrees of freedom
Multiple R-Squared:     1,      Adjusted R-squared:   NaN
F-statistic:   NaN on 6 and 0 DF,  p-value: NA

In this situation, step() or stepAIC() will not give any useful information.

I don't know why SAS could deal with this situation:
PROC REG;
 MODEL y=X1 X2 X3 X11 X22 X33 X12 X13 X23/SELECTION=STEPWISE;
RUN;

Any help will be really appreciated.

Wishes,

Jinsong Zhao


______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Reply | Threaded
Open this post in threaded view
|

Re: stepwise regression

Wincent
在 06-4-28,Jinsong Zhao<[hidden email]> 写道:
> Dear all,
>
> I have encountered a problem when perform stepwise regression.
> The dataset have more 9 independent variables, but 7 observation.
                        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~I think
this is the problem.

> In R, before performing stepwise, a lm object should be given.
> fm <- lm(y ~ X1 + X2 + X3 + X11 + X22 + X33 + X12 + X13 + X23)
>
> However, summary(fm) will give:
>
> Residual standard error: NaN on 0 degrees of freedom
> Multiple R-Squared:     1,      Adjusted R-squared:   NaN
> F-statistic:   NaN on 6 and 0 DF,  p-value: NA
>
> In this situation, step() or stepAIC() will not give any useful information.
>
> I don't know why SAS could deal with this situation:
> PROC REG;
>  MODEL y=X1 X2 X3 X11 X22 X33 X12 X13 X23/SELECTION=STEPWISE;
> RUN;
>
> Any help will be really appreciated.
>
> Wishes,
>
> Jinsong Zhao
>
>
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
>
>

--
黄荣贵
Deparment of Sociology
Fudan University


______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Reply | Threaded
Open this post in threaded view
|

Re: stepwise regression

Fox, John
In reply to this post by Jinsong Zhao
Dear Jinsong Zhao,

In proc reg in SAS, selection=stepwise does (modified) forward selection. In
step() in R, the default method is "backward" when the scope argument is
absent. To do (modified) forward selection, you can specify an initial model
with only a constant, and use the scope argument to specify all predictors.
See ?step for details.

It's hard to imagine, however, that it makes much sense to search for a
model with 9 predictors and 7 observations -- you'll just end up with a
model that fits perfectly.

I hope this helps,
 John

--------------------------------
John Fox
Department of Sociology
McMaster University
Hamilton, Ontario
Canada L8S 4M4
905-525-9140x23604
http://socserv.mcmaster.ca/jfox 
--------------------------------

> -----Original Message-----
> From: [hidden email]
> [mailto:[hidden email]] On Behalf Of Jinsong Zhao
> Sent: Thursday, April 27, 2006 7:58 PM
> To: r-help
> Subject: [R] stepwise regression
>
> Dear all,
>
> I have encountered a problem when perform stepwise regression.
> The dataset have more 9 independent variables, but 7 observation.
>
> In R, before performing stepwise, a lm object should be given.
> fm <- lm(y ~ X1 + X2 + X3 + X11 + X22 + X33 + X12 + X13 + X23)
>
> However, summary(fm) will give:
>
> Residual standard error: NaN on 0 degrees of freedom
> Multiple R-Squared:     1,      Adjusted R-squared:   NaN
> F-statistic:   NaN on 6 and 0 DF,  p-value: NA
>
> In this situation, step() or stepAIC() will not give any
> useful information.
>
> I don't know why SAS could deal with this situation:
> PROC REG;
>  MODEL y=X1 X2 X3 X11 X22 X33 X12 X13 X23/SELECTION=STEPWISE; RUN;
>
> Any help will be really appreciated.
>
> Wishes,
>
> Jinsong Zhao
>
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Reply | Threaded
Open this post in threaded view
|

Re: stepwise regression

Frank Harrell
In reply to this post by Jinsong Zhao
Jinsong Zhao wrote:
> Dear all,
>
> I have encountered a problem when perform stepwise regression.

You have more problems than you know.

> The dataset have more 9 independent variables, but 7 observation.

Why collect any data?  You can get great fits using random numbers using
this procedure.

Frank

>
> In R, before performing stepwise, a lm object should be given.
> fm <- lm(y ~ X1 + X2 + X3 + X11 + X22 + X33 + X12 + X13 + X23)
>
> However, summary(fm) will give:
>
> Residual standard error: NaN on 0 degrees of freedom
> Multiple R-Squared:     1,      Adjusted R-squared:   NaN
> F-statistic:   NaN on 6 and 0 DF,  p-value: NA
>
> In this situation, step() or stepAIC() will not give any useful information.
>
> I don't know why SAS could deal with this situation:
> PROC REG;
>  MODEL y=X1 X2 X3 X11 X22 X33 X12 X13 X23/SELECTION=STEPWISE;
> RUN;
>
> Any help will be really appreciated.
>
> Wishes,
>
> Jinsong Zhao

--
Frank E Harrell Jr   Professor and Chair           School of Medicine
                     Department of Biostatistics   Vanderbilt University


______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Frank Harrell
Department of Biostatistics, Vanderbilt University
Reply | Threaded
Open this post in threaded view
|

Re: stepwise regression

Thomas Lumley
In reply to this post by Jinsong Zhao
On Fri, 28 Apr 2006, Jinsong Zhao wrote:

> Dear all,
>
> I have encountered a problem when perform stepwise regression.
> The dataset have more 9 independent variables, but 7 observation.
>

The functions in the "leaps" package can do subset selection for data sets
with more variables than observations.

  -thomas


> In R, before performing stepwise, a lm object should be given.
> fm <- lm(y ~ X1 + X2 + X3 + X11 + X22 + X33 + X12 + X13 + X23)
>
> However, summary(fm) will give:
>
> Residual standard error: NaN on 0 degrees of freedom
> Multiple R-Squared:     1,      Adjusted R-squared:   NaN
> F-statistic:   NaN on 6 and 0 DF,  p-value: NA
>
> In this situation, step() or stepAIC() will not give any useful information.
>
> I don't know why SAS could deal with this situation:
> PROC REG;
> MODEL y=X1 X2 X3 X11 X22 X33 X12 X13 X23/SELECTION=STEPWISE;
> RUN;
>
> Any help will be really appreciated.
>
> Wishes,
>
> Jinsong Zhao
>
>

Thomas Lumley Assoc. Professor, Biostatistics
[hidden email] University of Washington, Seattle

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html