fuction to find outlier

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

fuction to find outlier

Methekar, Pushpa (GE Transportation, Non-GE)
Hi all, I am stuck on outlier, while doing regression analysis. I done up to modelling ,I got lm model for each y and x.
Now I want to find out outlier in that models. How do I find out outlier and remove them.


for(i in 1:10){
t1=print(outlierTest(fitted.modely1.temp.l ,cutoff=0.05, n.max=1, order=TRUE))
print(outlierTest(fitted.modely2.avg.pcp,cutoff=0.05, n.max=1, order=TRUE))
outlierTest(fitted.modely3.bshc,cutoff=0.05, n.max=1, order=TRUE )

outlierTest(fitted.modely4.bsco ,cutoff=0.05, n.max=1, order=TRUE)

outlierTest(fitted.modely5.gets ,cutoff=0.05, n.max=1, order=TRUE)

outlierTest(fitted.modely6.gimep,cutoff=0.05, n.max=1, order=TRUE )

outlierTest(fitted.modely7.ts ,cutoff=0.05, n.max=1, order=TRUE)

outlierTest(fitted.modely8.imp ,cutoff=0.05, n.max=1, order=TRUE)
outlierTest(fitted.modely9.maf ,cutoff=0.05, n.max=1, order=TRUE)
i<-i+1
}
fix(xsys)

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: fuction to find outlier

Fox, John
Dear Pushpa Methekar,

This apparently uses the outlierTest() function in the car package. I'm
afraid that I find your code incomprehensible and there's not enough
information here to reproduce what you've done. You're looping over the
index i in 1:10, explicitly incrementing the loop index at the end of the
loop (which, thankfully, will have no effect), and doing the same things in
each iteration -- i.e., repeatedly executing several commands without
modification. All this seems terribly confused and not clearly related to
locating and dealing with outliers.

Furthermore, it's not clear to me that what you propose is sensible -- to
delete outliers after performing statistical tests, in each case for a
single outlier, apparently mechanically and without investigating what's
going on in each case. It would probably be a good idea to consult a
competent statistician to help you decide how to proceed.

All that said, to answer your question directly, probably the easiest way to
remove an observation from a fitted model is to use update() with the subset
argument. For example model.1 <- update(model, subset = -6) would remove
case 6 from model and assign the result to model.1.

Best,
 John

-----------------------------------------------
John Fox, Professor
McMaster University
Hamilton, Ontario, Canada
http://socserv.socsci.mcmaster.ca/jfox/



> -----Original Message-----
> From: R-help [mailto:[hidden email]] On Behalf Of
> Methekar, Pushpa (GE Transportation, Non-GE)
> Sent: Monday, December 29, 2014 7:37 AM
> To: [hidden email]
> Subject: [R] fuction to find outlier
>
> Hi all, I am stuck on outlier, while doing regression analysis. I done
> up to modelling ,I got lm model for each y and x.
> Now I want to find out outlier in that models. How do I find out outlier
> and remove them.
>
>
> for(i in 1:10){
> t1=print(outlierTest(fitted.modely1.temp.l ,cutoff=0.05, n.max=1,
> order=TRUE))
> print(outlierTest(fitted.modely2.avg.pcp,cutoff=0.05, n.max=1,
> order=TRUE))
> outlierTest(fitted.modely3.bshc,cutoff=0.05, n.max=1, order=TRUE )
>
> outlierTest(fitted.modely4.bsco ,cutoff=0.05, n.max=1, order=TRUE)
>
> outlierTest(fitted.modely5.gets ,cutoff=0.05, n.max=1, order=TRUE)
>
> outlierTest(fitted.modely6.gimep,cutoff=0.05, n.max=1, order=TRUE )
>
> outlierTest(fitted.modely7.ts ,cutoff=0.05, n.max=1, order=TRUE)
>
> outlierTest(fitted.modely8.imp ,cutoff=0.05, n.max=1, order=TRUE)
> outlierTest(fitted.modely9.maf ,cutoff=0.05, n.max=1, order=TRUE)
> i<-i+1
> }
> fix(xsys)
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-
> guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: fuction to find outlier

glsnow
Pushpa,

To extend John Fox's answer a little.

Look at the "outliers" dataset in the TeachingDemos package, see
"?outliers" and run the examples on that help page.  Then ask yourself if
you are comfortable with the automatic outlier removal shown in the example.

On Mon, Dec 29, 2014 at 12:39 PM, John Fox <[hidden email]> wrote:

> Dear Pushpa Methekar,
>
> This apparently uses the outlierTest() function in the car package. I'm
> afraid that I find your code incomprehensible and there's not enough
> information here to reproduce what you've done. You're looping over the
> index i in 1:10, explicitly incrementing the loop index at the end of the
> loop (which, thankfully, will have no effect), and doing the same things in
> each iteration -- i.e., repeatedly executing several commands without
> modification. All this seems terribly confused and not clearly related to
> locating and dealing with outliers.
>
> Furthermore, it's not clear to me that what you propose is sensible -- to
> delete outliers after performing statistical tests, in each case for a
> single outlier, apparently mechanically and without investigating what's
> going on in each case. It would probably be a good idea to consult a
> competent statistician to help you decide how to proceed.
>
> All that said, to answer your question directly, probably the easiest way
> to
> remove an observation from a fitted model is to use update() with the
> subset
> argument. For example model.1 <- update(model, subset = -6) would remove
> case 6 from model and assign the result to model.1.
>
> Best,
>  John
>
> -----------------------------------------------
> John Fox, Professor
> McMaster University
> Hamilton, Ontario, Canada
> http://socserv.socsci.mcmaster.ca/jfox/
>
>
>
> > -----Original Message-----
> > From: R-help [mailto:[hidden email]] On Behalf Of
> > Methekar, Pushpa (GE Transportation, Non-GE)
> > Sent: Monday, December 29, 2014 7:37 AM
> > To: [hidden email]
> > Subject: [R] fuction to find outlier
> >
> > Hi all, I am stuck on outlier, while doing regression analysis. I done
> > up to modelling ,I got lm model for each y and x.
> > Now I want to find out outlier in that models. How do I find out outlier
> > and remove them.
> >
> >
> > for(i in 1:10){
> > t1=print(outlierTest(fitted.modely1.temp.l ,cutoff=0.05, n.max=1,
> > order=TRUE))
> > print(outlierTest(fitted.modely2.avg.pcp,cutoff=0.05, n.max=1,
> > order=TRUE))
> > outlierTest(fitted.modely3.bshc,cutoff=0.05, n.max=1, order=TRUE )
> >
> > outlierTest(fitted.modely4.bsco ,cutoff=0.05, n.max=1, order=TRUE)
> >
> > outlierTest(fitted.modely5.gets ,cutoff=0.05, n.max=1, order=TRUE)
> >
> > outlierTest(fitted.modely6.gimep,cutoff=0.05, n.max=1, order=TRUE )
> >
> > outlierTest(fitted.modely7.ts ,cutoff=0.05, n.max=1, order=TRUE)
> >
> > outlierTest(fitted.modely8.imp ,cutoff=0.05, n.max=1, order=TRUE)
> > outlierTest(fitted.modely9.maf ,cutoff=0.05, n.max=1, order=TRUE)
> > i<-i+1
> > }
> > fix(xsys)
> >
> >       [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/posting-
> > guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>



--
Gregory (Greg) L. Snow Ph.D.
[hidden email]

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.