Hey,
I am estiamting a partially linear model y=X\beta+f(\theta) where the f(\theta) is estiamted using wavelets. Has anyone heard of methods to test if the betas are significant or to address model fit? Thanks for any thoughts or comments. Elizabeth Lawson __________________________________________________ [[alternative HTML version deleted]] ______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html |
I have seen no replies to this post, and I don't know that I can
help, either. However, I wonder if you tried "RSiteSearch" with your favorite key words and phrases? For example, I just got 107 hits for 'RSiteSearch("wavelets")'. I wonder if any of them might help you. If you'd like further help from this list, please submit another post. However, before you do, I suggest you read the posting guide! "www.R-project.org/posting-guide.html". Anecdotal evidence suggests that posts more consistent with the guide tend to receive quicker, more useful replies. Best Wishes, spencer graves Elizabeth Lawson wrote: > Hey, > > I am estiamting a partially linear model y=X\beta+f(\theta) where the f(\theta) is estiamted using wavelets. > > Has anyone heard of methods to test if the betas are significant or to address model fit? > > Thanks for any thoughts or comments. > > Elizabeth Lawson > > __________________________________________________ > > > > [[alternative HTML version deleted]] > > ______________________________________________ > [hidden email] mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html -- Spencer Graves, PhD Senior Development Engineer PDF Solutions, Inc. 333 West San Carlos Street Suite 700 San Jose, CA 95110, USA [hidden email] www.pdf.com <http://www.pdf.com> Tel: 408-938-4420 Fax: 408-280-7915 ______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html |
In reply to this post by Elizabeth Lawson-2
This doesn't look like an R question, as I know of no pre-packaged
functionality publicly available that can fit the model that Elizabeth described, and it doesn't seem like she's particularly interested in an R-based answer, either. My gut feeling is that if there is a test of significance for beta in such a model, it probably shouldn't depend upon how f() is fitted, wavelets or otherwise. I.e., any test for the linear component in a partially linear model ought to do just fine. The main difference here, from a fully linear model, is that one no longer can estimate E(y) without bias, even with the assumption that the model is correct. What gets messier still is if data-dependent smoothing/de-noising is done in estimating f(), as that opens up a whole bucket of nasty creatures. I could be off, though, so take this with a truck-load of NaCl... Andy From: Spencer Graves > > I have seen no replies to this post, and I don't know > that I can > help, either. However, I wonder if you tried "RSiteSearch" with your > favorite key words and phrases? For example, I just got 107 hits for > 'RSiteSearch("wavelets")'. I wonder if any of them might help you. > > If you'd like further help from this list, please > submit another > post. However, before you do, I suggest you read the posting guide! > "www.R-project.org/posting-guide.html". Anecdotal evidence suggests > that posts more consistent with the guide tend to receive > quicker, more > useful replies. > > Best Wishes, > spencer graves > > Elizabeth Lawson wrote: > > > Hey, > > > > I am estiamting a partially linear model > y=X\beta+f(\theta) where the f(\theta) is estiamted using wavelets. > > > > Has anyone heard of methods to test if the betas are > significant or to address model fit? > > > > Thanks for any thoughts or comments. > > > > Elizabeth Lawson > > > > __________________________________________________ > > > > > > > > [[alternative HTML version deleted]] > > > > ______________________________________________ > > [hidden email] mailing list > > https://stat.ethz.ch/mailman/listinfo/r-help > > PLEASE do read the posting guide! > http://www.R-project.org/posting-guide.html > > -- > Spencer > Graves, PhD > Senior Development Engineer > PDF Solutions, Inc. > 333 West San Carlos Street Suite 700 > San Jose, CA 95110, USA > > [hidden email] > www.pdf.com <http://www.pdf.com> > Tel: 408-938-4420 > Fax: 408-280-7915 > > ______________________________________________ > [hidden email] mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide! > http://www.R-project.org/posting-guide.html > > ______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html |
"Liaw, Andy" <[hidden email]> writes:
> This doesn't look like an R question, as I know of no pre-packaged > functionality publicly available that can fit the model that Elizabeth > described, and it doesn't seem like she's particularly interested in an > R-based answer, either. > > My gut feeling is that if there is a test of significance for beta in such a > model, it probably shouldn't depend upon how f() is fitted, wavelets or > otherwise. I.e., any test for the linear component in a partially linear > model ought to do just fine. The main difference here, from a fully linear > model, is that one no longer can estimate E(y) without bias, even with the > assumption that the model is correct. What gets messier still is if > data-dependent smoothing/de-noising is done in estimating f(), as that opens > up a whole bucket of nasty creatures. > > I could be off, though, so take this with a truck-load of NaCl... Isn't it just a gam() model (package mgcv), if you replace the wavelets with splines? I haven't messed with this for a decade, but I seem to recall that there's a result to the effect that you need to undersmooth f slightly to get optimal inference for the beta. Perhaps look in Green & Silverman for the reference. > Andy > > From: Spencer Graves > > > > I have seen no replies to this post, and I don't know > > that I can > > help, either. However, I wonder if you tried "RSiteSearch" with your > > favorite key words and phrases? For example, I just got 107 hits for > > 'RSiteSearch("wavelets")'. I wonder if any of them might help you. > > > > If you'd like further help from this list, please > > submit another > > post. However, before you do, I suggest you read the posting guide! > > "www.R-project.org/posting-guide.html". Anecdotal evidence suggests > > that posts more consistent with the guide tend to receive > > quicker, more > > useful replies. > > > > Best Wishes, > > spencer graves > > > > Elizabeth Lawson wrote: > > > > > Hey, > > > > > > I am estiamting a partially linear model > > y=X\beta+f(\theta) where the f(\theta) is estiamted using wavelets. > > > > > > Has anyone heard of methods to test if the betas are > > significant or to address model fit? > > > > > > Thanks for any thoughts or comments. > > > > > > Elizabeth Lawson > > > > > > __________________________________________________ > > > > > > > > > > > > [[alternative HTML version deleted]] > > > > > > ______________________________________________ > > > [hidden email] mailing list > > > https://stat.ethz.ch/mailman/listinfo/r-help > > > PLEASE do read the posting guide! > > http://www.R-project.org/posting-guide.html > > > > -- > > Spencer > > Graves, PhD > > Senior Development Engineer > > PDF Solutions, Inc. > > 333 West San Carlos Street Suite 700 > > San Jose, CA 95110, USA > > > > [hidden email] > > www.pdf.com <http://www.pdf.com> > > Tel: 408-938-4420 > > Fax: 408-280-7915 > > > > ______________________________________________ > > [hidden email] mailing list > > https://stat.ethz.ch/mailman/listinfo/r-help > > PLEASE do read the posting guide! > > http://www.R-project.org/posting-guide.html > > > > > > ______________________________________________ > [hidden email] mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html > -- O__ ---- Peter Dalgaard Øster Farimagsgade 5, Entr.B c/ /'_ --- Dept. of Biostatistics PO Box 2099, 1014 Cph. K (*) \(*) -- University of Copenhagen Denmark Ph: (+45) 35327918 ~~~~~~~~~~ - ([hidden email]) FAX: (+45) 35327907 ______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html |
In reply to this post by Elizabeth Lawson-2
From: Peter Dalgaard
> > "Liaw, Andy" <[hidden email]> writes: > > > This doesn't look like an R question, as I know of no pre-packaged > > functionality publicly available that can fit the model > that Elizabeth > > described, and it doesn't seem like she's particularly > interested in an > > R-based answer, either. > > > > My gut feeling is that if there is a test of significance > for beta in such a > > model, it probably shouldn't depend upon how f() is fitted, > wavelets or > > otherwise. I.e., any test for the linear component in a > partially linear > > model ought to do just fine. The main difference here, > from a fully linear > > model, is that one no longer can estimate E(y) without > bias, even with the > > assumption that the model is correct. What gets messier still is if > > data-dependent smoothing/de-noising is done in estimating > f(), as that opens > > up a whole bucket of nasty creatures. > > > > I could be off, though, so take this with a truck-load of NaCl... > > Isn't it just a gam() model (package mgcv), if you replace the > wavelets with splines? I believe so. > I haven't messed with this for a decade, but I seem to recall that > there's a result to the effect that you need to undersmooth f slightly > to get optimal inference for the beta. Perhaps look in Green & > Silverman for the reference. A quote I heard from Prof. David Ruppert: "There are lies, damned lies, and then big O notations." I presume the need to undersmooth is to reduce the bias of the `smooth'. The problem is, by how much should one undersmooth, so the bias would go from O(k*n^-4) to O(k*n^-5) (I'm just making this up, but you get the idea)? Cheers, Andy > > > Andy > > > > From: Spencer Graves > > > > > > I have seen no replies to this post, and I don't know > > > that I can > > > help, either. However, I wonder if you tried > "RSiteSearch" with your > > > favorite key words and phrases? For example, I just got > 107 hits for > > > 'RSiteSearch("wavelets")'. I wonder if any of them might > help you. > > > > > > If you'd like further help from this list, please > > > submit another > > > post. However, before you do, I suggest you read the > posting guide! > > > "www.R-project.org/posting-guide.html". Anecdotal > evidence suggests > > > that posts more consistent with the guide tend to receive > > > quicker, more > > > useful replies. > > > > > > Best Wishes, > > > spencer graves > > > > > > Elizabeth Lawson wrote: > > > > > > > Hey, > > > > > > > > I am estiamting a partially linear model > > > y=X\beta+f(\theta) where the f(\theta) is estiamted using > wavelets. > > > > > > > > Has anyone heard of methods to test if the betas are > > > significant or to address model fit? > > > > > > > > Thanks for any thoughts or comments. > > > > > > > > Elizabeth Lawson > > > > > > > > __________________________________________________ > > > > > > > > > > > > > > > > [[alternative HTML version deleted]] > > > > > > > > ______________________________________________ > > > > [hidden email] mailing list > > > > https://stat.ethz.ch/mailman/listinfo/r-help > > > > PLEASE do read the posting guide! > > > http://www.R-project.org/posting-guide.html > > > > > > -- > > > Spencer > > > Graves, PhD > > > Senior Development Engineer > > > PDF Solutions, Inc. > > > 333 West San Carlos Street Suite 700 > > > San Jose, CA 95110, USA > > > > > > [hidden email] > > > www.pdf.com <http://www.pdf.com> > > > Tel: 408-938-4420 > > > Fax: 408-280-7915 > > > > > > ______________________________________________ > > > [hidden email] mailing list > > > https://stat.ethz.ch/mailman/listinfo/r-help > > > PLEASE do read the posting guide! > > > http://www.R-project.org/posting-guide.html > > > > > > > > > > ______________________________________________ > > [hidden email] mailing list > > https://stat.ethz.ch/mailman/listinfo/r-help > > PLEASE do read the posting guide! > http://www.R-project.org/posting-guide.html > > > > -- > O__ > ---- Peter Dalgaard Øster Farimagsgade 5, Entr.B > c/ /'_ --- Dept. of Biostatistics PO Box 2099, 1014 Cph. K > (*) \(*) -- University of Copenhagen Denmark Ph: > (+45) 35327918 > ~~~~~~~~~~ - ([hidden email]) FAX: > (+45) 35327907 > > ______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html |
"Liaw, Andy" <[hidden email]> writes:
> From: Peter Dalgaard > > > > "Liaw, Andy" <[hidden email]> writes: > > > > > This doesn't look like an R question, as I know of no pre-packaged > > > functionality publicly available that can fit the model > > that Elizabeth > > > described, and it doesn't seem like she's particularly > > interested in an > > > R-based answer, either. > > > > > > My gut feeling is that if there is a test of significance > > for beta in such a > > > model, it probably shouldn't depend upon how f() is fitted, > > wavelets or > > > otherwise. I.e., any test for the linear component in a > > partially linear > > > model ought to do just fine. The main difference here, > > from a fully linear > > > model, is that one no longer can estimate E(y) without > > bias, even with the > > > assumption that the model is correct. What gets messier still is if > > > data-dependent smoothing/de-noising is done in estimating > > f(), as that opens > > > up a whole bucket of nasty creatures. > > > > > > I could be off, though, so take this with a truck-load of NaCl... > > > > Isn't it just a gam() model (package mgcv), if you replace the > > wavelets with splines? > > I believe so. > > > I haven't messed with this for a decade, but I seem to recall that > > there's a result to the effect that you need to undersmooth f slightly > > to get optimal inference for the beta. Perhaps look in Green & > > Silverman for the reference. > > A quote I heard from Prof. David Ruppert: "There are lies, damned lies, and > then big O notations." > > I presume the need to undersmooth is to reduce the bias of the `smooth'. > The problem is, by how much should one undersmooth, so the bias would go > from O(k*n^-4) to O(k*n^-5) (I'm just making this up, but you get the idea)? > > Cheers, > Andy More like sacrificing the optimal O(n^-(2/5)) (?) convergence on the smooth part so that the bias is reduced below O(n^-(1/2)) at the expense of a bigger variance term in the MSE. The whole thing is controlled by having the bandwidth of the smoother shrink as O(n^-q) where q is, er, something... And of course the big lie is that there are some unknown multipliers that depend on the f that you are trying to estimate. > > > > > Andy > > > > > > From: Spencer Graves > > > > > > > > I have seen no replies to this post, and I don't know > > > > that I can > > > > help, either. However, I wonder if you tried > > "RSiteSearch" with your > > > > favorite key words and phrases? For example, I just got > > 107 hits for > > > > 'RSiteSearch("wavelets")'. I wonder if any of them might > > help you. > > > > > > > > If you'd like further help from this list, please > > > > submit another > > > > post. However, before you do, I suggest you read the > > posting guide! > > > > "www.R-project.org/posting-guide.html". Anecdotal > > evidence suggests > > > > that posts more consistent with the guide tend to receive > > > > quicker, more > > > > useful replies. > > > > > > > > Best Wishes, > > > > spencer graves > > > > > > > > Elizabeth Lawson wrote: > > > > > > > > > Hey, > > > > > > > > > > I am estiamting a partially linear model > > > > y=X\beta+f(\theta) where the f(\theta) is estiamted using > > wavelets. > > > > > > > > > > Has anyone heard of methods to test if the betas are > > > > significant or to address model fit? > > > > > > > > > > Thanks for any thoughts or comments. > > > > > > > > > > Elizabeth Lawson > > > > > > > > > > __________________________________________________ > > > > > > > > > > > > > > > > > > > > [[alternative HTML version deleted]] > > > > > > > > > > ______________________________________________ > > > > > [hidden email] mailing list > > > > > https://stat.ethz.ch/mailman/listinfo/r-help > > > > > PLEASE do read the posting guide! > > > > http://www.R-project.org/posting-guide.html > > > > > > > > -- > > > > Spencer > > > > Graves, PhD > > > > Senior Development Engineer > > > > PDF Solutions, Inc. > > > > 333 West San Carlos Street Suite 700 > > > > San Jose, CA 95110, USA > > > > > > > > [hidden email] > > > > www.pdf.com <http://www.pdf.com> > > > > Tel: 408-938-4420 > > > > Fax: 408-280-7915 > > > > > > > > ______________________________________________ > > > > [hidden email] mailing list > > > > https://stat.ethz.ch/mailman/listinfo/r-help > > > > PLEASE do read the posting guide! > > > > http://www.R-project.org/posting-guide.html > > > > > > > > > > > > > > ______________________________________________ > > > [hidden email] mailing list > > > https://stat.ethz.ch/mailman/listinfo/r-help > > > PLEASE do read the posting guide! > > http://www.R-project.org/posting-guide.html > > > > > > > -- > > O__ > > ---- Peter Dalgaard Øster Farimagsgade 5, Entr.B > > c/ /'_ --- Dept. of Biostatistics PO Box 2099, 1014 Cph. K > > (*) \(*) -- University of Copenhagen Denmark Ph: > > (+45) 35327918 > > ~~~~~~~~~~ - ([hidden email]) FAX: > > (+45) 35327907 > > > > > > > ------------------------------------------------------------------------------ > Notice: This e-mail message, together with any attachment...{{dropped}} ______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html |
Free forum by Nabble | Edit this page |