bowed linear approximations

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

bowed linear approximations

Evans, Richard K. (GRC-H000)
Hello,

Please run the following code snippet and note the resulting plot:

x <- c(10, 50)
y <- c(0.9444483, 0.7680123)
plot(x,y,type="b",log="x")
for(i in 1:50){
 xx <- exp(runif(1,log(min(x)),log(max(x)) ))
yy <- approx(x,y,xout=xx, method = "linear")
points(xx,yy$y)
}

notice the "log=x" plot parameter and the resulting "bow" in the linear approximation.

This makes sense when I realized that the plot command is first making the plot and then drawing straight lines between points on a log plot AFTER the plot is generated and that that's why the line is straight. I get that.
.. and it also makes sense that the bowed points are a result of the linear approximations being made BEFORE plotted in a logarithmic plot. I get that..

My goal is to make approximations that lie on the line produced on the plot as shown, so I realize that what I want to do is NOT linear approximations, but maybe "log" approximations?
However, the approximation methods are only "linear" and "constant" .. there isn't a "log" method to approximate with.

So can anyone tell me how to fix the code such that he approximated points DO lie on the line as plotted with the "log=x" plot parameter?
Oh, and they have to be uniformly distributed along the Log=x axis.. I think that's the tricky part.

Any help and/or insight would be greatly appreciated.

Thank you!
-Rich


        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: bowed linear approximations

Fox, John
Dear Rich,

Assuming that I understand what you want to do, try adding the following to your script (which, by the way, is more complicated that it needs to be):

xx <- 10:50
m <- lm(y ~ x)
yy <- predict(m, data.frame(x=xx))
lines(spline(xx, yy), col="blue")

m <- lm(y ~ log(x))
yy <- predict(m, data.frame(x=xx))
points(xx, yy, col="magenta")

The first set of commands adds a line corresponding to the points that you plotted, which if I understand right, is *not* what you want. The second set of commands shows how to find points along the diagonal straight line that you plotted, given their x-values, which is what I think you want.

If you examine the linear models fit, you'll see that they just interpolate between the two points, albeit differently.

I hope this helps,
 John

-----------------------------
John Fox, Professor Emeritus
McMaster University
Hamilton, Ontario, Canada
Web: socserv.mcmaster.ca/jfox




> -----Original Message-----
> From: R-help [mailto:[hidden email]] On Behalf Of Evans,
> Richard K. (GRC-H000)
> Sent: Monday, September 25, 2017 3:28 PM
> To: [hidden email]
> Subject: [R] bowed linear approximations
>
> Hello,
>
> Please run the following code snippet and note the resulting plot:
>
> x <- c(10, 50)
> y <- c(0.9444483, 0.7680123)
> plot(x,y,type="b",log="x")
> for(i in 1:50){
>  xx <- exp(runif(1,log(min(x)),log(max(x)) )) yy <- approx(x,y,xout=xx, method =
> "linear")
> points(xx,yy$y)
> }
>
> notice the "log=x" plot parameter and the resulting "bow" in the linear
> approximation.
>
> This makes sense when I realized that the plot command is first making the
> plot and then drawing straight lines between points on a log plot AFTER the
> plot is generated and that that's why the line is straight. I get that.
> .. and it also makes sense that the bowed points are a result of the linear
> approximations being made BEFORE plotted in a logarithmic plot. I get that..
>
> My goal is to make approximations that lie on the line produced on the plot as
> shown, so I realize that what I want to do is NOT linear approximations, but
> maybe "log" approximations?
> However, the approximation methods are only "linear" and "constant" .. there
> isn't a "log" method to approximate with.
>
> So can anyone tell me how to fix the code such that he approximated points
> DO lie on the line as plotted with the "log=x" plot parameter?
> Oh, and they have to be uniformly distributed along the Log=x axis.. I think
> that's the tricky part.
>
> Any help and/or insight would be greatly appreciated.
>
> Thank you!
> -Rich
>
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-
> guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: bowed linear approximations

Eric Berger
Hi Rich,
If I understand your comment about "uniformly distributed along the log=x
axis" then I think John's (second) set of commands needs a change to the
definition of xx, as follows:

xx <- exp(seq(from=log(min(x)),to=log(max(x)),length=50))
m <- lm(y ~ log(x))
yy <- predict(m, data.frame(x=xx))
points(xx, yy, col="red")

HTH,

Eric


On Mon, Sep 25, 2017 at 11:36 PM, Fox, John <[hidden email]> wrote:

> Dear Rich,
>
> Assuming that I understand what you want to do, try adding the following
> to your script (which, by the way, is more complicated that it needs to be):
>
> xx <- 10:50
> m <- lm(y ~ x)
> yy <- predict(m, data.frame(x=xx))
> lines(spline(xx, yy), col="blue")
>
> m <- lm(y ~ log(x))
> yy <- predict(m, data.frame(x=xx))
> points(xx, yy, col="magenta")
>
> The first set of commands adds a line corresponding to the points that you
> plotted, which if I understand right, is *not* what you want. The second
> set of commands shows how to find points along the diagonal straight line
> that you plotted, given their x-values, which is what I think you want.
>
> If you examine the linear models fit, you'll see that they just
> interpolate between the two points, albeit differently.
>
> I hope this helps,
>  John
>
> -----------------------------
> John Fox, Professor Emeritus
> McMaster University
> Hamilton, Ontario, Canada
> Web: socserv.mcmaster.ca/jfox
>
>
>
>
> > -----Original Message-----
> > From: R-help [mailto:[hidden email]] On Behalf Of Evans,
> > Richard K. (GRC-H000)
> > Sent: Monday, September 25, 2017 3:28 PM
> > To: [hidden email]
> > Subject: [R] bowed linear approximations
> >
> > Hello,
> >
> > Please run the following code snippet and note the resulting plot:
> >
> > x <- c(10, 50)
> > y <- c(0.9444483, 0.7680123)
> > plot(x,y,type="b",log="x")
> > for(i in 1:50){
> >  xx <- exp(runif(1,log(min(x)),log(max(x)) )) yy <- approx(x,y,xout=xx,
> method =
> > "linear")
> > points(xx,yy$y)
> > }
> >
> > notice the "log=x" plot parameter and the resulting "bow" in the linear
> > approximation.
> >
> > This makes sense when I realized that the plot command is first making
> the
> > plot and then drawing straight lines between points on a log plot AFTER
> the
> > plot is generated and that that's why the line is straight. I get that.
> > .. and it also makes sense that the bowed points are a result of the
> linear
> > approximations being made BEFORE plotted in a logarithmic plot. I get
> that..
> >
> > My goal is to make approximations that lie on the line produced on the
> plot as
> > shown, so I realize that what I want to do is NOT linear approximations,
> but
> > maybe "log" approximations?
> > However, the approximation methods are only "linear" and "constant" ..
> there
> > isn't a "log" method to approximate with.
> >
> > So can anyone tell me how to fix the code such that he approximated
> points
> > DO lie on the line as plotted with the "log=x" plot parameter?
> > Oh, and they have to be uniformly distributed along the Log=x axis.. I
> think
> > that's the tricky part.
> >
> > Any help and/or insight would be greatly appreciated.
> >
> > Thank you!
> > -Rich
> >
> >
> >       [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/posting-
> > guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: bowed linear approximations

Evans, Richard K. (GRC-H000)
Eric, John, thank you both very much for responding.

Ok.. I suppose I need to show more of the actual data.
I have this data:

freq <- c(2, 3, 5, 10, 50, 100, 200, 300, 500, 750, 1000, 1300, 1800, 2450, 2900, 3000, 4000, 5000, 6000, 7000, 8200, 9300, 10000, 11000, 18000, 26500, 33000, 40000)
mag <- c(1.9893038, 1.5088071, 1.1851947, 0.9444483, 0.7680123, 0.7458169, 0.7069638, 0.6393066, 0.6261539, 0.6263381, 0.7053774, 0.6900626, 0.6953527, 0.7843036, 0.9056359, 0.8867276, 0.8937421, 0.9492288, 0.9629118, 1.1972268, 1.0010515, 0.9945838, 1.0564356, 0.8733333, 1.1666667, 1.5366667, 1.4666667, 1.3166667)

that must be displayed this way:

plot(freq,mag,type="b",log="x")

Essentially, I just want to show that I can reliably approximate the magnitudes of new data for additional random frequencies uniformly lying on the frequency axis. And so I write the following:

for(i in 1:200){
xx <- exp(runif(1,log(min(freq)),log(max(freq)) ))
yy <- approx(freq,mag,xout=xx, method = "linear")
points(xx,yy$y,col=rgb(1,0,0))
}

And I have been puzzling over why the approximated points don't lie linearly over the original data set (especially prominent  in the bow between freq=10 and 50). Once I realized (and concurred with) why this bow exists, I have been struggling with how to make these approximations as expected.. In my original post, I think I oversimplified it too much by implying that my application was just 2 data points.

Are your suggestions still valid do you think?
-Rich
______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: bowed linear approximations

Evans, Richard K. (GRC-H000)
In reply to this post by Eric Berger
My apologies for the typos in the code.
Here is a corrected version you can copy/paste in R to see the issue.

freq <- c(2, 3, 5, 10, 50, 100, 200, 300, 500, 750, 1000, 1300, 1800, 2450, 2900, 3000, 4000, 5000, 6000, 7000, 8200, 9300, 10000, 11000, 18000, 26500, 33000, 40000);
mag <- c(1.9893038, 1.5088071, 1.1851947, 0.9444483, 0.7680123, 0.7458169, 0.7069638, 0.6393066, 0.6261539, 0.6263381, 0.7053774, 0.6900626, 0.6953527, 0.7843036, 0.9056359, 0.8867276, 0.8937421, 0.9492288, 0.9629118, 1.1972268, 1.0010515, 0.9945838, 1.0564356, 0.8733333, 1.1666667, 1.5366667, 1.4666667, 1.3166667);
plot(freq,mag,type="b",log="x");
for(i in 1:200){
xx <- exp(runif(1,log(min(freq)),log(max(freq)) ));
yy <- approx(freq,mag,xout=xx, method = "linear");
points(xx,yy$y,col=rgb(1,0,0));
}

For completeness, I have been puzzling over why the approximated points don't lie linearly over the original data set (especially prominent  in the bow between freq=10 and 50). Once I realized (and concurred with) why this bow exists, I have been struggling with how to make these approximations as expected.. In my original post, I think I oversimplified it too much by implying that my application was just 2 data points.

Are your suggestions still valid do you think?
-Rich
______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: bowed linear approximations

Fox, John
In reply to this post by Eric Berger
Dear Rich,

I think that it's generally a bad idea to give statistical (as opposed to simply technical) advice by email without knowing the context of the research. I think that you'd do well to seek help from a statistician, and not just do what I suggest below.

Interpolating the data only makes sense if there's no random component to the response (mag in your data). Otherwise, it makes more sense to get "predictions" from a statistical model that has an explicit error component for the response. In your case, a simple quadratic model in log(freq) seems to fit the data reasonably well.

To see what I mean, try

plot(log(freq), mag)
mod <- lm(mag ~ poly(log(freq), 2))
summary(mod)
points(log(freq), fitted(mod), pch=16)
lines(spline(log(freq), fitted(mod)))

Some basic regression diagnostics suggest that we can do better by taking the log of mag as well, producing a closer fit to the data and stabilizing the error variance:

plot(log(freq), log(mag))
mod2 <- lm(log(mag) ~ poly(log(freq), 2))
summary(mod2)
points(log(freq), fitted(mod2), pch=16)
lines(spline(log(freq), fitted(mod2)))

I have no idea whether this makes substantive sense in the context of your problem.

Best,
 John

> -----Original Message-----
> From: R-help [mailto:[hidden email]] On Behalf Of Evans,
> Richard K. (GRC-H000)
> Sent: Tuesday, September 26, 2017 10:01 AM
> To: Eric Berger <[hidden email]>; Fox, John <[hidden email]>
> Cc: [hidden email]
> Subject: Re: [R] bowed linear approximations
>
> My apologies for the typos in the code.
> Here is a corrected version you can copy/paste in R to see the issue.
>
> freq <- c(2, 3, 5, 10, 50, 100, 200, 300, 500, 750, 1000, 1300, 1800, 2450, 2900,
> 3000, 4000, 5000, 6000, 7000, 8200, 9300, 10000, 11000, 18000, 26500, 33000,
> 40000); mag <- c(1.9893038, 1.5088071, 1.1851947, 0.9444483, 0.7680123,
> 0.7458169, 0.7069638, 0.6393066, 0.6261539, 0.6263381, 0.7053774,
> 0.6900626, 0.6953527, 0.7843036, 0.9056359, 0.8867276, 0.8937421,
> 0.9492288, 0.9629118, 1.1972268, 1.0010515, 0.9945838, 1.0564356,
> 0.8733333, 1.1666667, 1.5366667, 1.4666667, 1.3166667);
> plot(freq,mag,type="b",log="x"); for(i in 1:200){ xx <-
> exp(runif(1,log(min(freq)),log(max(freq)) )); yy <- approx(freq,mag,xout=xx,
> method = "linear"); points(xx,yy$y,col=rgb(1,0,0)); }
>
> For completeness, I have been puzzling over why the approximated points
> don't lie linearly over the original data set (especially prominent  in the bow
> between freq=10 and 50). Once I realized (and concurred with) why this bow
> exists, I have been struggling with how to make these approximations as
> expected.. In my original post, I think I oversimplified it too much by implying
> that my application was just 2 data points.
>
> Are your suggestions still valid do you think?
> -Rich
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-
> guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: bowed linear approximations

Evans, Richard K. (GRC-H000)
[solved] -- As always, the solution is simple and I always feel foolish for not having seen it right away.

Thank you, John, for showing that the frequency data can be "converted" to log before the linear approximations are made.. once the linear approximations are done with log data, the approximations can be plotted into the "log=x" plot and everything is spot on :-)

Here is the updated code that does exactly what I had originally wanted:

freq <- c(2, 3, 5, 10, 50, 100, 200, 300, 500, 750, 1000, 1300, 1800, 2450, 2900, 3000, 4000, 5000, 6000, 7000, 8200, 9300, 10000, 11000, 18000, 26500, 33000, 40000);
mag <- c(1.9893038, 1.5088071, 1.1851947,  0.9444483, 0.7680123, 0.7458169, 0.7069638, 0.6393066, 0.6261539, 0.6263381, 0.7053774, 0.6900626, 0.6953527, 0.7843036, 0.9056359, 0.8867276, 0.8937421, 0.9492288, 0.9629118, 1.1972268, 1.0010515, 0.9945838, 1.0564356, 0.8733333, 1.1666667, 1.5366667, 1.4666667, 1.3166667);
flog <- log(freq);
plot(exp(flog),mag,type="b",log="x");
for(i in 1:2000){
xx <- runif(1,min(flog),max(flog));
yy <- approx(flog,mag,xout=xx, method = "linear");
points(exp(xx),yy$y,col=rgb(1,0,0));
}

Thank you!
-Rich

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.