Trend value from smoothing spline trend fit

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Trend value from smoothing spline trend fit

കുഞ്ഞായി kunjaai
Dear All,

I am trying to estimate the non -linear trend value from smooth spline
trend fit (using the generalized additive model (GAM)).

I want to estimate the trend value from a temperature dataset (spatial
averaged annual meantime from 1906 to 2005) as given in the Box 2.2, Table
1  in the attached Google doc pdf.  (That pages are from  IPCC Assessment
Report 5 chapter 2
<https://www.ipcc.ch/site/assets/uploads/2017/09/WG1AR5_Chapter02_FINAL.pdf>,
page number 21-22 )

I do not understand how they estimate the single value trend with 95%
confidence interval from a time-series data as given in the Box 2.2, Figure
1.  Is there any easy way to extract the trend value using mgcv library of
R.?

Google Doc Link:
https://drive.google.com/file/d/1z3XLW-154dsZE6GrvQ4rH_fev9lqdsho/view?usp=sharing

Thank you all in advance

Dileepkumar R

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Trend value from smoothing spline trend fit

Bert Gunter-2
Caveat: Did not look at any of your links.

However, the usual answer for this sort of question is ?predict.gam  (in
general, predict.whatevermethod)
Have you consulted the man page? If this is not what you want, you may need
to explain more carefully.

Bert Gunter

"The trouble with having an open mind is that people keep coming along and
sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Tue, Aug 11, 2020 at 5:21 AM Dileepkumar R <[hidden email]>
wrote:

> Dear All,
>
> I am trying to estimate the non -linear trend value from smooth spline
> trend fit (using the generalized additive model (GAM)).
>
> I want to estimate the trend value from a temperature dataset (spatial
> averaged annual meantime from 1906 to 2005) as given in the Box 2.2, Table
> 1  in the attached Google doc pdf.  (That pages are from  IPCC Assessment
> Report 5 chapter 2
> <
> https://www.ipcc.ch/site/assets/uploads/2017/09/WG1AR5_Chapter02_FINAL.pdf
> >,
> page number 21-22 )
>
> I do not understand how they estimate the single value trend with 95%
> confidence interval from a time-series data as given in the Box 2.2, Figure
> 1.  Is there any easy way to extract the trend value using mgcv library of
> R.?
>
> Google Doc Link:
>
> https://drive.google.com/file/d/1z3XLW-154dsZE6GrvQ4rH_fev9lqdsho/view?usp=sharing
>
> Thank you all in advance
>
> Dileepkumar R
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Trend value from smoothing spline trend fit

കുഞ്ഞായി kunjaai
Thank you for your reply.

Yes, we can get the curve fit values on the line (as the length of input
data frame) from predict.gam() function.  But I wish to get the trend value
(in Trends in °C per decade or °C per year) as given in the Box 2.2, Table
1.  But I couldn't find any option in GAM method.

Actually on of the reviewer of my paper suggested me to estimate the
non-linear trend as given in this Box 2.2, Table 1 and Figure 1.



Dileepkumar R



On Tue, Aug 11, 2020 at 8:01 PM Bert Gunter <[hidden email]> wrote:

> Caveat: Did not look at any of your links.
>
> However, the usual answer for this sort of question is ?predict.gam  (in
> general, predict.whatevermethod)
> Have you consulted the man page? If this is not what you want, you may
> need to explain more carefully.
>
> Bert Gunter
>
> "The trouble with having an open mind is that people keep coming along and
> sticking things into it."
> -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
>
>
> On Tue, Aug 11, 2020 at 5:21 AM Dileepkumar R <[hidden email]>
> wrote:
>
>> Dear All,
>>
>> I am trying to estimate the non -linear trend value from smooth spline
>> trend fit (using the generalized additive model (GAM)).
>>
>> I want to estimate the trend value from a temperature dataset (spatial
>> averaged annual meantime from 1906 to 2005) as given in the Box 2.2, Table
>> 1  in the attached Google doc pdf.  (That pages are from  IPCC Assessment
>> Report 5 chapter 2
>> <
>> https://www.ipcc.ch/site/assets/uploads/2017/09/WG1AR5_Chapter02_FINAL.pdf
>> >,
>> page number 21-22 )
>>
>> I do not understand how they estimate the single value trend with 95%
>> confidence interval from a time-series data as given in the Box 2.2,
>> Figure
>> 1.  Is there any easy way to extract the trend value using mgcv library of
>> R.?
>>
>> Google Doc Link:
>>
>> https://drive.google.com/file/d/1z3XLW-154dsZE6GrvQ4rH_fev9lqdsho/view?usp=sharing
>>
>> Thank you all in advance
>>
>> Dileepkumar R
>>
>>         [[alternative HTML version deleted]]
>>
>> ______________________________________________
>> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Trend value from smoothing spline trend fit

Bert Gunter-2
??

Change per year = (estimated end value - estimated begin value)/## years

Am I missing something subtle here?

Bert Gunter

"The trouble with having an open mind is that people keep coming along and
sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Tue, Aug 11, 2020 at 10:41 AM Dileepkumar R <[hidden email]>
wrote:

> Thank you for your reply.
>
> Yes, we can get the curve fit values on the line (as the length of input
> data frame) from predict.gam() function.  But I wish to get the trend value
> (in Trends in °C per decade or °C per year) as given in the Box 2.2, Table
> 1.  But I couldn't find any option in GAM method.
>
> Actually on of the reviewer of my paper suggested me to estimate the
> non-linear trend as given in this Box 2.2, Table 1 and Figure 1.
>
>
>
> Dileepkumar R
>
>
>
> On Tue, Aug 11, 2020 at 8:01 PM Bert Gunter <[hidden email]>
> wrote:
>
>> Caveat: Did not look at any of your links.
>>
>> However, the usual answer for this sort of question is ?predict.gam  (in
>> general, predict.whatevermethod)
>> Have you consulted the man page? If this is not what you want, you may
>> need to explain more carefully.
>>
>> Bert Gunter
>>
>> "The trouble with having an open mind is that people keep coming along
>> and sticking things into it."
>> -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
>>
>>
>> On Tue, Aug 11, 2020 at 5:21 AM Dileepkumar R <[hidden email]>
>> wrote:
>>
>>> Dear All,
>>>
>>> I am trying to estimate the non -linear trend value from smooth spline
>>> trend fit (using the generalized additive model (GAM)).
>>>
>>> I want to estimate the trend value from a temperature dataset (spatial
>>> averaged annual meantime from 1906 to 2005) as given in the Box 2.2,
>>> Table
>>> 1  in the attached Google doc pdf.  (That pages are from  IPCC Assessment
>>> Report 5 chapter 2
>>> <
>>> https://www.ipcc.ch/site/assets/uploads/2017/09/WG1AR5_Chapter02_FINAL.pdf
>>> >,
>>> page number 21-22 )
>>>
>>> I do not understand how they estimate the single value trend with 95%
>>> confidence interval from a time-series data as given in the Box 2.2,
>>> Figure
>>> 1.  Is there any easy way to extract the trend value using mgcv library
>>> of
>>> R.?
>>>
>>> Google Doc Link:
>>>
>>> https://drive.google.com/file/d/1z3XLW-154dsZE6GrvQ4rH_fev9lqdsho/view?usp=sharing
>>>
>>> Thank you all in advance
>>>
>>> Dileepkumar R
>>>
>>>         [[alternative HTML version deleted]]
>>>
>>> ______________________________________________
>>> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide
>>> http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>>
>>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Trend value from smoothing spline trend fit

കുഞ്ഞായി kunjaai
In reply to this post by കുഞ്ഞായി kunjaai
Dear Bert Gunter and Mathew Guilfoyle,

Thanks for the reply.

I also agree with you. But that is the actual slope of the straight line
connecting from the first estimated value to last estimated value ( like
dy/dx slope). I tried in that way also, but I couldn't replicate the
results as given in that  Box 2.2, Table 1 and Figure 1.  I can replicate
the line plot as in Box 2.2,  Figure 1 (a) (to verify my input data is the
same as in Figure 1 (a)).

You can see my plot here:
https://drive.google.com/file/d/14WDFFW5J69WvxnhbfdcOZKvGRk6HUWVn/view?usp=sharing


I have attached my code and data (as NetCDF file) along with this mail.

input Data (only 32Kb) :
https://drive.google.com/file/d/1Wt5sjVhWmjhRWOfdc6elUzwOvifRk-OI/view?usp=sharing

code:
https://drive.google.com/file/d/1r5mg1vcNmDCIf19jnMUAFEiLNOLKQiKq/view?usp=sharing

output figure:
https://drive.google.com/file/d/1bEXWCB-H5doVKXO8i_FLO7KZMW3qc9Cw/view?usp=sharing

Sincerely,

Dileepkumar R



On Tue, Aug 11, 2020 at 11:41 PM Mathew Guilfoyle <[hidden email]>
wrote:

> Looking at the report (specifically the legend for the table) I think all
> they have done is take the difference between fitted temperatures (given by
> linear regression or the GAM) at the start and end of each period (e.g. in
> 1901 and 2012), and then calculate the average change per year.
>
> In a way it ‘linearises’ the spline trend but I don’t think it’s really
> valid.  It just highlights how poorly a (simple) linear regression can fit
> a very non-linear trend.
>
> > On 11 Aug 2020, at 13:21, Dileepkumar R <[hidden email]> wrote:
> >
> > Dear All,
> >
> > I am trying to estimate the non -linear trend value from smooth spline
> > trend fit (using the generalized additive model (GAM)).
> >
> > I want to estimate the trend value from a temperature dataset (spatial
> > averaged annual meantime from 1906 to 2005) as given in the Box 2.2,
> Table
> > 1  in the attached Google doc pdf.  (That pages are from  IPCC Assessment
> > Report 5 chapter 2
> > <
> https://www.ipcc.ch/site/assets/uploads/2017/09/WG1AR5_Chapter02_FINAL.pdf
> >,
> > page number 21-22 )
> >
> > I do not understand how they estimate the single value trend with 95%
> > confidence interval from a time-series data as given in the Box 2.2,
> Figure
> > 1.  Is there any easy way to extract the trend value using mgcv library
> of
> > R.?
> >
> > Google Doc Link:
> >
> https://drive.google.com/file/d/1z3XLW-154dsZE6GrvQ4rH_fev9lqdsho/view?usp=sharing
> >
> > Thank you all in advance
> >
> > Dileepkumar R
> >
> >    [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.