Scaling - does it get any better results than not scaling?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Scaling - does it get any better results than not scaling?

Michael Thompson
Hi,
I seem to remember from classes that one effect of scaling / standardising data was to get better results in any analysis. But what I'm seeing when I study various explanations on scaling is that we get exactly the same results, just that when we look at standardised data it's easier to see proportionate effects.
This is all very well for the data scientist to further investigate, but from a practical point of view, (especially IF it doesn't improve the accuracy of the result) surely it adds complication to 'telling the story'
of the model to non-DS people?
So, is scaling a technique for the DS to use to find effects, while eventually delivering a non-scaled version to the users?
I'd like to be able to give the true story to my students, not some fairy story based on my misunderstanding. Hope you can help with this.
Michael

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Scaling - does it get any better results than not scaling?

Alex Zarebski
Hey,

Nice question, I'm interested to see what others have to say on this.
I'd like to point out a couple of algorithmic points:

- If you are using regularisation the scaling /will/ lead to different
results.
- If you are using an iterative method to estimate something, (yes very
vague but you get the gist), it can be very useful to know the data is
scaled in a particular way, i.e., it can inform an initial guess for the
iterative method.

On a pedagogical note, it might be interesting to point out to your
students that the act of choosing an scaling/transformation/preprocessing
can be useful as a way of understanding your data better.

Cheers,
Alex

On Tue, Jul 17, 2018 at 4:58 PM Michael Thompson <
[hidden email]> wrote:

> Hi,
> I seem to remember from classes that one effect of scaling / standardising
> data was to get better results in any analysis. But what I'm seeing when I
> study various explanations on scaling is that we get exactly the same
> results, just that when we look at standardised data it's easier to see
> proportionate effects.
> This is all very well for the data scientist to further investigate, but
> from a practical point of view, (especially IF it doesn't improve the
> accuracy of the result) surely it adds complication to 'telling the story'
> of the model to non-DS people?
> So, is scaling a technique for the DS to use to find effects, while
> eventually delivering a non-scaled version to the users?
> I'd like to be able to give the true story to my students, not some fairy
> story based on my misunderstanding. Hope you can help with this.
> Michael
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Scaling - does it get any better results than not scaling?

Roger Koenker-3
In reply to this post by Michael Thompson
In certain fields this sort of standardization has become customary based on some sort of (misguided) notion that it
induces “normality.”  For example, in anthropometric studies based on the international Demographic and Health
Surveys (DHS) childrens’ heights are often transformed to Z-scores prior to subsequent analysis under the dubious
presumption that variability around the Z-scores at various ages will be Gaussian.  In my experience this is rarely
justified, and analysts would be better off modeling the original data rather than doing the preliminary transformation.
This is discussed in further detail here:  https://projecteuclid.org/euclid.bjps/1313973394.

> On Jul 17, 2018, at 5:53 AM, Michael Thompson <[hidden email]> wrote:
>
> Hi,
> I seem to remember from classes that one effect of scaling / standardising data was to get better results in any analysis. But what I'm seeing when I study various explanations on scaling is that we get exactly the same results, just that when we look at standardised data it's easier to see proportionate effects.
> This is all very well for the data scientist to further investigate, but from a practical point of view, (especially IF it doesn't improve the accuracy of the result) surely it adds complication to 'telling the story'
> of the model to non-DS people?
> So, is scaling a technique for the DS to use to find effects, while eventually delivering a non-scaled version to the users?
> I'd like to be able to give the true story to my students, not some fairy story based on my misunderstanding. Hope you can help with this.
> Michael
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Scaling - does it get any better results than not scaling?

Bert Gunter-2
Prof. Koenker's response probably settles the matter, but if not, this
thread should really be taken offlist, as it is primarily about statistics
and not R programming.
stats.stackexchange.com might be an alternative place to post; indeed, I
suspect the issue has already been addressed in their archives.

Cheers,
Bert



Bert Gunter

"The trouble with having an open mind is that people keep coming along and
sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )

On Tue, Jul 17, 2018 at 1:02 AM, Roger Koenker <[hidden email]>
wrote:

> In certain fields this sort of standardization has become customary based
> on some sort of (misguided) notion that it
> induces “normality.”  For example, in anthropometric studies based on the
> international Demographic and Health
> Surveys (DHS) childrens’ heights are often transformed to Z-scores prior
> to subsequent analysis under the dubious
> presumption that variability around the Z-scores at various ages will be
> Gaussian.  In my experience this is rarely
> justified, and analysts would be better off modeling the original data
> rather than doing the preliminary transformation.
> This is discussed in further detail here:  https://projecteuclid.org/
> euclid.bjps/1313973394.
>
> > On Jul 17, 2018, at 5:53 AM, Michael Thompson <
> [hidden email]> wrote:
> >
> > Hi,
> > I seem to remember from classes that one effect of scaling /
> standardising data was to get better results in any analysis. But what I'm
> seeing when I study various explanations on scaling is that we get exactly
> the same results, just that when we look at standardised data it's easier
> to see proportionate effects.
> > This is all very well for the data scientist to further investigate, but
> from a practical point of view, (especially IF it doesn't improve the
> accuracy of the result) surely it adds complication to 'telling the story'
> > of the model to non-DS people?
> > So, is scaling a technique for the DS to use to find effects, while
> eventually delivering a non-scaled version to the users?
> > I'd like to be able to give the true story to my students, not some
> fairy story based on my misunderstanding. Hope you can help with this.
> > Michael
> >
> > ______________________________________________
> > [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Scaling - does it get any better results than not scaling?

Jeff Newmiller
In reply to this post by Michael Thompson
This question is interesting, but sadly off-topic here as there is nothing specific to R in it. Fortunately there are many resources for getting an answer... e.g. a quick search with Google finds [1] which addresses both centering and scaling.

[1] https://stats.stackexchange.com/questions/29781/when-conducting-multiple-regression-when-should-you-center-your-predictor-varia

On July 16, 2018 9:53:17 PM PDT, Michael Thompson <[hidden email]> wrote:

>Hi,
>I seem to remember from classes that one effect of scaling /
>standardising data was to get better results in any analysis. But what
>I'm seeing when I study various explanations on scaling is that we get
>exactly the same results, just that when we look at standardised data
>it's easier to see proportionate effects.
>This is all very well for the data scientist to further investigate,
>but from a practical point of view, (especially IF it doesn't improve
>the accuracy of the result) surely it adds complication to 'telling the
>story'
>of the model to non-DS people?
>So, is scaling a technique for the DS to use to find effects, while
>eventually delivering a non-scaled version to the users?
>I'd like to be able to give the true story to my students, not some
>fairy story based on my misunderstanding. Hope you can help with this.
>Michael
>
>______________________________________________
>[hidden email] mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.

--
Sent from my phone. Please excuse my brevity.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Scaling - does it get any better results than not scaling?

Richard M. Heiberger
In reply to this post by Michael Thompson
This is a variant of FAQ 7.31 on rounding.

For hand arithmetic, for example the variance of c(29,30,31), it was
easier to subtract the mean and work with c(-1,0,1).
For limited precision computers working directly with many-digit
numbers could lead to rounding in intermediate steps and catastrophic
cancellation.

For more information see FAQ 7.31 in file
system.file("../../doc/FAQ")
on your computer.  Open in your favorite text editor.

Here is a simple example using 5-bit arithmetic (rather than the R
standard double precision with 53 bits)  that shows catastrophic
cancellation.

library(Rmpfr)

NN <- 29:31
NN
NN^2
formatBin(NN)
formatBin(NN^2)

## 53 bit precision (double precision)
SSq <- NN[1]^2 +NN[2]^2 + NN[3]^2
SSq
CorrSSq <- SSq - ((NN[1]+NN[2]+NN[3])^2)/3
CorrSSq ## right answer
formatBin(CorrSSq)

## 5 bit precision
ONE <- mpfr(1, precBits=5)
NNO <- NN*ONE
NNO
NNO^2 ## note loss of precision
formatBin(NNO) ## 5-bit numbers.  Their squares require 10 bits.
formatBin(NNO^2) ## 10-bit squares rounded to 5 bits

SSqO <- NNO[1]^2 +NNO[2]^2 + NNO[3]^2
SSqO
CorrSSqO <- SSqO - ((NNO[1]+NNO[2]+NNO[3])^2)/3
CorrSSqO ## very wrong answer from catastrophic cancellation
formatBin(CorrSSqO)

## "normalizing" NNO  5 bit precision
NNOm30 <- NNO-30
NNOm30
NNOm30^2
SSqOm30 <- NNOm30[1]^2 +NNOm30[2]^2 + NNOm30[3]^2  ## 5 bit precision
SSqOm30 ## right answer, even with low-precision arithmetic
formatBin(SSqOm30)

formatBin(NNOm30)
formatBin(NNOm30^2)

On Tue, Jul 17, 2018 at 12:53 AM, Michael Thompson
<[hidden email]> wrote:

> Hi,
> I seem to remember from classes that one effect of scaling / standardising data was to get better results in any analysis. But what I'm seeing when I study various explanations on scaling is that we get exactly the same results, just that when we look at standardised data it's easier to see proportionate effects.
> This is all very well for the data scientist to further investigate, but from a practical point of view, (especially IF it doesn't improve the accuracy of the result) surely it adds complication to 'telling the story'
> of the model to non-DS people?
> So, is scaling a technique for the DS to use to find effects, while eventually delivering a non-scaled version to the users?
> I'd like to be able to give the true story to my students, not some fairy story based on my misunderstanding. Hope you can help with this.
> Michael
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Scaling - does it get any better results than not scaling?

Michael Thompson
In reply to this post by Bert Gunter-2
My thanks to all contributors, and while I was not in the right place, I certainly got the answers I needed. My students will benefit, so thank you all.

Regards,
Michael Thompson M.Prof.Studies Data Science
09 975 4678
Senior Lecturer, Digital Technologies
Manukau Campus
We all, like sheep, have gone astray Isaiah 53
Personal profile: https://www.manukau.ac.nz/about/faculties-schools/business-and-information-technology/more-information-for-students/lecturer-profiles/michael-thompson

From: Bert Gunter [mailto:[hidden email]]
Sent: Wednesday, 18 July 2018 3:02 AM
To: Roger Koenker <[hidden email]>
Cc: Michael Thompson <[hidden email]>; [hidden email]
Subject: Re: [R] Scaling - does it get any better results than not scaling?

Prof. Koenker's response probably settles the matter, but if not, this thread should really be taken offlist, as it is primarily about statistics and not R programming.
stats.stackexchange.com<http://stats.stackexchange.com> might be an alternative place to post; indeed, I suspect the issue has already been addressed in their archives.

Cheers,
Bert



Bert Gunter

"The trouble with having an open mind is that people keep coming along and sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )

On Tue, Jul 17, 2018 at 1:02 AM, Roger Koenker <[hidden email]<mailto:[hidden email]>> wrote:
In certain fields this sort of standardization has become customary based on some sort of (misguided) notion that it
induces “normality.”  For example, in anthropometric studies based on the international Demographic and Health
Surveys (DHS) childrens’ heights are often transformed to Z-scores prior to subsequent analysis under the dubious
presumption that variability around the Z-scores at various ages will be Gaussian.  In my experience this is rarely
justified, and analysts would be better off modeling the original data rather than doing the preliminary transformation.
This is discussed in further detail here:  https://projecteuclid.org/euclid.bjps/1313973394.

> On Jul 17, 2018, at 5:53 AM, Michael Thompson <[hidden email]<mailto:[hidden email]>> wrote:
>
> Hi,
> I seem to remember from classes that one effect of scaling / standardising data was to get better results in any analysis. But what I'm seeing when I study various explanations on scaling is that we get exactly the same results, just that when we look at standardised data it's easier to see proportionate effects.
> This is all very well for the data scientist to further investigate, but from a practical point of view, (especially IF it doesn't improve the accuracy of the result) surely it adds complication to 'telling the story'
> of the model to non-DS people?
> So, is scaling a technique for the DS to use to find effects, while eventually delivering a non-scaled version to the users?
> I'd like to be able to give the true story to my students, not some fairy story based on my misunderstanding. Hope you can help with this.
> Michael
>
> ______________________________________________
> [hidden email]<mailto:[hidden email]> mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email]<mailto:[hidden email]> mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.