Odd results with %% and conserving memory

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Odd results with %% and conserving memory

JustinNabble
Can anyone explain this?

I have a matrix with double components. It's taking up a lot of memory, so I want to multiply then turn it to integers. I'm pretty certain that there are only 2 decimal places, but I wanted to check by using modulo. E.g.

mat = matrix(11:50/100, ncol=4,nrow=10) #Matrix with values out to the hundredths
any((mat * 100)%%1!=0)

But oddly enough it doesn't work. Even in this simple example the result I get is this:
             [,1]         [,2] [,3] [,4]
 [1,] 0.000000e+00 0.000000e+00    0    0
 [2,] 0.000000e+00 0.000000e+00    0    0
 [3,] 0.000000e+00 0.000000e+00    0    0
 [4,] 1.776357e-15 0.000000e+00    0    0
 [5,] 0.000000e+00 0.000000e+00    0    0
 [6,] 0.000000e+00 0.000000e+00    0    0
 [7,] 0.000000e+00 0.000000e+00    0    0
 [8,] 0.000000e+00 3.552714e-15    0    0
 [9,] 0.000000e+00 1.000000e+00    0    0
[10,] 0.000000e+00 0.000000e+00    0    0

Two non-zero values are just very small, but one is value is actually 1. Can someone explain this?

If you pick just a single number you can see some odd results too.

> (4.1*100)%/%1
[1] 409
> (4.1*10*10)%/%1
[1] 410

Shouldn't the result be 410 each time?


I think in this case it should have returned all 0s, and I could have done something like

newmat = as.integer(mat*100)
dim(newmat) = dim(mat)
rm(mat)

Is there a better way to convert my double matrix to an integer matrix without losing precision? Or are there better ways to conserve memory? I'm at the limit.

Thanks,
Justin
Reply | Threaded
Open this post in threaded view
|

Re: Odd results with %% and conserving memory

Patrick Burns
You seem to be in Circle 1 of 'The R Inferno'.

Your technique does work, just not the
way that you expect.  Try doing:

range( (mat * 100) %% 1)

The 'zapsmall' function might be of interest
as well.

On 26/03/2010 21:05, JustinNabble wrote:

>
> Can anyone explain this?
>
> I have a matrix with double components. It's taking up a lot of memory, so I
> want to multiply then turn it to integers. I'm pretty certain that there are
> only 2 decimal places, but I wanted to check by using modulo. E.g.
>
> mat = matrix(11:50/100, ncol=4,nrow=10) #Matrix with values out to the
> hundredths
> any((mat * 100)%%1!=0)
>
> But oddly enough it doesn't work. Even in this simple example the result I
> get is this:
>               [,1]         [,2] [,3] [,4]
>   [1,] 0.000000e+00 0.000000e+00    0    0
>   [2,] 0.000000e+00 0.000000e+00    0    0
>   [3,] 0.000000e+00 0.000000e+00    0    0
>   [4,] 1.776357e-15 0.000000e+00    0    0
>   [5,] 0.000000e+00 0.000000e+00    0    0
>   [6,] 0.000000e+00 0.000000e+00    0    0
>   [7,] 0.000000e+00 0.000000e+00    0    0
>   [8,] 0.000000e+00 3.552714e-15    0    0
>   [9,] 0.000000e+00 1.000000e+00    0    0
> [10,] 0.000000e+00 0.000000e+00    0    0
>
> Two non-zero values are just very small, but one is value is actually 1. Can
> someone explain this?
>
> If you pick just a single number you can see some odd results too.
>
>> (4.1*100)%/%1
> [1] 409
>> (4.1*10*10)%/%1
> [1] 410
>
> Shouldn't the result be 410 each time?
>
>
> I think in this case it should have returned all 0s, and I could have done
> something like
>
> newmat = as.integer(mat*100)
> dim(newmat) = dim(mat)
> rm(mat)
>
> Is there a better way to convert my double matrix to an integer matrix
> without losing precision? Or are there better ways to conserve memory? I'm
> at the limit.
>
> Thanks,
> Justin

--
Patrick Burns
[hidden email]
http://www.burns-stat.com
(home of 'Some hints for the R beginner'
and 'The R Inferno')

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Odd results with %% and conserving memory

jholtman
In reply to this post by JustinNabble
FAQ 7.31

Follow the link to what you should know about floating point numbers.

On Fri, Mar 26, 2010 at 5:05 PM, JustinNabble <[hidden email]>wrote:

>
> Can anyone explain this?
>
> I have a matrix with double components. It's taking up a lot of memory, so
> I
> want to multiply then turn it to integers. I'm pretty certain that there
> are
> only 2 decimal places, but I wanted to check by using modulo. E.g.
>
> mat = matrix(11:50/100, ncol=4,nrow=10) #Matrix with values out to the
> hundredths
> any((mat * 100)%%1!=0)
>
> But oddly enough it doesn't work. Even in this simple example the result I
> get is this:
>             [,1]         [,2] [,3] [,4]
>  [1,] 0.000000e+00 0.000000e+00    0    0
>  [2,] 0.000000e+00 0.000000e+00    0    0
>  [3,] 0.000000e+00 0.000000e+00    0    0
>  [4,] 1.776357e-15 0.000000e+00    0    0
>  [5,] 0.000000e+00 0.000000e+00    0    0
>  [6,] 0.000000e+00 0.000000e+00    0    0
>  [7,] 0.000000e+00 0.000000e+00    0    0
>  [8,] 0.000000e+00 3.552714e-15    0    0
>  [9,] 0.000000e+00 1.000000e+00    0    0
> [10,] 0.000000e+00 0.000000e+00    0    0
>
> Two non-zero values are just very small, but one is value is actually 1.
> Can
> someone explain this?
>
> If you pick just a single number you can see some odd results too.
>
> > (4.1*100)%/%1
> [1] 409
> > (4.1*10*10)%/%1
> [1] 410
>
> Shouldn't the result be 410 each time?
>
>
> I think in this case it should have returned all 0s, and I could have done
> something like
>
> newmat = as.integer(mat*100)
> dim(newmat) = dim(mat)
> rm(mat)
>
> Is there a better way to convert my double matrix to an integer matrix
> without losing precision? Or are there better ways to conserve memory? I'm
> at the limit.
>
> Thanks,
> Justin
> --
> View this message in context:
> http://n4.nabble.com/Odd-results-with-and-conserving-memory-tp1692845p1692845.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html<http://www.r-project.org/posting-guide.html>
> and provide commented, minimal, self-contained, reproducible code.
>



--
Jim Holtman
Cincinnati, OH
+1 513 646 9390

What is the problem that you are trying to solve?

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Odd results with %% and conserving memory

JustinNabble
In reply to this post by Patrick Burns
Thanks for the response. I read through the Inferno and there's a lot of useful stuff in there. However I think you may have missed something.

Having some experience with other languages, I know that floating point numbers aren't represented exactly, the issue is that R is off by much more than would be expected. On my computer, cell 9,2 of (mat * 100) %% 1 is larger than .99 and the correct result is 0. That difference seems too large to be a problem with representing floating point numbers.

Likewise, identical( (4.1 * 100 ) %/% 1 , 409 ) returns TRUE on my computer, so the result is off by exactly 1, too larger to be explained by floating point error.

I think there's a problem with how the %% and %/% operators are getting their results.
Reply | Threaded
Open this post in threaded view
|

Re: Odd results with %% and conserving memory

Bert Gunter
Ummm... I think the "problem" is with _your_ inadequate understanding.

Consider: identical( (4.1 * 100 ) %/% 1 , 409 )

> 4.1*100 < 410
[1] TRUE

So, due to finite precision, 4.1*100 is slightly less than 410.
Whence the largest integer 1 can be divided into it is 409. i.e.

> (4.1*100) %/% 1
[1] 409

?"%/%"

Bert Gunter
Genentech Nonclinical Statistics

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On
Behalf Of JustinNabble
Sent: Tuesday, April 06, 2010 2:27 PM
To: [hidden email]
Subject: Re: [R] Odd results with %% and conserving memory


Thanks for the response. I read through the Inferno and there's a lot of
useful stuff in there. However I think you may have missed something.

Having some experience with other languages, I know that floating point
numbers aren't represented exactly, the issue is that R is off by much more
than would be expected. On my computer, cell 9,2 of (mat * 100) %% 1 is
larger than .99 and the correct result is 0. That difference seems too large
to be a problem with representing floating point numbers.

Likewise, identical( (4.1 * 100 ) %/% 1 , 409 ) returns TRUE on my computer,
so the result is off by exactly 1, too larger to be explained by floating
point error.

I think there's a problem with how the %% and %/% operators are getting
their results.
--
View this message in context:
http://n4.nabble.com/Odd-results-with-and-conserving-memory-tp1692845p175354
0.html
Sent from the R help mailing list archive at Nabble.com.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Odd results with %% and conserving memory

Thomas Lumley
In reply to this post by JustinNabble
On Tue, 6 Apr 2010, JustinNabble wrote:

> Likewise, identical( (4.1 * 100 ) %/% 1 , 409 ) returns TRUE on my computer,
> so the result is off by exactly 1, too larger to be explained by floating
> point error.

The result of (4.1 * 100 ) %/% 1 is an integer, so if it doesn't agree exactly it has to be off by at least 1, no matter how high the numerical precision.

    -thomas


Thomas Lumley Assoc. Professor, Biostatistics
[hidden email] University of Washington, Seattle

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Odd results with %% and conserving memory

Steve Lianoglou-6
In reply to this post by JustinNabble
Hi,

On Tue, Apr 6, 2010 at 5:26 PM, JustinNabble <[hidden email]> wrote:
>
> Thanks for the response. I read through the Inferno and there's a lot of
> useful stuff in there. However I think you may have missed something.
>
> Having some experience with other languages, I know that floating point
> numbers aren't represented exactly, the issue is that R is off by much more
> than would be expected. On my computer, cell 9,2 of (mat * 100) %% 1 is
> larger than .99 and the correct result is 0. That difference seems too large
> to be a problem with representing floating point numbers.

Like which languages?

python> 4.1 * 100
409.99999999999994

scala> 4.1 * 100
res0: Double = 409.99999999999994

ruby> 4.1 * 100 < 410
=> true

matlab> 4.1 * 100 < 410

ans =

     1

-steve

--
Steve Lianoglou
Graduate Student: Computational Systems Biology
 | Memorial Sloan-Kettering Cancer Center
 | Weill Medical College of Cornell University
Contact Info: http://cbio.mskcc.org/~lianos/contact

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.