# What is an alternative to expand.grid if create a long vector?

7 messages
Open this post in threaded view
|

## What is an alternative to expand.grid if create a long vector?

 Dear All, I would like to know that is there any problem in *expand.grid* function or it is a limitation of this function. I am trying to create a combination of elements using expand.grid function. A <- expand.grid( c(seq(0.001, 0.1, length.out = 100)), c(seq(0.0001, 0.001, length.out = 100)), c(seq(0.38, 0.42, length.out = 100)), c(seq(0.12, 0.18, length.out = 100))) Four combinations work fine. However, If I increase the combinations up to ten. The following error appears.  A <- expand.grid( c(seq(0.001, 1, length.out = 100)), c(seq(0.0001, 0.001, length.out = 100)), c(seq(0.38, 0.42, length.out = 100)), c(seq(0.12, 0.18, length.out = 100)), c(seq(0.01, 0.04, length.out = 100)), c(seq(0.0001, 0.001, length.out = 100)), c(seq(0.0001, 0.001, length.out = 100)), c(seq(0.001, 0.01, length.out = 100)), c(seq(0.01, 0.3, length.out = 100)) ) *Error in rep.int (rep.int (seq_len(nx), rep.int (rep.fac, nx)), orep) :   invalid 'times' value* After reducing the length to 10. It produced a different type of error A <- expand.grid( c(seq(0.001, 0.005, length.out = 10)), c(seq(0.0001, 0.0005, length.out = 10)), c(seq(0.38, 0.42, length.out = 5)), c(seq(0.12, 0.18, length.out = 7)), c(seq(0.01, 0.04, length.out = 5)), c(seq(0.0001, 0.001, length.out = 10)), c(seq(0.0001, 0.001, length.out = 10)), c(seq(0.001, 0.01, length.out = 10)), c(seq(0.1, 0.8, length.out = 8)) ) *Error: cannot allocate vector of size 1.0 Gb* What is an alternative to expand.grid if create a long vector based on 10 elements? With kind regards, Shah Alam         [[alternative HTML version deleted]] ______________________________________________ [hidden email] mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-helpPLEASE do read the posting guide http://www.R-project.org/posting-guide.htmland provide commented, minimal, self-contained, reproducible code.
Open this post in threaded view
|

## Re: What is an alternative to expand.grid if create a long vector?

 Hi Actually expand.grid produces data frame and not vector. And dimension of the data frame is "big" > dim(A) [1] 100000000         4 > str(A) 'data.frame':   100000000 obs. of  4 variables:  \$ Var1: num  0.001 0.002 0.003 0.004 0.005 0.006 0.007 0.008 0.009 0.01 ...  \$ Var2: num  1e-04 1e-04 1e-04 1e-04 1e-04 1e-04 1e-04 1e-04 1e-04 1e-04 ...  \$ Var3: num  0.38 0.38 0.38 0.38 0.38 0.38 0.38 0.38 0.38 0.38 ...  \$ Var4: num  0.12 0.12 0.12 0.12 0.12 0.12 0.12 0.12 0.12 0.12 ...  - attr(*, "out.attrs")=List of 2   ..\$ dim     : int [1:4] 100 100 100 100   ..\$ dimnames:List of 4   .. ..\$ Var1: chr [1:100] "Var1=0.001" "Var1=0.002" "Var1=0.003" "Var1=0.004" ...   .. ..\$ Var2: chr [1:100] "Var2=0.0001000000" "Var2=0.0001090909" "Var2=0.0001181818" "Var2=0.0001272727" ...   .. ..\$ Var3: chr [1:100] "Var3=0.3800000" "Var3=0.3804040" "Var3=0.3808081" "Var3=0.3812121" ...   .. ..\$ Var4: chr [1:100] "Var4=0.1200000" "Var4=0.1206061" "Var4=0.1212121" "Var4=0.1218182" ... > in case of 4 sequences 1e8 rows, 4 columns in case of 10 sequences 1e20 rows and 10 columns in your last example 1.4e8 rows and 10 columns which probably cross the memory capacity of your PC. Maybe you could increase memory of you PC. If I am correct to store the first you need about 3.2GB, to strore the last 11.2 GB. May I ask what you want to do with such a big object? Cheers Petr > -----Original Message----- > From: R-help <[hidden email]> On Behalf Of Shah Alam > Sent: Monday, April 19, 2021 2:36 PM > To: r-help mailing list <[hidden email]> > Subject: [R] What is an alternative to expand.grid if create a long vector? > > Dear All, > > I would like to know that is there any problem in *expand.grid* function or it > is a limitation of this function. > > I am trying to create a combination of elements using expand.grid function. > > A <- expand.grid( > c(seq(0.001, 0.1, length.out = 100)), > c(seq(0.0001, 0.001, length.out = 100)), c(seq(0.38, 0.42, length.out = 100)), > c(seq(0.12, 0.18, length.out = 100))) > > Four combinations work fine. However, If I increase the combinations up to > ten. The following error appears. > >  A <- expand.grid( > c(seq(0.001, 1, length.out = 100)), > c(seq(0.0001, 0.001, length.out = 100)), c(seq(0.38, 0.42, length.out = 100)), > c(seq(0.12, 0.18, length.out = 100)), c(seq(0.01, 0.04, length.out = 100)), > c(seq(0.0001, 0.001, length.out = 100)), c(seq(0.0001, 0.001, length.out = > 100)), c(seq(0.001, 0.01, length.out = 100)), c(seq(0.01, 0.3, length.out = 100)) > ) > > *Error in rep.int (rep.int (seq_len(nx), > rep.int (rep.fac, nx)), orep) :   invalid 'times' value* > > After reducing the length to 10. It produced a different type of error > > A <- expand.grid( > c(seq(0.001, 0.005, length.out = 10)), > c(seq(0.0001, 0.0005, length.out = 10)), c(seq(0.38, 0.42, length.out = 5)), > c(seq(0.12, 0.18, length.out = 7)), c(seq(0.01, 0.04, length.out = 5)), > c(seq(0.0001, 0.001, length.out = 10)), c(seq(0.0001, 0.001, length.out = 10)), > c(seq(0.001, 0.01, length.out = 10)), c(seq(0.1, 0.8, length.out = 8)) > ) > > *Error: cannot allocate vector of size 1.0 Gb* > > What is an alternative to expand.grid if create a long vector based on 10 > elements? > > With kind regards, > Shah Alam > > [[alternative HTML version deleted]] > > ______________________________________________ > [hidden email] mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help> PLEASE do read the posting guide http://www.R-project.org/posting-> guide.html > and provide commented, minimal, self-contained, reproducible code. ______________________________________________ [hidden email] mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-helpPLEASE do read the posting guide http://www.R-project.org/posting-guide.htmland provide commented, minimal, self-contained, reproducible code.
Open this post in threaded view
|

## Re: What is an alternative to expand.grid if create a long vector?

 In reply to this post by Shah Alam Hello, If you want to process the data by rows, then maybe you should consider a custom function that divides the problem in small chunks and process one chunk at a time. But even so, at 8 bytes per double, 100^10 rows is (100^10*8)/(1024^4)  # Tera bytes #[1] 727595761 It will take you a very, very long time to process. Revise the problem? Hope this helps, Rui Barradas Às 13:35 de 19/04/21, Shah Alam escreveu: > Dear All, > > I would like to know that is there any problem in *expand.grid* function or > it is a limitation of this function. > > I am trying to create a combination of elements using expand.grid function. > > A <- expand.grid( > c(seq(0.001, 0.1, length.out = 100)), > c(seq(0.0001, 0.001, length.out = 100)), > c(seq(0.38, 0.42, length.out = 100)), > c(seq(0.12, 0.18, length.out = 100))) > > Four combinations work fine. However, If I increase the combinations up to > ten. The following error appears. > >   A <- expand.grid( > c(seq(0.001, 1, length.out = 100)), > c(seq(0.0001, 0.001, length.out = 100)), > c(seq(0.38, 0.42, length.out = 100)), > c(seq(0.12, 0.18, length.out = 100)), > c(seq(0.01, 0.04, length.out = 100)), > c(seq(0.0001, 0.001, length.out = 100)), > c(seq(0.0001, 0.001, length.out = 100)), > c(seq(0.001, 0.01, length.out = 100)), > c(seq(0.01, 0.3, length.out = 100)) > ) > > *Error in rep.int (rep.int (seq_len(nx), > rep.int (rep.fac, nx)), orep) :   invalid 'times' value* > > After reducing the length to 10. It produced a different type of error > > A <- expand.grid( > c(seq(0.001, 0.005, length.out = 10)), > c(seq(0.0001, 0.0005, length.out = 10)), > c(seq(0.38, 0.42, length.out = 5)), > c(seq(0.12, 0.18, length.out = 7)), > c(seq(0.01, 0.04, length.out = 5)), > c(seq(0.0001, 0.001, length.out = 10)), > c(seq(0.0001, 0.001, length.out = 10)), > c(seq(0.001, 0.01, length.out = 10)), > c(seq(0.1, 0.8, length.out = 8)) > ) > > *Error: cannot allocate vector of size 1.0 Gb* > > What is an alternative to expand.grid if create a long vector based on 10 > elements? > > With kind regards, > Shah Alam > > [[alternative HTML version deleted]] > > ______________________________________________ > [hidden email] mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html> and provide commented, minimal, self-contained, reproducible code. > ______________________________________________ [hidden email] mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-helpPLEASE do read the posting guide http://www.R-project.org/posting-guide.htmland provide commented, minimal, self-contained, reproducible code.
Open this post in threaded view
|

## Re: What is an alternative to expand.grid if create a long vector?

Open this post in threaded view
|