nested for loop with data table

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

nested for loop with data table

Ek Esawi
I have a huge data file; a sample is listed below. I am using the package
data table to process the file and I am stuck on one issue and need some
feedback. I used fread to create a data table. Then I divided the data
table (named File1) into 10 general subsets using common table commands
such as:



AAA <- File1[Num<5&day>15]

BBB <- File1[Num>15&day<10]

…..

…..

…..

…..

…..

…..



I wanted to divide and count each of the above subsets based on a set of
parameters common to all subsets. I did the following to go through each
subset and it works:

For (I in 1: length (AAA)) {

              aa <- c(AAA[color==”green”&grade==”a”,month==”Januray” .N],[
AAA[color==”green”&grade==”b”& month==”June”’ .N])

}



The question: I don’t want to have a separate loop for each subset (10
loops). Instead, I was hoping to have 2 nested loops in the form below:



For (I in 1:N)){

              For (j in 1:M){



}

}



 Sample


Num

Color

Grade

Value

Month

Day

1

yellow

A

20

May

1

2

green

B

25

June

2

3

green

A

10

April

3

4

black

A

17

August

3

5

red

C

5

December

5

6

orange

D

0

January

13

7

orange

E

12

January

5

8

orange

F

11

February

8

9

orange

F

99

July

23

10

orange

F

70

May

7

11

black

A

77

June

11

12

green

B

87

April

33

13

black

A

79

August

9

14

green

A

68

December

14

15

black

C

90

January

31

16

green

D

79

January

11

17

black

E

101

February

17

18

red

F

90

July

21

19

red

F

112

February

13

20

red

F

101

July

20

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: nested for loop with data table

Boris Steipe
There's a lot that doesn't make sense here. I think what you need to do is produce a small, reproducible example, post that with dput() and state your question more clearly - including what you have tried and what didn't work. You'll probably be amazed how quickly you will get good advice if _you_only_follow_the_posting_guide_.

B.




> On May 2, 2017, at 12:35 PM, Ek Esawi <[hidden email]> wrote:
>
> I have a huge data file; a sample is listed below. I am using the package
> data table to process the file and I am stuck on one issue and need some
> feedback. I used fread to create a data table. Then I divided the data
> table (named File1) into 10 general subsets using common table commands
> such as:
>
>
>
> AAA <- File1[Num<5&day>15]
>
> BBB <- File1[Num>15&day<10]
>
> …..
>
> …..
>
> …..
>
> …..
>
> …..
>
> …..
>
>
>
> I wanted to divide and count each of the above subsets based on a set of
> parameters common to all subsets. I did the following to go through each
> subset and it works:
>
> For (I in 1: length (AAA)) {
>
>              aa <- c(AAA[color==”green”&grade==”a”,month==”Januray” .N],[
> AAA[color==”green”&grade==”b”& month==”June”’ .N])
>
> }
>
>
>
> The question: I don’t want to have a separate loop for each subset (10
> loops). Instead, I was hoping to have 2 nested loops in the form below:
>
>
>
> For (I in 1:N)){
>
>              For (j in 1:M){
>
>
>
> }
>
> }
>
>
>
> Sample
>
>
> Num
>
> Color
>
> Grade
>
> Value
>
> Month
>
> Day
>
> 1
>
> yellow
>
> A
>
> 20
>
> May
>
> 1
>
> 2
>
> green
>
> B
>
> 25
>
> June
>
> 2
>
> 3
>
> green
>
> A
>
> 10
>
> April
>
> 3
>
> 4
>
> black
>
> A
>
> 17
>
> August
>
> 3
>
> 5
>
> red
>
> C
>
> 5
>
> December
>
> 5
>
> 6
>
> orange
>
> D
>
> 0
>
> January
>
> 13
>
> 7
>
> orange
>
> E
>
> 12
>
> January
>
> 5
>
> 8
>
> orange
>
> F
>
> 11
>
> February
>
> 8
>
> 9
>
> orange
>
> F
>
> 99
>
> July
>
> 23
>
> 10
>
> orange
>
> F
>
> 70
>
> May
>
> 7
>
> 11
>
> black
>
> A
>
> 77
>
> June
>
> 11
>
> 12
>
> green
>
> B
>
> 87
>
> April
>
> 33
>
> 13
>
> black
>
> A
>
> 79
>
> August
>
> 9
>
> 14
>
> green
>
> A
>
> 68
>
> December
>
> 14
>
> 15
>
> black
>
> C
>
> 90
>
> January
>
> 31
>
> 16
>
> green
>
> D
>
> 79
>
> January
>
> 11
>
> 17
>
> black
>
> E
>
> 101
>
> February
>
> 17
>
> 18
>
> red
>
> F
>
> 90
>
> July
>
> 21
>
> 19
>
> red
>
> F
>
> 112
>
> February
>
> 13
>
> 20
>
> red
>
> F
>
> 101
>
> July
>
> 20
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: nested for loop with data table

Jim Lemon-4
In reply to this post by Ek Esawi
Hi Ek,
I think you want your example to look like this:

Sample<-read.table(text=
"Num Color Grade Value Month Day
1 yellow A 20 May 1
2 green B 25 June 2
3 green A 10 April 3
4 black A 17 August 3
5 red C 5 December 5
6 orange D 0 January 13
7 orange E 12 January 5
8 orange F 11 February 8
9 orange F 99 July 23
10 orange F 70 May 7
11 black A 77 June 11
12 green B 87 April 33
13 black A 79 August 9
14 green A 68 December 14
15 black C 90 January 31
16 green D 79 January 11
17 black E 101 February 17
18 red F 90 July 21
19 red F 112 February 13
20 red F 101 July 20",
header=TRUE)
AAA<-Sample[Sample$Num < 5 & Sample$Day < 3,]
BBB<-Sample[Sample$Num > 15 & Sample$Day > 13,]
for(i in 1:length(AAA)) {
 for(j in 1:length(BBB)) {
  ...
 }
}

except in data.table notation. However, I can't work out what you want
to do in the loop.

Jim


On Wed, May 3, 2017 at 2:35 AM, Ek Esawi <[hidden email]> wrote:

> I have a huge data file; a sample is listed below. I am using the package
> data table to process the file and I am stuck on one issue and need some
> feedback. I used fread to create a data table. Then I divided the data
> table (named File1) into 10 general subsets using common table commands
> such as:
>
>
>
> AAA <- File1[Num<5&day>15]
>
> BBB <- File1[Num>15&day<10]
>
> …..
>
> …..
>
> …..
>
> …..
>
> …..
>
> …..
>
>
>
> I wanted to divide and count each of the above subsets based on a set of
> parameters common to all subsets. I did the following to go through each
> subset and it works:
>
> For (I in 1: length (AAA)) {
>
>               aa <- c(AAA[color==”green”&grade==”a”,month==”Januray” .N],[
> AAA[color==”green”&grade==”b”& month==”June”’ .N])
>
> }
>
>
>
> The question: I don’t want to have a separate loop for each subset (10
> loops). Instead, I was hoping to have 2 nested loops in the form below:
>
>
>
> For (I in 1:N)){
>
>               For (j in 1:M){
>
>
>
> }
>
> }
>
>
>
>  Sample
>
>
> Num
>
> Color
>
> Grade
>
> Value
>
> Month
>
> Day
>
> 1
>
> yellow
>
> A
>
> 20
>
> May
>
> 1
>
> 2
>
> green
>
> B
>
> 25
>
> June
>
> 2
>
> 3
>
> green
>
> A
>
> 10
>
> April
>
> 3
>
> 4
>
> black
>
> A
>
> 17
>
> August
>
> 3
>
> 5
>
> red
>
> C
>
> 5
>
> December
>
> 5
>
> 6
>
> orange
>
> D
>
> 0
>
> January
>
> 13
>
> 7
>
> orange
>
> E
>
> 12
>
> January
>
> 5
>
> 8
>
> orange
>
> F
>
> 11
>
> February
>
> 8
>
> 9
>
> orange
>
> F
>
> 99
>
> July
>
> 23
>
> 10
>
> orange
>
> F
>
> 70
>
> May
>
> 7
>
> 11
>
> black
>
> A
>
> 77
>
> June
>
> 11
>
> 12
>
> green
>
> B
>
> 87
>
> April
>
> 33
>
> 13
>
> black
>
> A
>
> 79
>
> August
>
> 9
>
> 14
>
> green
>
> A
>
> 68
>
> December
>
> 14
>
> 15
>
> black
>
> C
>
> 90
>
> January
>
> 31
>
> 16
>
> green
>
> D
>
> 79
>
> January
>
> 11
>
> 17
>
> black
>
> E
>
> 101
>
> February
>
> 17
>
> 18
>
> red
>
> F
>
> 90
>
> July
>
> 21
>
> 19
>
> red
>
> F
>
> 112
>
> February
>
> 13
>
> 20
>
> red
>
> F
>
> 101
>
> July
>
> 20
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: nested for loop with data table

Ek Esawi
In reply to this post by Ek Esawi
Thank you both Boris and Jim. Thank you, Boris, for advising to read the
posting guide; I had and I just did.

Jim’s idea is exactly what I want; however, I could not pass sset1, sset2,
etc. to the j nested loop and collect the results in an vector.

Here attached my code, file, and my question which should be clear now. The
question again is instead of using separate loops for each sset1 and sset2,
I want one nested loop? Because I have at least 10 subsets
(sset1,sset2,sset3…..sset10).

Thanks again, EK


-------The code------

install.packages("data.table")
library(data.table)
File1 <-  "C:/Users/SampleData.csv"
DT <- fread(File1)
sset1 <- DT[Num<10&Day<10]
sset2 <- DT[Num>10&Day<15]

# Count how many combinations of A,B,C,D,E,F in each subset
for ( i in 1:length(sset1)){
  aa <- c(sset1[Grade=="A",.N],sset1[Grade=="D",.N])
  bb <- c(sset1[Grade=="B",.N],sset1[Grade=="F",.N])
  cc <- c(sset1[Grade=="C",.N],sset1[Grade=="A",.N])
  counts <- c(aa, bb,cc)
}

for ( i in 1:length(sset2)){
  aa1 <- c(sset2[Grade=="A",.N],sset2[Grade=="D",.N])
  bb1 <- c(sset2[Grade=="B",.N],sset2[Grade=="F",.N])
  cc1 <- c(sset2[Grade=="C",.N],sset2[Grade=="A",.N])
  counts <-  c(aa1,bb1,cc1)
}

-----------The File------------

   Num  Color Grade Value    Month Day
 1:   1 yellow     A    20      May   1
 2:   2  green     B    25     June   2
 3:   3  green     A    10    April   3
 4:   4  black     A    17   August   3
 5:   5    red     C     5 December   5
 6:   6 orange     D     0  January  13
 7:   7 orange     E    12  January   5
 8:   8 orange     F    11 February   8
 9:   9 orange     F    99     July  23
10:  10 orange     F    70      May   7
11:  11  black     A    77     June  11
12:  12  green     B    87    April  33
13:  13  black     A    79   August   9
14:  14  green     A    68 December  14
15:  15  black     C    90  January  31
16:  16  green     D    79  January  11
17:  17  black     E   101 February  17
18:  18    red     F    90     July  21
19:  19    red     F   112 February  13
20:  20    red     F   101     July  20

On Tue, May 2, 2017 at 12:35 PM, Ek Esawi <[hidden email]> wrote:

> I have a huge data file; a sample is listed below. I am using the package
> data table to process the file and I am stuck on one issue and need some
> feedback. I used fread to create a data table. Then I divided the data
> table (named File1) into 10 general subsets using common table commands
> such as:
>
>
>
> AAA <- File1[Num<5&day>15]
>
> BBB <- File1[Num>15&day<10]
>
> …..
>
> …..
>
> …..
>
> …..
>
> …..
>
> …..
>
>
>
> I wanted to divide and count each of the above subsets based on a set of
> parameters common to all subsets. I did the following to go through each
> subset and it works:
>
> For (I in 1: length (AAA)) {
>
>               aa <- c(AAA[color==”green”&grade==”a”,month==”Januray”
> .N],[ AAA[color==”green”&grade==”b”& month==”June”’ .N])
>
> }
>
>
>
> The question: I don’t want to have a separate loop for each subset (10
> loops). Instead, I was hoping to have 2 nested loops in the form below:
>
>
>
> For (I in 1:N)){
>
>               For (j in 1:M){
>
>
>
> }
>
> }
>
>
>
>  Sample
>
>
> Num
>
> Color
>
> Grade
>
> Value
>
> Month
>
> Day
>
> 1
>
> yellow
>
> A
>
> 20
>
> May
>
> 1
>
> 2
>
> green
>
> B
>
> 25
>
> June
>
> 2
>
> 3
>
> green
>
> A
>
> 10
>
> April
>
> 3
>
> 4
>
> black
>
> A
>
> 17
>
> August
>
> 3
>
> 5
>
> red
>
> C
>
> 5
>
> December
>
> 5
>
> 6
>
> orange
>
> D
>
> 0
>
> January
>
> 13
>
> 7
>
> orange
>
> E
>
> 12
>
> January
>
> 5
>
> 8
>
> orange
>
> F
>
> 11
>
> February
>
> 8
>
> 9
>
> orange
>
> F
>
> 99
>
> July
>
> 23
>
> 10
>
> orange
>
> F
>
> 70
>
> May
>
> 7
>
> 11
>
> black
>
> A
>
> 77
>
> June
>
> 11
>
> 12
>
> green
>
> B
>
> 87
>
> April
>
> 33
>
> 13
>
> black
>
> A
>
> 79
>
> August
>
> 9
>
> 14
>
> green
>
> A
>
> 68
>
> December
>
> 14
>
> 15
>
> black
>
> C
>
> 90
>
> January
>
> 31
>
> 16
>
> green
>
> D
>
> 79
>
> January
>
> 11
>
> 17
>
> black
>
> E
>
> 101
>
> February
>
> 17
>
> 18
>
> red
>
> F
>
> 90
>
> July
>
> 21
>
> 19
>
> red
>
> F
>
> 112
>
> February
>
> 13
>
> 20
>
> red
>
> F
>
> 101
>
> July
>
> 20
>
>
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: nested for loop with data table

Jeff Newmiller
You seem to be unaware of the "aggregate" data processing concept. There are many ways to accomplish aggregation, but I am not fluent in data.table methods but knowing the concept is the first step.

Perhaps look closely at [1], or Google for data table aggregation yourself?

[1] https://www.r-bloggers.com/efficient-aggregation-and-more-using-data-table/amp/
--
Sent from my phone. Please excuse my brevity.

On May 3, 2017 8:17:21 AM PDT, Ek Esawi <[hidden email]> wrote:

>Thank you both Boris and Jim. Thank you, Boris, for advising to read
>the
>posting guide; I had and I just did.
>
>Jim’s idea is exactly what I want; however, I could not pass sset1,
>sset2,
>etc. to the j nested loop and collect the results in an vector.
>
>Here attached my code, file, and my question which should be clear now.
>The
>question again is instead of using separate loops for each sset1 and
>sset2,
>I want one nested loop? Because I have at least 10 subsets
>(sset1,sset2,sset3…..sset10).
>
>Thanks again, EK
>
>
>-------The code------
>
>install.packages("data.table")
>library(data.table)
>File1 <-  "C:/Users/SampleData.csv"
>DT <- fread(File1)
>sset1 <- DT[Num<10&Day<10]
>sset2 <- DT[Num>10&Day<15]
>
># Count how many combinations of A,B,C,D,E,F in each subset
>for ( i in 1:length(sset1)){
>  aa <- c(sset1[Grade=="A",.N],sset1[Grade=="D",.N])
>  bb <- c(sset1[Grade=="B",.N],sset1[Grade=="F",.N])
>  cc <- c(sset1[Grade=="C",.N],sset1[Grade=="A",.N])
>  counts <- c(aa, bb,cc)
>}
>
>for ( i in 1:length(sset2)){
>  aa1 <- c(sset2[Grade=="A",.N],sset2[Grade=="D",.N])
>  bb1 <- c(sset2[Grade=="B",.N],sset2[Grade=="F",.N])
>  cc1 <- c(sset2[Grade=="C",.N],sset2[Grade=="A",.N])
>  counts <-  c(aa1,bb1,cc1)
>}
>
>-----------The File------------
>
>   Num  Color Grade Value    Month Day
> 1:   1 yellow     A    20      May   1
> 2:   2  green     B    25     June   2
> 3:   3  green     A    10    April   3
> 4:   4  black     A    17   August   3
> 5:   5    red     C     5 December   5
> 6:   6 orange     D     0  January  13
> 7:   7 orange     E    12  January   5
> 8:   8 orange     F    11 February   8
> 9:   9 orange     F    99     July  23
>10:  10 orange     F    70      May   7
>11:  11  black     A    77     June  11
>12:  12  green     B    87    April  33
>13:  13  black     A    79   August   9
>14:  14  green     A    68 December  14
>15:  15  black     C    90  January  31
>16:  16  green     D    79  January  11
>17:  17  black     E   101 February  17
>18:  18    red     F    90     July  21
>19:  19    red     F   112 February  13
>20:  20    red     F   101     July  20
>
>On Tue, May 2, 2017 at 12:35 PM, Ek Esawi <[hidden email]> wrote:
>
>> I have a huge data file; a sample is listed below. I am using the
>package
>> data table to process the file and I am stuck on one issue and need
>some
>> feedback. I used fread to create a data table. Then I divided the
>data
>> table (named File1) into 10 general subsets using common table
>commands
>> such as:
>>
>>
>>
>> AAA <- File1[Num<5&day>15]
>>
>> BBB <- File1[Num>15&day<10]
>>
>> …..
>>
>> …..
>>
>> …..
>>
>> …..
>>
>> …..
>>
>> …..
>>
>>
>>
>> I wanted to divide and count each of the above subsets based on a set
>of
>> parameters common to all subsets. I did the following to go through
>each
>> subset and it works:
>>
>> For (I in 1: length (AAA)) {
>>
>>               aa <- c(AAA[color==”green”&grade==”a”,month==”Januray”
>> .N],[ AAA[color==”green”&grade==”b”& month==”June”’ .N])
>>
>> }
>>
>>
>>
>> The question: I don’t want to have a separate loop for each subset
>(10
>> loops). Instead, I was hoping to have 2 nested loops in the form
>below:
>>
>>
>>
>> For (I in 1:N)){
>>
>>               For (j in 1:M){
>>
>>
>>
>> }
>>
>> }
>>
>>
>>
>>  Sample
>>
>>
>> Num
>>
>> Color
>>
>> Grade
>>
>> Value
>>
>> Month
>>
>> Day
>>
>> 1
>>
>> yellow
>>
>> A
>>
>> 20
>>
>> May
>>
>> 1
>>
>> 2
>>
>> green
>>
>> B
>>
>> 25
>>
>> June
>>
>> 2
>>
>> 3
>>
>> green
>>
>> A
>>
>> 10
>>
>> April
>>
>> 3
>>
>> 4
>>
>> black
>>
>> A
>>
>> 17
>>
>> August
>>
>> 3
>>
>> 5
>>
>> red
>>
>> C
>>
>> 5
>>
>> December
>>
>> 5
>>
>> 6
>>
>> orange
>>
>> D
>>
>> 0
>>
>> January
>>
>> 13
>>
>> 7
>>
>> orange
>>
>> E
>>
>> 12
>>
>> January
>>
>> 5
>>
>> 8
>>
>> orange
>>
>> F
>>
>> 11
>>
>> February
>>
>> 8
>>
>> 9
>>
>> orange
>>
>> F
>>
>> 99
>>
>> July
>>
>> 23
>>
>> 10
>>
>> orange
>>
>> F
>>
>> 70
>>
>> May
>>
>> 7
>>
>> 11
>>
>> black
>>
>> A
>>
>> 77
>>
>> June
>>
>> 11
>>
>> 12
>>
>> green
>>
>> B
>>
>> 87
>>
>> April
>>
>> 33
>>
>> 13
>>
>> black
>>
>> A
>>
>> 79
>>
>> August
>>
>> 9
>>
>> 14
>>
>> green
>>
>> A
>>
>> 68
>>
>> December
>>
>> 14
>>
>> 15
>>
>> black
>>
>> C
>>
>> 90
>>
>> January
>>
>> 31
>>
>> 16
>>
>> green
>>
>> D
>>
>> 79
>>
>> January
>>
>> 11
>>
>> 17
>>
>> black
>>
>> E
>>
>> 101
>>
>> February
>>
>> 17
>>
>> 18
>>
>> red
>>
>> F
>>
>> 90
>>
>> July
>>
>> 21
>>
>> 19
>>
>> red
>>
>> F
>>
>> 112
>>
>> February
>>
>> 13
>>
>> 20
>>
>> red
>>
>> F
>>
>> 101
>>
>> July
>>
>> 20
>>
>>
>>
>
> [[alternative HTML version deleted]]
>
>______________________________________________
>[hidden email] mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: nested for loop with data table

PIKAL Petr
In reply to this post by Ek Esawi
Hi

better to present us your data by dput, so they can be directly used.

> dput(dat)
dat <- structure(list(Num = 1:20, Color = structure(c(5L, 2L, 2L, 1L,
4L, 3L, 3L, 3L, 3L, 3L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 4L, 4L, 4L
), .Label = c("black", "green", "orange", "red", "yellow"), class = "factor"),
    Grade = structure(c(1L, 2L, 1L, 1L, 3L, 4L, 5L, 6L, 6L, 6L,
    1L, 2L, 1L, 1L, 3L, 4L, 5L, 6L, 6L, 6L), .Label = c("A",
    "B", "C", "D", "E", "F"), class = "factor"), value = c(20L,
    25L, 10L, 17L, 5L, 0L, 12L, 11L, 99L, 70L, 77L, 87L, 79L,
    68L, 90L, 79L, 101L, 90L, 112L, 101L), Month = structure(c(8L,
    7L, 1L, 2L, 3L, 5L, 5L, 4L, 6L, 8L, 7L, 1L, 2L, 3L, 5L, 5L,
    4L, 6L, 4L, 6L), .Label = c("April", "August", "December",
    "February", "January", "July", "June", "May"), class = "factor"),
    Day = c(1L, 2L, 3L, 3L, 5L, 13L, 5L, 8L, 23L, 7L, 11L, 33L,
    9L, 14L, 31L, 11L, 17L, 21L, 13L, 20L)), .Names = c("Num",
"Color", "Grade", "value", "Month", "Day"), class = "data.frame", row.names = c(NA,
-20L))
>

I do not know your exact intention and data.table commands. You can get some summary numbers simply by

table(dat$Grade[dat$Num<10 & dat$Day<10])

A B C D E F
3 1 1 0 1 1

It is probably preferable to obtain logical vectors for Num and Day before starting tabulation.

Cheers
Petr


> -----Original Message-----
> From: R-help [mailto:[hidden email]] On Behalf Of Ek Esawi
> Sent: Wednesday, May 3, 2017 5:17 PM
> To: [hidden email]
> Subject: Re: [R] nested for loop with data table
>
> Thank you both Boris and Jim. Thank you, Boris, for advising to read the
> posting guide; I had and I just did.
>
> Jim’s idea is exactly what I want; however, I could not pass sset1, sset2, etc.
> to the j nested loop and collect the results in an vector.
>
> Here attached my code, file, and my question which should be clear now.
> The question again is instead of using separate loops for each sset1 and
> sset2, I want one nested loop? Because I have at least 10 subsets
> (sset1,sset2,sset3…..sset10).
>
> Thanks again, EK
>
>
> -------The code------
>
> install.packages("data.table")
> library(data.table)
> File1 <-  "C:/Users/SampleData.csv"
> DT <- fread(File1)
> sset1 <- DT[Num<10&Day<10]
> sset2 <- DT[Num>10&Day<15]
>
> # Count how many combinations of A,B,C,D,E,F in each subset for ( i in
> 1:length(sset1)){
>   aa <- c(sset1[Grade=="A",.N],sset1[Grade=="D",.N])
>   bb <- c(sset1[Grade=="B",.N],sset1[Grade=="F",.N])
>   cc <- c(sset1[Grade=="C",.N],sset1[Grade=="A",.N])
>   counts <- c(aa, bb,cc)
> }
>
> for ( i in 1:length(sset2)){
>   aa1 <- c(sset2[Grade=="A",.N],sset2[Grade=="D",.N])
>   bb1 <- c(sset2[Grade=="B",.N],sset2[Grade=="F",.N])
>   cc1 <- c(sset2[Grade=="C",.N],sset2[Grade=="A",.N])
>   counts <-  c(aa1,bb1,cc1)
> }
>
> -----------The File------------
>
>    Num  Color Grade Value    Month Day
>  1:   1 yellow     A    20      May   1
>  2:   2  green     B    25     June   2
>  3:   3  green     A    10    April   3
>  4:   4  black     A    17   August   3
>  5:   5    red     C     5 December   5
>  6:   6 orange     D     0  January  13
>  7:   7 orange     E    12  January   5
>  8:   8 orange     F    11 February   8
>  9:   9 orange     F    99     July  23
> 10:  10 orange     F    70      May   7
> 11:  11  black     A    77     June  11
> 12:  12  green     B    87    April  33
> 13:  13  black     A    79   August   9
> 14:  14  green     A    68 December  14
> 15:  15  black     C    90  January  31
> 16:  16  green     D    79  January  11
> 17:  17  black     E   101 February  17
> 18:  18    red     F    90     July  21
> 19:  19    red     F   112 February  13
> 20:  20    red     F   101     July  20
>
> On Tue, May 2, 2017 at 12:35 PM, Ek Esawi <[hidden email]> wrote:
>
> > I have a huge data file; a sample is listed below. I am using the
> > package data table to process the file and I am stuck on one issue and
> > need some feedback. I used fread to create a data table. Then I
> > divided the data table (named File1) into 10 general subsets using
> > common table commands such as:
> >
> >
> >
> > AAA <- File1[Num<5&day>15]
> >
> > BBB <- File1[Num>15&day<10]
> >
> > …..
> >
> > …..
> >
> > …..
> >
> > …..
> >
> > …..
> >
> > …..
> >
> >
> >
> > I wanted to divide and count each of the above subsets based on a set
> > of parameters common to all subsets. I did the following to go through
> > each subset and it works:
> >
> > For (I in 1: length (AAA)) {
> >
> >               aa <- c(AAA[color==”green”&grade==”a”,month==”Januray”
> > .N],[ AAA[color==”green”&grade==”b”& month==”June”’ .N])
> >
> > }
> >
> >
> >
> > The question: I don’t want to have a separate loop for each subset (10
> > loops). Instead, I was hoping to have 2 nested loops in the form below:
> >
> >
> >
> > For (I in 1:N)){
> >
> >               For (j in 1:M){
> >
> >
> >
> > }
> >
> > }
> >
> >
> >
> >  Sample
> >
> >
> > Num
> >
> > Color
> >
> > Grade
> >
> > Value
> >
> > Month
> >
> > Day
> >
> > 1
> >
> > yellow
> >
> > A
> >
> > 20
> >
> > May
> >
> > 1
> >
> > 2
> >
> > green
> >
> > B
> >
> > 25
> >
> > June
> >
> > 2
> >
> > 3
> >
> > green
> >
> > A
> >
> > 10
> >
> > April
> >
> > 3
> >
> > 4
> >
> > black
> >
> > A
> >
> > 17
> >
> > August
> >
> > 3
> >
> > 5
> >
> > red
> >
> > C
> >
> > 5
> >
> > December
> >
> > 5
> >
> > 6
> >
> > orange
> >
> > D
> >
> > 0
> >
> > January
> >
> > 13
> >
> > 7
> >
> > orange
> >
> > E
> >
> > 12
> >
> > January
> >
> > 5
> >
> > 8
> >
> > orange
> >
> > F
> >
> > 11
> >
> > February
> >
> > 8
> >
> > 9
> >
> > orange
> >
> > F
> >
> > 99
> >
> > July
> >
> > 23
> >
> > 10
> >
> > orange
> >
> > F
> >
> > 70
> >
> > May
> >
> > 7
> >
> > 11
> >
> > black
> >
> > A
> >
> > 77
> >
> > June
> >
> > 11
> >
> > 12
> >
> > green
> >
> > B
> >
> > 87
> >
> > April
> >
> > 33
> >
> > 13
> >
> > black
> >
> > A
> >
> > 79
> >
> > August
> >
> > 9
> >
> > 14
> >
> > green
> >
> > A
> >
> > 68
> >
> > December
> >
> > 14
> >
> > 15
> >
> > black
> >
> > C
> >
> > 90
> >
> > January
> >
> > 31
> >
> > 16
> >
> > green
> >
> > D
> >
> > 79
> >
> > January
> >
> > 11
> >
> > 17
> >
> > black
> >
> > E
> >
> > 101
> >
> > February
> >
> > 17
> >
> > 18
> >
> > red
> >
> > F
> >
> > 90
> >
> > July
> >
> > 21
> >
> > 19
> >
> > red
> >
> > F
> >
> > 112
> >
> > February
> >
> > 13
> >
> > 20
> >
> > red
> >
> > F
> >
> > 101
> >
> > July
> >
> > 20
> >
> >
> >
>
>       [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-
> guide.html
> and provide commented, minimal, self-contained, reproducible code.

________________________________
Tento e-mail a jakékoliv k němu připojené dokumenty jsou důvěrné a jsou určeny pouze jeho adresátům.
Jestliže jste obdržel(a) tento e-mail omylem, informujte laskavě neprodleně jeho odesílatele. Obsah tohoto emailu i s přílohami a jeho kopie vymažte ze svého systému.
Nejste-li zamýšleným adresátem tohoto emailu, nejste oprávněni tento email jakkoliv užívat, rozšiřovat, kopírovat či zveřejňovat.
Odesílatel e-mailu neodpovídá za eventuální škodu způsobenou modifikacemi či zpožděním přenosu e-mailu.

V případě, že je tento e-mail součástí obchodního jednání:
- vyhrazuje si odesílatel právo ukončit kdykoliv jednání o uzavření smlouvy, a to z jakéhokoliv důvodu i bez uvedení důvodu.
- a obsahuje-li nabídku, je adresát oprávněn nabídku bezodkladně přijmout; Odesílatel tohoto e-mailu (nabídky) vylučuje přijetí nabídky ze strany příjemce s dodatkem či odchylkou.
- trvá odesílatel na tom, že příslušná smlouva je uzavřena teprve výslovným dosažením shody na všech jejích náležitostech.
- odesílatel tohoto emailu informuje, že není oprávněn uzavírat za společnost žádné smlouvy s výjimkou případů, kdy k tomu byl písemně zmocněn nebo písemně pověřen a takové pověření nebo plná moc byly adresátovi tohoto emailu případně osobě, kterou adresát zastupuje, předloženy nebo jejich existence je adresátovi či osobě jím zastoupené známá.

This e-mail and any documents attached to it may be confidential and are intended only for its intended recipients.
If you received this e-mail by mistake, please immediately inform its sender. Delete the contents of this e-mail with all attachments and its copies from your system.
If you are not the intended recipient of this e-mail, you are not authorized to use, disseminate, copy or disclose this e-mail in any manner.
The sender of this e-mail shall not be liable for any possible damage caused by modifications of the e-mail or by delay with transfer of the email.

In case that this e-mail forms part of business dealings:
- the sender reserves the right to end negotiations about entering into a contract in any time, for any reason, and without stating any reasoning.
- if the e-mail contains an offer, the recipient is entitled to immediately accept such offer; The sender of this e-mail (offer) excludes any acceptance of the offer on the part of the recipient containing any amendment or variation.
- the sender insists on that the respective contract is concluded only upon an express mutual agreement on all its aspects.
- the sender of this e-mail informs that he/she is not authorized to enter into any contracts on behalf of the company except for cases in which he/she is expressly authorized to do so in writing, and such authorization or power of attorney is submitted to the recipient or the person represented by the recipient, or the existence of such authorization is known to the recipient of the person represented by the recipient.
______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: nested for loop with data table

Ek Esawi
In reply to this post by Jeff Newmiller
Thank you Jeff. Your idea, as i mentioned on my previous posting, did
indeed work. I read somewhere that both data table dplyr do great many
things and i plan to learn both as much as i can. Suggestions on this list
either get you the answer you are looking for or give you lead to an answer.

Thanks again

On Thu, May 4, 2017 at 12:04 AM, Jeff Newmiller <[hidden email]>
wrote:

> You seem to be unaware of the "aggregate" data processing concept. There
> are many ways to accomplish aggregation, but I am not fluent in data.table
> methods but knowing the concept is the first step.
>
> Perhaps look closely at [1], or Google for data table aggregation yourself?
>
> [1] https://www.r-bloggers.com/efficient-aggregation-and-
> more-using-data-table/amp/
> --
> Sent from my phone. Please excuse my brevity.
>
> On May 3, 2017 8:17:21 AM PDT, Ek Esawi <[hidden email]> wrote:
> >Thank you both Boris and Jim. Thank you, Boris, for advising to read
> >the
> >posting guide; I had and I just did.
> >
> >Jim’s idea is exactly what I want; however, I could not pass sset1,
> >sset2,
> >etc. to the j nested loop and collect the results in an vector.
> >
> >Here attached my code, file, and my question which should be clear now.
> >The
> >question again is instead of using separate loops for each sset1 and
> >sset2,
> >I want one nested loop? Because I have at least 10 subsets
> >(sset1,sset2,sset3…..sset10).
> >
> >Thanks again, EK
> >
> >
> >-------The code------
> >
> >install.packages("data.table")
> >library(data.table)
> >File1 <-  "C:/Users/SampleData.csv"
> >DT <- fread(File1)
> >sset1 <- DT[Num<10&Day<10]
> >sset2 <- DT[Num>10&Day<15]
> >
> ># Count how many combinations of A,B,C,D,E,F in each subset
> >for ( i in 1:length(sset1)){
> >  aa <- c(sset1[Grade=="A",.N],sset1[Grade=="D",.N])
> >  bb <- c(sset1[Grade=="B",.N],sset1[Grade=="F",.N])
> >  cc <- c(sset1[Grade=="C",.N],sset1[Grade=="A",.N])
> >  counts <- c(aa, bb,cc)
> >}
> >
> >for ( i in 1:length(sset2)){
> >  aa1 <- c(sset2[Grade=="A",.N],sset2[Grade=="D",.N])
> >  bb1 <- c(sset2[Grade=="B",.N],sset2[Grade=="F",.N])
> >  cc1 <- c(sset2[Grade=="C",.N],sset2[Grade=="A",.N])
> >  counts <-  c(aa1,bb1,cc1)
> >}
> >
> >-----------The File------------
> >
> >   Num  Color Grade Value    Month Day
> > 1:   1 yellow     A    20      May   1
> > 2:   2  green     B    25     June   2
> > 3:   3  green     A    10    April   3
> > 4:   4  black     A    17   August   3
> > 5:   5    red     C     5 December   5
> > 6:   6 orange     D     0  January  13
> > 7:   7 orange     E    12  January   5
> > 8:   8 orange     F    11 February   8
> > 9:   9 orange     F    99     July  23
> >10:  10 orange     F    70      May   7
> >11:  11  black     A    77     June  11
> >12:  12  green     B    87    April  33
> >13:  13  black     A    79   August   9
> >14:  14  green     A    68 December  14
> >15:  15  black     C    90  January  31
> >16:  16  green     D    79  January  11
> >17:  17  black     E   101 February  17
> >18:  18    red     F    90     July  21
> >19:  19    red     F   112 February  13
> >20:  20    red     F   101     July  20
> >
> >On Tue, May 2, 2017 at 12:35 PM, Ek Esawi <[hidden email]> wrote:
> >
> >> I have a huge data file; a sample is listed below. I am using the
> >package
> >> data table to process the file and I am stuck on one issue and need
> >some
> >> feedback. I used fread to create a data table. Then I divided the
> >data
> >> table (named File1) into 10 general subsets using common table
> >commands
> >> such as:
> >>
> >>
> >>
> >> AAA <- File1[Num<5&day>15]
> >>
> >> BBB <- File1[Num>15&day<10]
> >>
> >> …..
> >>
> >> …..
> >>
> >> …..
> >>
> >> …..
> >>
> >> …..
> >>
> >> …..
> >>
> >>
> >>
> >> I wanted to divide and count each of the above subsets based on a set
> >of
> >> parameters common to all subsets. I did the following to go through
> >each
> >> subset and it works:
> >>
> >> For (I in 1: length (AAA)) {
> >>
> >>               aa <- c(AAA[color==”green”&grade==”a”,month==”Januray”
> >> .N],[ AAA[color==”green”&grade==”b”& month==”June”’ .N])
> >>
> >> }
> >>
> >>
> >>
> >> The question: I don’t want to have a separate loop for each subset
> >(10
> >> loops). Instead, I was hoping to have 2 nested loops in the form
> >below:
> >>
> >>
> >>
> >> For (I in 1:N)){
> >>
> >>               For (j in 1:M){
> >>
> >>
> >>
> >> }
> >>
> >> }
> >>
> >>
> >>
> >>  Sample
> >>
> >>
> >> Num
> >>
> >> Color
> >>
> >> Grade
> >>
> >> Value
> >>
> >> Month
> >>
> >> Day
> >>
> >> 1
> >>
> >> yellow
> >>
> >> A
> >>
> >> 20
> >>
> >> May
> >>
> >> 1
> >>
> >> 2
> >>
> >> green
> >>
> >> B
> >>
> >> 25
> >>
> >> June
> >>
> >> 2
> >>
> >> 3
> >>
> >> green
> >>
> >> A
> >>
> >> 10
> >>
> >> April
> >>
> >> 3
> >>
> >> 4
> >>
> >> black
> >>
> >> A
> >>
> >> 17
> >>
> >> August
> >>
> >> 3
> >>
> >> 5
> >>
> >> red
> >>
> >> C
> >>
> >> 5
> >>
> >> December
> >>
> >> 5
> >>
> >> 6
> >>
> >> orange
> >>
> >> D
> >>
> >> 0
> >>
> >> January
> >>
> >> 13
> >>
> >> 7
> >>
> >> orange
> >>
> >> E
> >>
> >> 12
> >>
> >> January
> >>
> >> 5
> >>
> >> 8
> >>
> >> orange
> >>
> >> F
> >>
> >> 11
> >>
> >> February
> >>
> >> 8
> >>
> >> 9
> >>
> >> orange
> >>
> >> F
> >>
> >> 99
> >>
> >> July
> >>
> >> 23
> >>
> >> 10
> >>
> >> orange
> >>
> >> F
> >>
> >> 70
> >>
> >> May
> >>
> >> 7
> >>
> >> 11
> >>
> >> black
> >>
> >> A
> >>
> >> 77
> >>
> >> June
> >>
> >> 11
> >>
> >> 12
> >>
> >> green
> >>
> >> B
> >>
> >> 87
> >>
> >> April
> >>
> >> 33
> >>
> >> 13
> >>
> >> black
> >>
> >> A
> >>
> >> 79
> >>
> >> August
> >>
> >> 9
> >>
> >> 14
> >>
> >> green
> >>
> >> A
> >>
> >> 68
> >>
> >> December
> >>
> >> 14
> >>
> >> 15
> >>
> >> black
> >>
> >> C
> >>
> >> 90
> >>
> >> January
> >>
> >> 31
> >>
> >> 16
> >>
> >> green
> >>
> >> D
> >>
> >> 79
> >>
> >> January
> >>
> >> 11
> >>
> >> 17
> >>
> >> black
> >>
> >> E
> >>
> >> 101
> >>
> >> February
> >>
> >> 17
> >>
> >> 18
> >>
> >> red
> >>
> >> F
> >>
> >> 90
> >>
> >> July
> >>
> >> 21
> >>
> >> 19
> >>
> >> red
> >>
> >> F
> >>
> >> 112
> >>
> >> February
> >>
> >> 13
> >>
> >> 20
> >>
> >> red
> >>
> >> F
> >>
> >> 101
> >>
> >> July
> >>
> >> 20
> >>
> >>
> >>
> >
> >       [[alternative HTML version deleted]]
> >
> >______________________________________________
> >[hidden email] mailing list -- To UNSUBSCRIBE and more, see
> >https://stat.ethz.ch/mailman/listinfo/r-help
> >PLEASE do read the posting guide
> >http://www.R-project.org/posting-guide.html
> >and provide commented, minimal, self-contained, reproducible code.
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.