Quantcast

(newbie) Accessing the pieces of a 'by' object

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

(newbie) Accessing the pieces of a 'by' object

Vivek Satsangi
Folks,
I know that I can do the following using a loop. That's been a lot
easier for me to write and understand. But I am trying to force myself
to use more vectorized / matrixed code so that eventually I will
become a better R programmer.

I have a dataframe that has some values by Year, Quarter and Ranking.
The variable of interest is the return (F3MRet), to be weighted
averaged within the year, quarter and ranking. At the end, we want to
end up with a table like this:
year  quarter ranking1 ranking2 ... ranking10
1987 1 1.33 1.45 ... 1.99
1987 2 6.45 3.22 ... 8.33
.
.
2005 4 2.22 3.33 ... 1.22

The dataset is too large to post and I can't come up with a small
working example very easily.

I tried the Reshape() package and also the aggregate and reshape
functions. Those don't work too well becuase of the need to pass
weighted.mean a weights vector. I tried the by() function, but now I
don't know how to coerce the returned object into a matrix so that I
can reshape it.

> fvs_weighted.mean <- function(y) weighted.mean(y$F3MRet, y$IndexWeight, na.rm=T);
> tmp_byRet <- by(dfReturns,
     list(dfReturns$Quarter,dfReturns$Year,dfReturns$Ranking),
     fvs_weighted.mean);

And various other ways to get the tmp_byRet object into a matrix were
tried, eg. unlist(), a loop like this:
dfRet <- data.frame(tmp_byRet);
for(i in 1:dim(dfRet)[2]){
       dfRet[ ,i] <- as.vector(dfRet[ ,i]);
}
In each case, I got some error or the other.

So, please help me get unstuck. How can I get the tmp_byRet() object
into a matrix or a dataframe?

--
-- Vivek Satsangi
Rochester, NY USA
"No amount of sophistication is going to allay the fact that all your
knowledge is about the past and all your decisions are about the
future." -- Ian Wilson

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: (newbie) Accessing the pieces of a 'by' object

Gabor Grothendieck
Try this:

iris.by <- do.call("rbind", by(iris[,-5], iris[,5,drop=FALSE],colSums))
do.call("rbind", iris.by)

On 3/7/06, Vivek Satsangi <[hidden email]> wrote:

> Folks,
> I know that I can do the following using a loop. That's been a lot
> easier for me to write and understand. But I am trying to force myself
> to use more vectorized / matrixed code so that eventually I will
> become a better R programmer.
>
> I have a dataframe that has some values by Year, Quarter and Ranking.
> The variable of interest is the return (F3MRet), to be weighted
> averaged within the year, quarter and ranking. At the end, we want to
> end up with a table like this:
> year  quarter ranking1 ranking2 ... ranking10
> 1987 1 1.33 1.45 ... 1.99
> 1987 2 6.45 3.22 ... 8.33
> .
> .
> 2005 4 2.22 3.33 ... 1.22
>
> The dataset is too large to post and I can't come up with a small
> working example very easily.
>
> I tried the Reshape() package and also the aggregate and reshape
> functions. Those don't work too well becuase of the need to pass
> weighted.mean a weights vector. I tried the by() function, but now I
> don't know how to coerce the returned object into a matrix so that I
> can reshape it.
>
> > fvs_weighted.mean <- function(y) weighted.mean(y$F3MRet, y$IndexWeight, na.rm=T);
> > tmp_byRet <- by(dfReturns,
>     list(dfReturns$Quarter,dfReturns$Year,dfReturns$Ranking),
>     fvs_weighted.mean);
>
> And various other ways to get the tmp_byRet object into a matrix were
> tried, eg. unlist(), a loop like this:
> dfRet <- data.frame(tmp_byRet);
> for(i in 1:dim(dfRet)[2]){
>       dfRet[ ,i] <- as.vector(dfRet[ ,i]);
> }
> In each case, I got some error or the other.
>
> So, please help me get unstuck. How can I get the tmp_byRet() object
> into a matrix or a dataframe?
>
> --
> -- Vivek Satsangi
> Rochester, NY USA
> "No amount of sophistication is going to allay the fact that all your
> knowledge is about the past and all your decisions are about the
> future." -- Ian Wilson
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: (newbie) Accessing the pieces of a 'by' object

Gabor Grothendieck
Sorry, that should be:

iris.by <- by(iris[,-5], iris[,5,drop=FALSE],colSums)
do.call("rbind", iris.by)

On 3/7/06, Gabor Grothendieck <[hidden email]> wrote:

> Try this:
>
> iris.by <- do.call("rbind", by(iris[,-5], iris[,5,drop=FALSE],colSums))
> do.call("rbind", iris.by)
>
> On 3/7/06, Vivek Satsangi <[hidden email]> wrote:
> > Folks,
> > I know that I can do the following using a loop. That's been a lot
> > easier for me to write and understand. But I am trying to force myself
> > to use more vectorized / matrixed code so that eventually I will
> > become a better R programmer.
> >
> > I have a dataframe that has some values by Year, Quarter and Ranking.
> > The variable of interest is the return (F3MRet), to be weighted
> > averaged within the year, quarter and ranking. At the end, we want to
> > end up with a table like this:
> > year  quarter ranking1 ranking2 ... ranking10
> > 1987 1 1.33 1.45 ... 1.99
> > 1987 2 6.45 3.22 ... 8.33
> > .
> > .
> > 2005 4 2.22 3.33 ... 1.22
> >
> > The dataset is too large to post and I can't come up with a small
> > working example very easily.
> >
> > I tried the Reshape() package and also the aggregate and reshape
> > functions. Those don't work too well becuase of the need to pass
> > weighted.mean a weights vector. I tried the by() function, but now I
> > don't know how to coerce the returned object into a matrix so that I
> > can reshape it.
> >
> > > fvs_weighted.mean <- function(y) weighted.mean(y$F3MRet, y$IndexWeight, na.rm=T);
> > > tmp_byRet <- by(dfReturns,
> >     list(dfReturns$Quarter,dfReturns$Year,dfReturns$Ranking),
> >     fvs_weighted.mean);
> >
> > And various other ways to get the tmp_byRet object into a matrix were
> > tried, eg. unlist(), a loop like this:
> > dfRet <- data.frame(tmp_byRet);
> > for(i in 1:dim(dfRet)[2]){
> >       dfRet[ ,i] <- as.vector(dfRet[ ,i]);
> > }
> > In each case, I got some error or the other.
> >
> > So, please help me get unstuck. How can I get the tmp_byRet() object
> > into a matrix or a dataframe?
> >
> > --
> > -- Vivek Satsangi
> > Rochester, NY USA
> > "No amount of sophistication is going to allay the fact that all your
> > knowledge is about the past and all your decisions are about the
> > future." -- Ian Wilson
> >
> > ______________________________________________
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
> >
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: (newbie) Accessing the pieces of a 'by' object

Vivek Satsangi
In reply to this post by Vivek Satsangi
I am writing to document the answer for the next poor sod who comes along.

To get tmp_byRet() into a multi-dimentional matrix, copy the object
using as.vector(), then copy the dim and dimnames from tmp_byRet into
the new object. However, this may not be what you want, since you
probably want the values of the factors within the object (i.e. it
should be a dataframe, not a matrix).

To get tmp_byRet into a dataframe, use unique() to create a dataframe
with just the unique values of your factors. Add a new column to the
dataframe, where you will store the summary stats. Use a loop to
populate this vector. Then use reshape() on the dataframe to get it to
the shape you want it in. It is difficult at best to vectorize this
and avoid the loop  -- and trying to do so will lead to probably less
transparent code.

Vivek


On 3/7/06, Vivek Satsangi <[hidden email]> wrote:

> Folks,
> I know that I can do the following using a loop. That's been a lot
> easier for me to write and understand. But I am trying to force myself
> to use more vectorized / matrixed code so that eventually I will
> become a better R programmer.
>
> I have a dataframe that has some values by Year, Quarter and Ranking.
> The variable of interest is the return (F3MRet), to be weighted
> averaged within the year, quarter and ranking. At the end, we want to
> end up with a table like this:
> year  quarter ranking1 ranking2 ... ranking10
> 1987 1 1.33 1.45 ... 1.99
> 1987 2 6.45 3.22 ... 8.33
> .
> .
> 2005 4 2.22 3.33 ... 1.22
>
> The dataset is too large to post and I can't come up with a small
> working example very easily.
>
> I tried the Reshape() package and also the aggregate and reshape
> functions. Those don't work too well becuase of the need to pass
> weighted.mean a weights vector. I tried the by() function, but now I
> don't know how to coerce the returned object into a matrix so that I
> can reshape it.
>
> > fvs_weighted.mean <- function(y) weighted.mean(y$F3MRet, y$IndexWeight, na.rm=T);
> > tmp_byRet <- by(dfReturns,
>      list(dfReturns$Quarter,dfReturns$Year,dfReturns$Ranking),
>      fvs_weighted.mean);
>
> And various other ways to get the tmp_byRet object into a matrix were
> tried, eg. unlist(), a loop like this:
> dfRet <- data.frame(tmp_byRet);
> for(i in 1:dim(dfRet)[2]){
>        dfRet[ ,i] <- as.vector(dfRet[ ,i]);
> }
> In each case, I got some error or the other.
>
> So, please help me get unstuck. How can I get the tmp_byRet() object
> into a matrix or a dataframe?
>
> --

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: (newbie) Accessing the pieces of a 'by' object

Gabor Grothendieck
Another solution (ex #3) is to return a data frame with at least two columns
and then remove the dummy one using drop = FALSE:

# ex #1. error
iris.by <- by(iris, iris[,5, drop = FALSE], length)
do.call("rbind", iris.by)

# ex #2. ok but no column heading
iris.by <- by(iris, iris[,5, drop=FALSE], function(x)
data.frame(length = length(x)))
do.call("rbind", iris.by)

# ex #3. ok
iris.by <- by(iris, iris[,5, drop=FALSE], function(x) data.frame(1,
length = length(x)))
do.call("rbind", iris.by)[,-1,drop=FALSE]


On 3/7/06, Vivek Satsangi <[hidden email]> wrote:

> I am writing to document the answer for the next poor sod who comes along.
>
> To get tmp_byRet() into a multi-dimentional matrix, copy the object
> using as.vector(), then copy the dim and dimnames from tmp_byRet into
> the new object. However, this may not be what you want, since you
> probably want the values of the factors within the object (i.e. it
> should be a dataframe, not a matrix).
>
> To get tmp_byRet into a dataframe, use unique() to create a dataframe
> with just the unique values of your factors. Add a new column to the
> dataframe, where you will store the summary stats. Use a loop to
> populate this vector. Then use reshape() on the dataframe to get it to
> the shape you want it in. It is difficult at best to vectorize this
> and avoid the loop  -- and trying to do so will lead to probably less
> transparent code.
>
> Vivek
>
>
> On 3/7/06, Vivek Satsangi <[hidden email]> wrote:
> > Folks,
> > I know that I can do the following using a loop. That's been a lot
> > easier for me to write and understand. But I am trying to force myself
> > to use more vectorized / matrixed code so that eventually I will
> > become a better R programmer.
> >
> > I have a dataframe that has some values by Year, Quarter and Ranking.
> > The variable of interest is the return (F3MRet), to be weighted
> > averaged within the year, quarter and ranking. At the end, we want to
> > end up with a table like this:
> > year  quarter ranking1 ranking2 ... ranking10
> > 1987 1 1.33 1.45 ... 1.99
> > 1987 2 6.45 3.22 ... 8.33
> > .
> > .
> > 2005 4 2.22 3.33 ... 1.22
> >
> > The dataset is too large to post and I can't come up with a small
> > working example very easily.
> >
> > I tried the Reshape() package and also the aggregate and reshape
> > functions. Those don't work too well becuase of the need to pass
> > weighted.mean a weights vector. I tried the by() function, but now I
> > don't know how to coerce the returned object into a matrix so that I
> > can reshape it.
> >
> > > fvs_weighted.mean <- function(y) weighted.mean(y$F3MRet, y$IndexWeight, na.rm=T);
> > > tmp_byRet <- by(dfReturns,
> >      list(dfReturns$Quarter,dfReturns$Year,dfReturns$Ranking),
> >      fvs_weighted.mean);
> >
> > And various other ways to get the tmp_byRet object into a matrix were
> > tried, eg. unlist(), a loop like this:
> > dfRet <- data.frame(tmp_byRet);
> > for(i in 1:dim(dfRet)[2]){
> >        dfRet[ ,i] <- as.vector(dfRet[ ,i]);
> > }
> > In each case, I got some error or the other.
> >
> > So, please help me get unstuck. How can I get the tmp_byRet() object
> > into a matrix or a dataframe?
> >
> > --
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Loading...