Two ALTREP questions

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Two ALTREP questions

Jiefei Wang
Hello,

I have two related ALTREP questions. It seems like there is no way to
assign attributes to an ALTREP vector without using C++ code. To be more
specifically, I want to make an ALTREP matrix, I have tried the following R
code but none of them work.
```
.Internal(inspect(1:6))
.Internal(inspect(matrix(1:6, 2,3)))
.Internal(inspect(as.matrix(1:6)))
.Internal(inspect(structure(1:6, dim = c(2L,3L))))
.Internal(inspect({x <- 1:6;attr(x, "dim") <- c(2L,3L);x}))
.Internal(inspect({x <- 1:6;attributes(x)<- list(dim = c(2L,3L));x}))
```

The only way to make an ALTREP matrix is to use the C level function
```
attachAttrib <- inline::cxxfunction( signature(x = "SEXP", attr = "SEXP" )
, '
SET_ATTRIB(x,attr);
return(R_NilValue);
')
x <- 1:6
attachAttrib(x, pairlist(dim = c(2L, 3L)))
.Internal(inspect(x))
```

Since the matrix, or adding attributes, is a common need for the object
operation, I wonder if this missing feature is intended? This also brings
my second question, it seems like the ALTREP coercion function does not
handle attributes correctly.  After the coercion, the ALTREP object will
lose its attributes.
```
coerceFunc <- inline::cxxfunction( signature(x = "SEXP", attr = "SEXP" ) , '
SET_ATTRIB(x,attr);
return(Rf_coerceVector(x, REALSXP));
')
> coerceFunc(1:6, pairlist(dim = c(2L, 3L)))
[1] 1 2 3 4 5 6
> coerceFunc(1:6 + 0L, pairlist(dim = c(2L, 3L)))
     [,1] [,2] [,3]
[1,]    1    3    5
[2,]    2    4    6
```
The problem is that the coercion function is directly dispatched to the
user-defined ALTREP coercion function, so the user is responsible to attach
the attributes after the coercion. If he forgets to do so, then the result
is a plain vector. Similar to the `Duplicate` and `DuplicateEX` functions
where the former one will attach the attributes by default, I feel that the
`Coerce` function should only return a plain vector and there should be a
`CoerceEx` function to do the attribute assignment, so the logic in the
no-EX ALTREP functions can be consistent. I do not know how dramastic the
change would be, so maybe this is too hard to do.

BTW, is there any way to contribute to the R source? I know R has a limited
resouces, so if possible, I will be happy to fix the matrix issue myself
and make some minor contributions to the R community.

Best,
Jiefei

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Two ALTREP questions

Dirk Eddelbuettel


On 21 November 2020 at 20:56, Jiefei Wang wrote:
| Hello,
|
| I have two related ALTREP questions. It seems like there is no way to
| assign attributes to an ALTREP vector without using C++ code. To be more
| specifically, I want to make an ALTREP matrix, I have tried the following R
| code but none of them work.
| ```
| .Internal(inspect(1:6))
| .Internal(inspect(matrix(1:6, 2,3)))
| .Internal(inspect(as.matrix(1:6)))
| .Internal(inspect(structure(1:6, dim = c(2L,3L))))
| .Internal(inspect({x <- 1:6;attr(x, "dim") <- c(2L,3L);x}))
| .Internal(inspect({x <- 1:6;attributes(x)<- list(dim = c(2L,3L));x}))
| ```
|
| The only way to make an ALTREP matrix is to use the C level function
| ```
| attachAttrib <- inline::cxxfunction( signature(x = "SEXP", attr = "SEXP" )
| , '
| SET_ATTRIB(x,attr);
| return(R_NilValue);
| ')
| x <- 1:6
| attachAttrib(x, pairlist(dim = c(2L, 3L)))
| .Internal(inspect(x))
| ```

(That's C code. The confusion here is partly our fault. When Romain and I
extended the inline package with 'cxxfunction' to support the then-young but
active Rcpp package, we picked C++. Strictly speaking that isn't required;
you are only in C++ here because ... "it made sense to us 10 years ago" and
it could generalized to C++ or C.  All ALTREP is, of course, purely C as it
is an R API.)
 
| Since the matrix, or adding attributes, is a common need for the object
| operation, I wonder if this missing feature is intended? This also brings
| my second question, it seems like the ALTREP coercion function does not
| handle attributes correctly.  After the coercion, the ALTREP object will
| lose its attributes.
| ```
| coerceFunc <- inline::cxxfunction( signature(x = "SEXP", attr = "SEXP" ) , '
| SET_ATTRIB(x,attr);
| return(Rf_coerceVector(x, REALSXP));
| ')
| > coerceFunc(1:6, pairlist(dim = c(2L, 3L)))
| [1] 1 2 3 4 5 6
| > coerceFunc(1:6 + 0L, pairlist(dim = c(2L, 3L)))
|      [,1] [,2] [,3]
| [1,]    1    3    5
| [2,]    2    4    6
| ```
| The problem is that the coercion function is directly dispatched to the
| user-defined ALTREP coercion function, so the user is responsible to attach
| the attributes after the coercion. If he forgets to do so, then the result
| is a plain vector. Similar to the `Duplicate` and `DuplicateEX` functions
| where the former one will attach the attributes by default, I feel that the
| `Coerce` function should only return a plain vector and there should be a
| `CoerceEx` function to do the attribute assignment, so the logic in the
| no-EX ALTREP functions can be consistent. I do not know how dramastic the
| change would be, so maybe this is too hard to do.
|
| BTW, is there any way to contribute to the R source? I know R has a limited
| resouces, so if possible, I will be happy to fix the matrix issue myself
| and make some minor contributions to the R community.

Yes. The generally recommended way is via a bug report at bugs.r-project.org
(for which you need an account there, and I always forget who the account
creation gatekeeper is ...) preferably with a patch. You effectively need an
R Core "sponsor" as someone has to actually put the code into R base.  So for
"everything else" the recommendation generally is to go via a package. Maybe
the easier route for you is to bundle a few ALTREP helper functions first?

Dirk

--
https://dirk.eddelbuettel.com | @eddelbuettel | [hidden email]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: [External] Two ALTREP questions

luke-tierney
In reply to this post by Jiefei Wang
On Sat, 21 Nov 2020, Jiefei Wang wrote:

> Hello,
>
> I have two related ALTREP questions. It seems like there is no way to
> assign attributes to an ALTREP vector without using C++ code. To be more
> specifically, I want to make an ALTREP matrix, I have tried the following R
> code but none of them work.
> ```
> .Internal(inspect(1:6))
> .Internal(inspect(matrix(1:6, 2,3)))
> .Internal(inspect(as.matrix(1:6)))
> .Internal(inspect(structure(1:6, dim = c(2L,3L))))
> .Internal(inspect({x <- 1:6;attr(x, "dim") <- c(2L,3L);x}))
> .Internal(inspect({x <- 1:6;attributes(x)<- list(dim = c(2L,3L));x}))
> ```

Some things that my help you:

- Try with 1:6 replaced by as.character(1:6), and look at the REF
   values in both cases.

- In particular, look at what this gives you:

     x <- as.character(1:6)
     attr(x, "dim") <- c(2, 3)

- Things can be a little different with larger vectors; try variants
   of your examples for more than 64 elements.

> This also brings
> my second question, it seems like the ALTREP coercion function does not
> handle attributes correctly.  After the coercion, the ALTREP object will
> lose its attributes.
> ```
> coerceFunc <- inline::cxxfunction( signature(x = "SEXP", attr = "SEXP" ) , '
> SET_ATTRIB(x,attr);
> return(Rf_coerceVector(x, REALSXP));
> ')
>> coerceFunc(1:6, pairlist(dim = c(2L, 3L)))
> [1] 1 2 3 4 5 6
>> coerceFunc(1:6 + 0L, pairlist(dim = c(2L, 3L)))
>     [,1] [,2] [,3]
> [1,]    1    3    5
> [2,]    2    4    6
> ```
> The problem is that the coercion function is directly dispatched to the
> user-defined ALTREP coercion function, so the user is responsible to attach
> the attributes after the coercion. If he forgets to do so, then the result
> is a plain vector. Similar to the `Duplicate` and `DuplicateEX` functions
> where the former one will attach the attributes by default, I feel that the
> `Coerce` function should only return a plain vector and there should be a
> `CoerceEx` function to do the attribute assignment, so the logic in the
> no-EX ALTREP functions can be consistent. I do not know how dramastic the
> change would be, so maybe this is too hard to do.

Since you raised this earlier I have been looking at it and also think
that this needs to he handled along the lines of
Duplicate/DuplicateEx. I need to find some time to think that through
and implement it; hopefully I'll get to it before the end of the year.

> BTW, is there any way to contribute to the R source? I know R has a limited
> resouces, so if possible, I will be happy to fix the matrix issue myself
> and make some minor contributions to the R community.

You can find the suggested process for contributing described in the
'Reporting Bugs' link on the R home page https://www.r-project.org/

Best,

luke

> Best,
> Jiefei
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

--
Luke Tierney
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa                  Phone:             319-335-3386
Department of Statistics and        Fax:               319-335-3017
    Actuarial Science
241 Schaeffer Hall                  email:   [hidden email]
Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: [External] Two ALTREP questions

Jiefei Wang
Thank Dirk and Luke for the answers!

(That's C code. The confusion here is partly our fault. When Romain and I
> extended the inline package with 'cxxfunction' to support the then-young
> but
> active Rcpp package, we picked C++. Strictly speaking that isn't required;
> you are only in C++ here because ... "it made sense to us 10 years ago" and
> it could generalized to C++ or C.  All ALTREP is, of course, purely C as it
> is an R API.)


Sometimes I forget to distinguish C/C++ code. Yes, this should be C code
and it is just C++ compatible.
Anyway, `inline` is a great package, `cxxfunction` makes life much easier
for reporting the low-level problem to the devel team.

- Try with 1:6 replaced by as.character(1:6), and look at the REF
>    values in both cases.
> - In particular, look at what this gives you:
>      x <- as.character(1:6)
>      attr(x, "dim") <- c(2, 3)
> - Things can be a little different with larger vectors; try variants
>    of your examples for more than 64 elements.


I see why we cannot change the attribute of the compact sequence(It is
shared, or at least marked as shared)
and the use of the wrapper for the large vector. Only the matrix function
needs to be patched.

The generally recommended way is via a bug report at bugs.r-project.org


You can find the suggested process for contributing described in the
> 'Reporting Bugs' link on the R home page https://www.r-project.org/


Bugzilla sounds like a good place to start, I will send an email to acquire
an account.

Best,
Jiefei


On Sun, Nov 22, 2020 at 6:57 AM <[hidden email]> wrote:

> On Sat, 21 Nov 2020, Jiefei Wang wrote:
>
> > Hello,
> >
> > I have two related ALTREP questions. It seems like there is no way to
> > assign attributes to an ALTREP vector without using C++ code. To be more
> > specifically, I want to make an ALTREP matrix, I have tried the
> following R
> > code but none of them work.
> > ```
> > .Internal(inspect(1:6))
> > .Internal(inspect(matrix(1:6, 2,3)))
> > .Internal(inspect(as.matrix(1:6)))
> > .Internal(inspect(structure(1:6, dim = c(2L,3L))))
> > .Internal(inspect({x <- 1:6;attr(x, "dim") <- c(2L,3L);x}))
> > .Internal(inspect({x <- 1:6;attributes(x)<- list(dim = c(2L,3L));x}))
> > ```
>
> Some things that my help you:
>
> - Try with 1:6 replaced by as.character(1:6), and look at the REF
>    values in both cases.
>
> - In particular, look at what this gives you:
>
>      x <- as.character(1:6)
>      attr(x, "dim") <- c(2, 3)
>
> - Things can be a little different with larger vectors; try variants
>    of your examples for more than 64 elements.
>
> > This also brings
> > my second question, it seems like the ALTREP coercion function does not
> > handle attributes correctly.  After the coercion, the ALTREP object will
> > lose its attributes.
> > ```
> > coerceFunc <- inline::cxxfunction( signature(x = "SEXP", attr = "SEXP" )
> , '
> > SET_ATTRIB(x,attr);
> > return(Rf_coerceVector(x, REALSXP));
> > ')
> >> coerceFunc(1:6, pairlist(dim = c(2L, 3L)))
> > [1] 1 2 3 4 5 6
> >> coerceFunc(1:6 + 0L, pairlist(dim = c(2L, 3L)))
> >     [,1] [,2] [,3]
> > [1,]    1    3    5
> > [2,]    2    4    6
> > ```
> > The problem is that the coercion function is directly dispatched to the
> > user-defined ALTREP coercion function, so the user is responsible to
> attach
> > the attributes after the coercion. If he forgets to do so, then the
> result
> > is a plain vector. Similar to the `Duplicate` and `DuplicateEX` functions
> > where the former one will attach the attributes by default, I feel that
> the
> > `Coerce` function should only return a plain vector and there should be a
> > `CoerceEx` function to do the attribute assignment, so the logic in the
> > no-EX ALTREP functions can be consistent. I do not know how dramastic the
> > change would be, so maybe this is too hard to do.
>
> Since you raised this earlier I have been looking at it and also think
> that this needs to he handled along the lines of
> Duplicate/DuplicateEx. I need to find some time to think that through
> and implement it; hopefully I'll get to it before the end of the year.
>
> > BTW, is there any way to contribute to the R source? I know R has a
> limited
> > resouces, so if possible, I will be happy to fix the matrix issue myself
> > and make some minor contributions to the R community.
>
> You can find the suggested process for contributing described in the
> 'Reporting Bugs' link on the R home page https://www.r-project.org/
>
> Best,
>
> luke
>
> > Best,
> > Jiefei
> >
> >       [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
>
> --
> Luke Tierney
> Ralph E. Wareham Professor of Mathematical Sciences
> University of Iowa                  Phone:             319-335-3386
> Department of Statistics and        Fax:               319-335-3017
>     Actuarial Science
> 241 Schaeffer Hall                  email:   [hidden email]
> Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel