Remove cases with -Inf from a data frame

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Remove cases with -Inf from a data frame

AbouEl-Makarim Aboueissa-3
Dear All: good morning


I have a log-transformed data frame with some *-Inf* data values.

*my question: *how to remove all rows with *-Inf* data value from that data
frame?


with many thanks
abou
______________________


*AbouEl-Makarim Aboueissa, PhD*

*Professor, Statistics and Data Science*
*Graduate Coordinator*

*Department of Mathematics and Statistics*
*University of Southern Maine*

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Remove cases with -Inf from a data frame

Michael Dewey-3
Dear Abou

Depends on exact details of your variables but

?is.finite

Gives you the basic tool.

On 16/02/2019 15:05, AbouEl-Makarim Aboueissa wrote:

> Dear All: good morning
>
>
> I have a log-transformed data frame with some *-Inf* data values.
>
> *my question: *how to remove all rows with *-Inf* data value from that data
> frame?
>
>
> with many thanks
> abou
> ______________________
>
>
> *AbouEl-Makarim Aboueissa, PhD*
>
> *Professor, Statistics and Data Science*
> *Graduate Coordinator*
>
> *Department of Mathematics and Statistics*
> *University of Southern Maine*
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

--
Michael
http://www.dewey.myzen.co.uk/home.html

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Remove cases with -Inf from a data frame

Martin Møller Skarbiniks Pedersen
In reply to this post by AbouEl-Makarim Aboueissa-3
On Sat, 16 Feb 2019 at 16:07, AbouEl-Makarim Aboueissa <
[hidden email]> wrote:
>
> I have a log-transformed data frame with some *-Inf* data values.
>
> *my question: *how to remove all rows with *-Inf* data value from that
data
> frame?


Hi,
  Here is a solution which uses apply.

First a data-frame as input:

set.seed(1)
df <- data.frame(w = sample(c(-Inf,1:20), 10),
                 x = sample(c(-Inf,1:20), 10),
                 y = sample(c(-Inf,1:20), 10),
                 z = sample(c(-Inf,1:20), 10))

df <- df[-(unlist(apply(df, 2, function(x) which(x == -Inf)))),]

Regards
Martin

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Remove cases with -Inf from a data frame

Bert Gunter-2
In reply to this post by AbouEl-Makarim Aboueissa-3
Many ways. I assume you know that Inf and -Inf are (special) numeric values
that can be treated like other numerics. i.e.

> 1 == - Inf
[1] FALSE

So straightforward indexing (selection) would do it.
But there is also ?is.infinite and ?is.finite, so

apply(yourdat, 1, function(x)all(is.finite(x)))

would produce the index vector to keep rows with only finite values
assuming yourdat contains only numeric data. If this is not the case, just
select the numeric columns to index on, i.e.

apply(yourdat[sapply(yourdat,is.numeric)], 1, function(x) all(is.finite(x)))

One possible problem here is handling of NA's:

is.finite(c(-Inf,NA))
[1] FALSE FALSE
... so rows containing NA's but no -Inf's would also get removed. If you
wish to keep rows with NA's but no -Inf's, then

function(x)(is.finite(x) | is.na(x) )

could be used.

Cheers,
Bert

Bert Gunter

"The trouble with having an open mind is that people keep coming along and
sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Sat, Feb 16, 2019 at 7:07 AM AbouEl-Makarim Aboueissa <
[hidden email]> wrote:

> Dear All: good morning
>
>
> I have a log-transformed data frame with some *-Inf* data values.
>
> *my question: *how to remove all rows with *-Inf* data value from that data
> frame?
>
>
> with many thanks
> abou
> ______________________
>
>
> *AbouEl-Makarim Aboueissa, PhD*
>
> *Professor, Statistics and Data Science*
> *Graduate Coordinator*
>
> *Department of Mathematics and Statistics*
> *University of Southern Maine*
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Remove cases with -Inf from a data frame

Bert Gunter-2
Sorry, that's

function(x)all(is.finite(x) | is.na(x) )

of course.


Bert Gunter

"The trouble with having an open mind is that people keep coming along and
sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Sat, Feb 16, 2019 at 8:25 AM Bert Gunter <[hidden email]> wrote:

> Many ways. I assume you know that Inf and -Inf are (special) numeric
> values that can be treated like other numerics. i.e.
>
> > 1 == - Inf
> [1] FALSE
>
> So straightforward indexing (selection) would do it.
> But there is also ?is.infinite and ?is.finite, so
>
> apply(yourdat, 1, function(x)all(is.finite(x)))
>
> would produce the index vector to keep rows with only finite values
> assuming yourdat contains only numeric data. If this is not the case, just
> select the numeric columns to index on, i.e.
>
> apply(yourdat[sapply(yourdat,is.numeric)], 1, function(x)
> all(is.finite(x)))
>
> One possible problem here is handling of NA's:
>
> is.finite(c(-Inf,NA))
> [1] FALSE FALSE
> ... so rows containing NA's but no -Inf's would also get removed. If you
> wish to keep rows with NA's but no -Inf's, then
>
> function(x)(is.finite(x) | is.na(x) )
>
> could be used.
>
> Cheers,
> Bert
>
> Bert Gunter
>
> "The trouble with having an open mind is that people keep coming along and
> sticking things into it."
> -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
>
>
> On Sat, Feb 16, 2019 at 7:07 AM AbouEl-Makarim Aboueissa <
> [hidden email]> wrote:
>
>> Dear All: good morning
>>
>>
>> I have a log-transformed data frame with some *-Inf* data values.
>>
>> *my question: *how to remove all rows with *-Inf* data value from that
>> data
>> frame?
>>
>>
>> with many thanks
>> abou
>> ______________________
>>
>>
>> *AbouEl-Makarim Aboueissa, PhD*
>>
>> *Professor, Statistics and Data Science*
>> *Graduate Coordinator*
>>
>> *Department of Mathematics and Statistics*
>> *University of Southern Maine*
>>
>>         [[alternative HTML version deleted]]
>>
>> ______________________________________________
>> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Remove cases with -Inf from a data frame

Rui Barradas
In reply to this post by Martin Møller Skarbiniks Pedersen
Hello,

An alternative, same dataset.

df[apply(df, 1, function(x) all(is.finite(x))), ]


Hope this helps,

Rui Barradas

Às 16:14 de 16/02/2019, Martin Møller Skarbiniks Pedersen escreveu:

> On Sat, 16 Feb 2019 at 16:07, AbouEl-Makarim Aboueissa <
> [hidden email]> wrote:
>>
>> I have a log-transformed data frame with some *-Inf* data values.
>>
>> *my question: *how to remove all rows with *-Inf* data value from that
> data
>> frame?
>
>
> Hi,
>    Here is a solution which uses apply.
>
> First a data-frame as input:
>
> set.seed(1)
> df <- data.frame(w = sample(c(-Inf,1:20), 10),
>                   x = sample(c(-Inf,1:20), 10),
>                   y = sample(c(-Inf,1:20), 10),
>                   z = sample(c(-Inf,1:20), 10))
>
> df <- df[-(unlist(apply(df, 2, function(x) which(x == -Inf)))),]
>
> Regards
> Martin
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Remove cases with -Inf from a data frame

AbouEl-Makarim Aboueissa-3
Dear Rui and All:

thank you very much for your very helpful responses.

with many thanks
abou
______________________


*AbouEl-Makarim Aboueissa, PhD*

*Professor, Statistics and Data Science*
*Graduate Coordinator*

*Department of Mathematics and Statistics*
*University of Southern Maine*



On Sat, Feb 16, 2019 at 11:36 AM Rui Barradas <[hidden email]> wrote:

> Hello,
>
> An alternative, same dataset.
>
> df[apply(df, 1, function(x) all(is.finite(x))), ]
>
>
> Hope this helps,
>
> Rui Barradas
>
> Às 16:14 de 16/02/2019, Martin Møller Skarbiniks Pedersen escreveu:
> > On Sat, 16 Feb 2019 at 16:07, AbouEl-Makarim Aboueissa <
> > [hidden email]> wrote:
> >>
> >> I have a log-transformed data frame with some *-Inf* data values.
> >>
> >> *my question: *how to remove all rows with *-Inf* data value from that
> > data
> >> frame?
> >
> >
> > Hi,
> >    Here is a solution which uses apply.
> >
> > First a data-frame as input:
> >
> > set.seed(1)
> > df <- data.frame(w = sample(c(-Inf,1:20), 10),
> >                   x = sample(c(-Inf,1:20), 10),
> >                   y = sample(c(-Inf,1:20), 10),
> >                   z = sample(c(-Inf,1:20), 10))
> >
> > df <- df[-(unlist(apply(df, 2, function(x) which(x == -Inf)))),]
> >
> > Regards
> > Martin
> >
> >       [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> >
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Remove cases with -Inf from a data frame

Ek Esawi
In reply to this post by AbouEl-Makarim Aboueissa-3
This is a similar versions of other answers.

df[apply(apply(df,2,is.finite),1,sum)==4,]

BOL---EK

On Sat, Feb 16, 2019 at 10:07 AM AbouEl-Makarim Aboueissa
<[hidden email]> wrote:

>
> Dear All: good morning
>
>
> I have a log-transformed data frame with some *-Inf* data values.
>
> *my question: *how to remove all rows with *-Inf* data value from that data
> frame?
>
>
> with many thanks
> abou
> ______________________
>
>
> *AbouEl-Makarim Aboueissa, PhD*
>
> *Professor, Statistics and Data Science*
> *Graduate Coordinator*
>
> *Department of Mathematics and Statistics*
> *University of Southern Maine*
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.