Quantcast

Imputing data below detection limit

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Imputing data below detection limit

aynumazi
Hello,

I'm trying to impute data below detection limit (with multiple detection limits)
so i need just a method or a code for imputation and then extract the complete dataset to do the analyses.
Is there any package which could do that simply as i'm a beginner in R

Thank you
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Imputing data below detection limit

Jessica Streicher
Tempting a use of let me google that for you..

Anyway, theres a package called Imputation. I myself used the zoo package. There are probably lots of others since its a real common problem.

They usually fill in places in you data that are designated as NA.

I do not completely understand what you mean with detection limit. If you do not have NAs, but rather some kind of threshold, i'd suggest going over the data and filling any applicable values with NAs, then use the library of your choice. I find that kind of weird though, if you haven't detected much you haven't detected much. Its part of the data, why impute?

On 11.08.2012, at 23:01, aynumazi wrote:

> Hello,
>
> I'm trying to impute data below detection limit (with multiple detection
> limits)
> so i need just a method or a code for imputation and then extract the
> complete dataset to do the analyses.
> Is there any package which could do that simply as i'm a beginner in R
>
> Thank you
>
>
>
> --
> View this message in context: http://r.789695.n4.nabble.com/Imputing-data-below-detection-limit-tp4640057.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Imputing data below detection limit

Bert Gunter
Yes, Jessica, the practice -- of which I also have been and continue
to be guilty -- does not really make a lot of sense. It usually
doesn't affect estimation all that much, but it can certainly mess up
inference. The proper approach is to use the proper approach: model it
as left-censored data. The problem with that is:

1. It's complicated, and is beyond the statistical background of most
folks who deal with such data -- it's a ubiquitous issue in science
and engineering after all.

2. Typically, the LOD isn't: that is, there often is not a well
defined value and that which is chosen is both arbitrary and
inaccurate. What one often sees is an increasing loss of relative
precision as one "approaches" the designated value. Modeling this
effectively gets even more complicated. David Rocke and colleagues has
published methodology on this, mostly in TECHNOMETRICS if memory
serves.

3. So, as in other situations, we muddle along with rather crude
statistical approaches and hope that they are adequate. Probably in
most circumstances they are, but ...

Cheers,
Bert

On Mon, Aug 13, 2012 at 1:15 AM, Jessica Streicher
<[hidden email]> wrote:

> Tempting a use of let me google that for you..
>
> Anyway, theres a package called Imputation. I myself used the zoo package. There are probably lots of others since its a real common problem.
>
> They usually fill in places in you data that are designated as NA.
>
> I do not completely understand what you mean with detection limit. If you do not have NAs, but rather some kind of threshold, i'd suggest going over the data and filling any applicable values with NAs, then use the library of your choice. I find that kind of weird though, if you haven't detected much you haven't detected much. Its part of the data, why impute?
>
> On 11.08.2012, at 23:01, aynumazi wrote:
>
>> Hello,
>>
>> I'm trying to impute data below detection limit (with multiple detection
>> limits)
>> so i need just a method or a code for imputation and then extract the
>> complete dataset to do the analyses.
>> Is there any package which could do that simply as i'm a beginner in R
>>
>> Thank you
>>
>>
>>
>> --
>> View this message in context: http://r.789695.n4.nabble.com/Imputing-data-below-detection-limit-tp4640057.html
>> Sent from the R help mailing list archive at Nabble.com.
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.



--

Bert Gunter
Genentech Nonclinical Biostatistics

Internal Contact Info:
Phone: 467-7374
Website:
http://pharmadevelopment.roche.com/index/pdb/pdb-functional-groups/pdb-biostatistics/pdb-ncb-home.htm

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Imputing data below detection limit

Rich Shepard
On Mon, 13 Aug 2012, Bert Gunter wrote:

> The proper approach is to use the proper approach: model it as
> left-censored data. The problem with that is:

>>> I'm trying to impute data below detection limit (with multiple detection
>>> limits) so i need just a method or a code for imputation and then
>>> extract the complete dataset to do the analyses. Is there any package
>>> which could do that simply as i'm a beginner in R

   This is the purpose of the NADA package. The package is based on Dennis
Helsel's "Statistics for Censored Environmental Data Using Minitab and R,
Second Edition."

Rich

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Imputing data below detection limit

Cade, Brian
In reply to this post by Bert Gunter
The below detection limit issue is similar to survival analysis with
censoring (but left rather than right censoring).   So many survival
estimation approaches are thus appropriate for analyses with below
detection limits (see NADA package, also censored quantile regression in
quantreg package, etc).

Brian

Brian S. Cade, PhD

U. S. Geological Survey
Fort Collins Science Center
2150 Centre Ave., Bldg. C
Fort Collins, CO  80526-8818

email:  [hidden email]
tel:  970 226-9326



From:
Bert Gunter <[hidden email]>
To:
Jessica Streicher <[hidden email]>
Cc:
[hidden email]
Date:
08/13/2012 09:28 AM
Subject:
Re: [R] Imputing data below detection limit
Sent by:
[hidden email]



Yes, Jessica, the practice -- of which I also have been and continue
to be guilty -- does not really make a lot of sense. It usually
doesn't affect estimation all that much, but it can certainly mess up
inference. The proper approach is to use the proper approach: model it
as left-censored data. The problem with that is:

1. It's complicated, and is beyond the statistical background of most
folks who deal with such data -- it's a ubiquitous issue in science
and engineering after all.

2. Typically, the LOD isn't: that is, there often is not a well
defined value and that which is chosen is both arbitrary and
inaccurate. What one often sees is an increasing loss of relative
precision as one "approaches" the designated value. Modeling this
effectively gets even more complicated. David Rocke and colleagues has
published methodology on this, mostly in TECHNOMETRICS if memory
serves.

3. So, as in other situations, we muddle along with rather crude
statistical approaches and hope that they are adequate. Probably in
most circumstances they are, but ...

Cheers,
Bert

On Mon, Aug 13, 2012 at 1:15 AM, Jessica Streicher
<[hidden email]> wrote:
> Tempting a use of let me google that for you..
>
> Anyway, theres a package called Imputation. I myself used the zoo
package. There are probably lots of others since its a real common
problem.
>
> They usually fill in places in you data that are designated as NA.
>
> I do not completely understand what you mean with detection limit. If
you do not have NAs, but rather some kind of threshold, i'd suggest going
over the data and filling any applicable values with NAs, then use the
library of your choice. I find that kind of weird though, if you haven't
detected much you haven't detected much. Its part of the data, why impute?
>
> On 11.08.2012, at 23:01, aynumazi wrote:
>
>> Hello,
>>
>> I'm trying to impute data below detection limit (with multiple
detection

>> limits)
>> so i need just a method or a code for imputation and then extract the
>> complete dataset to do the analyses.
>> Is there any package which could do that simply as i'm a beginner in R
>>
>> Thank you
>>
>>
>>
>> --
>> View this message in context:
http://r.789695.n4.nabble.com/Imputing-data-below-detection-limit-tp4640057.html

>> Sent from the R help mailing list archive at Nabble.com.
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.



--

Bert Gunter
Genentech Nonclinical Biostatistics

Internal Contact Info:
Phone: 467-7374
Website:
http://pharmadevelopment.roche.com/index/pdb/pdb-functional-groups/pdb-biostatistics/pdb-ncb-home.htm


______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.



        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Imputing data below detection limit

aynumazi
This post has NOT been accepted by the mailing list yet.
In reply to this post by Rich Shepard
Thank you very much for your reply,

I am triying to use the package NADA. however, i didn t found a function that could directly impute my data below LOD.
There is mainly functions that produce estimates or plots taking into account and modelizing these data below LOD.
My main objective is to impute them and then extract them for use with other packages : survey, mgcv

There is a software which do that using Regression on order statistics, but it doesn t work with my data.

Thank you again

Youssef
Loading...