Inter-rater reliability for 3 unique raters with binary outcome

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Inter-rater reliability for 3 unique raters with binary outcome

Lubo Larsson
Hello,

I would like to know if there is an R utility for computing some
measure of inter-rater reliability/agreement for 3 raters (columns),
where each rating is a binary assessment. Further, the three raters
are unique (the same rater's assessment in any single column) so that
ideally the statistic would account for this fact.

I see there was a package "concord" that had a variety of utilities
for computing kappa stats but looks to me that it is no longer
available.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Inter-rater reliability for 3 unique raters with binary outcome

Jim Lemon-4
Hi Lubo,
Have a look at the "irr" package. I transferred any unique functions
from concord to it to reduce duplication of functions in different
packages.

Jim

On Thu, Sep 12, 2019 at 7:03 PM Lubo Larsson <[hidden email]> wrote:

>
> Hello,
>
> I would like to know if there is an R utility for computing some
> measure of inter-rater reliability/agreement for 3 raters (columns),
> where each rating is a binary assessment. Further, the three raters
> are unique (the same rater's assessment in any single column) so that
> ideally the statistic would account for this fact.
>
> I see there was a package "concord" that had a variety of utilities
> for computing kappa stats but looks to me that it is no longer
> available.
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.