R › R help

no true negative data, need roc curve

Classic

List

Threaded

11 messages Options

vjyns

no true negative data, need roc curve

Hi,

I want to plot ROC curve for my detection algorithm which detects features in different images at two different thresholds.

6 different images used and obtained tp, fp and fn. No tn in my case.

in first threshold run i obtained 6 values of tp,fp and fn. In second threshold run agian i got 6 more.

i had calculated tpr and fpr. How to plot ROC in this case.

Is this possible to plot ROC curve in this case? If s, please guide me to plot that.

thank you.

Jessica Streicher

Re: no true negative data, need roc curve

To clarify:

Is TN = 0 or do you not know TN (N)?

On 16.08.2012, at 11:51, vjyns wrote:

> Hi,
>
> I want to plot ROC curve for my detection algorithm which detects
> features in different images at two different thresholds.
>
> 6 different images used and obtained tp, fp and fn. No tn in my case.
>
> in first threshold run i obtained 6 values of tp,fp and fn. In second
> threshold run agian i got 6 more.
>
> i had calculated tpr and fpr. How to plot ROC in this case.
>
> Is this possible to plot ROC curve in this case? If s, please guide me to
> plot that.
>
> thank you.
>
> http://r.789695.n4.nabble.com/file/n4640474/in1.png
>
>
>
> --
> View this message in context: http://r.789695.n4.nabble.com/no-true-negative-data-need-roc-curve-tp4640474.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

vjyns

RE: no true negative data, need roc curve

hi,

TN=0 in all cases, with this how can i plot the ROC, i need help in this regard.

thank you.

Date: Thu, 16 Aug 2012 09:47:20 -0700
From: [hidden email]
To: [hidden email]
Subject: Re: no true negative data, need roc curve

To clarify:

Is TN = 0 or do you not know TN (N)?

On 16.08.2012, at 11:51, vjyns wrote:

If you reply to this email, your message will be added to the discussion below:

http://r.789695.n4.nabble.com/no-true-negative-data-need-roc-curve-tp4640474p4640509.html

To unsubscribe from no true negative data, need roc curve, click here.
NAML

vjyns

Re: no true negative data, need roc curve

In reply to this post by Jessica Streicher

TN=0 in all cases, i had only tp, fp and fn for 6 images (two sets). suggest me how plot the roc curve.

Jessica Streicher

Re: no true negative data, need roc curve

In reply to this post by Jessica Streicher

Okay, first of a point in ROC space is spanned by true positive rate and false positive rate.
Now you need to decide if you want to plot a curve for each image or maybe a curve for the complete data (add all together).

In your case you only have 2 thresholds, so that makes the "curves" a little awkward because you only have 4 points total, but whatever.

Lets say you want a curve for image1:
tp1<-1066
fp1<-70
fn1<-116
tn1<-0
tpr1<-tp1/(tp1+fn1)
fpr1<-fp1/(fp1+tn1)

tp2<-446
fp2<-41
fn2<-55
tn2<-0
tpr2<-tp2/(tp2+fn2)
fpr2<-fp2/(fp2+tn2)

plot(0,0,type="n",ylim=c(0,1),xlim=c(0,1))
points(c(0,fpr1,fpr2,1),c(0,tpr1,tpr2,1))
lines(c(0,fpr1,fpr2,1),c(0,tpr1,tpr2,1))

Now i have to say though, after taking a look at that data you provided: THE DATA IS WRONG
If you add up all values (tp+tn+fp+fn), the number in threshold 2 is lower than in threshold 1.

It needs to be the same, so what the hell have you DONE? ^^

greetings
Jessi

On 16.08.2012, at 17:49, Jessica Streicher wrote:

> To clarify:
>
> Is TN = 0 or do you not know TN (N)?
>
> On 16.08.2012, at 11:51, vjyns wrote:
>
>> Hi,
>>
>> I want to plot ROC curve for my detection algorithm which detects
>> features in different images at two different thresholds.
>>
>> 6 different images used and obtained tp, fp and fn. No tn in my case.
>>
>> in first threshold run i obtained 6 values of tp,fp and fn. In second
>> threshold run agian i got 6 more.
>>
>> i had calculated tpr and fpr. How to plot ROC in this case.
>>
>> Is this possible to plot ROC curve in this case? If s, please guide me to
>> plot that.
>>
>> thank you.
>>
>> http://r.789695.n4.nabble.com/file/n4640474/in1.png
>>
>>
>>
>> --
>> View this message in context: http://r.789695.n4.nabble.com/no-true-negative-data-need-roc-curve-tp4640474.html
>> Sent from the R help mailing list archive at Nabble.com.
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

vjyns

Re: no true negative data, need roc curve

Hi,

thanks for the quick response, but as i said in my case due to two different threshold the detected features will differ. Moreover, there is some standard refined formula in calculating the tpr and fpr. herewith i had attached the refined formula from a standard international journal

when i used the above mentioned formula (fpr=fp/fp+tp) then i can able to see my point are distributed on the extreme left corner. Like this it is possible to put all the 6 images. Will you please suggest me now how to obtain the plot for different images of two threshold.

Jessica Streicher

Re: no true negative data, need roc curve

Above mentioned formula is wrong - maybe a typo
http://en.wikipedia.org/wiki/Receiver_operating_characteristic

The false positive rate is the rate of false positives, meaning how many of the total negatives (all in reality negatives(N), that is, all negatives falsely classified as positives(fp) and all negatives correctly classified as negatives(tn)) have been falsely classified as positive.

Also the authors obviously had (N+P=number of features), and therefore at least could have computed this properly. For example:

N+P=100
P=TP+FN
N=FP+TN
-> do the math with what you got

On 17.08.2012, at 11:13, vjyns wrote:

> Hi,
>
> thanks for the quick response, but as i said in my case due to two
> different threshold the detected features will differ. Moreover, there is
> some standard /refined/ formula in calculating the tpr and fpr. herewith i
> had attached the refined formula from a standard international journal
> http://r.789695.n4.nabble.com/file/n4640577/tpr_and_fpr.jpg
>
> when i used the above mentioned formula (fpr=fp/fp+tp) then i can able to
> see my point are distributed on the extreme left corner. Like this it is
> possible to put all the 6 images. Will you please suggest me now how to
> obtain the plot for different images of two threshold.
>
>
>
> --
> View this message in context: http://r.789695.n4.nabble.com/no-true-negative-data-need-roc-curve-tp4640474p4640577.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

vjyns

Re: no true negative data, need roc curve

This is not a typo error, in feature detection when it was not able to calculate 'tn' then they are using this formula and there are more papers (referred journal) which quote same formula. In the same manner i had also got the tp, fp and fn. Based on it can some one suggest me to plot the ROC curve for all the 6 data.

Jessica Streicher

Re: no true negative data, need roc curve

In reply to this post by Jessica Streicher

Just to say, i still consider it wrong (what is called the fpr there simply isn't the fpr, and the graph will not be what is commonly known as an ROC curve), at least the definition list puts down slightly different names for it.
What else exactly do you need then though? you now know how to plot points and lines, whats there more to explain?

You can use the commands ?plot ? points ?lines and so on to find help on it.

On 17.08.2012, at 13:18, Jessica Streicher wrote:

> Above mentioned formula is wrong - maybe a typo
> http://en.wikipedia.org/wiki/Receiver_operating_characteristic
>
> The false positive rate is the rate of false positives, meaning how many of the total negatives (all in reality negatives(N), that is, all negatives falsely classified as positives(fp) and all negatives correctly classified as negatives(tn)) have been falsely classified as positive.
>
> Also the authors obviously had (N+P=number of features), and therefore at least could have computed this properly. For example:
>
> N+P=100
> P=TP+FN
> N=FP+TN
> -> do the math with what you got
>
> On 17.08.2012, at 11:13, vjyns wrote:
>
>> Hi,
>>
>> thanks for the quick response, but as i said in my case due to two
>> different threshold the detected features will differ. Moreover, there is
>> some standard /refined/ formula in calculating the tpr and fpr. herewith i
>> had attached the refined formula from a standard international journal
>> http://r.789695.n4.nabble.com/file/n4640577/tpr_and_fpr.jpg
>>
>> when i used the above mentioned formula (fpr=fp/fp+tp) then i can able to
>> see my point are distributed on the extreme left corner. Like this it is
>> possible to put all the 6 images. Will you please suggest me now how to
>> obtain the plot for different images of two threshold.
>>
>>
>>
>> --
>> View this message in context: http://r.789695.n4.nabble.com/no-true-negative-data-need-roc-curve-tp4640474p4640577.html
>> Sent from the R help mailing list archive at Nabble.com.
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

vjyns

Re: no true negative data, need roc curve

Hi,

Thanks a lot. I learned that the ROC used in my area was a modified one like you mentioned, but i want to stick to that if i want to put my result to such community. I was new to this topic and R, you guided me a lot and made me to understand. thank you.

vjyns

Re: no true negative data, need roc curve

In reply to this post by Jessica Streicher

Hi,

again i had small clarification regarding the discussion. I had 6 images of two threshold test, so can i plot 6 roc for each individual image? or can i plot two roc curve (threshold 1 all images summed up and similar to threshold 2)? which is the correct one?

Please clarify me in this regard. You helped me a lot, know clarify this also

thank you.