Ask Me Help Desk

Ask Me Help Desk (https://www.askmehelpdesk.com/forum.php)
-   Other Subjects (https://www.askmehelpdesk.com/forumdisplay.php?f=406)
-   -   confusion matrix (https://www.askmehelpdesk.com/showthread.php?t=660069)

  • May 13, 2012, 10:48 PM
    ozanganeh
    confusion matrix
    1 down vote favorite
    share [g+] share [fb] share [tw]


    http://en.wikipedia.org/wiki/Confusion_matrix

    http://www2.cs.uregina.ca/~hamilton/courses/831/notes/confusion_matrix/confusion_matrix.html

    Hi guys, I want to calculate the true positive (TP), true negative (TN), false positive (FP) and false negative (FN) from confusion matrix. The question is, based on the above links, the formula to compute TP, TN, … are different. In Wikipedia link, TP, is one of the elements in confusion matrix without any additional computation, but in the other link:

    TP=d/(c+d),
    TN= ….

    I'm confused of two different values assigned to TP, TN, … in these two pages.

    Thanks for your help.
  • May 14, 2012, 04:05 AM
    camyyssa
    Hey,

    The confusion matrix is indeed the matrix with TP, FP, FN, TN, so if you have it you already have the values you need. The formulas you see in there are for the true positive rate, meaning the percentage of positives that are true out of all the positives returned. It's similar for the others.
  • May 14, 2012, 08:16 AM
    ozanganeh
    Quote:
    Originally Posted by camyyssa View Post
    Hey,

    The confusion matrix is indeed the matrix with TP, FP, FN, TN, so if you have it you already have the values you need. The formulas you see in there are for the true positive rate, meaning the percentage of positives that are true out of all the positives returned. It's similar for the others.

    Thanks Camyyssa,
    But what if we want to calculate the precision and recall from TP, TN,. in this case we use the rates (true positive rate, TN rate,. ) or just TP, TN,.

    It clearly says in:
    http://en.wikipedia.org/wiki/Precision_and_recall
    That we use TP not TP rate, but I was working on my friends code and he used the TP rate, TN rate and... to calculate precision, recall, and then Fmeasure and that's where all this come from!
  • May 14, 2012, 10:42 AM
    camyyssa
    If we want to calculate precision and recall we use the rates TP rate and TN rate.

    What the wikipedia article on precision and recall is saying corresponds to the formula in the University of Regina article.

    To show you this I'm going to take a careful look at the definition of Precision and Recall and the first example in the wikipedia article. I'll include the paragraph in here as reference:

    Quote:
    In pattern recognition and information retrieval, precision is the fraction of retrieved instances that are relevant, while recall is the fraction of relevant instances that are retrieved. Both precision and recall are therefore based on an understanding and measure of relevance. Suppose a program for recognizing dogs in scenes identifies seven dogs in a scene containing nine. If four of the identifications are correct, but three are actually cats, the program's precision is 4/7 while its recall is 4/9.
    Precision = (Relevant instances out of the retrieved ones)/(Retrieved instances)
    The retrieved instances are all instances that my program classifies as being part of the class I'm interested in (in our case dogs). Thus retrieved instances are all positives, true or not - TP+FP. Relevant instances are all positives that are actually relevant to my search, thus TP.
    So the formula they give is Precision = TP/(TP+FP)

    In other words precision is the answer to the question given n elements, how many am I actually going to be interested in?

    Recall = (Retrieved instances out of the relevant ones)/(Relevant instances)
    The first part is the number of instances that I retrieved that are also relevant (thus TP), and the second part is how many instances that I'm interested in are in the whole set, retrieved or not (the number of dogs in my set in this case is 9 = 4 that I retrieved + (9-4=5) that I missed - TP + FN = what I retrieved correctly + what I missed incorrectly). Recall = TP/(TP + FN)

    The confussion matrix that I got for the example:
    Predicted
    | Cats | Dogs
    --------------------------------------------------------
    Cats | ? | 5
    Actual --------------------------------------------------------
    Dogs | 3 | 4

    P = TP/(TP+FP) = 4/(4+3) = 4/7
    R = TP/(TP+FN) = 4/(4+5) = 4/9

    So wikipedia is by no means saying that P = TP. The formulas also coincide with the ones given on the university link that you put in in the original question.
  • May 14, 2012, 10:45 AM
    camyyssa
    Hmm.. the table got really confused. What I have in there in the order TN, FN, FP, TP is? 5, 3, 4
  • May 14, 2012, 07:24 PM
    ozanganeh
    Quote:
    Originally Posted by camyyssa View Post
    Hmm.. the table got really confused. What I have in there in the order TN, FN, FP, TP is ?, 5, 3, 4
    Thanks again for your quick and detailed answer.

    If we take a look, the result of recall = ( TP/(TP+FN) ) does not differ if we use TP rate or TP. The reason is:

    Recall = TPrate / (TPrate+FNrate) = (d/c+d) / ( d/(c+d) + c/(c+d) ) =
    = ( d/(c+d) ) / ( (d+c)/(c+d) ) = d/(c+d) = TP rate =
    definition of Recall in your example = TP/(TP+FN)

    but for Precision it differs if we use rates.

    So if I'm not missing any point, we can conclude that there are two different ways of calculating precision. Obviously, we get different results by these two different formulas, but the one that has been mostly used is by rates, TP rate and...
    Am I right?
  • May 15, 2012, 04:33 AM
    camyyssa
    I just noticed that on the university website they denote TP as the true positive rate. I was using it to denote the number of true positives, not the rate.

    I don't think either of the sources give the formulas based on rates, and it would be a rather circular way of defining at least the Recall, since it is equivalent to the TPRate.The reason it doesn't matter is because TPRate + FNRate = d/(c+d) + c/(c+d) = (c+d)/(c+d) = 1.

    The precision cannot be calculated using rates, and the recall, you already know it if you are trying to use rates. Hope that clears up the misunderstanding :D. Let me know how it goes!
  • May 15, 2012, 06:05 PM
    ozanganeh
    Quote:
    Originally Posted by camyyssa View Post
    I just noticed that on the university website they denote TP as the true positive rate. I was using it to denote the number of true positives, not the rate.

    I don't think either of the sources give the formulas based on rates, and it would be a rather circular way of defining at least the Recall, since it is equivalent to the TPRate.The reason it doesn't matter is because TPRate + FNRate = d/(c+d) + c/(c+d) = (c+d)/(c+d) = 1.

    The precision cannot be calculated using rates, and the recall, you already know it if you are trying to use rates. Hope that clears up the misunderstanding :D. Let me know how it goes!
    oooh, my bad!
    yeap, none of the recall and precision are calculated by rates.
    It's clear now!

    http://www2.cs.uregina.ca/~hamilton/courses/831/notes/confusion_matrix/confusion_matrix.html

    BTW, based on the above link, what is called "TP rate" differs from TP and...
  • May 15, 2012, 11:56 PM
    camyyssa
    Exactly. They're just using a rather stupid notation.

    • All times are GMT -7. The time now is 03:21 AM.