From: mailbox@dmitry-kazakov.de   
      
   On Fri, 25 Jun 2004 10:57:01 +0200, Peter Rijnbeek wrote:   
      
   > Can anyone help me with the following:   
   >   
   > In non-fuzzy classification systems the performance measures often used to   
   > compare systems are the accuracy, sensitivity and specificity as you all   
   > know. What kind of performance measures are used in the fuzzy environment?   
   > The performance measures above can only be used if the results are   
   > dichotimized to correct and incorrect cases. At the end of the inference in   
   > a fuzzy system you have to decide if the outcome is correct or not so you   
   > have to dichotimize the defuzzified outcome by saying > 0.5 is correct lower   
   > is incorrect? Isn't that a waist of information, e.g,   
   >   
   > we have two classes and after the defuzzification the following values are   
   > found:   
   >   
   > u(A)=0.4   
   > u(B)=0.6   
   >   
   > suppose the correct class is A than the outcome is not correct when you use   
   > a threshold of 0.5 but you can say that it did find some indications for   
   > class A. How do you account for that in the performance calculations of the   
   > classifier??   
      
   Similar, as in statistical approach. If you have a measure, say,   
   probability, possibility, necessity etc, then you can evaluate the   
   conditional measure of a given class of errors. For your classification   
   C={A:0.4,B:0.6}, "possibilistic" approach:   
      
      Pos(B|C)=0.6,   
      Nec(B|C)=1-Pos(~B|C)=1-Pos(A|C)=1-0.4=0.6.   
      
   So it is possible and necessary that the classifier is 0.6 wrong.   
      
   > Do you calculate the absolute difference between the outcome and correct   
   > membership of the three classes is that the solution and is that the   
   > performance measure used in most published articles (no one tells that   
   > explicitly?):   
   >   
   > say u(A)=1, u(B)=0 is the correct outcome than this case gives an error of:   
   >   
   >|1-0.4|=0.6 which is summed for all casses and divided by the number of   
   > cases is that the trick?   
      
   It depends on the measure or else set distance you apply. That should be   
   consistent with the premises your model is built upon.   
      
   > The big problem is that, OK with fuzzy logic you can use all fuzziness of   
   > the problem in the calculations, but in the end you have to make a decision   
   > what the class is so you have to make a non-fuzzy decision which overrules   
   > all advantages and only puts the crisp threshold in the end!!??   
      
   First of all, what makes you sure that classes do not intersect and are   
   crisp? Consider yourself as a classifier answering the question "how do you   
   do?". Is the answer always crisp? Does "good" exclude "fine"?   
      
   Second. When dealing with a truly crisp case, there could still be an   
   advantage. Because the classifier itself can be based on incomplete   
   knowledge of the domain. In this case, the data might be crisp, but the   
   rules, are not. Consider, you know that the rule is A if x>N, but you do   
   not know N for sure, then there might be no strict order relation on x etc.   
   You can view x>N as a random variable and that is the way statistical   
   approach goes. Alternatively you can consider it fuzzy. That the fuzzy way.   
      
   --   
   Regards,   
   Dmitry A. Kazakov   
   www.dmitry-kazakov.de   
      
   --- SoupGate-Win32 v1.05   
    * Origin: you cannot sedate... all the things you hate (1:229/2)