Hi all,   
      
   I have a general question, I hope you guys could help me.   
      
   Suppose I have a classifier A that discriminates between two classes:   
   class W and B (White balls and Black balls, respectively).   
      
   Suppose I have to run the classifier on a vast set of balls (:= P), in   
   which the distribution of White and Black balls is unknown (Which   
   means I don't know the a-priori probability of getting a white or a   
   black ball to examine).   
      
   Now I would like to test the classifier. I choose a subset of P (:=N)   
   that consists of N balls and run the experiment to get the ROC curve   
   of the classifier.   
      
   My question is: What is the best way to set the distribution of White   
   and Black balls in N if the distribution of P is unknown? 0.5*N Black   
   balls and 0.5*N White balls sounds right, but is it really right?! And   
   how would the answer change if P can be determined?   
      
   [ comp.ai is moderated ... your article may take a while to appear. ]   
      
   --- SoupGate-Win32 v1.05   
    * Origin: you cannot sedate... all the things you hate (1:229/2)