From: "a_jakulin@"@hotmail.com   
      
   Assuming that we're doing exponential families and ML fitting: The   
   ability to select features gives you additional freedom in creating   
   the model. This can be used to:   
   * gain in simplicity,   
   * achieve an improvement in generalized fit, using the assumption that   
   weak improvements on the training data are often due to randomness,   
   * risk overfitting.   
      
   Certain learning methods, such as the naive Bayes make assumptions   
   about the independence of features. If these assumptions are violated,   
   you lose in performance. So it's a slightly different subject there.   
      
   There are two notable recent pieces of work. Grunwald and Halpern had   
   a colorful paper at UAI 2004 titled "Ignorance is Bliss" on how   
   additional information may sometimes hurt you. There was a NIPS   
   feature selection competition last year, and there were a bunch of   
   uninformative features injected in the data set. The winner was R.   
   Neal with his student(s). What's interesting about this is that they   
   did not perform any feature selection: they just didn't overfit. In   
   all, Bayesians tend to scoff at feature selection. They just integrate   
   the irrelevance out.   
      
   --   
   mag. Aleks Jakulin   
   http://www.ailab.si/aleks/   
   Artificial Intelligence Laboratory,   
   Faculty of Computer and Information Science,   
   University of Ljubljana, Slovenia.   
      
   [ comp.ai is moderated.  To submit, just post and be patient, or if ]   
   [ that fails mail your article to , and ]   
   [ ask your news administrator to fix the problems with your system. ]   
      
   --- SoupGate-Win32 v1.05   
    * Origin: you cannot sedate... all the things you hate (1:229/2)