From: ted.dunning@gmail.com   
      
   On Aug 13, 6:01 am, Shah  wrote:   
      
   > I am working on a project that intends to investigate the   
   > implementation of a modern heuristic  (e.g. simulated annealing,   
   > genetic algorithms or local search) to search through a space of   
   > polynomial transformations and assign selections for a linear   
   > regression.   
   >   
   > I have read that standard statistical methods for finding suitable   
   > transformations of regressors use hill-climbing algorithms to search   
   > for the correct transformations for linear modelling. I have found   
   > that alot of times techniques such as stepwise regression have been   
   > used to select a subset of regressors  using a greedy algorithm.   
   >   
   > BUT when this technique is used on a more complex model these   
   > algorithms would fail to reach a global optimum.   
      
   I have had very good results using Bayesian estimates of posterior   
   marginal likelihood as a model selection method.   
      
   The literature on Dirichlet and Gaussian processes is also very   
   exciting in this regard.  Essentially what happens in these methods is   
   that you don't actually select just a single model form.  Instead,   
   what you estimate is the distribution over all possible models and   
   then you can use this distribution to compute a distribution of   
   results when you use the regression.  This is related to the method of   
   decision forests where a distribution over decision trees is estimated   
   and the result is the probabilistic combination of the output of all   
   of the trees.   
      
   This is obviously more computationally demanding than running a single   
   model.  Potentially, the difference can be several orders of   
   magnitudes.  Since the actually running of the model is typically to   
   massively dominated by other considerations, this is rarely material.   
      
   [ comp.ai is moderated ... your article may take a while to appear. ]   
      
   --- SoupGate-Win32 v1.05   
    * Origin: you cannot sedate... all the things you hate (1:229/2)