From: ted.dunning@gmail.com
On Nov 2, 3:50 pm, ItChy^D wrote:
> hi, i'm an informatics student that doing reseach about boosting
> algorithm for my final project, i read many paper about variant of
> boosting algorithm especially AdaBoost, but i'm getting confused about
> example that can be reweighting or resampling in the next round that
> depends on error that the example got.
> my questions is:
> 1. what is the meaning of reweighting? and is there any method for
> reweighting?
> 2. what kind of algorithm that can used weight for its training,
> because in WEKA, when i'm using AdaBoost.M1 and decision stumps for
> its weak learner, Decision Stumps can received weight for its
> training, i think decision stumps only use entropy calculation for its
> output (hypothesis), so how come decision stumps use weight in the
> training process? or i'm wrong?
> could anyone help me? thx... (btw, sorry if my english isn't good)
>
You should also be looking at random forests. Random forests are an
extension and simplification of tree boosting and bagging techniques.
They should provide a very interesting contrast with the boosting
techniques you are looking at.
The basic intuition behind reweighting is to increase the trust that
the model puts into trees that seem to work well. This makes sense,
but the results from random forests where there is no boosting at all
makes the quality of this intuition suspect.
Also, I would recommend using R for these experiments. Weka has a
very idiosyncratic interface while R is a full blown language that is
relatively simple to learn. More importantly, R is much more the
focus of the statistical community when it comes to porting
interesting algorithms. For instance, R has a port of random forests,
but I don't think that weka has one yet.
[ comp.ai is moderated ... your article may take a while to appear. ]
--- SoupGate-Win32 v1.05
* Origin: you cannot sedate... all the things you hate (1:229/2)
|
