Gradient boosting is a technique which produces a prediction model in the form of an ensemble of "weak" prediction models (small decision trees).

The concept is to train a set of decision trees (weak learners) to create a final strong learner. This is an iterative method. After each tree is trained, the data is reweighted: samples that were misclassified gain weight while the correctly classified ones lose weight. This allows future weak learners to focus on the "difficult" examples that the previous weak learners missed.

Gradient Boosted Trees is a generalization of boosting to arbitrary differentiable loss functions. GBT is an accurate and effective off-the-shelf procedure that can be used for both regression and classification problems. Gradient Boosted Trees models are used in a variety of areas including Web search ranking and ecology. The advantages of GBRT are:

• Natural handling of data of mixed type (= heterogeneous features)
• Predictive power
• Robustness to outliers in output space (via robust loss functions)

Due to the iterative nature of boosting, it is not very parallelizable and is less scalable than other algorithms.