Random Forest

Contrary to some of the other algorithms random forest is actually fairly easy to understand. Random forest just merges a collection of independent decision trees to get a more accurate and stable prediction.
 
Random forest is a type of ensemble method. Ensemble combines several machine learning models to decrease both bias and variance. The overall idea here is that multiple models are fit and each key in the different aspect of the data. By combining these models, ultimately generate better prediction together than any of those single models on their own.
 
Let's look at a quick example. We will start with what the training looks like, and then we will look at what it is like to test the model. In training, this algorithm will take N samples from the training data. This is sampling with replacement. That means one single sample is likely to appear in multiple different samples. Then, the next step is to take a sample of features to be used for each of the data samples. In the Titanic dataset, if we have 7 features, each of these data samples, maybe we'll sample four or five of those features to be used. So now, we have N subsets of our overall data. And those subsets contain both a subset of the rows and also a subset of the columns. Then with the features and the data subset, the algorithm will build a decision tree and try to generate the most accurate results. It's important to realize that these decision trees are all developed on their own. They do not know what the other trees are doing.
 
In summary, we would have N trees that are all build independently on different subsets of data. It's important that these trees are all independent because you want each decision tree to key in on different relationships in the data. This will ultimately generate the best prediction.