In this lesson we will learn how to traverse a tree to make predictions.

At the end of the previous lesson we displayed the values of a few of the the leafs of the tree. Let's return to using the sci-kit learn library to visualize the values of all of the nodes in tree. In the below code editor we use the plot_tree function in scikit-learn's tree library to visualize the tree.

Let's go over the image. Starting with the root node, this is the rectangle at the top of the image.

• x[0] ≤ 5.55: The x[0] is the first feature in the feature set, in this case since we only have one independent variable used in fitting the tree, this is sepal length.
• Squared error: This is the model evaluation metric associated with this split value.
• Samples: The number of observations that fall into this node.
• Value: The value of the outcome variable at this node.

The other nodes have the same interpretation. The only difference is that the x[0] value is different leading to different results in the other attributes. For example, the left leaf node has x[0] ≤ 5.45, an improved squared error, 59 samples (observations), that moved from the root node to this node, and a value of 1.942. The left most leaf node has no x[0] listed as there are no further splits beyond it. The value here would be the predicted value of the outcome variable for all observations that fall within this leaf node.

Prediction with decision trees is the process of starting at the top of the tree and going down traversing the tree following the path the data unit belongs to until we reach a leaf. The value of the leaf is the prediction. And we do this for each row of the data. For example for the following five data points, we would predict using the previous tree the following values filled in to the Predicted column.

Observing that the first three observations have the same predicted value, we can see one of the limitations of decision trees, which is that with a shallow depth, we can only make a limited number of predictions. This is why in our original plot of the predicted line, we saw the jagged nature of the line, as we had made that tree with a depth of 3, leading to a maximum of 2^3=8 unique predicted values.