In this lesson we begin learning an alternative modeling strategy to linear regression called decision trees. We will use this modelling strategy in the same way we used linear regression to make the comparison between the two more clear.

Decision trees like linear regression use a set of independent variables to predict a dependent variable. However, unlike linear regression, decision trees are not based on a mathematical equation. Instead, they are based on a series of if-then statements called nodes, that are used to conditionally split the data into smaller and smaller groups creating a branch structure with final nodes called leaves. The final result is a tree-like structure that can be used to make predictions. Let's take a look at these predictions on the iris dataset just as we did with linear regression, before going over the details of how decision trees work.

We can see that the decision tree fit the data in a much different way from the linear regression model. Instead of forming a straight line, the decision tree is a series of horizontal and vertical lines. By the end of this lessen we will understand why this predicted line takes such an odd shape. So let's dig into the details by going over the general structure of a decision tree and its main components.

Formulating the decision tree

A decision tree is composed of three things: nodes, branches, and leaves. The following decision tree starts with a node labelled 1, since it is the first node it is called the root node, it has two children nodes, 2 and 3 with the line connecting them the branches. Then continuing, node 2 has two children nodes, 4 and 5, since these nodes have no children they are the leaves. Node 3 has one child node, 6, with 6 being the final leaf in the decision tree.

Let's go over the components of a decision tree in the next few sections.

The root node

The root node, node 1 at the top of the tree, is the starting point of our decision tree. It represents the entire dataset, containing all the instances that we aim to analyze or predict. The root node is critical because it makes the first decision, effectively initiating the tree's branching process. This initial decision splits the data based on a variable from our dataset that best separates the data into two groups, maximizing the difference between these groups according to a criterion specific on the decision tree algorithm in use.

Intermediary nodes

Intermediary nodes, nodes 2 and 3, situated between the root and the leaves, represent the subsequent decisions made after the initial split at the root. Each intermediary node evaluates either the same or another variable, further subdividing the dataset into more homogeneous groups.

Branches

The branches, the lines connecting the nodes, in a decision tree represent the decision rules or conditions that lead from one node to another. Each branch corresponds to an outcome of the decision made at a node. In essence, they are the paths that connect the nodes and illustrate the flow of decisions. The branches guide the movement from the root to the leaves, dictating the path followed based on the specific features of the data point being considered.

Leaves

The leaves, nodes 4,5, and 6, also called terminal nodes, represent the final outcomes or predictions of the decision tree. Data that falls within a leaf node is groups the data in as a close way as it can. In regression tasks, the value of the dependent variable at these points becomes the predicted values.

The code that produced the plot

To conclude this lesson let's go over the code line-by-line that produced the plot. Note that the libraries we imported are hidden in the code below, if you want to see what they are, check out the Code environment drop-down menu at the top of the page.

• Line 2: This plots the iris dataset with sepal length on the x-axis and petal length on the y-axis.
• Lines 4 & 5: Collect the x and y values into their own variables.
• Line 7: Create a decision tree object.
• Line 8: Fit the decision tree using its fit method.
• Line 9: Add the predicted values to the dataframe using the decision tree's predict method.
• Lines 12: We add the predicted fit line to the plot.