Chapter 2: Supervised Learning Foundations

Supervised learning is one of the most used branches of machine learning when we are talking about day to day life, from filtering spam emails or calls to your tiktok for you page algorithm. In the background of all of these things supervised learning is making predictions based on previous observations.

2.1 What is Supervised Learning?

Supervised learning involves training a model on a labeled dataset, meaning each example in the training data comes with a known correct output (a label). You can think of it like teaching a child with flashcards: Show them a card (input), tell them the answer (output), repeat until they can guess the right output without help.

2.2 Types of Supervised Learning

There are two main types of supervised learning and that is regression and classification.

Regression is used when the output label is a continuous number, for example predicting the price of a house based on different attributes such as number of bedrooms, square footage, and location.

Classification is used when the output label is a category such as determining if an email is spam or not. Much like the name it classifies different types of emails into different categories (spam or not).

2.3 Training a Model

Training a model is one of the most important aspects when it comes to making a good model. In supervised learning, the goal is to find a function f that maps inputs X to outputs y as accurately as possible:

\( \hat{y} = f(X) \)

The process of training a model:

  • Feed the model a dataset of inputs and known outputs.
  • The model makes predictions.
  • Compare the predictions with the true outputs using a loss function.
  • Adjust the model to reduce the loss.
  • Repeat until the model performs well.

We will go over these steps in more detail in future chapters, but for now it is important to understand the general process of training a model.

Example: Let’s say that we are trying to predict the price of a house which has 3 bedrooms, 1500 sqft, and has a location score of 7. The model that we have gives us a prediction price of $475,000. Now we compare it to the actual price of $450,000 and then we begin to adjust the model and features accordingly to get as close to the actual price as possible.

2.4 How Do We Know If the Model Is Good?

We use a few key steps:

2.4.1 Train-Test Split

We divide our dataset:

  • Training set (e.g. 80% of the data): to train the model.
  • Test set (e.g. 20% of the data): to see how well the model performs on unseen data.

This helps prevent overfitting (when a model memorizes the training data but fails on new data).

2.4.2 Evaluation Metrics

For regression:

  • Mean Squared Error (MSE): Average of squared prediction errors.
  • Mean Absolute Error (MAE): Average of absolute differences.
  • R-squared (R^2): Proportion of variance explained by the model.

For classification:

  • Accuracy
  • Precision
  • Recall
  • F1 Score

These words and phrases may sound like a lot right now, but as we dive deeper into the chapters and models you will understand all of these phrases and be able to work with them in order to create your own model.

2.5 Linking Back to Regression

If you haven't yet, check out our earlier Regression Chapter to learn how supervised learning begins with ideas you may already know: drawing lines through points to predict outcomes. There, we explain: What linear regression is, how predictions work using a simple equation, and how to evaluate your regression model with training and test subsets.

Next Chapter →