Features & Labels

1. Introduction

"Features" and "labels" are two of the most frequently used words in Machine Learning — you'll encounter them in nearly every topic from here onward. This short but essential topic makes sure you have a rock-solid understanding of exactly what these terms mean and how they're identified in a real dataset.

2. What are Features and Labels?

Simple definition: Features are the input information used to make a prediction (like a house's size and location). A label is the correct answer or output you're trying to predict (like the house's actual price).

Technical explanation: In a supervised learning dataset, features (also called independent variables or inputs, typically denoted X) are the measurable properties used by the model, while the label (also called the dependent variable, target, or output, typically denoted y) is the value the model learns to predict.

3. Why is it Important?

• Every supervised learning model you build starts with correctly identifying X (features) and y (label) in your dataset.
• Choosing the right features has a massive impact on model performance — arguably more so than the choice of algorithm itself.
• Misidentifying features or labels is one of the most common beginner mistakes when setting up a new ML project.

4. Prerequisites

Comfort with Pandas (Module 1, Topic 6) and Supervised Learning (Topic 5 of this module).

5. Core Concepts

1. Features (inputs, X)
2. Labels (output/target, y)
3. Types of features: numerical vs categorical
4. Identifying features and labels in a real dataset

6. Detailed Explanation

a) Features

Features are the pieces of information the model uses to make its prediction. In a house price dataset, features might include square footage, number of bedrooms, location, and age of the house.

b) Labels

The label is what you're actually trying to predict. In the same house price example, the label is the house's actual sale price.

c) Types of Features

• Numerical features are actual numbers (e.g., square footage, age).
• Categorical features represent categories or labels themselves (e.g., location name, house type) — these often need special encoding before being used in ML models (covered in Module 3 — Data Preprocessing).

d) Identifying Features and Labels in a Dataset

In a Pandas DataFrame, you typically select all columns except the target as your features (X), and the single target column as your label (y).

7. How It Works

1. Examine your dataset and identify which column represents the outcome you want to predict — this becomes your label (y).
2. All other relevant columns become your features (X).
3. Feed X and y together into a supervised learning algorithm during training, so the model can learn the relationship between them.

8. Real-World Example

In a dataset used to predict whether a student will pass an exam, features might include "hours studied," "attendance percentage," and "previous exam score." The label would be "pass" or "fail" — the actual outcome we want the model to predict for new students.

9. Python Example

python
import pandas as pd

data = {
    "hours_studied": [1, 3, 5, 7, 9],
    "attendance_percent": [60, 65, 80, 90, 95],
    "passed": [0, 0, 1, 1, 1]
}
df = pd.DataFrame(data)

# Separating features (X) and label (y)
X = df[["hours_studied", "attendance_percent"]]
y = df["passed"]

print("Features (X):")
print(X)
print("\nLabel (y):")
print(y)

Expected Output:

text
Features (X):
   hours_studied  attendance_percent
0              1                  60
1              3                  65
2              5                  80
3              7                  90
4              9                  95

Label (y):
0    0
1    0
2    1
3    1
4    1
Name: passed, dtype: int64

10. Code Explanation

• df[["hours_studied", "attendance_percent"]] selects two columns as the features (X) — notice the double square brackets, which select multiple columns as a DataFrame.
• df["passed"] selects the single target column as the label (y) — using single square brackets returns it as a Series.
• This X/y split is the standard first step before feeding data into any Scikit-learn supervised learning algorithm.

11. Advantages

• A clear, consistent framework (X and y) makes supervised learning code predictable and easy to follow.
• Separating features from labels makes it straightforward to swap in different algorithms without restructuring your data.

12. Limitations

• Not every dataset comes with an obvious label — sometimes deciding what should be the target requires careful thought about the actual business problem.
• Poorly chosen or irrelevant features can significantly hurt model performance, regardless of how good the algorithm is.

13. Common Mistakes

• Accidentally including the label column as one of the features, which can cause a model to "cheat" by learning from information it shouldn't have access to.
• Confusing categorical features with the label, especially in datasets with multiple category-like columns.
• Not considering whether some features might be irrelevant or redundant (explored further in Module 3 — Feature Selection).

14. Best Practices

• Always clearly define your target variable (label) before selecting features.
• Double-check that no feature column accidentally contains information that leaks the label.
• Use domain knowledge to judge whether a feature is likely to be genuinely useful for prediction.

15. Real-World Applications

• Every supervised ML project — house price prediction, spam detection, disease diagnosis — starts with defining features and labels.
• Feature selection and engineering (explored later in Modules 3 and 7) directly build on a solid understanding of this topic.

16. Interview-Oriented Points

• Be ready to clearly define "features" and "labels" using a simple example.
• Understand the common notation: X for features, y for the label/target.
• Be able to explain why feature quality matters as much as (or more than) algorithm choice.

17. Exam-Oriented Points

• Features (X) are the input variables used to make predictions.
• The label (y) is the known correct output the model is trained to predict.
• Features can be numerical or categorical.

18. Comparison Table — Features vs Labels

19. Quick Revision

• Features (X) are the inputs used to make a prediction; the label (y) is the correct output the model learns to predict.
• Features can be numerical (actual numbers) or categorical (category names).
• In Pandas, features are typically selected as multiple columns, and the label as a single target column.
• Choosing high-quality, relevant features is often just as important as the choice of ML algorithm.