Basic Statistics

1. Introduction

Machine learning is, at its core, built on statistics — it's how we summarize data, understand patterns, and make sense of numbers before feeding them into any model. Before touching ML algorithms, you need to be comfortable with a handful of basic statistical concepts: measures that describe the "center" of data and measures that describe how "spread out" it is.

2. What is Basic Statistics?

Simple definition: Basic statistics refers to simple numerical tools used to summarize and describe a set of data — such as its average value, its middle value, and how spread out the values are.

Technical explanation: Descriptive statistics involves measures of central tendency (mean, median, mode) that summarize the "typical" value in a dataset, and measures of dispersion (range, variance, standard deviation) that describe how much the data values vary from that typical value.

3. Why is it Important?

• ML algorithms rely heavily on statistical summaries to understand and process data (e.g., scaling features using mean and standard deviation).
• Statistics help identify unusual values (outliers) and understand the overall "shape" of a dataset before modeling.
• Nearly every evaluation metric in ML (like MAE, MSE, R²) is built on statistical foundations.

4. Prerequisites

Basic arithmetic and comfort with Python basics and NumPy (helpful for calculations), though this topic can also be understood using plain math.

5. Core Concepts

1. Mean (average)
2. Median (middle value)
3. Mode (most frequent value)
4. Range
5. Variance
6. Standard deviation

6. Detailed Explanation

a) Mean

The mean is the sum of all values divided by the number of values — the everyday "average."

b) Median

The median is the middle value when data is arranged in order. If there's an even number of values, it's the average of the two middle ones. The median is useful because, unlike the mean, it isn't heavily affected by extreme outliers.

c) Mode

The mode is the value that appears most frequently in a dataset. A dataset can have one mode, multiple modes, or no mode at all (if every value is unique).

d) Range

The range is the difference between the maximum and minimum values — a very simple measure of spread.

e) Variance

Variance measures how far, on average, each data point is from the mean — but in squared units, which makes it harder to interpret directly.

f) Standard Deviation

Standard deviation is simply the square root of variance, bringing the measure of spread back into the same units as the original data — making it much easier to interpret. A small standard deviation means values are clustered close to the mean; a large one means they're spread out widely.

7. How It Works

To calculate variance and standard deviation step by step:

1. Calculate the mean of the dataset.
2. Subtract the mean from each value, then square the result (this removes negative signs and emphasizes larger differences).
3. Average all these squared differences — this gives the variance.
4. Take the square root of the variance to get the standard deviation.

8. Real-World Example

Imagine two cricket batsmen who both average 50 runs per match (same mean). But Batsman A consistently scores between 45–55 runs each match, while Batsman B sometimes scores 0 and sometimes scores 100. Even though their averages (means) are identical, Batsman B has a much higher standard deviation — his performance is far less consistent.

9. Technical Example

For the dataset [2, 4, 4, 4, 5, 5, 7, 9]:

• Mean = (2+4+4+4+5+5+7+9) / 8 = 40 / 8 = 5
• Variance = average of squared differences from the mean = 4
• Standard Deviation = √4 = 2

10. Mathematical Explanation

Mean Formula:

Mean (μ) = (Σx) / n

Where:

• Σx = sum of all values
• n = total number of values

Variance Formula (Population):

Variance (σ²) = Σ(x − μ)² / n

Where:

• x = each individual value
• μ = the mean
• n = total number of values

Standard Deviation Formula:

Standard Deviation (σ) = √(Variance)

Numerical Example:

Dataset: [2, 4, 4, 4, 5, 5, 7, 9]

1. Mean = 40 / 8 = 5
2. Squared differences from mean: (2-5)²=9, (4-5)²=1, (4-5)²=1, (4-5)²=1, (5-5)²=0, (5-5)²=0, (7-5)²=4, (9-5)²=16
3. Sum of squared differences = 9+1+1+1+0+0+4+16 = 32
4. Variance = 32 / 8 = 4
5. Standard Deviation = √4 = 2

Interpreting the Result: A standard deviation of 2 means that, on average, each value in this dataset is about 2 units away from the mean of 5 — indicating the data is moderately close to the average value.

11. Python Example

python
import numpy as np
from statistics import mode

data = [2, 4, 4, 4, 5, 5, 7, 9]

mean_value = np.mean(data)
median_value = np.median(data)
mode_value = mode(data)
range_value = max(data) - min(data)
variance_value = np.var(data)
std_dev_value = np.std(data)

print("Data:", data)
print("Mean:", mean_value)
print("Median:", median_value)
print("Mode:", mode_value)
print("Range:", range_value)
print("Variance:", variance_value)
print("Standard Deviation:", std_dev_value)

Expected Output:

text
Data: [2, 4, 4, 4, 5, 5, 7, 9]
Mean: 5.0
Median: 4.5
Mode: 4
Range: 7
Variance: 4.0
Standard Deviation: 2.0

12. Code Explanation

• np.mean(data) calculates the average of all values.
• np.median(data) sorts the data internally and finds the middle value (or the average of the two middle values, since there are 8 values here).
• mode(data) (from Python's built-in statistics module) returns the most frequently occurring value — here, 4 appears three times.
• max(data) - min(data) calculates the range directly.
• np.var(data) and np.std(data) compute variance and standard deviation using NumPy's built-in, optimized functions, matching our manual calculation above.

13. Advantages

• Provides quick, meaningful summaries of large datasets.
• Helps detect unusual data points (outliers) before modeling.
• Forms the mathematical foundation for many ML algorithms and evaluation metrics.

14. Limitations

• Mean is heavily influenced by extreme outliers, which can give a misleading picture.
• A single statistic (like mean or standard deviation) can hide important details about the actual shape of the data.
• Mode can be unhelpful or ambiguous when there are multiple equally frequent values.

15. Common Mistakes

• Assuming mean and median will always be close — they can differ significantly with skewed data.
• Forgetting that variance is in "squared units," making standard deviation the more interpretable measure.
• Confusing population variance/standard deviation (dividing by n) with sample variance/standard deviation (dividing by n-1) — these are related but slightly different formulas.
• Ignoring outliers that may be distorting the mean.

16. Best Practices

• Always look at both the mean and median together to spot potential skew or outliers.
• Use standard deviation (not variance) when you need an easily interpretable measure of spread.
• Visualize data (e.g., with a histogram) alongside statistical summaries for a fuller picture.
• Be aware of which variance formula (population vs sample) is appropriate for your context.

17. Real-World Applications

• Feature scaling in ML (standardization) uses mean and standard deviation directly.
• Detecting outliers in datasets before training a model.
• Summarizing survey results, exam scores, or sensor readings for quick insights.

18. Interview-Oriented Points

• Be ready to explain why median is more "robust" to outliers than mean.
• Understand the relationship between variance and standard deviation.
• Know the formulas for mean, variance, and standard deviation, and be able to calculate them manually for a small dataset.
• Be able to explain a real-world scenario where mean and median would differ significantly.

19. Exam-Oriented Points

• Mean = average; Median = middle value; Mode = most frequent value.
• Range = maximum − minimum.
• Variance = average of squared differences from the mean.
• Standard deviation = square root of variance.

20. Comparison Table — Mean vs Median vs Mode

21. Quick Revision

• Mean, median, and mode all describe the "center" of a dataset, but behave differently with outliers.
• Range, variance, and standard deviation describe how spread out the data is.
• Variance is in squared units; standard deviation converts it back to the original units, making it easier to interpret.
• A small standard deviation means data is tightly clustered around the mean; a large one means it's widely spread.