📘 Hypothesis Testing — Introduction

It provides a scientific framework for testing claims, validating assumptions, and making evidence-based conclusions under uncertainty.

🎯 Why Hypothesis Testing is Needed

In real-world situations, we often encounter claims such as:

• A new medicine is more effective than the old one
• A teaching method improves student performance
• A manufacturing process reduces defects
• An AI model performs better than previous models

Since studying entire populations is impractical, we use sample data to evaluate whether such claims are supported by evidence.

⚖️ Decision-Making Under Uncertainty

Sample results naturally vary due to randomness.

Hypothesis testing helps answer:

🧠 Key Terminology

1️⃣ Statistical Hypothesis

A statement or claim about a population parameter.

Examples:

• The average height is 170 cm
• The defect rate is less than 2%
• The model accuracy exceeds 90%

2️⃣ Null Hypothesis (H₀)

The default assumption that there is no effect, no difference, or no change.

Examples:

• The medicine has no effect
• The average score has not changed
• The model accuracy is equal to previous performance

3️⃣ Alternative Hypothesis (H₁ or Hₐ)

The competing claim that suggests a real effect or difference exists.

Examples:

• The medicine improves recovery
• The average score has increased
• The model performs better than before

⚔️ Null vs Alternative Hypothesis

🔍 Example 1: Exam Performance

A school claims that a new teaching method increases average exam scores.

Step 1: Define Hypotheses

• H₀: The new method does not improve scores
• H₁: The new method improves scores

We collect sample data and evaluate whether evidence supports rejecting H₀.

🔍 Example 2: AI Model Accuracy

An AI company claims that a new model has higher accuracy than the previous version.

• H₀: New model accuracy = Old model accuracy
• H₁: New model accuracy > Old model accuracy

Testing determines whether the observed improvement is statistically significant.

🧪 Hypothesis Testing as a Legal Trial (Analogy)

• Null Hypothesis → Defendant is innocent
• Alternative Hypothesis → Defendant is guilty
• Evidence → Sample data
• Decision → Reject or fail to reject innocence

🎯 Outcomes of Hypothesis Testing

📊 Relationship with Confidence Intervals

Confidence intervals estimate plausible ranges for population parameters.

Hypothesis tests check whether a specific claimed value is plausible.

🤖 Importance in Machine Learning

• Comparing model performances
• A/B testing algorithms
• Validating feature importance
• Testing improvement claims
• Experimental design

🧠 Key Insights

• Hypothesis testing evaluates population claims using samples
• Null hypothesis assumes no effect
• Alternative hypothesis suggests real effect
• Evidence determines statistical decisions
• It is a framework for scientific validation