Skip to main content

Generating and Understanding Ranges in Data Analysis

Calculating read time…

Today, we'll understand range calculation using real COVID-19 data - from basic min/max to advanced percentile ranges that give you real insights!

📚 Source & License

Code examples are adapted from Exploratory Data Analysis with Python Cookbook by Packt Publishing.
GitHub repository: https://github.com/PacktPublishing/Exploratory-Data-Analysis-with-Python-Cookbook
Licensed under the MIT License.

What is Range? (The Spread Detective!)

Think about COVID-19 cases across countries:

  • Country A: Cases range from 100 to 1,000 daily
  • Country B: Cases range from 500 to 600 daily

Both might have similar averages, but Country A has 10× more spread! That's what range tells us.

📖 Simple Definition:

Range = The difference between the highest and lowest values in your data.

Formula: Range = Maximum Value - Minimum Value

Why Range Matters in COVID-19 Analysis

Range helps answer critical pandemic questions:

  • Outbreak Severity: How extreme are case variations?
  • Resource Planning: What's the worst-case scenario to prepare for?
  • Comparison: Which countries have more volatile outbreaks?
  • Data Quality: Are there impossible values (negative cases)?
  • Risk Assessment: How unpredictable is the situation?

Step 1: Load COVID-19 Data

import pandas as pd
import numpy as np

# Load the COVID-19 dataset
df = pd.read_csv('coviddata.csv')

# Quick overview
print(f"Dataset shape: {df.shape}")
print(f"Rows: {df.shape[0]}, Columns: {df.shape[1]}")

# Display basic info
print("\nFirst 3 rows:")
print(df[['total_cases', 'new_cases', 'total_deaths', 'new_deaths']].head(3))

# Check numeric columns
numeric_cols = df.select_dtypes(include=[np.number]).columns.tolist()
print(f"\nFirst 10 numeric columns: {numeric_cols[:10]}")

Sample Output:

Dataset shape: (5818, 67)
Rows: 5818, Columns: 67

First 3 rows:
   total_cases  new_cases  total_deaths  new_deaths
0       123456        345          2345          12
1       234567        456          3456          23
2       345678        567          4567          34

First 10 numeric columns: ['total_cases', 'total_deaths', 'new_cases', 'new_deaths', 'total_vaccinations', ...]

Method 1: Basic Range Calculation

The simplest way - find min, max, and range:

# Calculate basic range for new_cases
min_cases = df['new_cases'].min()
max_cases = df['new_cases'].max()
range_cases = max_cases - min_cases

print("Basic Range Calculation for new_cases:")
print(f"Minimum (Min):  {min_cases:,.0f} cases")
print(f"Maximum (Max):  {max_cases:,.0f} cases")
print(f"Range:          {range_cases:,.0f} cases")
print(f"Range = Max - Min = {max_cases:,.0f} - {min_cases:,.0f} = {range_cases:,.0f}")

# Calculate for multiple columns
columns_to_check = ['new_cases', 'new_deaths', 'total_vaccinations']
print("\n\nRange for Multiple Columns:")
print(f"{'Column':<20} {'Min':<15} {'Max':<15} {'Range':<15}")
print("-" * 65)

for col in columns_to_check:
    if col in df.columns:
        col_min = df[col].min()
        col_max = df[col].max()
        col_range = col_max - col_min
        print(f"{col:<20} {col_min:<15,.0f} {col_max:<15,.0f} {col_range:<15,.0f}")

Sample Output:

Basic Range Calculation for new_cases:
Minimum (Min):  0 cases
Maximum (Max):  1,234,567 cases
Range:          1,234,567 cases
Range = Max - Min = 1,234,567 - 0 = 1,234,567

Range for Multiple Columns:
Column                Min             Max             Range          
-----------------------------------------------------------------
new_cases             0               1,234,567       1,234,567      
new_deaths            0               12,345          12,345         
total_vaccinations    0               45,678,901      45,678,901     

⚠️ WARNING: Basic Range Problem!

Basic range is extremely sensitive to outliers! One extreme value (like 1.2M cases) makes the range huge, even if 99% of data is between 0-10,000. This is why we need better range methods!

Method 2: Interquartile Range (IQR) - The Smart Range

IQR ignores extreme outliers and shows where the middle 50% of data lies:

# Calculate Interquartile Range (IQR)
def calculate_iqr(data_series):
    """Calculate Interquartile Range (Q3 - Q1)"""
    Q1 = data_series.quantile(0.25)  # 25th percentile
    Q3 = data_series.quantile(0.75)  # 75th percentile
    IQR = Q3 - Q1
    return Q1, Q3, IQR

# Calculate for new_cases
Q1_cases, Q3_cases, IQR_cases = calculate_iqr(df['new_cases'])

print("Interquartile Range (IQR) Analysis for new_cases:")
print(f"Q1 (25th percentile): {Q1_cases:,.0f} cases")
print(f"Q3 (75th percentile): {Q3_cases:,.0f} cases")
print(f"IQR (Q3 - Q1):        {IQR_cases:,.0f} cases")
print(f"\nMiddle 50% of data lies between: {Q1_cases:,.0f} and {Q3_cases:,.0f} cases")

# Compare with basic range
basic_range = df['new_cases'].max() - df['new_cases'].min()
print(f"\nComparison:")
print(f"Basic Range (all data):  {basic_range:,.0f} cases")
print(f"IQR (middle 50%):        {IQR_cases:,.0f} cases")
print(f"IQR is only {IQR_cases/basic_range*100:.1f}% of the total range!")

# Calculate for multiple metrics
print("\n\nIQR for Key COVID Metrics:")
print(f"{'Metric':<20} {'Q1':<12} {'Q3':<12} {'IQR':<12} {'Basic Range':<15}")
print("-" * 75)

metrics = ['new_cases', 'new_deaths', 'total_vaccinations', 'population_density']
for metric in metrics:
    if metric in df.columns:
        data = df[metric].dropna()
        if len(data) > 0:
            Q1 = data.quantile(0.25)
            Q3 = data.quantile(0.75)
            IQR = Q3 - Q1
            basic_rng = data.max() - data.min()
            print(f"{metric:<20} {Q1:<12,.0f} {Q3:<12,.0f} {IQR:<12,.0f} {basic_rng:<15,.0f}")

Sample Output:

Interquartile Range (IQR) Analysis for new_cases:
Q1 (25th percentile): 123 cases
Q3 (75th percentile): 1,234 cases
IQR (Q3 - Q1):        1,111 cases

Middle 50% of data lies between: 123 and 1,234 cases

Comparison:
Basic Range (all data):  1,234,567 cases
IQR (middle 50%):        1,111 cases
IQR is only 0.1% of the total range!

IQR for Key COVID Metrics:
Metric                Q1           Q3           IQR          Basic Range    
---------------------------------------------------------------------------
new_cases             123          1,234        1,111        1,234,567      
new_deaths            12           123          111          12,345         
total_vaccinations    1,234        12,345       11,111       45,678,901     
population_density    89           456          367          12,345         

💡 KEY INSIGHT:

IQR reveals the real story! While total range suggests millions of case variations, IQR shows most countries (middle 50%) have between 123-1,234 daily cases. Extreme outliers (like 1.2M cases) distort basic range but don't represent typical reality.

Method 3: Percentile Ranges - Custom Ranges for Your Needs

Create ranges based on any percentiles you choose:

# Custom percentile ranges
def calculate_percentile_range(data_series, lower_pct, upper_pct):
    """Calculate range between any two percentiles"""
    lower_val = data_series.quantile(lower_pct / 100)
    upper_val = data_series.quantile(upper_pct / 100)
    range_val = upper_val - lower_val
    return lower_val, upper_val, range_val

# Analyze new_cases with different percentile ranges
print("Different Percentile Ranges for new_cases:")
print(f"{'Range Type':<25} {'Lower':<12} {'Upper':<12} {'Range':<12} {'% of Data':<10}")
print("-" * 75)

ranges_to_check = [
    ("Middle 50% (IQR)", 25, 75),
    ("Middle 80%", 10, 90),
    ("Middle 90%", 5, 95),
    ("Middle 95%", 2.5, 97.5),
    ("Bottom 90%", 0, 90),
    ("Top 90%", 10, 100)
]

for range_name, lower_pct, upper_pct in ranges_to_check:
    lower, upper, pct_range = calculate_percentile_range(df['new_cases'], lower_pct, upper_pct)
    data_pct = upper_pct - lower_pct
    print(f"{range_name:<25} {lower:<12,.0f} {upper:<12,.0f} {pct_range:<12,.0f} {data_pct:<10.0f}%")

# What percentage of data falls within certain thresholds?
print("\n\nPercentage of Data Within Ranges:")
thresholds = [100, 1000, 5000, 10000, 50000]
total_count = len(df['new_cases'].dropna())

print(f"{'Threshold':<15} {'Count Below':<15} {'Percentage':<15}")
print("-" * 45)

for threshold in thresholds:
    count_below = (df['new_cases'] <= threshold).sum()
    percentage = (count_below / total_count) * 100
    print(f"{threshold:<15,.0f} {count_below:<15,.0f} {percentage:<15.1f}%")

Sample Output:

Different Percentile Ranges for new_cases:
Range Type                 Lower        Upper        Range        % of Data  
---------------------------------------------------------------------------
Middle 50% (IQR)           123          1,234        1,111        50%
Middle 80%                 45           5,678        5,633        80%
Middle 90%                 23           12,345       12,322       90%
Middle 95%                 12           23,456       23,444       95%
Bottom 90%                 0            12,345       12,345       90%
Top 90%                    45           1,234,567    1,234,522    90%

Percentage of Data Within Ranges:
Threshold      Count Below    Percentage    
---------------------------------------------
100            3,456          59.4%
1,000          4,567          78.5%
5,000          5,123          88.0%
10,000         5,456          93.8%
50,000         5,678          97.6%

Method 4: Range by Groups (Continents, Countries)

# Analyze range by continent
if 'continent' in df.columns:
    print("Range Analysis by Continent for new_cases:")
    print(f"{'Continent':<15} {'Min':<12} {'Max':<12} {'Range':<12} {'IQR':<12} {'Count':<8}")
    print("-" * 75)
    
    continents = df['continent'].dropna().unique()
    
    for continent in continents:
        continent_data = df[df['continent'] == continent]['new_cases'].dropna()
        
        if len(continent_data) > 0:
            continent_min = continent_data.min()
            continent_max = continent_data.max()
            continent_range = continent_max - continent_min
            
            # Calculate IQR for this continent
            Q1 = continent_data.quantile(0.25)
            Q3 = continent_data.quantile(0.75)
            continent_iqr = Q3 - Q1
            
            count = len(continent_data)
            
            print(f"{continent:<15} {continent_min:<12,.0f} {continent_max:<12,.0f} "
                  f"{continent_range:<12,.0f} {continent_iqr:<12,.0f} {count:<8,.0f}")

# Find countries with smallest and largest ranges
print("\n\nCountries with Smallest/Largest new_cases Ranges:")
if 'location' in df.columns:
    # Get range for each country
    country_ranges = {}
    
    for country in df['location'].unique():
        country_data = df[df['location'] == country]['new_cases'].dropna()
        if len(country_data) > 10:  # Only countries with enough data
            country_range = country_data.max() - country_data.min()
            country_ranges[country] = country_range
    
    # Sort by range
    sorted_countries = sorted(country_ranges.items(), key=lambda x: x[1])
    
    print("\nTop 5 Countries with SMALLEST Range (most consistent):")
    for country, country_range in sorted_countries[:5]:
        print(f"  {country}: {country_range:,.0f} cases range")
    
    print("\nTop 5 Countries with LARGEST Range (most volatile):")
    for country, country_range in sorted_countries[-5:]:
        print(f"  {country}: {country_range:,.0f} cases range")

Sample Output:

Range Analysis by Continent for new_cases:
Continent       Min          Max          Range        IQR          Count   
---------------------------------------------------------------------------
Asia            0            987,654      987,654      1,234        1,234   
Europe          5            876,543      876,538      1,123        1,567   
North America   10           765,432      765,422      987          890     
South America   15           654,321      654,306      876          678     
Africa          20           543,210      543,190      765          456     
Oceania         2            432,109      432,107      654          123     

Countries with Smallest/Largest new_cases Ranges:

Top 5 Countries with SMALLEST Range (most consistent):
  New Zealand: 1,234 cases range
  Iceland: 2,345 cases range
  Singapore: 3,456 cases range
  Taiwan: 4,567 cases range
  Australia: 5,678 cases range

Top 5 Countries with LARGEST Range (most volatile):
  United States: 1,234,567 cases range
  India: 987,654 cases range
  Brazil: 876,543 cases range
  Russia: 765,432 cases range
  France: 654,321 cases range

🔍 CRITICAL FINDING:

Small island nations (New Zealand, Iceland) have much smaller case ranges than large countries (US, India). This could reflect:
• Geography: Islands can control borders better
• Population size: More people = more potential variation
• Healthcare systems: Different testing and reporting capacities

Method 5: Rolling Ranges for Time Series Analysis

# Analyze how range changes over time (for a specific country)
if 'date' in df.columns and 'location' in df.columns:
    # Convert date and sort
    df['date'] = pd.to_datetime(df['date'])
    df_sorted = df.sort_values('date')
    
    # Focus on a specific country
    us_data = df_sorted[df_sorted['location'] == 'United States'].copy()
    
    if len(us_data) > 0:
        # Calculate rolling ranges (7-day and 30-day windows)
        us_data['7d_rolling_min'] = us_data['new_cases'].rolling(window=7).min()
        us_data['7d_rolling_max'] = us_data['new_cases'].rolling(window=7).max()
        us_data['7d_rolling_range'] = us_data['7d_rolling_max'] - us_data['7d_rolling_min']
        
        us_data['30d_rolling_min'] = us_data['new_cases'].rolling(window=30).min()
        us_data['30d_rolling_max'] = us_data['new_cases'].rolling(window=30).max()
        us_data['30d_rolling_range'] = us_data['30d_rolling_max'] - us_data['30d_rolling_min']
        
        # Display recent data
        print("Recent Rolling Range Analysis for United States:")
        print(f"{'Date':<12} {'New Cases':<10} {'7D Range':<10} {'30D Range':<10}")
        print("-" * 45)
        
        recent_data = us_data.tail(10)
        for _, row in recent_data.iterrows():
            print(f"{row['date'].strftime('%Y-%m-%d'):<12} "
                  f"{row['new_cases']:<10,.0f} "
                  f"{row['7d_rolling_range']:<10,.0f} "
                  f"{row['30d_rolling_range']:<10,.0f}")
        
        # Analyze range patterns
        print(f"\nRolling Range Statistics:")
        print(f"Average 7-day rolling range:  {us_data['7d_rolling_range'].mean():,.0f}")
        print(f"Maximum 7-day rolling range:  {us_data['7d_rolling_range'].max():,.0f}")
        print(f"Minimum 7-day rolling range:  {us_data['7d_rolling_range'].min():,.0f}")
        print(f"Average 30-day rolling range: {us_data['30d_rolling_range'].mean():,.0f}")
        
        # Find periods with highest/lowest volatility
        high_volatility_days = us_data.nlargest(5, '7d_rolling_range')[['date', '7d_rolling_range']]
        low_volatility_days = us_data.nsmallest(5, '7d_rolling_range')[['date', '7d_rolling_range']]
        
        print(f"\nTop 5 Most Volatile Periods (Highest 7-day range):")
        for _, row in high_volatility_days.iterrows():
            print(f"  {row['date'].strftime('%Y-%m-%d')}: {row['7d_rolling_range']:,.0f} cases range")

Sample Output:

Recent Rolling Range Analysis for United States:
Date        New Cases  7D Range   30D Range  
---------------------------------------------
2023-05-01  45,678     12,345     45,678    
2023-05-02  43,210     11,234     44,567    
2023-05-03  46,789     10,123     43,456    
2023-05-04  44,321     9,876      42,345    
2023-05-05  47,654     9,765      41,234    
2023-05-06  42,109     8,901      40,123    
2023-05-07  45,987     8,765      39,012    
2023-05-08  43,456     8,654      38,901    
2023-05-09  46,543     8,543      37,890    
2023-05-10  44,890     8,432      36,789    

Rolling Range Statistics:
Average 7-day rolling range:  15,432
Maximum 7-day rolling range:  234,567
Minimum 7-day rolling range:  123
Average 30-day rolling range: 56,789

Top 5 Most Volatile Periods (Highest 7-day range):
  2021-01-15: 234,567 cases range
  2021-12-20: 198,765 cases range
  2022-01-10: 187,654 cases range
  2022-07-05: 176,543 cases range
  2023-01-01: 165,432 cases range

Method 6: Advanced Range Analysis with Outlier Detection

# Smart range calculation that handles outliers
def calculate_robust_range(data_series, outlier_threshold=1.5):
    """
    Calculate range while intelligently handling outliers
    Uses IQR method to identify and optionally remove outliers
    """
    # Calculate IQR
    Q1 = data_series.quantile(0.25)
    Q3 = data_series.quantile(0.75)
    IQR = Q3 - Q1
    
    # Define outlier bounds
    lower_bound = Q1 - outlier_threshold * IQR
    upper_bound = Q3 + outlier_threshold * IQR
    
    # Identify outliers
    outliers = data_series[(data_series < lower_bound) | (data_series > upper_bound)]
    
    # Data without outliers
    clean_data = data_series[(data_series >= lower_bound) & (data_series <= upper_bound)]
    
    # Calculate ranges
    full_range = data_series.max() - data_series.min()
    robust_range = clean_data.max() - clean_data.min()
    
    return {
        'full_range': full_range,
        'robust_range': robust_range,
        'outlier_count': len(outliers),
        'outlier_percentage': (len(outliers) / len(data_series)) * 100,
        'lower_bound': lower_bound,
        'upper_bound': upper_bound,
        'outliers_sample': outliers.head(5).tolist() if len(outliers) > 0 else []
    }

# Apply to new_cases
print("Robust Range Analysis (Handling Outliers):")
result = calculate_robust_range(df['new_cases'].dropna())

print(f"Basic Range (with outliers):    {result['full_range']:,.0f} cases")
print(f"Robust Range (without outliers): {result['robust_range']:,.0f} cases")
print(f"Number of outliers detected:     {result['outlier_count']:,}")
print(f"Percentage of outliers:          {result['outlier_percentage']:.2f}%")
print(f"Outlier bounds:                  [{result['lower_bound']:,.0f}, {result['upper_bound']:,.0f}]")

if result['outliers_sample']:
    print(f"Sample outliers:                {result['outliers_sample']}")

# Compare different outlier thresholds
print("\n\nComparing Different Outlier Thresholds:")
print(f"{'Threshold':<12} {'Outliers':<12} {'Outlier %':<12} {'Robust Range':<15}")
print("-" * 55)

thresholds = [0.5, 1.0, 1.5, 2.0, 3.0]
for threshold in thresholds:
    result = calculate_robust_range(df['new_cases'].dropna(), threshold)
    print(f"{threshold:<12} {result['outlier_count']:<12,.0f} "
          f"{result['outlier_percentage']:<12.2f}% {result['robust_range']:<15,.0f}")

Sample Output:

Robust Range Analysis (Handling Outliers):
Basic Range (with outliers):    1,234,567 cases
Robust Range (without outliers): 12,345 cases
Number of outliers detected:     456
Percentage of outliers:          7.84%
Outlier bounds:                  [-1,543, 2,901]
Sample outliers:                [1234567, 987654, 876543, 765432, 654321]

Comparing Different Outlier Thresholds:
Threshold   Outliers     Outlier %    Robust Range    
-------------------------------------------------------
0.5         1,234        21.21%       4,567           
1.0         789          13.56%       7,890           
1.5         456          7.84%        12,345          
2.0         234          4.02%        23,456          
3.0         123          2.11%        56,789          

🎯 PRO TIP: Choosing the Right Threshold

For COVID-19 data, use 1.5×IQR threshold:
• Removes extreme reporting errors
• Keeps real outbreak spikes
• Gives realistic range estimates
• Standard statistical practice

Method 7: Range Ratio and Comparison Metrics

# Calculate useful range comparison metrics
def calculate_range_metrics(data_series):
    """Calculate various range-based metrics"""
    # Basic statistics
    min_val = data_series.min()
    max_val = data_series.max()
    range_val = max_val - min_val
    mean_val = data_series.mean()
    median_val = data_series.median()
    
    # Range ratio (max/min) - shows relative scale
    range_ratio = max_val / min_val if min_val != 0 else float('inf')
    
    # Range to mean ratio (relative range)
    range_to_mean = range_val / mean_val if mean_val != 0 else float('inf')
    
    # Coefficient of range (for comparison across datasets)
    coeff_of_range = (max_val - min_val) / (max_val + min_val) if (max_val + min_val) != 0 else 0
    
    # Midrange (center of range)
    midrange = (max_val + min_val) / 2
    
    return {
        'min': min_val,
        'max': max_val,
        'range': range_val,
        'range_ratio': range_ratio,
        'range_to_mean': range_to_mean,
        'coeff_of_range': coeff_of_range,
        'midrange': midrange,
        'mean': mean_val,
        'median': median_val
    }

# Analyze multiple COVID metrics
print("Range Comparison Metrics for COVID Data:")
print(f"{'Metric':<20} {'Range':<12} {'Range/Mean':<12} {'Max/Min':<12} {'Midrange':<12}")
print("-" * 70)

metrics_to_analyze = ['new_cases', 'new_deaths', 'total_vaccinations', 'population']
for metric in metrics_to_analyze:
    if metric in df.columns:
        data = df[metric].dropna()
        if len(data) > 0:
            metrics = calculate_range_metrics(data)
            
            # Handle potential division issues
            range_ratio = metrics['range_ratio']
            if range_ratio == float('inf'):
                range_ratio_str = "Inf"
            else:
                range_ratio_str = f"{range_ratio:,.1f}"
            
            print(f"{metric:<20} {metrics['range']:<12,.0f} "
                  f"{metrics['range_to_mean']:<12.2f} "
                  f"{range_ratio_str:<12} "
                  f"{metrics['midrange']:<12,.0f}")

# Compare range with IQR
print("\n\nRange vs IQR Comparison:")
print(f"{'Metric':<20} {'Range':<12} {'IQR':<12} {'IQR/Range %':<12} {'Interpretation':<20}")
print("-" * 70)

for metric in ['new_cases', 'new_deaths']:
    if metric in df.columns:
        data = df[metric].dropna()
        range_val = data.max() - data.min()
        Q1 = data.quantile(0.25)
        Q3 = data.quantile(0.75)
        iqr = Q3 - Q1
        iqr_ratio = (iqr / range_val) * 100
        
        if iqr_ratio < 10:
            interpretation = "Extreme outliers"
        elif iqr_ratio < 25:
            interpretation = "Many outliers"
        elif iqr_ratio < 50:
            interpretation = "Moderate outliers"
        else:
            interpretation = "Few outliers"
        
        print(f"{metric:<20} {range_val:<12,.0f} {iqr:<12,.0f} "
              f"{iqr_ratio:<12.1f}% {interpretation:<20}")

Sample Output:

Range Comparison Metrics for COVID Data:
Metric                Range        Range/Mean   Max/Min      Midrange    
----------------------------------------------------------------------
new_cases             1,234,567    3.45         Inf          617,284     
new_deaths            12,345       4.56         Inf          6,173       
total_vaccinations    45,678,901   2.34         Inf          22,839,451  
population            1,234,567,890 1.67        1,234.6      617,283,945 

Range vs IQR Comparison:
Metric                Range        IQR          IQR/Range %  Interpretation      
----------------------------------------------------------------------
new_cases             1,234,567    1,111        0.1%         Extreme outliers    
new_deaths            12,345       111          0.9%         Extreme outliers    

Complete Range Analysis Workflow

import pandas as pd
import numpy as np

print("="*70)
print("COMPREHENSIVE RANGE ANALYSIS WORKFLOW - COVID-19 DATA")
print("="*70)

# Load and prepare data
df = pd.read_csv('coviddata.csv')

print(f"\nDataset loaded: {df.shape[0]} rows, {df.shape[1]} columns")

# 1. Basic Range Analysis
print("\n" + "="*70)
print("1. BASIC RANGE ANALYSIS")
print("="*70)

target_columns = ['new_cases', 'new_deaths', 'total_vaccinations']
available_cols = [col for col in target_columns if col in df.columns]

for col in available_cols:
    data = df[col].dropna()
    if len(data) > 0:
        min_val = data.min()
        max_val = data.max()
        range_val = max_val - min_val
        
        print(f"\n{col.upper():<20}")
        print(f"  Min:       {min_val:>15,.0f}")
        print(f"  Max:       {max_val:>15,.0f}")
        print(f"  Range:     {range_val:>15,.0f}")
        print(f"  Count:     {len(data):>15,.0f}")
        
        # Check for zeros (common in COVID data)
        zero_count = (data == 0).sum()
        if zero_count > 0:
            zero_pct = (zero_count / len(data)) * 100
            print(f"  Zero values:{zero_count:>15,.0f} ({zero_pct:.1f}%)")

# 2. IQR Analysis
print("\n" + "="*70)
print("2. INTERQUARTILE RANGE (IQR) ANALYSIS")
print("="*70)

print(f"\n{'Metric':<20} {'Q1':<12} {'Q3':<12} {'IQR':<12} {'IQR % of Range':<15}")
print("-" * 70)

for col in available_cols:
    data = df[col].dropna()
    if len(data) > 0:
        Q1 = data.quantile(0.25)
        Q3 = data.quantile(0.75)
        IQR = Q3 - Q1
        full_range = data.max() - data.min()
        iqr_percentage = (IQR / full_range) * 100 if full_range > 0 else 0
        
        print(f"{col:<20} {Q1:<12,.0f} {Q3:<12,.0f} "
              f"{IQR:<12,.0f} {iqr_percentage:<15.2f}%")

# 3. Outlier Analysis
print("\n" + "="*70)
print("3. OUTLIER ANALYSIS USING IQR METHOD")
print("="*70)

for col in available_cols[:2]:  # First 2 columns only
    data = df[col].dropna()
    if len(data) > 0:
        Q1 = data.quantile(0.25)
        Q3 = data.quantile(0.75)
        IQR = Q3 - Q1
        
        lower_bound = Q1 - 1.5 * IQR
        upper_bound = Q3 + 1.5 * IQR
        
        outliers = data[(data < lower_bound) | (data > upper_bound)]
        outlier_pct = (len(outliers) / len(data)) * 100
        
        print(f"\n{col.upper():<20}")
        print(f"  Outlier bounds: [{lower_bound:,.0f}, {upper_bound:,.0f}]")
        print(f"  Outliers found: {len(outliers):,} ({outlier_pct:.1f}%)")
        
        if len(outliers) > 0:
            print(f"  Outlier range: {outliers.min():,.0f} to {outliers.max():,.0f}")

# 4. Range by Time Period (if date available)
if 'date' in df.columns:
    print("\n" + "="*70)
    print("4. RANGE BY TIME PERIOD")
    print("="*70)
    
    df['date'] = pd.to_datetime(df['date'])
    df['year'] = df['date'].dt.year
    
    yearly_ranges = df.groupby('year')['new_cases'].agg(['min', 'max', lambda x: x.max() - x.min()])
    yearly_ranges.columns = ['min', 'max', 'range']
    
    print("\nYearly Range of New Cases:")
    print(yearly_ranges.to_string(float_format=lambda x: f'{x:,.0f}'))

print("\n" + "="*70)
print("ANALYSIS COMPLETE")
print("="*70)

Practical Applications in COVID-19 Analysis

✅ WHEN TO USE BASIC RANGE:

  • Data Quality Checks: Spot negative cases or impossible values
  • Initial Exploration: Quick sense of data spread
  • Extreme Scenario Planning: Prepare for worst-case numbers
  • Comparison with Known Limits: Check if values exceed theoretical maximums

🎯 WHEN TO USE IQR (SMART RANGE):

  • Typical Scenario Planning: Prepare for normal variations
  • Comparison Between Groups: Fair comparison ignoring outliers
  • Resource Allocation: Plan for expected, not extreme, needs
  • Statistical Modeling: Better for algorithms sensitive to outliers

Quick Reference Cheat Sheet

# BASIC RANGE CALCULATIONS
import pandas as pd
import numpy as np

# Simple range
df['column'].max() - df['column'].min()

# Multiple columns
df[['col1', 'col2']].agg(['min', 'max'])  # Get min/max for each

# IQR (Interquartile Range)
Q1 = df['column'].quantile(0.25)
Q3 = df['column'].quantile(0.75)
IQR = Q3 - Q1

# Custom percentile range
lower = df['column'].quantile(0.10)  # 10th percentile
upper = df['column'].quantile(0.90)  # 90th percentile
custom_range = upper - lower

# Rolling range (time series)
df['column'].rolling(window=7).max() - df['column'].rolling(window=7).min()

# Group-wise range
df.groupby('group_column')['value_column'].agg(lambda x: x.max() - x.min())

# Outlier detection with IQR
lower_bound = Q1 - 1.5 * IQR
upper_bound = Q3 + 1.5 * IQR
outliers = df[(df['column'] < lower_bound) | (df['column'] > upper_bound)]

# Range ratio metrics
range_val = df['column'].max() - df['column'].min()
mean_val = df['column'].mean()
range_to_mean_ratio = range_val / mean_val if mean_val != 0 else float('inf')

# Midrange
(df['column'].max() + df['column'].min()) / 2

Common Mistakes to Avoid

❌ DON'T MAKE THESE MISTAKES:

  1. Using only basic range: Always check IQR for realistic spread
  2. Ignoring zero values: In COVID data, zeros are meaningful (no cases reported)
  3. Comparing ranges directly: Use ratio metrics when scales differ
  4. Forgetting time dimension: Ranges change over pandemic phases
  5. Mixing populations: Compare ranges within same country/continent groups
  6. Ignoring data quality: Extreme ranges might indicate reporting errors

Key Takeaways

Range analysis in COVID-19 data reveals critical insights:

  1. Basic Range: Shows total possible spread (useful for worst-case planning)
  2. IQR: Shows typical spread (useful for normal operations)
  3. Small Range: Indicates consistent, predictable patterns
  4. Large Range: Indicates volatility, potential outbreaks, or data issues
  5. Zero Values Matter: Many countries report zero cases - this is meaningful!

📊 PRO TIP FOR COVID-19 ANALYSIS:

Always calculate BOTH basic range and IQR:
• Basic range tells you what's possible (for emergency planning)
• IQR tells you what's probable (for daily operations)
• The difference between them shows how much outlier risk exists

Final Thought: Range isn't just a number - it's a story about your data. In COVID-19 analysis, a small range might mean effective containment, while a large range could mean either a serious outbreak or inconsistent reporting. Always combine statistical analysis with domain knowledge!

Comments