Imputation Methods - Filling Missing Values

Filling Missing Values in Pandas-fillna()

Imagine your list of numbers has some empty spots (like missing puzzle pieces). We call them "NaN" (Not a Number). Instead of throwing them away, we can fill them with something smart. Pandas has a magic tool called fillna() to do this. Let's learn it like a game — super easy!

Our Example List

import pandas as pd
import numpy as np

s = pd.Series([1, np.nan, 3, np.nan, 5, np.nan, 7])
print(s)

Original (with gaps):

0    1.0
1    NaN
2    3.0
3    NaN
4    5.0
5    NaN
6    7.0
dtype: float64

The gaps are at positions 1, 3, and 5. Now, let's fill them in different fun ways! 🎨

1. Fill with a Fixed Thing (like 0 or a word)

Just put the same thing in every gap.

print(s.fillna(0))          # Fill with number 0

0    1.0
1    0.0
2    3.0
3    0.0
4    5.0
5    0.0
6    7.0

print(s.fillna('missing'))  # Fill with a word

0        1.0
1    missing
2        3.0
3    missing
4        5.0
5    missing
6        7.0

Easy, right? Like coloring all blanks with the same crayon.

2. Forward Fill (Copy from Above)

Copy the last good number forward — like repeating the previous answer.

print(s.ffill())  # New way (recommended!)
# Old way: s.fillna(method='ffill')

0    1.0
1    1.0
2    3.0
3    3.0
4    5.0
5    5.0
6    7.0

You can limit it: s.ffill(limit=1) — only fill one gap at a time (same here since no big gaps).

3. Backward Fill (Copy from Below)

Copy the next good number backward — like peeking ahead!

print(s.bfill())  # New way

0    1.0
1    3.0
2    3.0
3    5.0
4    5.0
5    7.0
6    7.0

4. Fill with Average (Mean or Median)

Use the average of the good numbers — fair and smart!

print(s.fillna(s.mean()))   # Mean = 4

0    1.0
1    4.0
2    3.0
3    4.0
4    5.0
5    4.0
6    7.0

Median (middle value) gives the same here: 4.

5. Fill Different Spots Differently

Choose exactly what to put in each gap (using a dictionary).

fill_values = {1: 100, 3: 300, 5: 500}
print(s.fillna(fill_values))

0      1.0
1    100.0
2      3.0
3    300.0
4      5.0
5    500.0
6      7.0

Perfect for when you know special values!

Quick Tips for You 🚀

• Use s.ffill() or s.bfill() instead of the old method= way — it's newer and better.
• Mean/median is great for numbers.

Keep playing with Pandas — it's fun! 🐼✨