Detecting Missing Values in Pandas 🕵️♂️
Missing data is everywhere in real-world datasets — think NaN, None, or Pandas' newer pd.NA. Pandas makes it super easy to spot them in a Series using dedicated methods that return a boolean mask.
Key Detection Methods
.isna()/.isnull(): ReturnsTruefor missing values..notna()/.notnull(): ReturnsTruefor valid (non-missing) values.
Important notes:
isna()andisnull()are exact aliases — they do the same thing. Same fornotna()/notnull().- Pandas officially recommends
.isna()and.notna()for clarity. - These methods detect
np.nan,None, andpd.NAas missing.
Practical Example
import pandas as pd
import numpy as np
# A Series with various types of missing values
s = pd.Series([1, np.nan, 3, None, 5, pd.NA])
print(s)
Output (the Series itself):
0 1
1 NaN
2 3
3 None
4 5
5
dtype: object
Detecting Missing Values
print(s.isna()) # Recommended
print(s.isnull()) # Alias — identical result
Output:
0 False
1 True
2 False
3 True
4 False
5 True
dtype: bool
print(s.notna()) # Inverse
Output:
0 True
1 False
2 True
3 False
4 True
5 False
dtype: bool
All missing indicators (np.nan, None, pd.NA) are correctly flagged as True in isna().
Bonus Tips 🚀
- Count missing values:
s.isna().sum()→ 3 in this example. - Get missing values only:
s[s.isna()]→ returns the missing entries. - Get indices of missing values:
s.index[s.isna()]→ Index([1, 3, 5]).
These simple tools are your first step in any data cleaning workflow. Master them, and handling missing data becomes painless!
Happy Pandas-ing!
Comments
Post a Comment