Pandas & Data Analysis·Pandas from first Series to analysis project

Cleaning data

Handle missing values, duplicates and wrong types deliberately.

Beginner 14 min read Lesson 4 of 300 Challenge included

What you will be able to do

Count missing values before deciding.
Drop duplicates with subset and keep rules.
Convert types with astype or to_numeric.
Document every cleaning decision.

In plain English

Cleaning is a series of explicit decisions. Dropping rows is sometimes right; filling with a median or a sentinel is sometimes right. Record why you chose each one so the analysis is reproducible.

clean.pyPython

import pandas as pd

df = pd.DataFrame({
    'course': ['CSS', 'CSS', 'SQL', None],
    'minutes': [18, 18, None, 9],
})

df = df.drop_duplicates()
df['minutes'] = df['minutes'].fillna(df['minutes'].median())
df = df.dropna(subset=['course'])
print(df)

Your turn

Drop incomplete rows

Easy

Remove rows where the course column is missing.

PYTHON · starter

df = df.dropna()

Check your understanding before moving on.

Knowledge checkQuestion 1

What does fillna do?

Select an answer to continue

Study tools

Make it yours.

Keep a private note for this lesson, then turn the key ideas into flashcards you can revisit.

Sign in to save notes and flashcards.Both are private to your account and stay with your progress.

Finished reading?

Lock in what you just learned.

Mark this lesson complete to save your place and move one step closer to the certificate.

Next: Selecting and filtering

Lesson 4 of 300 · 0% of course