# How to clean data in pandas with dropna()

The [Python pandas](https://www.ionos.ca/digitalguide/websites/web-development/python-pandas/) `DataFrame.dropna()` function is used to remove all rows or columns containing missing values (NaN) from a DataFrame. This makes it especially crucial for **preparing and cleaning data**.

## What is the syntax for pandas `dropna()`?

The `dropna()` function accepts **up to five parameters**. Here’s its syntax:

```python
DataFrame.dropna(axis=0, how='any', thresh=None, subset=None, inplace=False, ignore_index=False)
```

### Important parameters for `dropna()`

You can **use parameters to influence the behavior** of the pandas `DataFrame.dropna()` function. Here’s an overview of the most important ones:

<table>
  <thead>
    <tr>
      <th><strong>Parameter</strong></th>
      <th><strong>Description</strong></th>
      <th><strong>Default Value</strong></th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>`axis`</td>
      <td>Determines whether rows (0 or `index`) or columns (1 or `columns`) will be removed</td>
      <td>0</td>
    </tr>
    <tr>
      <td>`how`</td>
      <td>Specifies whether all (`all`) or only some (`any`) values must be NaN</td>
      <td>`any`</td>
    </tr>
    <tr>
      <td>`thresh`</td>
      <td>Specifies the minimum number of non-NaN values a row or column must have to avoid being removed; cannot be combined with `how`</td>
      <td>optional</td>
    </tr>
    <tr>
      <td>`subset`</td>
      <td>Specifies which rows or columns should be considered</td>
      <td>optional</td>
    </tr>
    <tr>
      <td>`inplace`</td>
      <td>Determines whether the operation is performed on the original DataFrame</td>
      <td>`False`</td>
    </tr>
    <tr>
      <td>`ignore_index`</td>
      <td>If `True`, the remaining axis is labeled from 0 to n-1</td>
      <td>`False`</td>
    </tr>
  </tbody>
</table>

## How to use pandas `DataFrame.dropna()`

Pandas `dropna()` is used to clean data before it’s analyzed. The removal of rows or columns with missing values helps to **prevent biases** in statistical evaluations. Since missing values can also lead to problems with data visualization, using the function is also **advantageous when creating charts** and reports.

### Removing rows with missing values

In the following example, we’ll take a look at a DataFrame containing NaN values:

```python
import pandas as pd
import numpy as np
# Creating a DataFrame with sample data
data = {
    'A': [1, 2, np.nan, 4],
    'B': [5, np.nan, np.nan, 8],
    'C': [9, 10, 11, 12]
}
df = pd.DataFrame(data)
print(df)
```

The DataFrame looks like this:

```none
A    B   C
0  1.0  5.0   9
1  2.0  NaN  10
2  NaN  NaN  11
3  4.0  8.0  12
```

Next, we’re going to apply the pandas `dropna()` function:

```python
## Remove all rows that contain at least one NaN value
df_cleaned = df.dropna()
print(df_cleaned)
```

Running the code above produces the following result:

```none
A    B  C
0  1.0  5.0  9
3  4.0  8.0 12
```

Since all the other rows contain NaN values, only the zeroth and third rows remain.

### Removing columns with missing values

Similarly, you can remove columns with missing values by setting the `axis` parameter to 1:

```python
## Remove all columns that contain at least one NaN value
df_cleaned_columns = df.dropna(axis=1)
print(df_cleaned_columns)
```

Column C is the only column that remains, since it’s the only one that doesn’t contain NaN values:

```none
C
0   9
1  10
2  11
3  12
```

### Using `thresh`

If you want to remove rows that contain fewer than two non-NaN values, you can use the `thresh` parameter:

```python
## Only keeps rows that have 2 or more non-NaN values
df_thresh = df.dropna(thresh=2)
print(df_thresh)
```

Running the code produces the following output:

```none
A    B   C
0  1.0  5.0   9
1  2.0  NaN  10
3  4.0  8.0  12
```

Row 1 is not removed from the output because it contains 2 non-NaN values (2.0 and 10).

### Using `subset`

The `subset` parameter allows you to specify the columns where the program should look for missing values. Only rows that contain missing values in the columns that have been specified will be removed.

```python
## Removes all rows where column A contains a NaN value
df_subset = df.dropna(subset=['A'])
print(df_subset)
```

Here, only the second row is removed. The NaN value in the first row is ignored due to the subset parameter, which only takes column A into consideration:

```none
A    B   C
0  1.0  5.0   9
1  2.0  NaN  10
3  4.0  8.0  12
```


This is a markdown version of: [https://www.ionos.ca/digitalguide/websites/web-development/python-pandas-dataframe-dropna/](https://www.ionos.ca/digitalguide/websites/web-development/python-pandas-dataframe-dropna/) for AI/LLM consumption.